National Academies Press: OpenBook

Performance Measures for Freight Transportation (2011)

Chapter: Chapter 2 - Performance Measurement Lessons from the Private Sector

« Previous: Chapter 1 - Research Objective
Page 27
Suggested Citation:"Chapter 2 - Performance Measurement Lessons from the Private Sector." National Academies of Sciences, Engineering, and Medicine. 2011. Performance Measures for Freight Transportation. Washington, DC: The National Academies Press. doi: 10.17226/14520.
×
Page 27
Page 28
Suggested Citation:"Chapter 2 - Performance Measurement Lessons from the Private Sector." National Academies of Sciences, Engineering, and Medicine. 2011. Performance Measures for Freight Transportation. Washington, DC: The National Academies Press. doi: 10.17226/14520.
×
Page 28
Page 29
Suggested Citation:"Chapter 2 - Performance Measurement Lessons from the Private Sector." National Academies of Sciences, Engineering, and Medicine. 2011. Performance Measures for Freight Transportation. Washington, DC: The National Academies Press. doi: 10.17226/14520.
×
Page 29
Page 30
Suggested Citation:"Chapter 2 - Performance Measurement Lessons from the Private Sector." National Academies of Sciences, Engineering, and Medicine. 2011. Performance Measures for Freight Transportation. Washington, DC: The National Academies Press. doi: 10.17226/14520.
×
Page 30
Page 31
Suggested Citation:"Chapter 2 - Performance Measurement Lessons from the Private Sector." National Academies of Sciences, Engineering, and Medicine. 2011. Performance Measures for Freight Transportation. Washington, DC: The National Academies Press. doi: 10.17226/14520.
×
Page 31

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

25 c h a p t e r 2 Performance Measurement Lessons from the Private sector The following chapter examines the private sector’s use of performance measures, and how that use has evolved over the preceding five decades. The evolution of the private sector’s use of measures was examined to anticipate how a national reporting system may need to be structured to meet evolv- ing measurement needs. For instance, a key lesson from the private sector is that lagging measures alone soon prove to be inadequate for decision making. Therefore, the proposed framework includes a strong component of leading indica- tors. That and other private-sector findings are described. The Evolution of Private- Sector Measurement The development of performance measures for the national freight network is belated in comparison to the extensive development of performance measures at state departments of transportations. The transportation agencies, in turn, were belated in comparison to the private sector. Business literature extensively discussed performance measures in the 1950s. By 1993, the Government Performance and Results Act (the Results Act), required federal departments and programs to adopt goals and performance measures to track progress toward those goals. By the late 1990s, transportation agencies were regularly developing performance metrics. This late development of measures for the freight sector has one significant advantage. It allows the nascent effort to benefit from the evolution, mistakes, missteps, and lessons of several generations of performance measure development in other sectors. The research effort examined private-sector performance management for two reasons. First, the research statement called for performance measures that would be of interest to the private sector. Second, the lessons learned during the long evolution of private-sector performance measure ment provides insights into how to approach the development of measures for this project. The review of the private sector’s literature reveals a con- tinuous path of development and evolution as measures mature and evolve to address shortcomings perceived in earlier generations of measures. The literature reveals that the individual measures used in the private sector may have little direct relevance for national freight performance mea- sures. However, the process by which private-sector decision makers identify, collect, and assess measures holds significant relevance for the public-sector decision makers who want to create a set of national freight performance measures. From Lagging to Leading Indicators Peter Drucker1 wrote that 70 years ago General Motors pioneered modern cost accounting systems and used its performance output for important resource-allocation and decision-support performance measures. The “Manage- ment by Objective” that Drucker popularized arose from his 1954 book, The Practice of Management. In later years after decades of observation, Drucker wrote that it is possible to define predictable evolutionary paths in organizations that have embraced performance measures. Initially, organiza- tions embraced financial measures, such as Internal Rate of Return, Cash Flow, Liquidity, Return on Assets, and other similar measures. He labeled these “foundational” measures. He noted that they are inherently “backward looking” and lacking in granularity. They may tell if the firm is performing poorly or well but not why. The lack of specific performance insight led to the next evolutionary stage of measurement, which was “Productivity” measurement. These measures were intended to “drill down” into productivity within an organi- zation and date from approximately the WWII era. The third set of measures evolved in the 1990s and are what Drucker described as “Competency” or “Innovation” measures. These are most common in the private sector and relate to whether a company possesses “best in class” or unique skills that dif-

26 ferentiate it from the competition. In the public sector, these sets of measures may be similar to benchmarking measures with other comparable organizations. The fourth and final evolution of private-sector performance measures relates to “Resource Allocation.” Those measures evaluate different sets of potential investments to determine which provide the optimum return. To restate, Drucker described four types of private- sector performance measures, which evolved in approximate order of: • Foundational or basic financial measures; • Productivity or internal performance measures; • Competency or innovation measures comparing to exter- nal performance; • Resource allocation or investment-tradeoff allocation measures. Drucker and others have noted that this evolution is the result of trial and error over decades of well-intentioned efforts by decision makers to understand which measures provide critical insight into their companies. In reviewing decades of private-sector performance measure develop- ment, Drucker, Porter (2002), and Frigo (2002) stressed the need for performance measures to be properly aligned with the strategic direction or desired strategic outcomes of the organization. All three noted that organizations have developed performance measures only to be frus- trated that they did not provide true insight, they created unintended disincentives, or they failed to measure cus- tomer satisfaction. Drucker’s findings that executives quickly grow dissatisfied with backward-looking, or lagging, indicators influenced the development of the Freight System Report Card. The report’s inclusion of predictive indicators is a direct result of the Drucker finding. From Measuring Process to Measuring Strategic Outcomes That strategy and performance measurement are insepa- rable2, 3 is another lesson from the private-sector experience with performance measurement. Because performance measures drive organizational behavior, a clear linkage between the organization’s goals and the activities the organization encourages is critical. Several private-sector authors emphasized the need to first conduct a strategic planning exercise to clarify organizational goals before identifying measures to gauge organizational effectiveness. Effectiveness should be considered in terms of achievement of institutional goals. The first questions some managers ask when embarking on a performance measure initiative are “What should we measure?” or “How should we measure performance in a given area?” In fact, these are the last questions management should focus on. Strategic performance measurement systems, like the balanced score card, are first and foremost about strategy. Strategic performance measurement begins with a sound philosophy pertaining to and a sound judgment surrounding how strategic decisions will be made and how performance mea- surement will be used to make decisions and execute strategy. Management must be vigilant in aligning performance measures with the strategy of the organization. . . .4 Wade and Recardo5 described the common reaction of sea- soned corporate managers who quickly grew disenchanted with performance measurement systems in the 1990s: Traditional corporate-level performance measures—financial and gross productivity results—have failed most corporations. Managers have become disillusioned with these “trailing” per- formance measures, because they have not helped them run the business. Savvy companies have learned that performance mea- sures, used diligently, significantly affect organizational align- ment. CEOs want performance measures that offer predictive power and provide a better understanding of the real costs asso- ciated with each process. Authors describe at least four crucial strategy- development steps that need to precede the identification of measures so that the measures do more than only look backward. First, identify proper goals that serve the customer. Second, identify the different aspects of the organization or system and have goals and strategy for each. Third, understand how the orga- nization serves its industry. Fourth, understand that many discrete activities must work in harmony to create an organi- zation’s success.6 By having predictive measures that provide insight as to whether the organization’s current activities are leading it to future success has improved the usefulness of many organization’s performance measurement systems. These writers conclude that more than five decades of Performance Measurement in the private sector led prac- titioners to reach three overriding conclusions. First, truly sound and effective measures must relate directly to cus- tomer satisfaction. Measuring success of processes and sub- processes that do not directly relate to satisfied customers does not guarantee success. Only satisfied customers guar- antee an organization’s success. Second, measures need to be balanced. That is they need to allow a holistic understanding of financials, processes, comparisons to peers, and customer satisfaction. Third, measures must capture an organization’s ability to learn, innovate, and improve the quality of its products. “Quality measures represent the most positive step taken to date in broadening the basis of business performance measurement,” was one typical conclusion.7

27 From Skewing Performance to Balancing Competing Objectives A common shortcoming described in private-sector per- formance measurement literature related to skewing of behavior or “suboptimization.” This refers to organizational performance focusing inordinately upon achieving narrow, measured activities to the detriment of other important organizational goals. For instance, an unbalanced focus upon product cost could lead to fatal lack of product quality, which dooms the business. Or a unit can be measured for timeliness of a process, but not the cost or quality of the process. Orga- nizations also frequently measured performance of individ- ual divisions, or “silos,” which can cause a disincentive for the divisions to devote resources to collaborating with other divisions. If the performance measurement incentives did not reward cross-divisional collaboration, then such collabora- tion was less likely to be achieved. Thus, in 1992, evolved the Balance Scorecard.8 The score includes at least four categories of measures, which reflect the tension between important considerations, such as cost ver- sus quality or timeliness versus completeness or profitability versus customer satisfaction. The scorecard was developed to address the type of trade-off analysis and balancing of com- peting values that organizations frequently confront. The Balanced Scorecard also was proposed because man- agers complained of being swamped with too many mea- sures. A proliferation of measures left executives data rich and information poor. The Balanced Scorecard was created to answer four basic questions: • How do customers see the organization? • What must the organization excel at? • How can the organization continue to improve and create value? • How does the organization fare financially? The Balanced Scorecard attempts to assemble in a single report the disparate and often competing values that must be addressed. Inherent in the Balanced Scorecard is the recogni- tion that judgments must be made by executives. Although metrics provide insight, ultimately judgments are made to balance issues such as cost versus quality, profitability versus social obligations, and between customer satisfaction and available resources. See Figure 2.1. This new generation of performance measurement as reflected in the Balanced Scorecard does not abandon the earlier four types of measures that Drucker had written about. Foundational measures are still used to measure basic financial and performance outputs. Operational measures still allow managers to drill down into areas that don’t meet customer needs. Competency or benchmarking measures are used in the “Innovating and Learning Perspective.” Finally, the Resource Allocation measures still are inherent within all four sectors as measures to help make intelligent investment decisions. What the Balance Scorecard evolution has done is to: • Sharpen measures into a “critical few”; • Acknowledge the need for artful trade-offs to achieve opti- mum overall performance; 4 versus completeness or profitability versus customer satisfaction. The scorecard was developed to address the type of trade-off analysis and balancing of competing values that organizations frequently confront. The Balanced Scorecard also was proposed because managers complained of being swamped with too many measures. A proliferation of measures left executives data rich and information poor. The Balanced Scorecard was created to answer four basic questions: • How do customers see the organization? • What must the organization excel at? • How can the organization continue to improve and create value? • How does the organization fare financially? The Balanced Scorecard attempts to assemble in a single report the disparate and often competing values which must be addressed. Inherent in the Balanced Scorecard is the recognition that judgments must be made by executives. Although metrics provide insight, ultimately judgments are made to balance issues such as cost versus quality, profitability versus social obligations and between customer satisfaction and available resources. See Figure 2.1. Figure 2.1. A Balanced Scorecard. This new generation of Performance Measurement as reflected in the Balanced Scorecard does not abandon the earlier four types of measures that Drucker had written about. Foundational measures are still used to measure basic financial and performance outputs. Operational measures still allow managers to drill down into areas that don’t meet customer needs. Competency or benchmarking measures are used in the “Innovating and Learning Perspective.” Finally, the Resource Allocation measures still are inherent within all four sectors as measures to help make intelligent investment decisions. What the Balance Scorecard evolution has done is to: • Sharpen measures into a “critical few” ; Financial Perspective Goals Measures Innovation and Learning Goals Measures Business Operations Goals Measures Customer Perspective Goals Measures Figure 2.1. A Balanced Scorecard.

28 • Have measures reflect customers’ perspectives; • Have measures derived directly from the organization’s strategic goals; • Incorporate the dynamic “continuous improvement” ethos of the “Quality Movement” as a basic measure of a success- ful organization. Learning to Support Measurement Systems An important lesson that was repeatedly emphasized by the private-sector literature is that a Balanced Scorecard and other performance measurement processes are systems.9 Like all systems, they need constant maintenance, support, and refreshing to keep them current. At least five critical steps are necessary to establish a performance measurement system: 1. Developing an information architecture; 2. Putting technology in place to support the architecture; 3. Aligning incentives with the new system; 4. Drawing on outside resources, such as benchmarking or customer-survey resources; 5. Designing an ongoing maintenance and support process to perpetuate steps 1–4. An information architecture has been described as, “an umbrella term for the categories of information needed to manage a company’s businesses, the methods the company uses to generate this information, and the rules regulating its flow.”10 Key steps in developing the architecture are the definition of measures that translate the organization’s goals into specific actions and identification of a reporting process to capture those carefully defined measures. It is necessary to develop a set of definitions, a taxonomy, and even technical manuals to clarify how to identify, collect, and classify the results of activities into the proper set of measures. Putting technology in place requires integrating existing data and creating processes to capture needed but lacking data. In the private sector, financial measures tend to be the most accessible because of the long history of public account- ing rules. In comparison, measures such as customer satisfac- tion, quality, and innovation are harder to obtain because of their lack of maturity in most organizations. These data also tend to be captured less frequently, such as annually or quar- terly through sample-based surveys. Also necessary is a set of rules and protocols about who collects the data, who gener- ates them, who receives and analyzes them, who can change the architecture rules, and who takes action when the data reveal a problem.11 Once the architecture is in place, then the data-system compatibility processes must be addressed. Most organizations are large, with multiple systems developed in different years, with different technology. Reconciling and integrating legacy systems into a common performance- measurement reporting process can require significant effort. Aligning incentives or consequences is important because, if the measurement results lack consequences or do not spur improvement efforts, the measures lack relevance and tend to atrophy. In the private sector, consequences can be in the form of profit, loss, or market share. In the public sector, the consequences can come in the form of executives initiat- ing improvement efforts if the measures indicate that perfor- mance targets have not been reached. It is generally agreed in the literature that performance measurement systems that do not relate directly to key organizational consequences or outcomes tend to atrophy. In a Balanced Scorecard framework another step comes from collecting data from outside points of comparison for benchmarking, peer comparison, or customer-satisfaction surveys. This can come in a variety of forms such as opinion surveys and comparative analyses with peer groups. The final step is creation of an ongoing process to sus- tain and perpetuate the performance measurement system. Because the measurement process is a system, it requires ongoing resources to perpetually collect data, categorize it, review its quality, and disseminate the data. Brue12 stresses that measures must be customer-focused and succinct—yet at the same time allow granularity when necessary to drill down into problems. He describes efforts by companies to select the correct measures after reviewing up to 1,000 internal processes, each of which had up to 120 internal technical specifications. Such a volume of measures is impractical, and they need to be consolidated into compos- ite measures that allow the high-level tracking of two major issues—customer satisfaction and financial viability. The lower-level process information is critical, but only to those process owners. Brue describes what could be termed the “accordion” syn- drome, which was frequently alluded to in the 1990s litera- ture and which is being addressed by many organizations in the 2000s. First, managers hungrily consumed measures and kept broadly expanding them across a wide array of activi- ties until the number of measures swamped organizational decision makers. Then, in reaction, the managers narrowed the array of measures that they tracked regularly. However, a deep and detailed array of “process” measures need to be available when processes break down and those processes needed to be reviewed. These process measures may not be regularly reviewed by senior leadership but are drawn on when needed.

29 From Measuring Performance to Improving Performance The private sector’s collection and review of performance measures beginning in the 1950s led to the eventual develop- ment of “quality” systems in the 1980s. The “Total Quality Management” concepts developed by W. Edwards Deming resulted in large part by a rigorous review of performance data. As the source of failure to achieve targets was analyzed systematically, then “continuous improvement” developed. The continuous improvement processes took the form of Total Quality Management, International Standards Organi- zation (ISO) processes, Six Sigma, the Baldrige process, or the Japanese Kaizen process. As noted earlier, measurement systems that are not tied to some consequence or action tend to atrophy. Those systems that are tied to consequences have contributed to continuous improvement efforts. Metrics necessary for continuous improvement fall into two categories—those to measure customer satisfaction in its various forms and those to measure the processes that create the products the customers use. When customers are dissatisfied, investigation occurs into the organizational per- formance that led to the dissatisfaction. Therefore, many modern performance measurement sys- tems include both quantitative and qualitative measures.13 The quantified measures will be based on process, outcome, or financial measurements, while the qualitative ones will be based upon the perceived satisfaction of customers, employ- ees, and other stakeholders. Employee satisfaction and cus- tomer satisfaction are the most common qualitative key per- formance measures.14 Relevance of the Private- Sector Lessons The private-sector performance measurement lessons include: • Organizations that are highly experienced in evolving gen- erations of performance metrics have learned that they soon grow dissatisfied with static, lagging indicators that measure only past performance. Such measures may be all organizations initially have, but they quickly prove inad- equate to provide insight into future performance; • Current performance measurement is heavily invested in measuring customer satisfaction and system performance from the customer perspective; • Leading indicators that provide insight into likely future performance are strongly desired; • Performance measures must be part of an ongoing sys- tem that has its own architecture, data system rules, and grammar and a control process that keeps it accurate, current, and relevant; • Successful measurement systems overcome a contradic- tion. They must be high-level and simple while allowing granularity to drill down into processes if the high-level measures indicate a breakdown in performance; • Private companies struggle to get good performance data from within their own organizations, which only further highlights the challenge of getting consistent data from public and private sources for a set of national freight per- formance measures; • Modern private-sector performance measures are used to drive organizational strategy; • Performance measurement systems that become integral to an organization tend to drive “continuous improvement” efforts, while systems that are not integral tend to atrophy. Endnotes 1 Drucker, Peter. The Information Executives Truly Need (Harvard Business Review, Jan./Feb. 1992). In Harvard Business Review on Measuring Corporate Performance, Harvard Business School Press, Boston, Mass., 1998, pp. 1–24. 2 Frigo, M. L. Strategy-Focused Performance Measures, Strategic Finance, Sept. 2002. 3 Porter, Michael. The Importance of Being Strategic, Balanced Scorecard Re- port, March/April 2002, p. 9. 4 Frigo, Strategy-Focused Performance Measures, accessed at http://www.all- business.com/management/benchmarking-strategic-planning/265889-1. html, Feb. 14, 2011. 5 Wade, David, and Ronald Recardo. Corporate Performance Management, Butterworth Heinemann, Burlington, Mass., 2001, p 2. 6 Porter, The Importance of Being Strategic, p. 5. 7 Eccles, Robert G. The Performance Measurement Manifesto 1991 (Harvard Business Review, 1991). In Harvard Business Review on Measuring Corporate Performance, 1998, pp. 25–45. 8 Kaplan, Robert S., and David P. Norton. The Balanced Scorecard – Measures that Drive Performance (Harvard Business Review, Feb. 1992). In Harvard Business Review on Measuring Corporate Performance, Harvard Business School Press, Boston, Mass., 1998. 9 Eccles, The Performance Measurement Manifesto, pp. 25–45. 10 Eccles, The Performance Measurement Manifesto, pp. 25–45. 11 Eccles, The Performance Measurement Manifesto, pp. 25–45. 12 Brue, Greg. Six Sigma for Managers, McGraw-Hill, 2002, pp. 36–61. 13 Wade and Recardo, Corporate Performance Management, p. 12. 14 Wade and Recardo, Corporate Performance Management, p. 13.

Next: Chapter 3 - Performance Measurement Experience in the Public Sector »
Performance Measures for Freight Transportation Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Freight Research Program (NCFRP) Report 10: Performance Measures for Freight Transportation explores a set of measures to gauge the performance of the freight transportation system.

The measures are presented in the form of a freight system report card, which reports information in three formats, each increasingly detailed, to serve the needs of a wide variety of users from decision makers at all levels to anyone interested in assessing the performance of the nation’s freight transportation system.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!