National Academy of Sciences | 150 Year Anniversary

Questions? Call 800-624-6242

| Items in cart [0]

The National Academies Press

Rights & Permissions

topleft topright

NCFRP Report 10: Performance Measures for Freight Transportation (2011)
National Cooperative Freight Research Program (NCFRP)

Citation Manager

Transportation Research Board. "Learning to Support Measurement Systems." NCFRP Report 10: Performance Measures for Freight Transportation. Washington, DC: The National Academies Press, 2011.

Please select a format:

BibTeX EndNote RefMan


Page
28
bottomleft bottomright
Page
28
Front Matter (R1-R9)
Summary (1-18)
Background on Research Need (19-21)
Research Approach (22-22)
Anticipated Use of Research Findings (23-23)
Endnotes (24-24)
From Lagging to Leading Indicators (25-25)
From Measuring Process to Measuring Strategic Outcomes (26-26)
From Skewing Performance to Balancing Competing Objectives (27-27)
Learning to Support Measurement Systems (28-28)
Endnotes (29-29)
Migration of Performance Measures from the Private to Public Sectors (30-30)
The Maturation of Public-Sector Performance Measurement (31-31)
States Use Only a Handful of Freight Measures (32-33)
Endnotes (34-34)
Rail Data (35-35)
Emerging but Incomplete National Measures (36-36)
Lack of Well-Defined Goals (37-37)
Endnotes (38-39)
Private-Sector Perspectives (40-40)
Responses to Individual Measures (41-43)
State Perspectives (44-44)
Federal Agency Perspectives (45-45)
Railroad Industry Perspectives (46-46)
Maritime Industry Perspectives (47-47)
Additional Practitioners (48-48)
Endnotes (49-49)
Freight Data Issues (50-52)
Case Studies (53-56)
Data Considerations for the Freight Report Card (57-59)
Endnotes (60-60)
Overcoming Impediments (61-61)
Creating a Coalition (62-63)
Appendix A - Summaries of Freight Performance Information for National Report Card Performance Summaries (64-94)
Appendix B - Statewide and Metropolitan Freight Performance Metrics Examples (95-110)
Appendix C - State-Level Freight Performance Measures: State of Practice (111-114)
Appendix D - National-Level Performance Measures: State of Practice (115-117)
Appendix E - Modal Freight Performance Measures: State of Practice (118-128)
Appendix F - Environmental Freight Performance Measures: State of Practice (129-133)
Appendix G - Stakeholder Perspectives (134-159)
Abbreviations and acronyms (160-160)

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 28
28 · Have measures reflect customers' perspectives; data-system compatibility processes must be addressed. Most · Have measures derived directly from the organization's organizations are large, with multiple systems developed in strategic goals; different years, with different technology. Reconciling and · Incorporate the dynamic "continuous improvement" ethos integrating legacy systems into a common performance- of the "Quality Movement" as a basic measure of a success- measurement reporting process can require significant effort. ful organization. Aligning incentives or consequences is important because, if the measurement results lack consequences or do not spur improvement efforts, the measures lack relevance and tend to atrophy. In the private sector, consequences can be in the Learning to Support form of profit, loss, or market share. In the public sector, Measurement Systems the consequences can come in the form of executives initiat- An important lesson that was repeatedly emphasized by ing improvement efforts if the measures indicate that perfor- the private-sector literature is that a Balanced Scorecard and mance targets have not been reached. It is generally agreed other performance measurement processes are systems.9 Like in the literature that performance measurement systems that all systems, they need constant maintenance, support, and do not relate directly to key organizational consequences or refreshing to keep them current. At least five critical steps are outcomes tend to atrophy. necessary to establish a performance measurement system: In a Balanced Scorecard framework another step comes from collecting data from outside points of comparison for 1. Developing an information architecture; benchmarking, peer comparison, or customer-satisfaction 2. Putting technology in place to support the architecture; surveys. This can come in a variety of forms such as opinion 3. Aligning incentives with the new system; surveys and comparative analyses with peer groups. 4. Drawing on outside resources, such as benchmarking or The final step is creation of an ongoing process to sus- customer-survey resources; tain and perpetuate the performance measurement system. 5. Designing an ongoing maintenance and support process Because the measurement process is a system, it requires to perpetuate steps 1­4. ongoing resources to perpetually collect data, categorize it, review its quality, and disseminate the data. An information architecture has been described as, "an Brue12 stresses that measures must be customer-focused umbrella term for the categories of information needed to and succinct--yet at the same time allow granularity when manage a company's businesses, the methods the company necessary to drill down into problems. He describes efforts uses to generate this information, and the rules regulating by companies to select the correct measures after reviewing its flow."10 Key steps in developing the architecture are the up to 1,000 internal processes, each of which had up to 120 definition of measures that translate the organization's goals internal technical specifications. Such a volume of measures into specific actions and identification of a reporting process is impractical, and they need to be consolidated into compos- to capture those carefully defined measures. It is necessary to ite measures that allow the high-level tracking of two major develop a set of definitions, a taxonomy, and even technical issues--customer satisfaction and financial viability. The manuals to clarify how to identify, collect, and classify the lower-level process information is critical, but only to those results of activities into the proper set of measures. process owners. Putting technology in place requires integrating existing Brue describes what could be termed the "accordion" syn- data and creating processes to capture needed but lacking drome, which was frequently alluded to in the 1990s litera- data. In the private sector, financial measures tend to be the ture and which is being addressed by many organizations in most accessible because of the long history of public account- the 2000s. First, managers hungrily consumed measures and ing rules. In comparison, measures such as customer satisfac- kept broadly expanding them across a wide array of activi- tion, quality, and innovation are harder to obtain because of ties until the number of measures swamped organizational their lack of maturity in most organizations. These data also decision makers. Then, in reaction, the managers narrowed tend to be captured less frequently, such as annually or quar- the array of measures that they tracked regularly. However, terly through sample-based surveys. Also necessary is a set of a deep and detailed array of "process" measures need to be rules and protocols about who collects the data, who gener- available when processes break down and those processes ates them, who receives and analyzes them, who can change needed to be reviewed. These process measures may not be the architecture rules, and who takes action when the data regularly reviewed by senior leadership but are drawn on reveal a problem.11 Once the architecture is in place, then the when needed.