Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 17
17
Goals/Objectives
Program Development
System Performance Measures to Monitor Progress
Component I
on Meeting Goals/Objectives
Performance Targets for Planning/Programming
Given
Resource Availability
Program Development and Project Selection
Specific Set of Programs and Projects with a Defined
Budget, Schedule, Scope
Project Delivery
Component II
Performance Measures to Track Delivery
Program/Project Implementation
System Monitoring
Component III
and Reporting
Result = System Performance
Source: Hendren, Neumann, and Pickrell, 2004.
Figure 2.3. Developing performance-based program and project delivery.
The focus also changes from considerations of system per- Literature Review
formance (mobility, safety, etc.) to organizational performance
(project delivery, quality, etc.). The literature review for this effort focused on materials rel-
Based on this review and, in following the principles above, evant to development of performance measures to support
an initial draft framework was developed for the SHRP 2 C02 capacity decision making. It considered measures relevant to
project. This initial framework had two basic dimensions the stages of the planning and project development process
(Table 2.1): from long-range planning through environmental review. It
primarily considers measures of physical impacts (i.e., amount
· The stage of project development (across the columns); and of congestion, level of environmental harm or benefit), and
· The factors considered such as mobility, safety, environment, not measures that address process and project delivery. Five
economics, and community (down the rows). key subject areas were reviewed:
These broad dimensions were adapted and refined as 1. General Use of Performance Measures by Transportation
described in chapter 3. Agencies;
OCR for page 18
18
Project Development Phases Relevant to SHRP 2 C02 Phases Potentially Relevant Phases With Limited Relevance
Long-Range Preprogram Environmental Monitoring and
Factors Planning Studies Programming Review Design Permitting Construction Operations
Objective Identify system needs Specify projects/ Identify mix of Select preferred Specify project Permits from Construct Operate the
and projects alternatives projects to be alternative completely resource capital system in
constructed agencies projects real time
Mobility Measures that identify Detailed project Select projects Measures of Work zone and Evaluate mobility
and prioritize needs analysis into program mobility reliability impacts
to meet needs impacts issues
Safety Measures that identify Detailed project Select projects Detailed measures Work zone Evaluate safety
and prioritize needs analysis into program of safety issues impacts
to meet needs impacts
Environment Early issue overview More detailed Criteria avoid Detailed Use of environ- Permit delivery Environmental
(check boxes) overview, but projects with environmental mental best monitoring
still summary fatal flaws impact practices
Economics Economic development/ Potential
impact measures selection
criteria
Community Early overview of issues Criteria avoid Detailed measures Use of public
(check boxes) in projects with of community involvement
regional/corridor negative impacts best practices
studies impacts
Other Reliability Project delivery Project delivery Operational
(let on time, (on time/on measures of
on budget) budget) reliability
Table 2.1. Initial Performance Measures Framework Issues
OCR for page 19
19
2. Performance Measures for Transportation System En- communicate with external audiences, and manage towards
hancement; strategic outcomes. A state DOT's strategic mission, vision,
3. Performance Measures for Environmental Stewardship; goals, and objectives are tied to day-to-day activities via regular
4. Performance Measures for Community Enhancement; and review of performance results in core DOT business functions
5. Performance Measures for Economic Impacts and Devel- such as pavement and bridge preservation, transportation
opment. safety, traffic operations, and infrastructure maintenance.
Performance-based management has three essential and
The first subject area describes the state of the practice of interconnected components: program development, project
transportation performance measurement. It reviews the most delivery, and system monitoring and reporting:
useful literature and describes the best practices and lessons
learned as performance measurement has matured in the trans- 1. Program Development Program Development typically
portation industry. The following four subject areas are tied begins with establishing agency goals and objectives that
to sections of the performance measurement framework, and are in turn monitored through performance measures.
are discussed in the relevant section of the report that addresses Taking into account resource constraints, performance
the specific issue (chapters 4 through 8). targets are set and projects and programs are identified
and selected based on performance criteria aimed to lead
a transportation agency toward its goals/objectives. For
Performance Management
example, a state DOT could identify the goal "preserve
No state DOT conducts highway capacity decision making the existing system" with the related performance mea-
in a vacuum. The performance measurement framework sure "percentage of highway miles with acceptable pave-
described in this section for supporting highway capacity ment condition." In turn, a project selection (or program
decision making is envisioned as part of a broader agencywide budgeting) criterion would be its estimated impact on
performance management system of the kind that some state highway pavement condition. The relationship between
DOTs are adopting. Performance management is a business performance targets and project selection is an iterative
process that links agency goals and objectives to resources process based on changing needs, available resources, and
and results. Figure 2.4 provides two examples of performance political support.
measure categories often used in making highway capacity 2. Program Delivery Program Delivery begins when a set
investment decisions safety and mobility and shows how of preselected projects are "passed off" to the delivery team.
performance management is used to link goals and outcomes A performance-based process uses a set of measures to
via use of measures. evaluate and monitor project implementation (e.g., per-
With a comprehensive performance management system in centage of construction contracts completed on time).
place, DOTs use performance measures systematically across Figure 2.2 illustrates not only the separation between the
a full range of core business functions to support decisions, selection and delivery of projects and programs, but also
Safety Mobility
Ensure High Standards of Provide for Efficient Movement
Goal Safety in the System of People and Goods
Reduce Rate of Motor Decrease Travel Times
Objective Vehicle Crashes for Commuting
Performance Crashes per VMT Hours-of-Delay
Measures Crashes per Capita Travel Time Index
Performance Reduce Crashes per VMT Reduce Delay by Two Percent
Target by One Percent per Year per Year Travel Time Index = 1.25
Figure 2.4. Example measures/targets for highway capacity investments.
OCR for page 20
20
that there are two distinct groups of performance measures. Transportation practitioners in both the private and pub-
One set of measures relates to project selection and is lic sectors have worked diligently to achieve the current
linked to agency goals and objectives, while the other set degree of performance measurement competency. Numer-
focuses on delivering projects. ous peer exchanges and three international invitation-only
3. System Monitoring and Reporting System Monitoring conferences have been devoted to the subject. The FHWA
and Reporting measures the changes that occur due to maintains a web-based Performance Measurement Exchange,
implemented projects and programs. It is the delivery of a thorough and well-maintained list of resources, including
projects that produces the "result", i.e., system performance. discussion boards, citations and links to journal articles,
Thus, this component of the process indicates whether the reports and studies, and a directory of practitioners (Perfor-
intended goals and objectives of the project have been met mance Measurement Exchange, 2007). Furthermore, literature
(e.g., reduction in travel delays). on the use of performance measures by transportation agen-
cies is extensive. It documents the wide range of activities in
The outcomes of performance-based management include which measures are relevant and informative. Several efforts
more efficient distribution of limited resources and increased comprehensively review the history, development, state of the
accountability and credibility in government. practice, best practices, and recent trends in performance-
based planning (Cambridge Systematics, Inc., 2000; Hendren
et al., 2005; Pickrell and Neumann, 2001; Poister, 1997; Pad-
General Use of Performance Measures gett, 2006). Strategic Performance Measures for State Depart-
by Transportation Agencies ments of Transportation: A Handbook for CEOs and Executives
Performance-based decision making has increasingly been (TransTech Management Inc., 2003) describes the core areas
used by transportation agencies over the past 10 years in a in which most DOTs use measures, and the three primary
reasons they use measures: communication, management,
range of contexts and applications. Defined as a systemic,
and decision making. NCHRP Report 551: Performance Mea-
ongoing process integrated into an agency's planning, manage-
sures and Targets for Transportation Asset Management reviews
ment, and decision-making activities (Pickrell and Neumann,
the literature and addresses several key questions:
2001), it involves a continuous effort of monitoring and feed-
back that improves decision-making capabilities over time.
· What criteria are used by agencies to select performance
Though transportation agencies have employed performance
measures?
measures since the 1950s, the most dramatic evolution of the
· How are current performance measurement frameworks
state of the practice has occurred in the last two decades.
structured, and what kinds of measures do they include?
Improvements have been driven by:
· How are performance measures being used to gauge the
impacts of transportation investments, support resource
· Demands from the public and elected officials for increased
allocation and utilization decisions, and assess agency per-
accountability and performance; formance in program delivery and cost-effectiveness?
· Emergence of management systems for pavement, bridges,
· How are measures being tailored for different levels of trans-
and congestion; portation organizations?
· Strong leadership within the agencies themselves that bor- · How are measures being used to communicate program
rowed from private sector-driven initiatives like "Six Sigma" status both internally and externally?
and "Baldridge Awards"; and
· Recognition that the decision-making environment within NCHRP Report 446: A Guidebook for Performance-Based
which agencies operate has become more complex (Poister, Transportation Planning provides practitioners an extensive
2005; Larson, 2005; Bremmer et al., 2005). library of measures organized by the following categories:
Examining the use of performance measures by transporta- · Accessibility;
tion agencies is particularly informative. The Government Per- · Mobility;
formance Project (GPP), a nonpartisan, independent research · Economic development;
program of the Pew Center on the States, acknowledges many · Quality of life;
DOTs are leaders in their use of performance measurement and · Environmental and resource conservation;
can serve as models for other state agencies (Government Per- · Safety;
formance Project, 2005). Federal transportation agencies are · Operation efficiency;
often asked to pilot performance measure processes, and state · System preservation; and
DOTs are often asked to help other agencies with performance- · Measures relevant to multiple goal categories (Cambridge
based initiatives (Poister, 2005). Systematics, Inc., 2000).
OCR for page 21
21
Key Findings measures become focused on the "vital few," and technical
capabilities advance. Performance-based processes should
The extensive literature on transportation performance mea-
inform the decision-making process, not replace it, and initial
sures points to several general trends.
impacts on final decisions may be subtle. Decision makers
Performance measures should be driven by strategically
may initially be slow to accept performance-based recom-
aligned goals and objectives. Performance measures should
mendations of staff, but they will adopt the additional infor-
be identified in response to goals and objectives and not the
mation into their decision-making process at varying rates as
other way around. An agency's goals should reflect the most
agencywide support for the approach builds. Staff should not
important aspects of what it wishes to accomplish. Perfor-
give up as providing better quality data from performance
mance measures are the primary means of assessing how suc-
measuring efforts can only improve the decision-making
cessful an agency is in accomplishing its goals. Therefore, it
process (Pickrell and Neumann, 2001).
should be clear what goal(s) each measure illustrates advance-
Performance measurement programs are most effective
ment of. Failure to properly align measures with goals and
when integrated throughout an organization. The linkage
objectives can result in tracking measures that have little to do
between program development and project delivery is a vital
with performance of the organization or transportation sys-
tem (Pickrell and Neumann, 2001; Poister et al., 2004; Kittelson component to mature, integrated, performance-based method-
& Associates, Inc. et al., 2003). ologies. Few state transportation agencies have implemented
Input, output, and outcome measures should all be both, and fewer have linked the two approaches together. In
included in performance measurement. Many of the mea- isolation, a performance-based program development process
sures used to monitor systems operations are derived from may identify the best projects to fund but may not guard
those developed in the 1950s, and these measures reflect the against excessive scope creep, schedule slippage, or cost esca-
values of that time, which have evolved significantly. As nega- lation. Conversely, a performance-based delivery process in
tive impacts from transportation and other infrastructure isolation may result in efficient delivery of a program that
investments have become apparent, mainstream concerns includes marginal projects. By executing both performance-
have changed to consider a wider range of ways in which based program development and project delivery, the most
transportation affects our communities (Meyer, 2001). The effective set of projects is selected and implemented efficiently
type of measures considered for transportation planning has (Hendren et al., 2005; Padgett, 2006).
grown to include not only those that consider input (time, Performance measurement reporting should be appro-
capital, resources) and output (speed, throughput, conges- priately tailored to intended audiences. The informational
tion), but also those that consider outcomes to communities needs of technical staff, decision makers, and the public are
and the environment (Poister, 1997; Poister and Van Slyke, different, and presentation and depth of reporting should
2001). Consideration of both output and outcome measures reflect these needs (Shaw, 2003; Cambridge Systematics, Inc.,
reflect the differences in perspectives of those who manage 2006; Poister, 2005). TCRP Report 88, A Guidebook for Devel-
the system and those who use it (Shaw, 2003; Kittelson & oping a Transit Performance-Measurement System highlights
Associates, Inc. et al., 2003). this diversity, noting the difference in perspective of four
Performance measurement efforts should concentrate different transit stakeholders: the customer, community,
on the "vital few." Agencies must use simple, understand- agency, and driver. In addition to delivering the right data
able measures and avoid attempting to measure everything to the right people, reporting should be as simple and con-
(TransTech Management, Inc., 2006; Larson, 2005; Kassof, sistent as possible. Aside from providing clarity for decision
2001). Providing excessive or redundant measures may over- makers, simplicity and regularity facilitate process improve-
whelm the end user and obscure key drivers of service quality. ment of each iteration of the performance-based process
As cited in TCRP Report 88, Brown (1996) describes this as (Larson, 2005).
choosing between "the vital few measures and the trivial many," Successful performance measurement programs require
and suggests an upper limit of 20 measures. Some of the desired high-level buy-in. A performance-based approach must have
"vital few" may not be available during the first iteration of widespread and deep-rooted support to withstand significant
a performance-based process. This deficiency should be changes in leadership. Five key stakeholder groups must accept
addressed in future efforts (Larson, 2005). a program for it to have long-term viability:
Early attempts at performance measurement should
emphasize process as well as results. Management and staff · Agency management;
should set realistic expectations about first iteration results as · Agency staff;
performance-based planning is an inherently incremental · Customers;
process. The implementation strategy will evolve over time as · Agency's governing body; and
stakeholder and leadership buy-in improve, performance · Senior contractors (Kittelson & Associates, Inc. et al., 2003).
