Click for next page ( 35

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 34
34 APPENDIX A Key Literature This section provides an excerpt of the literature that was the public are increasingly aware of the external impacts of reviewed for this project with a focus on material that was the transportation system on the economy, the environ- used to produce the guidebook. A more comprehensive liter- ment, and surrounding communities. With limited trans- ature review is available on request as part of the final report. portation budgets, there is increasing pressure on public of- ficials and transportation agencies to ensure that projects that are funded are those with the best overall value and least The Case for System-Level negative externalities. Selecting projects in this context re- Performance Measures quires broad knowledge of existing system performance and Developing and monitoring network performance mea- the ability to evaluate the costs and benefits of alternatives. sures requires communication and coordination between in- Consideration of operations solutions over new construc- dividuals who plan the transportation system, develop policy, tion. As construction costs climb, federal and state trans- and manage operations. Existing system-level performance portation trust funds decline, and highway systems become measures have come together through various collaborations built out, the focus of most transportation agencies is shift- between mixed groups of state agencies, MPOs, local govern- ing from capacity improvements to maximizing operational ments, transit agencies, and others. Several common ele- efficiency (Brydia et al., 2007; Cambridge Systematics, 2005, ments exist between these collaborations, which may begin 2007; Hendren and Myers, 2006; Meyer, 2001; Randall, to provide a framework that guides agencies when develop- 2007). With this change, DOTs and MPOs have begun ex- ing and implementing system-level performance measures. amining the linkages between operations and other agency Some of the common experiences leading to collaboration functions (e.g., capacity building, maintenance, and preser- and development of system-level performance measurement vation) and reevaluating funding for different categories of include improvements. This approach requires new measures that capture the impact of operational improvements more Demands from elected officials and the public for in- accurately than do traditional engineering measures and a creased accountability and performance. AASHTO's State more system-oriented performance measurement strategy. DOT Performance Management Programs: Select Examples The FHWA's primer, Opportunities for Linking Planning and (AASHTO, 2007) includes several case studies illustrat- Operations, provides a framework for how performance ing how DOTs manage their agencies using performance measures can be used to link planning and operations depart- measures. Although system-level management was not listed ments and therefore policies and decisions. as a primary use of performance measures by agencies, Recognition of the complex nature of organizational deci- "ensuring accountability and responsiveness to stakehold- sion making and policy setting. Performance measurement ers," which involves increasing network connectivity, is is a constantly evolving process. State DOTs and other trans- included as a fundamental reason for implementing and portation agencies are under substantial political pressure to expanding performance management programs. improve accountability and performance for system users. Many of the transportation issues of greatest concern to Several studies have examined the decision-making process the public today are those that require the ability to ad- within transportation agencies and their methods for devel- dress different systems as a single network (e.g., conges- oping performance measures (Bremmer et al., 2005; Larson, tion, safety, and security). In addition, elected officials and 2005; Poister, 2005). As organizations' understanding of the

OCR for page 34
35 complex interaction between different elements of the trans- that more efficiently set goals and track progress to improve portation system and surrounding environmental, economic, overall user experience. and social systems increases, organizations' decision-making processes also change. One result of this evolution has been Best Practices in System-Level an increased awareness of the need for performance mea- Performance Measures sures and collaborations that span modes, agencies, and jurisdictions. For system-level performance measures to be successful, NCHRP Project 8-36A, Multimodal Tradeoffs Framework strong partnerships, solid policies, and implementable prac- Development for Statewide Transportation Planning, tices must be in place. The literature highlights the specific provides guidance on conducting multimodal tradeoffs conditions that must exist regarding these important factors as part of the state planning process. The Strategic High- in the development and implementation of performance mea- way Research Program (SHRP 2) C02 project developed sures across modes and jurisdictions. a framework for performance measurement of highway ca- pacity projects that provides linkages to measures in the Partners transportation planning and programming processes and across a range of impact areas. Together, these studies can A major concern when developing system-level perfor- act as a basis for developing performance measures that pro- mance measures is determining what stakeholders should be vide meaningful comparisons across modes and jurisdictions involved in the process and the respective roles of each partic- and assist agencies in responding to the demands of elected ipant in implementing and monitoring measures once they officials and the public. have been established. Existing studies of such performance Attempts to balance agency and user needs and perspec- measures suggest that in order to be successful these programs tives on system performance. Multiple studies show that require both traditional and nontraditional participation and system-level performance measures provide a means to link support. Stakeholders involved in most successful system-level organizations' perspectives with the experience of those performance measurement programs include the following: who use the transportation system (Adams et al., 2005; Cambridge Systematics, 2007; Hendren and Meyers, 2006; Entities accountable for network results. Those involved Shaw, 2003). in how the network operates should be the ones to decide Common metrics, measures, and technology to span what to measure, how to measure, and how to convey re- modes and jurisdictions. The emphasis on operations over sults. In the context of system performance measurement, the past decade as a means to make more efficient use of ex- this group of stakeholders could include isting capacity has resulted in the growth of methods and Federal, state, or local governments and departments; technology to monitor system operation in real time. This MPOs; movement, combined with the development of travel- Transit agencies; and based performance measures, has resulted in means and Nonprofit organizations (e.g., economic development, methods of comparing mobility efficiency that is adaptable environmental, transportation, and other interest to multiple modes and can easily span jurisdictional groups). boundaries. The wireless data age is putting increased pres- Staff in departments throughout participating agencies. sure on transportation agencies to provide real-time data Like all successful performance measurement programs, in the hands of users who expect accurate measurement of system-level ventures require deep-rooted buy-in from existing mobility conditions (NCHRP Project 207). staff in all levels of participating agencies. Working coop- eratively with other agencies can lead to more robust data An increasing number of transportation agencies are uti- and perspectives to make system-level measures work most lizing performance-based management and planning. As this efficiently. Cascading systems that link performance at all trend and those discussed above continue, recognition of the levels to high-level strategic goals of all organizations in- need for common measurable indicators that can be shared volved in a collaboration have been effective at building own- across organizational and modal boundaries increases. ership among staff. The most important step in perpetuating Through the process of collaboration, staff from different staff buy-in is to create practical measures tied to compelling agencies, jurisdictions, and modes bring together different priorities that are meaningful for all partner agencies. data, expertise, and methods. This is both a challenge and op- High-level, committed leaders in partner agencies. Sup- portunity for system-level performance measurement, pre- port from high-level leadership is necessary for measures senting communication challenges while creating opportuni- to withstand changes in leadership, political relationships, ties to combine resources and perspectives to create measures or policies. Just as performance measurement within a

OCR for page 34
36 single organization often needs a champion to succeed, using them for program evaluation, transit agencies have committed leadership is required to promote incorporation had difficulty using performance measures to make fund- of system-level measurement into organizational decision ing and programming decisions. These differences make making. The champion should be someone familiar with developing and implementing performance measures the principles of social impacts, distribution of impacts, or across agencies difficult and time-consuming. Similar relationships between transportation and other systems issues have been identified in studies of interagency envi- (Cervero et al., 2004; TransTech, 2004). Another approach ronmental streamlining efforts. A 2004 Gallup survey of to ensure commitment from agency leaders is to create a transportation agencies involved in these efforts found that memorandum of understanding (MOU) among collabo- collaborating organizations had notably different percep- rating agencies and organizations, modeled after the MOU tions of how well efforts were working. Another survey of signed by 23 state agencies in support of the Efficient Trans- streamlining projects indicated that collaboration is hard portation Decision-Making System (Edwards et al., 2005). work, time-consuming, labor-intensive, and expensive Legislators and policymakers. One common motivation (Bracaglia, 2005). for creating system-level performance measures is the result Political barriers. Transportation decision making is a of calls for increased accountability and performance from complex and highly political process. Project selection and legislators and policymakers. These decisionmakers should prioritization in particular is an issue of interest to the pub- be regularly updated on steps to develop and monitor lic and one that can engage many vocal and passionate in- system-level performance management programs and in- terest groups. Agencies and local governments often com- formed about the benefits of these efforts for system users. pete for the same limited funding pools and are pressured Support from legislators can help programs to withstand to prioritize local projects and performance. Similarly, changes in organizational leadership and policies and also changes in administration or policy within one jurisdic- can help agencies to obtain or maintain funding for per- tion can cause tension, limit resources, and make system- formance measurement programs. level performance measurement difficult (Cambridge Systematics, 1999). These challenges can be overcome to some degree with strong leadership and broad support for Challenges the value of quantitative and performance-driven inputs Performance management is a complex and evolving pro- into the decision-making process. cess. Expanding performance measurement programs to in- Speed of implementation. Partner agencies will incorpo- clude system-level considerations creates additional complex- rate performance data into their decision making at vari- ities that accompany any coordination of activities among ous rates based on their level of buy-in and organizational multiple actors and stakeholder groups with divergent inter- structure (Pickrell and Neumann, 2001). Private-sector ests. The successful development and implementation of per- businesses tend to make decisions and implement changes formance measurement at the organizational level involves more quickly while public-sector agencies tend to have many challenges. System-level measurement attempts face slower, more complex decision-making processes and may many of the same challenges but require even stronger com- be more resistant to change. This tendency has made im- munication and collaboration skills to address. The most plementing performance measures at any level a challenge common challenges to system-level performance measure- for public agencies (Cambridge Systematics, 1999). Differ- ment identified in the literature include ences in speed of implementation among different agen- cies present a particular challenge and point of tension for Divergent priorities, goals, and funding among partner system-level performance measure programs. agencies. The primary obstacle to interagency collabora- Data compatibility. Data fuels performance-based man- tions--around performance measurement or any other agement and transportation decision making. Complex topic--is the time-consuming nature of developing partner- transportation decisions involving system-level thinking ships (Venner, 2005). Transportation agencies have differ- require information that is timely, understandable, and ing priorities, tight restrictions on the types and locations of standardized. Creating accurate, consistent data collection projects that funding can be used for, and different motiva- and reporting mechanisms to support performance man- tions for participating in system-level performance measure- agement is a complex task for any organization. Develop- ment. For example, several studies have focused on transit ing efficient data-sharing processes, eliminating redundant agencies' and state DOTs' approaches to performance-based data collection and storage, and streamlining workflows planning and management. The reports show that while is difficult even within different departments of a single DOTs are increasingly relying on performance measures as agency. These issues become even more important and management tools and are becoming more sophisticated in complex when multiple agencies are involved.

OCR for page 34
37 Data sharing and compatibility have received much at- sures and decide on the most efficient improvement option tention as a means to increase the efficiency, sustain- if there is no way to compare user benefit-costs of signal ability, and proactive thinking of management programs improvements versus transit service enhancement. Accord- (Halfawy, 2008). However, implementing data sharing ing to several studies, measures that use "common denomi- and collection across multiple organizations remains a nators" such as speed, acceptable travel time, and person major challenge. In 2007, TRB hosted "Information As- throughput are needed to facilitate system-level and multi- sets to Support Transportation Decision Making," a peer modal management (Pratt and Lomax, 1996; Shaw, 2003). exchange organized to identify data gaps and best prac- Aggressive yet realistic targets. Agencies need to make tices in data sharing in the transportation sector. The progress toward goals to get buy-in from partners and the most successful examples of data collecting, sharing, and public. If no progress is made or the goal is unobtainable, use at the system level that were identified in this ex- the program will fail. System-level performance measures change came from the specializations of safety and secu- need to address issues that partner agencies have the power rity. This work has been motivated by recent events that to address. If targets are easily achieved and do not challenge highlighted failings in existing processes and resulted in agencies or influence decision making, data collection and increased recognition of the need for evacuation routes measurement will be perceived as irrelevant. and other plans that require intensive collaboration across modes and jurisdictions (TRB, 2007). Best practices iden- Examples of System-Level tified by organizations involved in this work include the Performance Measures following: in the Literature Communicate opportunities and limitations of data as- sets to managers and partners; Traditional performance measures are discussed at length in Provide easy access to data and metadata; the literature (Brydia et al., 2007; Cambridge Systematics, 2000, Develop data business plans; 2005, 2007; Shaw, 2003). Multiple catalogs of established mea- Standardize linear referencing systems to support inte- sures for specific modes (e.g., freight) and types of agencies (e.g., gration; and DOTs and MPOs) have been published (Harrison et al., 2006). Conduct benchmarking analyses using national data- The literature describing specific system-level performance bases. measures, however, is limited. These studies focus primarily These approaches begin to provide a data-sharing frame- on the collaborative elements of system-level performance work to support system-level performance measurement. measurement, such as best practices in developing system- Unfortunately, many of the methods outlined in the current level performance management programs and facilitating literature are costly or time- and labor-intensive to develop communication between partner agencies and jurisdictions. and implement. As a result, standards for collecting, sharing, Very little is written about the actual performance measures and using data to support system-level performance mea- used to successfully monitor system-level performance. This surement should be agreed upon by all partners and docu- section will highlight some common system-level perfor- mented in the early stages of measure development. mance measures identified in the current literature. Lack of common terminology. Many transportation agen- cies have implemented similar performance management A major criticism of traditional, non-system-level per- programs but use different lexicons to describe the same formance measures used today is that many are descendents inputs, outputs, and processes. For example, many mu- of measures conceived in the 1950s (Meyer, 2001). Many nicipalities use "dashboards" to track performance while of these measures were developed with an engineering, others use "scorecards." The systems are very similar, but capacity-building view in mind and focus on facility-type- the difference in terminology impedes communication be- specific measures of performance on individual segments tween municipal staff that could help both organizations to of the transportation network. share their experiences and improve their systems. One of In recent years, the types of performance measures used in the first steps in any attempt to develop performance mea- transportation planning and management have expanded sures across agencies or jurisdictions should be to agree on to address a growing range of issues. These measures not only a common set of terminology understood by all participants consider inputs (e.g., time, staff, and funding) and out- (TRB, 2005). puts (e.g., pavement quality and congestion) but increas- Cross-modal comparisons. There is a lack of common per- ingly focus on measuring outcomes from the perspective of formance measures that allow accurate comparisons across both system managers and system users (Kittelson & Asso- modes in terms of service levels, quality, travel times, and ciates, Inc., et al., 2003; Poister, 1997; Poister and Van Slyke, cost. It is difficult to create corridor-level performance mea- 2001; Shaw, 2003).

OCR for page 34
38 Alternative Performance Measures for Transportation Plan- Percentage of jobs, dwelling units, and population within ning: Evolution Toward Multimodal Planning states that system one-quarter and one-half mile of transit; performance can be defined based on what is important to Percent growth in areas with good/poor accessibility; the owner and user of the transportation system. In the au- Accessibility and number of destinations within 15 to thors' view, both system- and lower-level measures are needed 30 minutes of travel; and for effective performance measurement yet should be distin- Overall density and density of approved development. guished from one another. According to several multimodal studies, mobility and acces- Environmental impacts include sibility should be incorporated as key measures of system performance (Meyer, 1995). For example Wetlands and forest developed; VMT and VMT per capita; Travel time and modal availability should be the founda- Emissions and emissions per capita; tion for mobility performance measures. Gallons of gas consumed; Accessibility measures should be incorporated into project Percentage of new roads with sidewalk and bike lane/path; planning and system evaluation approaches. Nonauto trips, transportation alternatives; Market segmentation and distributional effects of mobility Modal share for all trips; and accessibility changes should be part of measuring system Water quality; performance. Storm runoff (quantity and quality); Wildlife/habitat impacted; Additional guidance in creating system-level performance Visual quality/aesthetics; measures comes from several specializations within transporta- Cultural resources; and tion agencies that have led the way in developing innovative Geologic resources. performance measures that cross boundaries between agencies, specializations, and jurisdictions. These collaborations have pri- Many of these measures have been used to measure the marily surrounded several issues. performance of individual links/jurisdictions in the past but are potentially powerful system-level measures. Models re- quiring the use of quantitative input measures also have been Environment and Land Use used to measure and predict transportation and land use Beginning with the Intermodal Surface Transportation interactions (ICF Consulting, 2005). Efficiency Act and National Environmental Policy Act, federal legislation requires consideration of land use and environmen- Community Impacts tal impacts of transportation projects. These considerations are in their essence system-level measurements. To capitalize on Several efforts have attempted to provide guidance for the relationship between transportation and land use, trans- quantitatively measuring community impacts of transporta- portation agencies must collaborate with surrounding munic- tion projects and their distribution among segments of the ipalities. To measure environmental impacts agencies must population (Cambridge Systematics, 2002, 2004; Edwards, consider larger natural systems and often partner with envi- 2004; Forkenbrock and Weisbrod, 2001; The Louis Berger ronmentally focused organizations such as watershed districts Group, Inc., 2002; TransTech Management, Inc., 2004; Ward, and the department of natural resources (Cambridge System- 2005). Types of community impacts and possible system-level atics, 2004; Cervero et al., 2004; Rose et al., 2005). Examples of performance measures include the following: integrated planning efforts in this area and possible system- level performance measures are provided below. Number of residents exposed to noise in excess of estab- Land use impacts include lished thresholds; Number of opportunities within a specific distance on a Corridor/access management; specific mode; and Number of street connections per 100 acres; Results of visual preference surveys. Smart-growth policies; Acres of mixed-use or transit-oriented development; Context-sensitive solutions and distribution of benefits Open space and farmland developed; measures include Amount of land developed and developed per capita; Job/housing balance; Number of displaced persons; Percentage of workers within 15 to 30 minutes of their job; Number and value of displaced homes;

OCR for page 34
39 Neighborhood cohesion; Net present value; Accessibility to community services; Rate of return; Use of multidisciplinary teams; Benefitcost ratio; Measures of public engagement; and First-year benefit ratio; Definition and adherence to vision, goals, and objectives Payback period; (TransTech Management, Inc., 2004). Financial feasibility; Cost per new person-trip; Number and value of displaced businesses; Economic Development Accessibility to employment, retail, new/planned devel- The methods used to determine economic impacts of trans- opment; portation investments often result in performance measures Jobs created; that aid decisionmakers in project or program selection. Many Gross regional product; and of these processes rely on lower-level performance measures as Change in personal income (AASHTO, 1977; FHWA, 2003; inputs (e.g., mobility through monetized travel-time savings Lewis, 1991; Shaw, 2003). and safety through crash reductions and associated costs) and as a result are easily adapted to measuring performance at the The measures listed above are a sample of those being used system level. These methods include by organizations at the system level. Additional measures can be found in the discussion boards and literature available on Lifecycle cost; the FHWA's Performance Measurement Exchange, System Lifecycle benefit; Performance Measurement Group website.