Measuring Transportation Network Performance (2010)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

34 This section provides an excerpt of the literature that was reviewed for this project with a focus on material that was used to produce the guidebook. A more comprehensive liter- ature review is available on request as part of the final report. The Case for System-Level Performance Measures Developing and monitoring network performance mea- sures requires communication and coordination between in- dividuals who plan the transportation system, develop policy, and manage operations. Existing system-level performance measures have come together through various collaborations between mixed groups of state agencies, MPOs, local govern- ments, transit agencies, and others. Several common ele- ments exist between these collaborations, which may begin to provide a framework that guides agencies when develop- ing and implementing system-level performance measures. Some of the common experiences leading to collaboration and development of system-level performance measurement include • Demands from elected officials and the public for in- creased accountability and performance. AASHTO’s State DOT Performance Management Programs: Select Examples (AASHTO, 2007) includes several case studies illustrat- ing how DOTs manage their agencies using performance measures. Although system-level management was not listed as a primary use of performance measures by agencies, “ensuring accountability and responsiveness to stakehold- ers,” which involves increasing network connectivity, is included as a fundamental reason for implementing and expanding performance management programs. • Many of the transportation issues of greatest concern to the public today are those that require the ability to ad- dress different systems as a single network (e.g., conges- tion, safety, and security). In addition, elected officials and the public are increasingly aware of the external impacts of the transportation system on the economy, the environ- ment, and surrounding communities. With limited trans- portation budgets, there is increasing pressure on public of- ficials and transportation agencies to ensure that projects that are funded are those with the best overall value and least negative externalities. Selecting projects in this context re- quires broad knowledge of existing system performance and the ability to evaluate the costs and benefits of alternatives. • Consideration of operations solutions over new construc- tion. As construction costs climb, federal and state trans- portation trust funds decline, and highway systems become built out, the focus of most transportation agencies is shift- ing from capacity improvements to maximizing operational efficiency (Brydia et al., 2007; Cambridge Systematics, 2005, 2007; Hendren and Myers, 2006; Meyer, 2001; Randall, 2007). With this change, DOTs and MPOs have begun ex- amining the linkages between operations and other agency functions (e.g., capacity building, maintenance, and preser- vation) and reevaluating funding for different categories of improvements. This approach requires new measures that capture the impact of operational improvements more accurately than do traditional engineering measures and a more system-oriented performance measurement strategy. The FHWA’s primer, Opportunities for Linking Planning and Operations, provides a framework for how performance measures can be used to link planning and operations depart- ments and therefore policies and decisions. • Recognition of the complex nature of organizational deci- sion making and policy setting. Performance measurement is a constantly evolving process. State DOTs and other trans- portation agencies are under substantial political pressure to improve accountability and performance for system users. Several studies have examined the decision-making process within transportation agencies and their methods for devel- oping performance measures (Bremmer et al., 2005; Larson, 2005; Poister, 2005). As organizations’ understanding of the A P P E N D I X A Key Literature

35 complex interaction between different elements of the trans- portation system and surrounding environmental, economic, and social systems increases, organizations’ decision-making processes also change. One result of this evolution has been an increased awareness of the need for performance mea- sures and collaborations that span modes, agencies, and jurisdictions. • NCHRP Project 8-36A, Multimodal Tradeoffs Framework Development for Statewide Transportation Planning, provides guidance on conducting multimodal tradeoffs as part of the state planning process. The Strategic High- way Research Program (SHRP 2) C02 project developed a framework for performance measurement of highway ca- pacity projects that provides linkages to measures in the transportation planning and programming processes and across a range of impact areas. Together, these studies can act as a basis for developing performance measures that pro- vide meaningful comparisons across modes and jurisdictions and assist agencies in responding to the demands of elected officials and the public. • Attempts to balance agency and user needs and perspec- tives on system performance. Multiple studies show that system-level performance measures provide a means to link organizations’ perspectives with the experience of those who use the transportation system (Adams et al., 2005; Cambridge Systematics, 2007; Hendren and Meyers, 2006; Shaw, 2003). • Common metrics, measures, and technology to span modes and jurisdictions. The emphasis on operations over the past decade as a means to make more efficient use of ex- isting capacity has resulted in the growth of methods and technology to monitor system operation in real time. This movement, combined with the development of travel- based performance measures, has resulted in means and methods of comparing mobility efficiency that is adaptable to multiple modes and can easily span jurisdictional boundaries. The wireless data age is putting increased pres- sure on transportation agencies to provide real-time data in the hands of users who expect accurate measurement of existing mobility conditions (NCHRP Project 20–7). An increasing number of transportation agencies are uti- lizing performance-based management and planning. As this trend and those discussed above continue, recognition of the need for common measurable indicators that can be shared across organizational and modal boundaries increases. Through the process of collaboration, staff from different agencies, jurisdictions, and modes bring together different data, expertise, and methods. This is both a challenge and op- portunity for system-level performance measurement, pre- senting communication challenges while creating opportuni- ties to combine resources and perspectives to create measures that more efficiently set goals and track progress to improve overall user experience. Best Practices in System-Level Performance Measures For system-level performance measures to be successful, strong partnerships, solid policies, and implementable prac- tices must be in place. The literature highlights the specific conditions that must exist regarding these important factors in the development and implementation of performance mea- sures across modes and jurisdictions. Partners A major concern when developing system-level perfor- mance measures is determining what stakeholders should be involved in the process and the respective roles of each partic- ipant in implementing and monitoring measures once they have been established. Existing studies of such performance measures suggest that in order to be successful these programs require both traditional and nontraditional participation and support. Stakeholders involved in most successful system-level performance measurement programs include the following: • Entities accountable for network results. Those involved in how the network operates should be the ones to decide what to measure, how to measure, and how to convey re- sults. In the context of system performance measurement, this group of stakeholders could include – Federal, state, or local governments and departments; – MPOs; – Transit agencies; and – Nonprofit organizations (e.g., economic development, environmental, transportation, and other interest groups). • Staff in departments throughout participating agencies. Like all successful performance measurement programs, system-level ventures require deep-rooted buy-in from staff in all levels of participating agencies. Working coop- eratively with other agencies can lead to more robust data and perspectives to make system-level measures work most efficiently. Cascading systems that link performance at all levels to high-level strategic goals of all organizations in- volved in a collaboration have been effective at building own- ership among staff. The most important step in perpetuating staff buy-in is to create practical measures tied to compelling priorities that are meaningful for all partner agencies. • High-level, committed leaders in partner agencies. Sup- port from high-level leadership is necessary for measures to withstand changes in leadership, political relationships, or policies. Just as performance measurement within a

36 single organization often needs a champion to succeed, committed leadership is required to promote incorporation of system-level measurement into organizational decision making. The champion should be someone familiar with the principles of social impacts, distribution of impacts, or relationships between transportation and other systems (Cervero et al., 2004; TransTech, 2004). Another approach to ensure commitment from agency leaders is to create a memorandum of understanding (MOU) among collabo- rating agencies and organizations, modeled after the MOU signed by 23 state agencies in support of the Efficient Trans- portation Decision-Making System (Edwards et al., 2005). • Legislators and policymakers. One common motivation for creating system-level performance measures is the result of calls for increased accountability and performance from legislators and policymakers. These decisionmakers should be regularly updated on steps to develop and monitor system-level performance management programs and in- formed about the benefits of these efforts for system users. Support from legislators can help programs to withstand changes in organizational leadership and policies and also can help agencies to obtain or maintain funding for per- formance measurement programs. Challenges Performance management is a complex and evolving pro- cess. Expanding performance measurement programs to in- clude system-level considerations creates additional complex- ities that accompany any coordination of activities among multiple actors and stakeholder groups with divergent inter- ests. The successful development and implementation of per- formance measurement at the organizational level involves many challenges. System-level measurement attempts face many of the same challenges but require even stronger com- munication and collaboration skills to address. The most common challenges to system-level performance measure- ment identified in the literature include • Divergent priorities, goals, and funding among partner agencies. The primary obstacle to interagency collabora- tions—around performance measurement or any other topic—is the time-consuming nature of developing partner- ships (Venner, 2005). Transportation agencies have differ- ing priorities, tight restrictions on the types and locations of projects that funding can be used for, and different motiva- tions for participating in system-level performance measure- ment. For example, several studies have focused on transit agencies’ and state DOTs’ approaches to performance-based planning and management. The reports show that while DOTs are increasingly relying on performance measures as management tools and are becoming more sophisticated in using them for program evaluation, transit agencies have had difficulty using performance measures to make fund- ing and programming decisions. These differences make developing and implementing performance measures across agencies difficult and time-consuming. Similar issues have been identified in studies of interagency envi- ronmental streamlining efforts. A 2004 Gallup survey of transportation agencies involved in these efforts found that collaborating organizations had notably different percep- tions of how well efforts were working. Another survey of streamlining projects indicated that collaboration is hard work, time-consuming, labor-intensive, and expensive (Bracaglia, 2005). • Political barriers. Transportation decision making is a complex and highly political process. Project selection and prioritization in particular is an issue of interest to the pub- lic and one that can engage many vocal and passionate in- terest groups. Agencies and local governments often com- pete for the same limited funding pools and are pressured to prioritize local projects and performance. Similarly, changes in administration or policy within one jurisdic- tion can cause tension, limit resources, and make system- level performance measurement difficult (Cambridge Systematics, 1999). These challenges can be overcome to some degree with strong leadership and broad support for the value of quantitative and performance-driven inputs into the decision-making process. • Speed of implementation. Partner agencies will incorpo- rate performance data into their decision making at vari- ous rates based on their level of buy-in and organizational structure (Pickrell and Neumann, 2001). Private-sector businesses tend to make decisions and implement changes more quickly while public-sector agencies tend to have slower, more complex decision-making processes and may be more resistant to change. This tendency has made im- plementing performance measures at any level a challenge for public agencies (Cambridge Systematics, 1999). Differ- ences in speed of implementation among different agen- cies present a particular challenge and point of tension for system-level performance measure programs. • Data compatibility. Data fuels performance-based man- agement and transportation decision making. Complex transportation decisions involving system-level thinking require information that is timely, understandable, and standardized. Creating accurate, consistent data collection and reporting mechanisms to support performance man- agement is a complex task for any organization. Develop- ing efficient data-sharing processes, eliminating redundant data collection and storage, and streamlining workflows is difficult even within different departments of a single agency. These issues become even more important and complex when multiple agencies are involved.

• Data sharing and compatibility have received much at- tention as a means to increase the efficiency, sustain- ability, and proactive thinking of management programs (Halfawy, 2008). However, implementing data sharing and collection across multiple organizations remains a major challenge. In 2007, TRB hosted “Information As- sets to Support Transportation Decision Making,” a peer exchange organized to identify data gaps and best prac- tices in data sharing in the transportation sector. The most successful examples of data collecting, sharing, and use at the system level that were identified in this ex- change came from the specializations of safety and secu- rity. This work has been motivated by recent events that highlighted failings in existing processes and resulted in increased recognition of the need for evacuation routes and other plans that require intensive collaboration across modes and jurisdictions (TRB, 2007). Best practices iden- tified by organizations involved in this work include the following: – Communicate opportunities and limitations of data as- sets to managers and partners; – Provide easy access to data and metadata; – Develop data business plans; – Standardize linear referencing systems to support inte- gration; and – Conduct benchmarking analyses using national data- bases. • These approaches begin to provide a data-sharing frame- work to support system-level performance measurement. Unfortunately, many of the methods outlined in the current literature are costly or time- and labor-intensive to develop and implement. As a result, standards for collecting, sharing, and using data to support system-level performance mea- surement should be agreed upon by all partners and docu- mented in the early stages of measure development. • Lack of common terminology. Many transportation agen- cies have implemented similar performance management programs but use different lexicons to describe the same inputs, outputs, and processes. For example, many mu- nicipalities use “dashboards” to track performance while others use “scorecards.” The systems are very similar, but the difference in terminology impedes communication be- tween municipal staff that could help both organizations to share their experiences and improve their systems. One of the first steps in any attempt to develop performance mea- sures across agencies or jurisdictions should be to agree on a common set of terminology understood by all participants (TRB, 2005). • Cross-modal comparisons. There is a lack of common per- formance measures that allow accurate comparisons across modes in terms of service levels, quality, travel times, and cost. It is difficult to create corridor-level performance mea- sures and decide on the most efficient improvement option if there is no way to compare user benefit-costs of signal improvements versus transit service enhancement. Accord- ing to several studies, measures that use “common denomi- nators” such as speed, acceptable travel time, and person throughput are needed to facilitate system-level and multi- modal management (Pratt and Lomax, 1996; Shaw, 2003). • Aggressive yet realistic targets. Agencies need to make progress toward goals to get buy-in from partners and the public. If no progress is made or the goal is unobtainable, the program will fail. System-level performance measures need to address issues that partner agencies have the power to address. If targets are easily achieved and do not challenge agencies or influence decision making, data collection and measurement will be perceived as irrelevant. Examples of System-Level Performance Measures in the Literature Traditional performance measures are discussed at length in the literature (Brydia et al., 2007; Cambridge Systematics, 2000, 2005, 2007; Shaw, 2003). Multiple catalogs of established mea- sures for specific modes (e.g., freight) and types of agencies (e.g., DOTs and MPOs) have been published (Harrison et al., 2006). The literature describing specific system-level performance measures, however, is limited. These studies focus primarily on the collaborative elements of system-level performance measurement, such as best practices in developing system- level performance management programs and facilitating communication between partner agencies and jurisdictions. Very little is written about the actual performance measures used to successfully monitor system-level performance. This section will highlight some common system-level perfor- mance measures identified in the current literature. • A major criticism of traditional, non-system-level per- formance measures used today is that many are descendents of measures conceived in the 1950s (Meyer, 2001). Many of these measures were developed with an engineering, capacity-building view in mind and focus on facility-type- specific measures of performance on individual segments of the transportation network. • In recent years, the types of performance measures used in transportation planning and management have expanded to address a growing range of issues. These measures not only consider inputs (e.g., time, staff, and funding) and out- puts (e.g., pavement quality and congestion) but increas- ingly focus on measuring outcomes from the perspective of both system managers and system users (Kittelson & Asso- ciates, Inc., et al., 2003; Poister, 1997; Poister and Van Slyke, 2001; Shaw, 2003). 37

38 Alternative Performance Measures for Transportation Plan- ning: Evolution Toward Multimodal Planning states that system performance can be defined based on what is important to the owner and user of the transportation system. In the au- thors’ view, both system- and lower-level measures are needed for effective performance measurement yet should be distin- guished from one another. According to several multimodal studies, mobility and acces- sibility should be incorporated as key measures of system performance (Meyer, 1995). For example • Travel time and modal availability should be the founda- tion for mobility performance measures. • Accessibility measures should be incorporated into project planning and system evaluation approaches. • Market segmentation and distributional effects of mobility and accessibility changes should be part of measuring system performance. Additional guidance in creating system-level performance measures comes from several specializations within transporta- tion agencies that have led the way in developing innovative performance measures that cross boundaries between agencies, specializations, and jurisdictions. These collaborations have pri- marily surrounded several issues. Environment and Land Use Beginning with the Intermodal Surface Transportation Efficiency Act and National Environmental Policy Act, federal legislation requires consideration of land use and environmen- tal impacts of transportation projects. These considerations are in their essence system-level measurements. To capitalize on the relationship between transportation and land use, trans- portation agencies must collaborate with surrounding munic- ipalities. To measure environmental impacts agencies must consider larger natural systems and often partner with envi- ronmentally focused organizations such as watershed districts and the department of natural resources (Cambridge System- atics, 2004; Cervero et al., 2004; Rose et al., 2005). Examples of integrated planning efforts in this area and possible system- level performance measures are provided below. Land use impacts include • Corridor/access management; • Number of street connections per 100 acres; • Smart-growth policies; • Acres of mixed-use or transit-oriented development; • Open space and farmland developed; • Amount of land developed and developed per capita; • Job/housing balance; • Percentage of workers within 15 to 30 minutes of their job; • Percentage of jobs, dwelling units, and population within one-quarter and one-half mile of transit; • Percent growth in areas with good/poor accessibility; • Accessibility and number of destinations within 15 to 30 minutes of travel; and • Overall density and density of approved development. Environmental impacts include • Wetlands and forest developed; • VMT and VMT per capita; • Emissions and emissions per capita; • Gallons of gas consumed; • Percentage of new roads with sidewalk and bike lane/path; • Nonauto trips, transportation alternatives; • Modal share for all trips; • Water quality; • Storm runoff (quantity and quality); • Wildlife/habitat impacted; • Visual quality/aesthetics; • Cultural resources; and • Geologic resources. Many of these measures have been used to measure the performance of individual links/jurisdictions in the past but are potentially powerful system-level measures. Models re- quiring the use of quantitative input measures also have been used to measure and predict transportation and land use interactions (ICF Consulting, 2005). Community Impacts Several efforts have attempted to provide guidance for quantitatively measuring community impacts of transporta- tion projects and their distribution among segments of the population (Cambridge Systematics, 2002, 2004; Edwards, 2004; Forkenbrock and Weisbrod, 2001; The Louis Berger Group, Inc., 2002; TransTech Management, Inc., 2004; Ward, 2005). Types of community impacts and possible system-level performance measures include the following: • Number of residents exposed to noise in excess of estab- lished thresholds; • Number of opportunities within a specific distance on a specific mode; and • Results of visual preference surveys. Context-sensitive solutions and distribution of benefits measures include • Number of displaced persons; • Number and value of displaced homes;

• Neighborhood cohesion; • Accessibility to community services; • Use of multidisciplinary teams; • Measures of public engagement; and • Definition and adherence to vision, goals, and objectives (TransTech Management, Inc., 2004). Economic Development The methods used to determine economic impacts of trans- portation investments often result in performance measures that aid decisionmakers in project or program selection. Many of these processes rely on lower-level performance measures as inputs (e.g., mobility through monetized travel-time savings and safety through crash reductions and associated costs) and as a result are easily adapted to measuring performance at the system level. These methods include • Lifecycle cost; • Lifecycle benefit; • Net present value; • Rate of return; • Benefit–cost ratio; • First-year benefit ratio; • Payback period; • Financial feasibility; • Cost per new person-trip; • Number and value of displaced businesses; • Accessibility to employment, retail, new/planned devel- opment; • Jobs created; • Gross regional product; and • Change in personal income (AASHTO, 1977; FHWA, 2003; Lewis, 1991; Shaw, 2003). The measures listed above are a sample of those being used by organizations at the system level. Additional measures can be found in the discussion boards and literature available on the FHWA’s Performance Measurement Exchange, System Performance Measurement Group website. 39