Engineering Economic Analysis Practices for Highway Investment (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

90 chapter five ConClusions study objeCtives and Case development objectives and scope The objective of this project was to study current practices by U.S. state transportation agencies in performing engineer- ing economic analyses (EEAs) to evaluate highway invest- ment decisions. The scope of these investigations included analysis techniques, processes, and resources currently being used, knowledge of which might be helpful to other agencies considering the use of economic methods. According to the Scope of Work for this synthesis, past research had indicated that certain transportation agencies are proficient in integrating economic analyses of their investment options into their asset management strategy using a variety of tools and processes. Other agencies were found to lack the resources, guidance, and understanding to perform such evaluations for other than a few, limited applications. The purpose of this study was to build on these earlier assessments and develop specific find- ings on how proficient agencies—those state DOTs and other transportation agencies that are conversant with economic methods—apply EEAs. A series of case examples would be used to demonstrate methods, data, assumptions, results, and applications to highway investment decisions that typified unique, comprehensive, or innovative uses of EEA. The Scope of Work made the case examples central to the conduct and the findings of the study. It was agreed with the topic panel that the synthesis would present only a brief over- view of engineering economic methodology and data, refer- ring readers to other references for additional information on these subjects. The value-added products of this study would be the descriptions of how successful state departments of transportation (DOTs) and other transportation agencies go about the steps of EEA: articulating the highway system need or problem to be investigated; defining alternative solu- tions to be assessed; quantifying the parameters of the analy- ses; setting economic and engineering criteria for decisions; introducing other, noneconomic or nonquantitative factors affecting the outcome; completing the analysis; and inter- preting results. Case identification and development Based on the literature review, the most widely used applica- tions of economic methods in the United States today focus on project-level engineering decisions related to pavement and bridge preservation, spot-location safety improvements, evaluation of major projects, and urban highway analyses [e.g., mobility improvements and the growing use of Intelli- gent Transportation System (ITS) devices]. Of key interest for investigation in this synthesis were unique applications (e.g., problems not widely addressed, as in evaluation of dif- ferent methods of project delivery); broad, comprehensive applications (e.g., applying EEAs across a range of decisions in the project life cycle and at different levels of the high- way system); and innovative applications (e.g., applying util- ity theory to represent benefits that are otherwise not easily quantified). The other criterion in selecting case studies was the ability and willingness of the host agency to describe the case and provide needed data. The study employed several mechanisms to identify can- didates for case examples. These included a literature review, presentations before two committees at the TRB Annual Meeting, interviews with topic panel members and other industry experts in the field, and a screening survey. The screening survey questionnaire was sent to state DOT repre- sentatives on AASHTO’s Standing Committee on Highways, with copies to each state DOT’s Research Advisory Com- mittee member, and a corresponding questionnaire to each state’s FHWA division office. Responses were received from 17 state DOTs and 8 FHWA division offices. For two states responses were received from both the DOT and the FHWA division office; these were combined to produce a single response for each state, resulting in 23 unique state responses. Of these 23 responses, 20 states reported using EEAs, whereas 3 described themselves as not using such analyses. The 20 affirmatively responding states were con- sidered as candidates for case examples; the 3 states that responded negatively provided reasons why their state high- way agency did not use economic analysis on a routine basis. The survey results were used solely as a screening device to identify candidates for case examples and to obtain back- ground information on usage patterns. The background infor- mation was not used as a statistically significant indication of nationwide practice. The survey results were consistent, however, with previous research and with findings of other surveys having higher response rates. Case examples The findings of all of the sources of information converged on a particular set of case examples, which are presented in

91 chapter three. An extension of all of these cases to discuss implementation issues and considerations is presented in chapter four. The categories of case examples in this synthe- sis include the following: • Critical Interstate Facilities, illustrating economics- based planning analyses by the Port Authority of New York and New Jersey of its interstate bridges and tun- nels, with an accompanying economic analysis by the U.S. Army Corps of Engineers of changes in future maritime shipping as the result of the Panama Canal capacity expansion. • Mobility Planning, illustrating planning-level tools now being put in place by the Washington State DOT (WSDOT) as part of their technical and economic analy sis of their highway improvement program. • Mobility Programming, illustrating WSDOT’s benefit– cost analysis (BCA) of highway improvements for proj- ect prioritization and building the highway improvement capital program. The case explains screening criteria applied to project candidates, multi-objective evaluation criteria (economic benefit–cost, environmental impact, stakeholder response, and interaction of project design with factors such as land use and use of alternative modes including nonmotorized transportation), analytic proce- dures, and application of results to prioritization, program building, and budget recommendation. • Safety Programming, illustrating WSDOT’s benefit– cost approach to systematic, low-cost safety improve- ments, with explanations of geographic information system-based capabilities for problem diagnosis, pri- oritization, and selection of proposed solutions. The case explains WSDOT’s Target Zero strategic highway safety plan, its safety project priority levels, methods of analyzing safety needs and solutions, and organi- zational and institutional relationships that support a coordinated and effective statewide safety program. • Bridge Programming and Permitting, illustrating a unique approach by the California DOT (Caltrans) to analyzing bridge project priorities regarding critical, risk-related needs that are not addressed in Caltrans’ bridge management system. Caltrans applies utility theory to gauge the benefits of reduced risk of scour, reduced risk of seismic damage, and bridge railing safety. Results are applied to bridge project prioritiza- tion using a multistep inspection, evaluation, and peer review process, conducted in the context of environ- mental permitting requirements for bridges that are near coastal zones, in environmentally sensitive areas, or are of historic importance. • Economics-Based Tradeoffs, illustrating the develop- ment of a prototype, economics-based tradeoff analy- sis by the New York State DOT. The benefits measure used is excess road user costs (or costs avoided). • Pavement Type Selection, illustrating the economic evaluation of competing pavement alternatives by Caltrans and the Colorado DOT. Explanations of each state’s approach to determining a discount rate are pro- vided in this case. • Value Engineering (VE), illustrating methods of conducting VE by the Florida DOT and Caltrans, two agencies that account for the largest number of the state DOT VE studies in the United States. The case is noteworthy in its description of the well-structured VE approach used by Caltrans, including data on the duration of these reviews and quantification of resulting project cost savings. • Accelerated Project Delivery, illustrating the devel- opment of a new methodology by the Minnesota DOT to analyze options for speeding completion of project construction (or design and construction, as in design– build). This method applies a BCA of project phases during construction and after construction, which has been built using data, models, and risk assessments developed during a recently completed highway expan- sion project. Each case addresses a number of aspects suggested by the Scope of Work; for example, an Introduction describing the case and highlighting ways in which it is unique, com- prehensive, or innovative; the Role of Economic Analysis in Highway Investment, laying the groundwork for the case description and the approach to be used in its analysis; Meth- ods and Measures, developing the methodological aspects, key variables, and measures to characterize results; Decision Support, describing how the economic analysis results are applied to decision making, what other factors (including noneconomic and nonquantifiable considerations) are incor- porated in the decision process, and the resulting decision if available; and Resources Needed and Other Information, including any unique capabilities or resources required for the analysis. Findings Case example Results The case examples apply conventional methods of EEA, but each is tailored to the particular problem at hand, and several employ software (including spreadsheet workbooks) developed by an agency or its consultants. Project costs include typical costs of work on transportation facilities, encompassing preliminary engineering and project prepara- tion; design; construction; and maintenance and operation as appropriate. Project benefits typically include savings in travel time, in vehicle operating costs, and in costs of colli- sions, but selected cases (such as the Bridge Programming and Permitting example) use specialized benefits measures: in the bridge case, a measure of bridge health encompassing vulnerability to catastrophic damage, expressed as a utility function. The quantification of other input variables is cov- ered as applicable to each case. Certain cases also describe the agency’s method of determining the value of the discount rate that is used in the economic analysis.

92 Similar explanations are given in each case example for the measures of results. Where estimates of life-cycle benefits and costs are both available, a BCA is applied with the economic results metric expressed as a net present value, benefit–cost ratio, or in some cases an internal rate of return. If a monetized benefit stream is not available, a cost-effectiveness analysis is applied, as for the Bridge Programming and Permitting exam- ple in which benefits are expressed through changes in the bridge health utility function. Other measures of benefits used in the case examples are road user costs avoided (or excess road user costs) and collision frequency and severity for safety analyses, which can also be used in a cost-effectiveness cal- culation. Several cases also illustrate the application of risk analyses, using sensitivity and scenario analyses or probabi- listic input values. Characteristics of proficient agencies Viewed individually, the case examples show how engi- neering knowledge and a need to understand the impacts of particular decisions can be organized within a practical economic framework. Viewed collectively, however, the case examples begin to reveal characteristics that are held in common among agencies successfully applying engineering economic practices: • Establishment of solid, understandable, and up-to-date agency guidance on how, when, where, and why to apply economic techniques, with the support and participation of executive leadership. • Development and use of analytic tools, accompanied by the definition, collection, and use of quality data, needed to support economic analyses. • Incorporation of these analytic tools within day-to-day business processes, with an agency expectation of their effective use backed by executive leadership. • Effective integration of engineering as well as economic logic within analytic procedures. • Use of cost-effective software such as spreadsheet workbooks and databases in which to develop, use, and communicate EEAs. • A willingness to experiment and innovate when exist- ing data and analytic capabilities do not fit a unique situation that requires a decision. • The capability and willingness to undertake reasonable, realistic estimates where hard data are not available. • Maintaining a healthy perspective on EEAs, viewing results as information—not an automated decision—that becomes part of an overall understanding of a problem. • Providing staff training in economic methods and tools, and encouraging personnel to apply these capabilities in their daily work. • A willingness to engage experts from all quarters—other public agencies, private-sector consultants, academia, stakeholders—to assist in improving agency capabilities, analytic tools, and data supporting economic analyses. value of economic analyses The case examples, together with follow-on surveys of the literature and interviews with agency personnel, demonstrate two broad classes of benefit of performing economic analyses. • The first is the direct or tangible benefit of having obtained an economic result that shows the value or merit of a highway investment. This value may be in benefits received by road users or costs avoided by them and by the agency. There is generally a link between economic performance (benefits versus costs) and engi- neering or technical performance of the highway facil- ity. Monetized benefits help in understanding tradeoffs among competing alternatives. The process of prepar- ing an economic analysis also imposes a discipline to account for all costs and all benefits in as comprehensive and accurate a way as possible. • The second is the indirect or intangible benefit of encour- aging a better decision process within the organization. This benefit and its implications are discussed at length in chapter four, but generally relate to an incentive to iden- tify all realistic alternatives for solution, to maintain focus on the purpose of the proposed investment and avoid “scope creep,” to avoid biases toward familiar or tradi- tional options (such as particular paving materials), and to support these objectives through clear agency guidance and communication, backed by the availability of analytic tools and effective data collection and processing. Resources Technical resources are available to assist agencies in strengthening their own capabilities in conducting economic analyses. The case examples in chapter three have identi- fied a broad range of engineering economic applications to a number of highway investment analyses occurring at vari- ous points of the highway infrastructure life cycle. The case examples illustrate the type and sources of data that can be used in these analyses. Chapter two of this report identifies a number of guides and other literature related specifically to highway transportation. There is also a considerable body of more general literature on engineering economic con- cepts and methods. Some of the agencies cited in the case examples maintain extensive websites on economic concepts and techniques, or on the use of such methods within their business processes, to produce results that are displayed and discussed on the website. FHWA provides downloadable software to support some of the economic methods discussed in this report. Current activities Current activities are extending economic methods to addi- tional types of highway features, systems, and information (with benefit–cost techniques applied to assess the value of

93 the information). These extensions include other highway features besides pavements and bridges (such as roadside hardware, pavement markings, and traffic signals), ITS devices, Road Weather Information Systems for automated control of anti-icing application or to inform managerial decisions on anti-icing application, electronic toll collection systems, freeway management systems, IntelliDrive deploy- ment for wireless networking of data among road vehicles, road infrastructure, and passengers’ personal communica- tions devices, and use of information from full-scale acceler- ated pavement testing facilities. european and Canadian experience Guidance documents pertaining to economic analysis used by the European Union and Canadian transportation agen- cies were reviewed in chapter two. As a general comment, these guidelines have a broader scope than comparable U.S. documents, encompassing either several transportation modes (as in the Transport Canada Guide) or several infra- structure sectors (as in the European Commission Guide). In other respects, however, the economic analyses used by these countries are more circumscribed, with less latitude afforded the performing agency. For example, European analyses are for project appraisals: they tend to be done early in a project’s life cycle and are performed for analysis periods of 20 years. Canadian analyses are currently constrained to a discount rate of 10%; although there appears to be recogni- tion that this rate may at some point need to be reviewed; that guidance remains in effect as of the writing of this report. By contrast, the U.S. case examples in chapter three entail analy- sis periods of 20 years or longer, and discount rates from 3% to 4% at the lower end up to 7% at the higher end. The case examples occur throughout the project life cycle, encompass- ing project/program development as well as delivery. Apart from these differences, the mechanics of the benefit–cost methods used are similar across all regions. As one example, the U.S., Canadian, and European guides all agree that the economic impacts of highway investments (e.g., job creation and employment increases) are secondary benefits that might not be included in the BCAs of these projects. European, New Zealand, and Australian experience has also been cited in chapters three and four. Australian research papers have supported the Safety Programming case exam- ple in chapter three, and an international scan of perfor- mance management practices in Australia, New Zealand, Great Britain, and Sweden has informed the discussion of economic analysis and performance management in chapter four. A major contribution from the European Commission, however, is also described in chapter four: the EVA-TREN project, which has investigated the use of ex-post (or post- construction) analyses of the actual benefit–cost results gener- ated by the completed project as compared with the estimates during project appraisal. The United States does not have a formal policy guideline requiring post-construction analyses, as pointed out by the European sources. However, based on published studies, such a requirement could strengthen future BCAs and improve the quality of data, parameter values, and assumptions. Diligent application of post-construction analyses could conceivably remove many of the reservations some agency personnel have about the assumptions and data in existing economic analyses (refer to the following section on Impediments to Wider Application). Accordingly, post- construction (or ex-post) analyses are included as a research recommendation later in this chapter. The international scan likewise included ex-post analyses as one of the practices recommended to be adopted by U.S. agencies. impediments to WideR appliCation Several studies by the General Accountability Office (GAO) and FHWA have identified a number of impediments to wider use of EEAs. These are discussed in two categories: weaknesses inherent in the methods themselves, and short- comings in how the methods are applied. Weaknesses in the methods Concerns about economic methods, BCAs in particular, include the following: • BCA tallies net benefits in the aggregate, without regard to the equity of the distribution of these benefits. • Although impacts such as travel-time saved, reductions in emissions, and reductions in accident fatalities and injuries can be monetized, not all researchers may agree on these valuations of impacts. • Existing models are unable to predict accurately certain key effects of transportation investments; for example, changes in land use, driver behavior, or diversion to alternate routes or modes. This problem is aggravated by the diverse set of models employed by local agencies (which can affect the coordination of state and local policy and project recommendations). shortcomings in How methods are applied Observed shortcomings in how benefit–cost methods are applied include the following: • The way in which projects are scoped may affect results. One of the GAO studies provided examples with group- ing of independent projects, where not all might have survived a BCA individually. Another problem con- cerned how to account correctly for interactions between complementary projects, where benefits of each project estimated individually might underestimate the total ben- efit of the projects collectively if they were coordinated. • A BCA that considers impacts in several areas creates the need to forecast data in these areas, tasks that are subject to uncertainty.

94 • Benefits that are difficult to quantify may be overlooked or eliminated from the analysis. • Lack of complete, accurate data may distort forecasts and lead to erroneous results. Surveys of travel demand are getting more difficult to fund and to conduct. • Certain benefits are double-counted; certain expendi- tures are counted as benefits. Sometimes future benefits are cited at their nominal value, not discounted to pres- ent value. The avoided cost of another project may be counted as a benefit of the project that is being analyzed. • The definition of alternatives may miss viable options in the current mode or in other modes (e.g., failure to compare a highway project with a transit option). It can be noted that these shortcomings can be addressed through staff training. There are also methods and proce- dures to mitigate the risks identified previously; these are discussed in the methodological sections of chapter two and are illustrated in some of the case examples in chapter three. assessments of the tigeR grant program The economic analyses that accompanied the TIGER grant applications in 2009 have been of interest to U.S. General Accountability Office (GAO) and FHWA. The program has been proposed as a model for a new type of performance- based or merit-based competitive funding mechanism to replace some of the formulaic funding distributions now used in transportation programs. In its review of the TIGER grant program (together with the High-Speed Intercity Passenger Rail Program), GAO found that benefit–cost assessments were not comprehensive and monetization of benefits varied widely among applications. Many applications also did not follow existing federal guidelines by failing to provide infor- mation concerning risk and uncertainty, data limitations, and assumptions inherent in the methodology. The findings of the FHWA review of TIGER grant applications echoed some of these concerns, noting the following errors in the economic analyses: • Incorrect treatment of some economic development and local construction impacts as project benefits. • Improper accounting of project costs. • Unrealistic base cases and project lifetimes. • Incorrect treatment of discounting and inflation. • Incorrectly treating initial-year or design-year numbers as a stream of constant annual amounts. • Incorrectly estimating safety benefits. The resulting conclusions by the TIGER grant reviewers were the following: • A wide disparity in the depth and quality of BCAs. • Insufficient information on expected project outcomes, accompanied by inadequately supported assertions of project benefits. survey Responses Responses to the synthesis survey by those states that do not routinely use economic analysis provided further reasons for the lack of wider application of these methods. The reserva- tions about economic methods included the following: • Other factors driving investment decisions, including political forces and the need to consider initial project costs rather than life-cycle costs in distributing scarce project funding throughout the state. (The agency making this latter point plans to investigate economic methods in the future, but it would be a difficult concept to sell in the current funding climate.) • Perceived problems with models and data; for example, questionable assumptions, inadequate available data, and lack of confidence in data (engineering judgment is preferred). • The complexity and time involved in developing an EEA. • The perceived tendency of a BCA to favor a single type of safety treatment rather than a more diverse program of multiple types of treatments. • Litigation concerns with safety programs evaluated through BCA. ReseaRCH suggestions Several gaps in current knowledge and limitations of existing analytic models have been identified throughout this report. These needs for more and better information have been translated into the following recommendations for further research. 1. The errors and deficiencies in the 2009 TIGER grant applications have highlighted weaknesses in nation- wide practices regarding the conduct of BCA. More general findings by GAO and others reinforce the need for better understanding of the limitations of engineer- ing economic methods and potential errors in applying these methods. These weaknesses may take on higher priority for correction if the TIGER grant program becomes a model for a future mechanism of federal transportation funding. Several of the errors observed by FHWA in its review of grant applications are basic in nature; for example, erroneous treatment of some economic development and local construction impacts as project benefits, incorrect treatment of discounting and inflation, and unrealistic base cases and project lifetimes. Research is needed to understand the best way in which these types of errors can be corrected, whether through training, peer exchanges, webinars, or development of self-training tools (software), to name a few candidates. Further research would develop the recommended approach as a product for use by trans- portation agencies.

95 2. The FHWA review of the TIGER grant applications generated a topic for future research: to understand bet- ter the nonroad user benefits and nontraditional impacts of transportation investments. The research could also address the distribution of these benefits and impacts; for example, to public versus private entities; local ver- sus national perspectives; and by income level or popu- lation group. This research could structure findings in a manner suitable for use in BCAs of future highway investments. 3. International experience has highlighted the value of ex-post or post-construction BCAs to assess the actual economic results of completed operational projects and compare these values to the estimates made when appraising the project during planning or design. A diligent program of post-construction analyses can improve future benefit–cost calculations and promote the use of higher quality data and better estimates of parameter values. Research is needed to define a rec- ommended structure and protocol for these analyses. A protocol is needed because the post-construction analy- sis should agree in its computational structure with the original benefit–cost estimate made during project appraisal, and the appraisal estimate must therefore be correctly done. For example, it would not include the types of errors that have been observed in FHWA’s and GAO’s reviews of the TIGER grant applications. As a final note, the international scan on performance management discussed in chapter four likewise recom- mended that U.S. agencies consider applying a post- construction, or ex-post, evaluation of their project’s actual economic results. 4. Agencies internationally have shown interest in better modeling the variability in travel time, a factor impor- tant for high-value and time-critical categories of freight. Research is needed to investigate and develop analytic procedures for this purpose, together with recommenda- tions on setting a value of time for time-critical goods. 5. With the exception of Caltrans’ documentation of its value analysis program, there is little information on how long it takes an agency to conduct EEAs. This duration could also vary by the relative experience and familiarity of agency personnel with economic meth- ods, the type of analysis and stage of decision making involved, the status of agency practices in data collec- tion and the degree to which they could support eco- nomic analyses, the analytic tools available to assist in these analyses, training and other startup activi- ties needed, and other factors. Research is needed to quantify the expected time commitment that would be needed for preparation and then to conduct these analyses on a routine basis. 6. An alternate approach is to study how agencies that are conversant with economic analyses have integrated these methods within their business processes. Caltrans and WSDOT provide examples of state DOTs that have done this successfully. Such integration implies economy of activities such as data collection and pro- cessing that simultaneously serve the requirements of design engineering, traffic engineering, economic analyses, and performance monitoring. Departments that use economic analysis effectively and efficiently assign high value to automating data collection, per- forming the economic analyses, and reporting results. These agencies also rely on other resources for special- ized knowledge and tasks such as consultants, univer- sity researchers, and other public agencies. Research is needed to understand the paths by which these agencies transform themselves into effective users of economic methods, and the time, costs, and other resources required. It would also be instructive to identify what human, organizational, and technical factors within this integrated approach would enable agency person- nel to exhibit greater team creativity and insight (e.g., in defining alternative solutions for analysis), leading to more cost-effective solutions. 7. Bias in forecasting trends has been observed in research by Flyvbjerg et al. (2005) and Kriger et al. (2006). As an analytic solution, Flyvbjerg proposes a newer method: reference class forecasting. Research is needed to inves- tigate and describe this method as it would be used for the types of analyses prevalent in highway investments: typically, forecasts of travel demand and of revenues on toll roads. The research would include identification of changes in current state DOT practices, if any, that are needed to accommodate the new method, and any issues in realigning data to support the new approach. 8. TRB Special Report 288 notes the limitations of the current four-step model for travel demand estima- tion. This model is not inherently behavioral in nature and therefore may not accurately capture road users’ responses to the types of policy initiatives now being explored by transportation agencies and political bod- ies. These current studies might include applications of economic analysis that would require a network capability (e.g., ITS projects, high-occupancy vehi- cle projects, interchange additions or improvements, and addition of significant new road capacity). Fur- thermore, existing models begin to break down when disaggregate, individual-level responses must be pre- dicted. The latter requirements would be needed in esti- mating the time chosen for travel, individual responses to policies such as congestion pricing and telecommut- ing initiatives, nonmotorized travel, and freight and commercial vehicle movements. Research is needed to develop new travel demand models. Organizational and institutional changes might also need to be researched if model development will affect coordination among the state DOT, MPOs, the federal government, and any intergovernmental agreements. 9. Agencies differ in their characteristics, complicating the problem of transferring knowledge and technol- ogy from one agency to another. This issue has been observed by the European Commission as well as in the

96 United States. An approach to this problem regarding asset management techniques involved creating agency profiles in the form of a series of tables by state DOT, listing factors regarded as important to asset manage- ment practice. A similar approach might be applicable to EEA. Research is needed to investigate this idea, or to propose and investigate alternatives for transferring technology and disseminating information among state DOTs and other transportation agencies. The character- istics relevant to profiles related to economic analysis require further thought in the recommended research effort, but helpful starting points are suggested in the structuring of information in chapter two and the case examples in chapter three. These items might include but are not limited to the following: • Identification of how each agency has implemented relevant federal and national-level guidance, includ- ing federal law, FHWA regulations and guidelines, Office of Management and Budget guidelines, AAS- HTO guidelines, and findings in TRB studies. • Documentation of relevant state requirements and general agency policies and practices regarding the use of economic methods (e.g., statewide or agency determination of a discount rate; identification of recommended methods; and description of analytic tools already in use). • Categorization of projects and programs as focal points for EEA. For instance, Table 5 organizes eco- nomic analyses by stage of decision making within the project/program life cycle and Table 6 adds the system level(s) at which each is analyzed and the particular economic method used. • The specific information technology products or analytic tools that are used to conduct each type of analysis indicated earlier. Input data and their sources (with the responsible organizational unit) can also be included. • Noneconomic factors that are included with eco- nomic criteria in informing decisions. • Particular challenges that agencies have identified in the use of their analyses.