Framework and Tools for Estimating Benefits of Specific Freight Network Investments (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

52 The Freight Evaluation Framework, described in detail in this chapter, provides a consistent approach to evaluating freight investments that is sensitive to the different spatial scales of freight improvement projects, the different benefits between the public and private sectors, and the different plan- ning and investment decision-making process used by pri- vate- and public-sector entities, all of which were described earlier. This chapter describes the principles used to guide Framework development, introduces the Framework concept and methodology, and describes how risk was incorporated within the Framework. 4.1 Developing the Framework Guiding Principles The literature review, interviews, and case studies described earlier led to the development of several principles that were used to steer the development of the Freight Evaluation Framework. These guiding principles are described as follows: 1. The Freight Evaluation Framework should be capable of evaluating freight investments competing with other freight investments across modal boundaries. Many freight investment stakeholders have developed and imple- mented approaches to evaluate different freight investments occurring on the same mode (i.e., deciding between com- peting rail investments or highway projects). Missing is an approach that will allow stakeholders to evaluate the poten- tial benefits of highway, rail, seaport, and intermodal con- nector projects on an apples-to-apples basis. This requires the development of common metrics that are meaningful across all of the modes and from the perspective of the affected stakeholders. 2. The Freight Evaluation Framework should be capable of evaluating projects that span all of the different freight modes, across all different levels of geography (local/site, regional/state, and national). As described earlier, the scale of freight projects can vary enormously—from localized projects completed over several weeks, to regional or national mega-projects that are measured in terms of years or decades. The Freight Evaluation Framework must be sensitive to these different scales, and be capable of being applied to local, statewide/regional, and multistate/national scales. 3. The Freight Evaluation Framework must use existing data and tools to the degree possible. Substantial public- and private-sector investment has gone into the develop- ment of various tools, methods, and approaches for assess- ing the benefits of freight investments. The Framework must complement and enhance these previous efforts without reinventing the wheel. The Framework should be developed in a manner that builds upon existing tools and leverages investments made to date (by U.S.DOT, NCHRP, AASHTO, various state DOTs, and universities) rather than appearing to compete against them. 4. The Freight Evaluation Framework must be consistent with existing decision-making processes used by freight stakeholders. As described earlier, different stakeholders clearly use different methods and processes to answer the question “Is this a good investment?” Similar to Guiding Principle No. 3, the Framework must not be developed in such a way that it competes with, or usurps, these existing processes. Rather, it should be developed so that it sup- ports the investment decision-making processes already employed by different freight stakeholders. The Frame- work should be a decision-support tool, not the ultimate decisionmaker. 5. The Freight Evaluation Framework should use a few good measures. Although the Framework should use quantitative performance measures, these measures should reflect the impact or benefit categories that are likely to be most important to different freight stake- holders in determining whether the project is beneficial C H A P T E R 4 Development of the Freight Evaluation Framework

53 from that group’s perspective. Although it may be tempt- ing to expand the number of overall indicators to more comprehensively understand the potential benefits and impacts of a proposed freight investment, some prospec- tive measures may be too peripheral to offer value and actually reduce the overall effectiveness of the assessment. 6. The Freight Evaluation Framework should allow for qual- itative assessment of investments. Although it is critically important to allow quantitative assessments of project ben- efits to drive the Framework, it also needs to be flexible enough to incorporate qualitative assessments. These qual- itative assessments will be helpful in fatal flaw analyses— reviews to ensure that the proposed project is practical and fits within the goals of affected stakeholders (particularly those without a direct financial stake in the investment). 7. The Freight Evaluation Framework should target freight investments that are on the left side of the benefit assess- ment spectrum. This spectrum, described in Figure 3.6, describes the types of investment decisions made by freight stakeholders and their associated timeframes. Because the types of tools and processes used by different freight stake- holders within this spectrum varies considerably, the Framework should be developed to meet the decisions that are strategic or tactical in nature, with a timeframe mea- sured in years or decades. Framework Concepts and Elements Using these principles as a guide, the Freight Evaluation Framework consists of four key elements: identify benefit cat- egories and metrics, calculate project costs, calculate benefits and impacts, and assess risks. Figure 4.1 describes how these individual elements link to the guiding principles described above; Figure 4.2 provides a more detailed description of the entire framework. A detailed overview of these four modules is provided in the following sections. Benefit Categories and Metrics The guiding principles underlying the development of the Framework call for the identification of a few good measures that represent the benefits that are most important to the var- ious stakeholders. In addition, the measures should be com- parable across modes and types of investments to allow for apples-to-apples comparisons of investment opportunities. The benefit types that are meaningful to the various stake- holders have been identified previously. Table 4.1 presents potential metrics to capture the benefits that are of concern to different freight investment stakeholders. There are a few mea- sures, such as transportation cost savings, crash reductions, emission reductions, and pavement/track conditions that will be important across a wide array of stakeholders. Others, such as maintenance savings and asset velocity, will be relevant to a small set of stakeholders but are likely to drive that stakeholder’s decision on whether to participate in the investment. Costs The costs of a constructed facility or implemented technol- ogy to the owner include both the initial capital cost and the subsequent operation and maintenance costs. Each of these major cost categories consists of a number of cost components. The magnitude of each of these cost components depends on the nature, size, and location of the project as well as the own- ing organization (i.e., public or private). • Capital costs for a project include the expenses related to the initial establishment of the facility, such as – Land acquisition, including assembly, holding, and improvement; – Construction, including materials, equipment, and labor; – Field supervision of construction; – Construction financing; – Insurance and taxes during construction; and – Inspection and testing. • Operation and maintenance costs in subsequent years over the project lifecycle typically include – Land rent, if applicable; – Operating staff; – Labor and material for maintenance and repairs; – Periodic renovations; – Insurance and taxes; – Financing costs; – Utilities; and – Owner’s other expenses. Identify Benefit Categories and Categories Metrics Guiding Principles 1, 2, 5, 6 Calculate Costs Guiding Principles 1, 3, 4 Calculate Benefits and Impacts Guiding Principles 3, 4 Assess Risks Guiding Principles 1, 4 Figure 4.1. Framework elements and guiding principles.

Assess Risks Capital Costs O&M Costs Other Costs Benefit Cost Metrics (NPV, BC ratio, ROI) Total Costs Economic Total Economic Impact Metrics (jobs, income, output, tax base) Change in Travel Distance Change in Travel Time Change in Travel Quality Change in User Operating Costs (by truck, auto, rail, aircraft, and vessel) Change in Crash/ Incident Costs (by truck, auto, and rail) Change in VMT for existing traffic (truck, auto, rail) Change in delay, average speeds, and reliability (trucks, autos, and rail) Change in Environment Costs (by truck, auto, rail, aircraft, and vessel) Change in Time Costs of Delay/ Unreliability (by truck, auto, rail) Change in Time Costs of Transportation (by truck, auto, rail) Change in Loss/ Damage of Cargo Freight Transportation Investment Direct Economic Impacts (value of out-of-pocket benefits) Total Efficiency Benefits (public and private) ID Stakeholders and Modes Benefit Categories and Metrics Costs Benefits and Impacts Risks Figure 4.2. Freight Evaluation Framework.

The Freight Evaluation Framework recognizes that although construction cost may be the single largest component of capi- tal cost, other cost components are not insignificant. For exam- ple, land acquisition costs are a major expenditure for building new or expanding existing facilities in high-density urban areas, and construction financing costs can be significant. From the owner’s perspective, it is equally important to estimate the cor- responding operation and maintenance cost of each alternative for a proposed facility in order to analyze the lifecycle costs. The large expenditures needed for facility maintenance, espe- cially for publicly owned infrastructure, necessitate the need to include operation and maintenance cost in the design stage. Cost information changes depending on the stage of the plan- ning process. There are three categories of cost estimates— design, bid, and control. Design estimates will be the most common source of cost data for use in the Framework. Design Estimates In the planning and design stages of a project, various design estimates reflect the progress of the design. At the very early stage, the screening estimate or order-of-magnitude estimate is usually made before the facility is designed, and must, there- fore, rely on the cost data of similar facilities built in the past. A preliminary estimate or conceptual estimate is based on the conceptual design of the facility at the stage when the basic technologies for the design are known. The detailed estimate or definitive estimate is made when the scope of work is clearly defined and the detailed design is in progress so that the essen- tial features of the facility are identifiable. The engineer’s esti- mate is based on the completed plans and specifications when they are ready for the owner to solicit bids from construction contractors.(9) Design-level cost estimates can be calculated using unit costing (e.g., cost per mile), historical cost data, or computer-aided costing. To account for risks involved in cost estimating, most con- struction cost estimates include an allowance for contingen- cies or unexpected costs occurring during construction. This contingency amount may be included within each cost item or may be included in a single category of construction con- tingency. The amount of contingency is based on historical experience and the expected difficulty of a particular project. Examples of various risk factors in cost estimating include the following: • Design development changes, • Schedule adjustments, • Cost of materials, • Site conditions that differ from those expected, and • Third-party requirements imposed during construction (such as new permits). Calculating Benefits and Impacts Freight transportation investments are designed to bring about changes in travel patterns, and these changes can yield benefits and economic impacts locally, regionally, and nation- ally. To properly account for these effects and to account for Benefit Type Benefit Metric Public Sector Service Provider Shipper/ End User Other Impacted Party Private-Sector Asset Provider Capacity Transportation Cost Savings Safety Crash Reductions Environmental Quality Emission Reductions Scheduling/ Reliability Reliability Improvements Facility Maintenance Costs Pavement/Track Maintenance Savings Loss and Damage Pavement/Track Conditions Productivity Asset Velocity Economic Development Jobs, Income, Industry Output Tax Revenue Tax Base Impact Facility Capital Costs Facility Costs Table 4.1. Benefit metrics by benefit and stakeholder type. 55

56 the benefits most relevant to various stakeholders and geo- graphic interests, the Freight Evaluation Framework proceeds in two parallel tracks: benefit/cost analysis (BCA) and eco- nomic impact analysis (EIA). Benefit/cost analysis identifies the benefits of investing (as compared with not investing), and compares these to the project costs. It includes both actual or out-of-pocket cost savings (e.g., reduced spending on fuel) and the broader social benefits (e.g., reduced vehicle emissions). Benefits and costs can be analyzed based on geography and to whom the benefits and costs accrue. Moreover, the direct travel efficiencies represent productivity gains that are a net benefit to the national economy, making BCA an appropriate analysis for national or federal investment decisions. In addition, con- ducting parallel benefit/cost analyses based on travel efficiency analysis for alternative projects using common metrics allows for the comparison of investments across modes. Economic impact analysis, in contrast, compares the over- all economic growth (e.g., employment, income, and output) in the specified study region with or without investing. Because this method focuses on regional economic growth, certain classes of benefits accounted for in benefit/cost analysis are excluded. Specifically, only those travel changes that affect the actual flow of dollars through the regional economy are considered, thus excluding social benefits and personal travel- time savings. Generally, EIA is useful for local-, regional-, and state-level analysis because the measures in economic growth often represent a redistribution of economic activity from one location to another resulting from increased competitiveness. The Framework addresses both BCA and EIA because utilizing both types of analysis provides two sets of metrics for evaluat- ing freight investments to meet the needs of various stakehold- ers and to reflect local-, regional-, and national-level benefits. As illustrated in Figure 4.2, there are significant distinctions between benefit/cost analysis and economic impact analysis. First, BCA weighs the costs of a given investment initiative against the benefits it provides to their users. It involves iden- tification and estimation of all private, public, and social costs and benefits of an investment to derive a measure of net ben- efit or a benefit/cost ratio that measures the value of benefits received per dollar in costs. Another major distinction is that private and public expen- ditures/investments and business output and jobs that result from those expenditures are viewed as costs in BCA. This is because they consume societal resources that could have alter- native uses. Other possible costs include ongoing operation and maintenance costs, as well externalities such as pollution, noise, and reduction of property values. Reduction of these (or other) costs is viewed as a benefit, and a reduction of some existing benefits is viewed as a cost. In EIA, the cost of the investment is often regarded as an injection of construction spending that gives rise to immediate, short-term increases in employment and other economic benefits. For the purpose of both BCA and EIA, all costs and benefits are measured over the project lifecycle to capture the timing of costs and benefits. Then the NPV of the costs and benefits is calculated using the appropriate discount rate. 4.2 Incorporating Risk The incorporation of risk into the Freight Evaluation Frame- work represents a significant enhancement to the freight investment analysis tools, methods, and processes that have been developed by U.S.DOT, NCHRP, AASHTO, various state DOTs, and universities. Risk in the context of a freight invest- ment refers to downside outcomes due to uncertainty. From a financial perspective, investors or bond holders may experi- ence weaker-than-anticipated returns on their investment. Weak returns can be the result of weaker-than-expected demand for a facility’s services, or higher-than-expected capi- tal or operating costs, or a combination of the two. From the public’s perspective, the project may not yield its anticipated benefits in the form of congestion mitigation or job creation. Risk Factors Risk assessment has been a critical component of private- sector investment decision-making for a long time because sizable losses can be devastating to firms of all types and sizes. Risk management metrics also have a role in customer satis- faction, potential market development, and market access. All of the functions in this category can have a direct cost— insurance, employee safety and retention, financial penalties and down time, etc. On the public-sector side, risk manage- ment techniques are typically included in asset management strategies for pavements, bridges, and other investments. Rarely are risk management techniques employed as part of the investment decision-making activities of these agencies, including freight investments. Although considering risk in the Freight Evaluation Frame- work does add a degree of complexity, it is warranted. Although simplicity in analysis is desirable, the research team also sought to avoid analyses that are more simple than necessary in order to arrive at a decision. The “correct” level of detail is one that addresses the problem while not overburdening the analysis. To provide this balance, the Framework assesses a limited number of risk factors that most significantly impact freight investment decisions. These include the following: • Market risks, or those that relate to overall demand on the potential investment. Factors to consider include confidence in forecast growth in population and business activity, development of competing facilities and services, applica- tion of new technology, or other external factors (i.e., rela- tive prices).

• Cost risks, or those that relate to cost overruns associated with scope creep, cost overruns associated with price increases of raw materials or labor availability, or unantic- ipated delay costs. • Methodological risks, or those that significantly affect the conclusions of an analysis, such as approximation introduced by the level of aggregation or level of detail included in the analysis (e.g., assumptions about “peak- ing,” spatial resolution of geography); value judgments and policy variables (e.g., prices assigned to emissions or the value of life); or uncertainties about technical, eco- nomic, and political quantities (e.g., future vehicle fuel burn and emission rates, future inflation rates, potential impacts of new regulations). • Moral hazard risks, or those related to an individual or orga- nization’s inadequate incentive to guard against a risk when there is protection against it. For instance, in a public- private partnership where the government funds the invest- ment and the private partner manages and operates it, there is some likelihood that the private partner will not ade- quately represent the public sector’s interests. It is crucial that the selection of the private partner and the public- private partnership contract ensure that the public’s inter- ests are not compromised. One of the problems inherent in public-private partnerships is that the pre-decision analysis will typically assume that an identity of interests exists among the partners. However, this will only be the case in practice if contracts and incentives are structured in a way that supports this goal. Accounting for Risk The following two cases of risky outcomes are incorporated within the Freight Evaluation Framework: • Case 1: The project is operational, but disappoints at least one stakeholder and fails to realize the hurdle rate that was established in the planning stage and upon which the deci- sion to proceed was based. (Hurdle rate refers to a breakeven threshold—in the case of benefit/cost analysis, NPV=0 is the hurdle rate that a project is expected to exceed. A private entity will typically seek a rate of return in excess of some value equal to the entity’s opportunity cost of capital.) • Case 2: The project fails overwhelmingly so that it is either abandoned or bailed out with unanticipated public fund- ing to keep it operating. The process for assessing risk in the Framework screens and rejects grand failures of the Case 2 type. Projects that pass the first level of screening will be subject to additional scrutiny and risk analysis, permitting informed decisions on the level of risk and its acceptability to the project stakeholders. The risk assessment component of the Framework is shown in Figure 4.3. This process conducts an initial screen for potential Case 2 failure to identify these potential catastrophic failures early in the process. The analysis of Case 1 risk is an additional dimension to the evaluation and the go/no-go bot- tom line analytic metrics for each stakeholder group and is expressed as risk analysis results (i.e., a range of results with an associated probability for attaining a particular result). Risk Case Scenarios and Stakeholder Impacts Case 1 risk analysis screening involves assessing different project alternatives for downside risk using different scenarios. The scenarios correspond to specific future outlooks in which a combination of unanticipated negative events, or risk cases, Identify Soft Factors and Weak Assumptions in Analysis Assess Probability of Case 2 Failure Is Less Than Lower Acceptable Threshold? Reject Project Apply Risk Analysis Scenarios Accept Project Alternative Find Probability Distributions of all Key Outcomes Confidence Acceptable? Refine Alternative and Seek to Mitigate Risks yes no yes no Figure 4.3. Freight Evaluation Framework risk assessment component. 57

58 occur with adverse impacts on the project. Evaluating different scenarios also allows for an assessment of how different risk cases impact different stakeholder types. This is critically important because downside risk may not be borne equally among different stakeholders and the decision-making process will need to ensure that the opportunities for each party and the risks assumed by each are acceptable. Table 4.2 describes sample alternative scenarios for a freight transportation investment being funded by the private sector and the severity of the downside risk on various parties. (The indicators of severity are illustrative. In principle, contractual arrangements could shift risk across parties. For instance, an infrastructure provider could have contractual guarantees that cover its prospective losses in the event of facility underutiliza- tion.) These examples show that the asset provider (who builds and operates the new facility) has the most at stake. In the event of a catastrophic failure with little or no return on its invest- ment, the asset provider’s financial loss could be very large. At the other end of the spectrum, prospective facility users may have little at stake should a project fail because there may be existing alternatives or the users may be able to shift their activ- ity so that the impact is minimal. Because the different toler- ances for risk across different project and stakeholder types factor significantly into eventual decisions, such risk scenario assessments are a critical component of the overall implemen- tation of the Framework. Risk Case Case 1 Operational, but Falls Short of Expectations Case 2 Fails Overwhelmingly Minimal Impact Scenario Example Weak demand due to economic conditions or competing modes and facilities Price shocks Demand dramatically less than forecast Projected local componennt of demand fails to materialize Region fails to proceed with complementary projects that were critical success components Slight Impact Severe Impact Provider Stakeholders and Impacts of Downside Risk Asset Service Provider End User Other Impacted Party Table 4.2. Risk scenarios and stakeholder impacts.