Return on Investment in Transportation Asset Management Systems and Practices (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

167 A P P E N D I X G Supplemental Guidance on Use of Simulation Results in ROI Analysis As described in the ROI calculation guidance in Chapter 3 and in several of the references included in the review described in Appendix A, often simulation tools can be used to help predict the effects of implementing a new TAM system or improved process. The calculation tool documented in Chapter 4 specifically accommodates simulation results from three possible systems: HERS-ST, NBIAS, and an agency PMS. This appendix provides additional guidance on use of simulation tools for supporting an ROI analysis. The term “simulation” in this context refers to simulating over some period of time the effects of different investment strategies on asset conditions, agency and user costs, and/or other measures. The following subsections describe general considerations, guidance concerning the three systems listed above, and a checklist summarizing issues to address in configuring and running a simulation model in support of an ROI analysis. GENERAL CONSIDERATIONS The following are general rules of thumb to consider when using any investment simulation tool to support the analysis of ROI of a TAM system or process improvement. 1. Use of a simulation model should be considered if the TAM system or process improvement is expected to impact how an agency invests its capital resources. As discussed in Chapter 2, implementation of a TAM system or process improvement may yield a range of types of benefits. Where the improved benefit of a new system or improved process is an efficiency gain, resulting in a reduction of agency costs for performing a given set of functions, there is no particular need to complicate the ROI analysis with a management system simulation. Likewise, benefits from an investment in better systems and/or data that reduces the probability of asset failure can be captured in the ROI calculation tool without supplemental simulation. However, if the improvement is further expected to impact the nature of the agency’s capital program, such as through shifting resources to emphasize on preserving existing assets, then one should consider using a simulation model to address the impacts of the system or process improvement. The Western State and Eastern State case studies described in Chapter 3 both illustrate such cases. 2. There are inherent challenges in using a TAM system to analyze investing in a TAM system. Once one has established that a given TAM system may impact capital investment, a “chicken and egg” challenge presents itself. That is, an agency seeking to implement a new TAM system to improve its capital decisions most likely lacks a system to simulate the effects of different capital investment strategies. There is really no easy way around this conundrum, but there are certain situations where the necessary simulation tool may be available. Specifically, the FHWA tools HERS-ST and NBIAS are publicly-available tools for simulating results of different highway and bridge investment strategies at a network level that may aid an analyst in performing ROI analyses for TAM system and process

168 Return on Investment in Transportation Asset Management Systems and Practices improvements. Another alternative is to utilize TAM system results from an agency with a similar asset inventory and operating conditions as the case to be analyzed. 3. It is particularly difficult to identify existing asset simulation tools for assets other than pavement and bridges. There are several reasons for this. First, the readily available tools listed above are designed primarily for analyzing pavement and bridges. Second, capital investments in other assets are often impacted by other factors that are difficult to capture in a management system simulation. For instance, many traffic and safety features improved when a pavement need is identified. Third, many simulation tools and approaches require are “data hungry,” requiring data to run that realistically can be captured for long- lived assets such as pavement and bridges. However, other recent research provides some approaches that can be adapted for analysis of other assets. NCHRP Report 713 describes approaches for modeling asset life, addressing a number of different highway asset types. NCHRP Project 14-20A, scheduled for completion in 2017, will provide example models illustrating effects of delaying maintenance for assets such as culverts and highway lighting that could be adapted for use in an ROI analysis. 4. Management system results should be carefully validated. Put simply, one should not take the results generated by any management system at face value – the results should be validated in some fashion. This is always true, but is particularly true in cases where the management system one is using to perform an analysis is the very system in which one is contemplating an investment! One approach for validating a system is to use it to predict current asset conditions starting with historic conditions (i.e., conditions from 10 years ago) and actual agency expenditures, approximating the agency’s investment strategy. If the system correctly predicts actual conditions starting from historic data then this may provide a measure of confidence in its ability to predict future conditions. Likewise, if the system predicts conditions at great variance from reality, this should serve to identify a need for further work to review model assumptions and parameter values. 5. A key issue in using a simulation tool is to define a realistic base and investment case. The base case in the analysis should describe “business as usual” while the investment case should describe what would happen if the new system/process were implemented. Both of these may be a challenge to define in a simulation tool. If one has performed model validation using historic data, then this may be used to help establish the base case. But how will an agency’s investment strategy actually change given the availability of a new system? It may not be realistic that an agency will suddenly change its investment strategy and comply with the recommendations of a new TAM system. However, as illustrated in the case studies, a new system may be accompanied by a shift in policy or outlook that does have significant positive impacts, however difficult these may be to forecast. Ultimately the analyst’s responsibility is to develop as realistic a set of assumptions as he or she can, and carefully document these to support the decision-making process. TOOL-SPECIFIC GUIDANCE Additional guidance in using the tools identified in Chapter 4 is provided below. HERS-ST: this system is designed to model highway investment needs broadly, including needs for highway widening and improvement, in addition to pavement preservation and rehabilitation. Consequently, its approach to modeling pavement treatments is relatively coarse compared to that of a PMS. However, a significant benefit of using HERS-ST is that

Appendix G 169 it has a comprehensive set of models for predicting user and social costs that can be integrated directly into an ROI analysis. The basic approach recommended for using HERS-ST is to first calibrate the system’s parameters to predict current conditions using historic data and costs. The same parameters used for the validation can be used to develop the base case. For the investment case one can evaluate a shift to greater emphasis on preservation by adjusting the thresholds in the system for rehabilitation and reconstruction, and/or pavement treatment costs (e.g., to approximate greater use of lower-cost treatments). Alternatively, if the effect of implementing a system or process improvement is to provide increased funds for highways, this can be simulated simply by increasing the budget. Although the system is intended for use in simulating a full range of highway improvements, the system parameters can be adjusted to suppress most or all highway improvements if these should be excluded from an analysis by changing the criteria for triggering improvements, or specifying that widening an existing highway is infeasible. References (24) and (25) described in Appendices A and B detail the use of HERS-ST for calculating the benefits of TAM investments. NBIAS: this system is used by FHWA to analyze bridge investment needs at a national level (run in conjunction with HERS for analyzing highway investment needs). As of 2017 FHWA was using Version 4.2 of the system for its analyses, and was testing Version 5.1 of the system. NBIAS simulates need for bridge maintenance, repair and rehabilitation, as well as for functional improvements including bridge widening (of existing lanes and shoulders), raising, strengthening and replacement. It requires only the data available in the FHWA National Bridge Inventory (NBI) to run, and thus can be used to simulate investment needs and conditions for all U.S. highway bridges on the public road system. Similar to the case with HERS-ST, an advantage of using NBIAS is that it predicts a range of agency and user costs that can be integrated directly into an ROI analysis. Further, the user cost parameters in the system are populated by FHWA and derived from those used in HERS. Although NBIAS is populated with state-specific cost and deterioration models, calibration of the system using historic data is both highly recommended and highly feasible (given historic NBI data are available on the FHWA web site). As in the case of HERS-ST, once an analyst has calibrated the system based on historic data, the parameters used for the calibration can be applied going forward to establish the base case. Approaches to adjusting an agency’s investment strategy within the system include selecting an alternative policy for when to perform bridge preservation treatments (the system comes populated with four default models), or adjusting the business rules defined in the system for triggering bridge replacement. The system does not model preventive maintenance treatments performed on bridge elements in good condition, such as bridge washing, but preventive treatments can be simulated by changing the assumed rate of bridge deterioration (e.g., assuming a lower rate of deterioration resulting from increased preventive maintenance). As in the case of HERS- ST, if the effect of implementing a system or process improvement is to provide increased funds for bridges, this can be simulated simply by increasing the budget. PMS: if one chooses to use an agency’s PMS to simulate condition, then the ROI calculation tool requires specification of the International Roughness Index predicted over time by pavement group. The tool utilizes user cost models extracted from HERS to predict costs as a function of roughness for each group. Different pavement groups are defined in the tool to allow for difference in functional class, traffic, and other key parameters, but it is not

170 Return on Investment in Transportation Asset Management Systems and Practices necessary to define different pavement groups for each functional class to perform an analysis. Typically a PMS is configured with one or more major systems (e.g., Interstates, non-Interstate National Highway System roadways, and other roadways). The tool will allow for as few as one and as many as 12 pavement groups (plus “Other”). If a PMS is used to support the ROI analysis, the details of how to configure the PMS and define the scenarios for analysis will necessarily depend on the specific system used. However, suffice it so say that validating results for a comparable system or through use of historic data is particularly important to provide a basis for relying on the system’s results. One approach for validating results generated by a PMS is to compare the amount of work predicted by treatment time to that expected and consider whether the work being simulated by the system is realistic. If the system is simulating an inordinate amount of work be performed for a given treatment type, this may trigger a need for closer investigation of the results. Once the PMS generates a credible base case, changes in investment strategy can be simulated through adjusting constraints on how much work is performed by treatment type, changing the system’s decision trees, or adjusting the budget.

Appendix G 171 CHECKLIST Table G-1 is a checklist the analyst should review when using HERS-ST, NBIAS, an agency PMS or other simulation tool to support calculating ROI of a TAM system or process improvement. Table G-1. Checklist for Use when Utilizing a Simulation Tool for an ROI Analysis. Description Notes Current asset inventory and conditions Highway Performance Monitoring System (HPMS) data used for HERS-ST and National Bridge Inventory (NBI) data used for NBIAS are readily available – other data may be difficult to obtain particularly if implementing a new system. Historic asset inventory and conditions Needed if validating the model using historic data. Asset deterioration rates Default values populated in HERS-ST and NBIAS. Asset treatment policies Specified through parameters, preservation policies, and/or decision trees. Default values populated in HERS- ST and NBIAS. Asset treatment costs These should be validated even when populated by default. Asset life cycle policies Specified through parameters, preservation policies, and/or decision trees. Default values populated in HERS- ST and NBIAS. User cost parameters Required but populated with default values in HERS-ST and NBIAS. May or may not be present in other systems. Model validation If feasible validation should be performed through use of historic data or comparison to a similar system. Base case definition Can utilize parameters for validation with historic data if performed. Includes definition of expected budget. Investment case definition Typically varies from the base case through the asset life cycle policies or budget. Results review Review results for feasibility, overall reasonableness prior to incorporation in the ROI analysis. Documentation of model assumptions Key parameters and model assumptions should be documented to support further investigation and analysis.

Abbreviations and acronyms used without definitions in TRB publications: A4A Airlines for America AAAE American Association of Airport Executives AASHO American Association of State Highway Officials AASHTO American Association of State Highway and Transportation Officials ACI–NA Airports Council International–North America ACRP Airport Cooperative Research Program ADA Americans with Disabilities Act APTA American Public Transportation Association ASCE American Society of Civil Engineers ASME American Society of Mechanical Engineers ASTM American Society for Testing and Materials ATA American Trucking Associations CTAA Community Transportation Association of America CTBSSP Commercial Truck and Bus Safety Synthesis Program DHS Department of Homeland Security DOE Department of Energy EPA Environmental Protection Agency FAA Federal Aviation Administration FAST Fixing America’s Surface Transportation Act (2015) FHWA Federal Highway Administration FMCSA Federal Motor Carrier Safety Administration FRA Federal Railroad Administration FTA Federal Transit Administration HMCRP Hazardous Materials Cooperative Research Program IEEE Institute of Electrical and Electronics Engineers ISTEA Intermodal Surface Transportation Efficiency Act of 1991 ITE Institute of Transportation Engineers MAP-21 Moving Ahead for Progress in the 21st Century Act (2012) NASA National Aeronautics and Space Administration NASAO National Association of State Aviation Officials NCFRP National Cooperative Freight Research Program NCHRP National Cooperative Highway Research Program NHTSA National Highway Traffic Safety Administration NTSB National Transportation Safety Board PHMSA Pipeline and Hazardous Materials Safety Administration RITA Research and Innovative Technology Administration SAE Society of Automotive Engineers SAFETEA-LU Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (2005) TCRP Transit Cooperative Research Program TDC Transit Development Corporation TEA-21 Transportation Equity Act for the 21st Century (1998) TRB Transportation Research Board TSA Transportation Security Administration U.S.DOT United States Department of Transportation

TRA N SPO RTATIO N RESEA RCH BO A RD 500 Fifth Street, N W W ashington, D C 20001 A D D RESS SERV ICE REQ U ESTED N O N -PR O FIT O R G . U .S. PO STA G E PA ID C O LU M B IA , M D PER M IT N O . 88 Return on Investm ent in Transportation A sset M anagem ent System s and Practices N CH RP Research Report 866 TRB ISBN 978-0-309-44676-1 9 7 8 0 3 0 9 4 4 6 7 6 1 9 0 0 0 0