Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
157 A P P E N D I X F Pilot of ROI Guidance and ROI Calculation Tool (ROI Tool) Context A New England state agency was chosen for the pilot based on the fact that the agency indicated an interest in participating in the pilot, was considering multiple investments to improve its asset management systems and processes, and appeared to have the data required to support the testing. The agency operates a highway system with over 4,500 miles of road, as well as numerous other assets, including culverts and closed drainage systems (referred to as culverts). Analysis Scope The project team decided to pilot the ROI guidance by calculating ROI for investments in the agencyâs culvert asset data and management system. The agency currently has detailed data on its bridge-length culverts, but incomplete data on shorter culverts. These culverts occasionally fail and may result in roadway closure while emergency repairs are performed. The agency is interested in improving its culvert inventory to support improved inspection and maintenance of culverts. This analysis focuses on the potential installation of a culvert data collection and inventory system. The TAM investment should reduce the number of failure incidents, resulting in agency, user, and social cost savings. Analysis Approach The validation testing followed the basic ROI calculation steps defined in Chapter 4 â ROI Calculation Guidance. The steps are illustrated below and include the following: Step 1: Describe the Purpose of the Study and the Scope of the TAM Investment. This step first involves an evaluation of the planned ROI analysis to establish the purpose and extent of the ROI analysis, as well as the scope of the potential investment. Step 2: Identify Likely Impacts. âImpactsâ are defined as changes that occur as a result of adopting a TAM system or process, while âbenefitsâ are defined as impacts that have a measureable value to the agency, assets, users, or community. Not all impacts will be easily quantified or monetized. Step 3: Assess Available Data. Review the impacts listed in Step 2 and assess the availability of the data needed to estimate them. This data assessment will identify gaps in the data available for carrying out the analysis. STEP 7 Estimate ROI and Summarize Results STEP 6 Conduct Analysis STEP 5 Collect Necessary Data STEP 4 Establish Modeling Framework STEP 3 Assess Available Data STEP 2 Identify Likely Impacts STEP 1 Define Purpose of Study
158 Return on Investment in Transportation Asset Management Systems and Practices Step 4: Establish Modeling Approach and Identify Tools. Define the investment case and the base case, establish which benefits to include in the analysis, choose an analytical method for quantifying impacts, and establish key modeling parameters. Step 5: Collect Necessary Data. Collect all data necessary for the analysis, building on Steps 3 and 4. Usually, this will entail compiling existing data from multiple sources rather than original data collection in the field. Step 6: Conduct Analysis. Conduct the analysis defined in Step 4. The analysis will yield an estimate of the impacts and monetized value of the investment. Step 7: Estimate ROI and Summarize Results. Calculate an estimated ROI based on the results of the analysis conducted in Step 6. Four ROI metrics to calculate are net present value (NPV), benefit-cost ratio, internal rate of return (IRR), and payback period. In addition to the quantitative results of the analysis, this step should summarize the qualitative benefits of the investment. This final step should account for uncertainty in the analysis and include a review for errors and biases. The project team began the pilot by meeting with the agency to review the pilot scope, select potential TAM investments, and begin gathering data. The agency and the project team agreed to evaluate a potential investment in a culvert management system. The agency currently has no system for culvert management for culverts with spans under 10 feet. Culverts with a span 10 feet or longer are classified as bridges and included in the state bridge management program. The project team used the ROI calculation tool developed during this project to estimate the costs and benefits of a potential investment in a culvert management system for the agency. The agency provided data on culvert inventory, inspection, maintenance, failures and costs. The project team used estimates based on other sources where data was unavailable. TAM Investment Costs The first step in calculating the ROI of the potential TAM investment was estimating the costs of a culvert management system in the base case and investment case. The ROI tool includes three types of costs: agency, user, and social (emissions) costs. In order to run the ROI tool, the project team needed to include data for agency costs such as labor costs, failure incident costs, and investment costs; user costs such as travel-time costs and failure incident costs; and social costs such as emissions costs. The inputs sheet of the ROI tool is shown below in Figure F-1.
Appendix F 159 Figure F-1. Inputs Worksheet. The general parameters of the analysis are set at the default values of the tool, with three exceptions as shown in Figure F-2. First, the analysis period is ten years. Second, the annual cost per full-time equivalent (FTE) is estimated to be $119,993. This value is based on the average annual cost to an employer for workers in computer and mathematical occupations. The analysis assumes that agency labor involved in installing and operating a culvert management system would fall under this labor category. Third, the agency cost per incident is estimated to be $1,000,000. The analysis defines âincidentâ as unplanned, emergency culvert work on a major road. This value is an estimate based on the costs of recent culvert failures, recent culvert work performed, and programmed funds for future culvert projects.
160 Return on Investment in Transportation Asset Management Systems and Practices Figure F-2. General Parameters. The analysis modified two other incident cost model parameters from the default values offered in the tool, shown in Figure F-3. The project team estimates the average incident duration to be 540 minutes. This estimate was discussed by the project team and the agency and based on recent culvert failures. The analysis also includes the estimate that AADT is 14,260, based on the AADT of state- programmed culvert projects. All other parameters with default values are unchanged.
Appendix F 161 Figure F-3. Incident Cost Model. The analysis assumes no costs associated with a culvert management system in the base case, because no such system exists at the agency currently. For simplicity of analysis, the analysis assumes 1.1 culvert failure incidents will occur each year on average in the base case. This means $1,100,000 in annual agency costs due to incidents. The base case is shown below in Figure F-4. Figure F-4. Base Case.
162 Return on Investment in Transportation Asset Management Systems and Practices The costs of the investment case are shown below in Figure F-5. The initial cost of acquiring a culvert inventory and data collection system in the investment case is estimated to be $50,000. This cost occurs in the first year of the analysis and again in the sixth year of the analysis. The acquisition cost is assumed to recur every five years due to new software being available. The project team reviewed public records documenting transportation asset management system procurement contracts for public entities to help define the acquisition cost estimate. In many of the examples of asset management systems reviewed by the project team, the products include maintenance management system functionality similar to what the agency already has. The $50,000 cost estimate used in this analysis is for an asset inventory and data collection system without extraneous capabilities. As described in Chapter 5 â Using the ROI Calculation Tool, future costs and benefits are discounted at a rate of 4%. The selection of a 4% discount rate was explained in Chapter 2. In brief, the rate is in accordance with historic real discount rates between 3%-5%. A rate of 4% also avoids overstating potential benefits and is consistent with the rate used in the case studies. Discounted at a rate of 4%, the present value of the acquisition costs of a culvert data collection and inventory system is estimated to be $91,096. Figure F-5. Investment Case. In addition to the acquisition cost, the analysis assumes a $35,000 annual contractor cost. This cost is an estimate of software support, maintenance, and update costs. The estimate is based on the review of system costs noted previously. Discounted at a rate of 4%, the present value of the contractor costs of a culvert data collection and inventory system is estimated to be $295,237. The analysis also includes recurring costs of $270,000, representing the estimated cost of data collection efforts for the agency. The estimate was developed by assuming that data collection is performed by three two-person teams working 21 days a month, six months a year (during summer months), with each team covering approximately 2.4 miles per day. The analysis assumes that data collection activities cover the road system every five years, meaning that one-fifth of the 4,597 miles of state-owned road is surveyed in any given year. Assuming that the cost of an FTE performing data collection duties is $90,000, a value based on state wage data and agency experience, data collection for culverts will cost roughly $270,000 per year. Discounted at a rate of 4%, the present value of the recurring costs of a culvert data collection and inventory system is estimated to be $2,277,540. The calculation includes agency labor costs associated with operating a culvert management system. As noted previously, the annual cost per full-time equivalent (FTE) for Computer and Mathematical Occupations is estimated to be $119,993. The analysis assumes that 1.0 FTE would be required in years that the software is installed and 0.5 FTEs would be required in all other years. Discounted at a rate of 4%, the present value of the agency labor costs associated with a culvert data collection and inventory system is estimated to be $615,400.
Appendix F 163 In sum, the present value of the total cost to install and operate a culvert data collection and inventory system is estimated to be $3,279,272. TAM Investment Eï¬ects The second step in calculating the ROI of the potential TAM investment was estimating the effects of the TAM investment in the investment case. The primary effect of an investment in a system for culvert data collection and inventory would be the reduction in culvert failure incidents. The analysis assumes that culverts which are inspected will not be subject to avoidable failure. With additional inventory and condition information about culverts, agency engineers could make a better case for maintenance and avoid surprise culvert failures. Given the assumption that data would be collected for 1/5 of the system every year, the analysis assumes that predicted failure incidents will decline 0.2 each year, dropping from 1.1 failure incidents in the first year of the analysis to 0.1 failure incidents after five years. Note that this decline is modeling the effects of a decline of 1 incident per year, rather than modeling a reduction of incidents to 0. The analysis uses a reduction from 1.1 to 0.1 for simplicity, but the results would be the same if the decline was from 10 to 9 or from 50 to 49. The 0.1 annual failure incidents after five years represent the unknown number of unavoidable failure incidents. This decline, shown above in Figure F-5, would create a series of trickle-down effects, leading to reductions in traffic, property damages, human injuries, environmental emissions and more. These effects are included in the analysis and documented further in the next section. In addition to the effects captured and quantified in the tool, one would expect to see additional effects that are more difficult to measure. One example is the bundling of paving projects with culvert projects. Currently, much of the culvert work is driven by the paving program for the agency. Maintenance personnel discover culvert problems while scouting roads ahead of pavement work. This means that culvert problems are sometimes discovered without sufficient time to bundle the pavement projects with the culvert work. This results in additional and unnecessary contracting costs. Consistent inspection and data collection would allow the agency to change programming tendencies and to bundle paving and culvert projects. Beneï¬ts The final step in calculating the ROI of the potential TAM investment was estimating the benefits of the TAM investment by comparing the investment case to the base case. The reduction in culvert failure incidents generates agency, user, and social benefits, summarized below in Figure F-6. Figure F-6. Benefits of Incident Reduction. As predicted failure incidents decrease from 1.1 to 0.1 in the investment case, agency cost decreases from $1,100,000 to $100,000 over five years. In other words, the undiscounted, annual agency benefit increases from $0 to $1,000,000 over five years. Discounted at a rate of 4%, the present
164 Return on Investment in Transportation Asset Management Systems and Practices value of agency benefits resulting from a culvert data collection and inventory system is estimated to be $5,584,808. The reduction in failure incidents also reduces user costs such as travel-time cost caused by traffic; vehicle operating costs; and fatality, injury, and property damage costs. As described in Chapter 5, using the ROI Calculator, the default values for cost per fatality, cost per injury, and cost per property damage incident are estimates based on values used by FHWA for national level analyses in HERS and NBIAS as of 2016, specified in 2014 dollars. Discounted at a rate of 4%, the present value of user benefits resulting from a culvert data collection and inventory system is estimated to be $257,522. The reduction in failure incidents and subsequent reduction in traffic results in fewer detours, delays, and harmful emissions. Discounted at a rate of 4%, the present value of user benefits resulting from a culvert data collection and inventory system is estimated to be $4,646. The present value of the total benefit of a culvert data collection and inventory system is estimated to be $5,846,976. In addition to the quantifiable benefits, the project team expects that the TAM investment would also reduce administrative & contracting costs due to bundling of pavement and culvert projects. These potential savings cannot be modeled by the tool. Results The results of the analysis are shown below in Figure F-7. Figure F-7. Summary Results Page.
Appendix F 165 The tool summarizes the results in four measures of return on investment. These measures and the breakdown of total costs and benefits are shown below in Figure F-8. Figure F-8. Summary Measures. The net present value is calculated by subtracting the present value of the total costs from the present value of the total benefits. The Benefit/Cost ratio is calculated by dividing the present value of the total benefits by the present value of the total costs. Internal Rate of Return is the hypothetical discount rate at which the NPV of the investment is $0. The payback period is the length of time required for the investment to break even. According to the estimates of the ROI tool, net present value is $2,567,704, the Benefit/Cost ratio is 1.78; the Internal Rate of Return is 40.05%; and the payback period is 4 years. This analysis is subject to a number of assumptions and qualifications, including the following: The results do not include unknown benefits and costs that are not represented in the model. For example, the agency expects benefits of bundling pavement and culvert projects together, saving money on administrative and contractor costs. Quantifying other benefits or costs could skew or change the results of the analysis. The cost assumptions used in the analysis are based on expert judgement, consultation with the agency, and public data. However, assumptions may turn out to be inaccurate, either by understating or overstating the costs. For example, the costs of a project resulting from an emergency can be unpredictable: costs related to permitting may be reduced due to expedited approval, while labor costs may increase due to overtime. If acquisition costs,
166 Return on Investment in Transportation Asset Management Systems and Practices labor costs, or incident costs vary from the estimates, the results of the analysis may be affected. The analysis assumes a linear decline in culvert failure incidents, paralleling the development of the culvert inventory and inspection system. Culvert failures may not decline as quickly as anticipated if maintenance policy and programming does not adapt at the same rate. Alternatively, an increased focus on culvert management that accompanies a new management system may result in an accelerated decrease in culvert failure incidents. Depending on the rate of culvert failure incidents, the results of the analysis may be affected. The analysis does not take potential changes in funding into account. If the culvert management system is installed, but there is no funding for culvert maintenance, the benefits will likely be diminished. Alternatively, increased funding for culvert maintenance may result in greater benefits than estimated in the tool. Depending on the funding for culvert projects, the results of the analysis may be affected.
