Benefit–Cost Analyses Guidebook for Airport Stormwater (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

46 By this point in the analysis, the airport has identified and, to the extent possible, monetized the benefits (Chapter 4) and costs (Chapter 5) of one or more project alternatives. With this information, the airport is ready to compare the benefits and costs of each project to identify the alternative that will result in the highest value or otherwise meet the airport’s goals. There are several metrics that measure value. As discussed in previous chapters, these metrics can be quantitative or qualitative, and they will enhance the airport’s understanding of the outcomes of each project alternative. This chapter summarizes the final steps needed to prepare for comparing benefits and costs. A summary is then provided of the quantitative and qualitative measures that will likely be most useful to an airport evaluating a stormwater project. This chapter also addresses the importance of risk and uncertainty in the results of the BCA and provides information on how airports can manage these issues in their analyses. The risk due to uncertainty is separate from, but related to, the risked posed by stormwater. As described in the following, the analysis must make some assumptions about unknown variables, which can affect the analysis’s conclusions about the best approach for mitigating the risk posed by stormwater. This chapter concludes with a demonstration of a BCA using the hypothetical airport featured throughout this guidebook. Figure 7 shows a flowchart that depicts the process of comparing benefits and costs. This flowchart is a general example; in the real world, the BCA process will vary from analysis to analysis. 6.1 Preparing for the Analysis The airport staff should begin the analysis by reviewing the decisions made at the start of the planning process (identified in Chapter 2). If needed, the airport can revise these elements of the analysis based on the information gathered during the evaluation of benefits and costs (Chapters 4 and 5). These decisions include the definition and descriptions of alternatives and the term of the analysis. In addition, the airport should continue to seek input from staff with the skills needed to evaluate the alternatives and results of the analysis. The airport staff will also need to determine whether the analysis will use real or nominal dollars and whether cost escalation will be applicable. To ensure consistent comparisons of benefits and costs over time, the analysis needs to account for inflation and escalation. While the analysis can use nominal prices (i.e., increasing prices for inflation over time and using a nominal discount rate), it is usually easiest to use real prices and discount rates. For the purpose of these BCA analyses, it is suggested that the airport use real prices and not apply an infla­ tion rate for future benefits and costs. The only time inflation should be applied is to inflate C H A P T E R 6 Compare Benefits and Costs

Compare Benefits and Costs 47 past benefits and costs to current dollar values. The airport staff should determine whether any benefits or costs are expected to increase by more than inflation. If so, the airport should identify appropriate escalation rates for those items. (For more detail on the use of interest and escalation rates, see Appendix A.) Based on the research on benefits and costs discussed in Chapters 4 and 5, the airport should have been able to divide benefits and costs into two groups: those that can be expressed in monetary terms and those that cannot. Benefits and costs that can be expressed in monetary terms can be evaluated using quantitative measures (as described in the next section). Benefits and costs that are difficult or infeasible to monetize can be considered in the qualitative or non­ monetary quantitative analysis. Figure 7. Flowchart for the process of comparing benefits and costs.

48 Benefit–Cost Analyses Guidebook for Airport Stormwater 6.2 Measures of Value 6.2.1 Quantitative Monetary Metrics Once an airport has identified which benefits and costs can be expressed in monetary terms, it is ready to develop the quantitative metrics that are most effective in helping it evaluate its project(s). While it is often preferable to select the project with the highest NPV, airports may find that using a suite of other metrics will provide them with a greater depth of understand­ ing and equip them with effective communication tools when justifying their decisions. (See Chapter 7 for a more detailed discussion on how to communicate the results of a BCA.) This section will discuss the basics of each metric and compare the advantages and drawbacks of each (summarized in Table 17). A more detailed description of these metrics as well as their formulas are included in Appendix A. 6.2.1.1 Net Present Value The NPV is equal to the present value (PV) of benefits minus the PV of costs. Costs are incurred and benefits accrue over the life of a stormwater infrastructure project. Consequently, a BCA must account for the concept that money in hand today is worth more than if that same Metric Definition Use Limitations NPV Overall value of the project (present value of benefits minus present value of costs) Primary measure used to understand the overall value of the project • Does not take into account feasibility limitations such as funding up-front costs B/C Overall return on investment (present value of benefits divided by present value of costs) To understand the monetary return on each dollar invested in the project • Does not indicate the magnitude/scale in overall value of the project Internal rate of return The discount rate at which the present value of benefits equals the present value of costs To understand profitability or yield of a project • Does not indicate the magnitude/scale in overall value of the project Payback period The period required for the airport to recoup the funds invested in the project To understand timing of benefits and how far into the future they accrue • Does not use discounted benefits and costs • Does not demonstrate the overall value of the project • Does not account for opportunity costs Cost-effectiveness Cost per unit of measured outcomes To understand how efficiently the project achieves goals, targets, or priorities defined by the airport • Does not communicate the overall effectiveness of the project in terms of meeting the goal or target • Does not demonstrate the overall value of the project Table 17. Summary of quantitative metrics to represent value.

Compare Benefits and Costs 49 amount of money were to be received in the future (the time value of money). Adjusting the future values of benefits and costs to PVs is referred to as “discounting,” and the rate used to discount future values is the discount rate. Representing benefits and costs that are anticipated over the life of the project in terms of PV enables them to be compared against each other. Airports should choose a discount rate that reflects the investment philosophy of the owners. For example, a private, for­profit airport may use the weighted average of its cost of capital (i.e., the rate it pays to fund investment projects) to reflect the expected return to investors. The lower the weighted average cost of capital, the cheaper it is to fund a new project. For more information on discount rates and how to choose an appropriate rate, see Appendix A. The NPV is the best metric for evaluating the overall value of a project. A positive NPV indi­ cates that the benefits are greater than the costs and that the project is worth pursuing. In tra­ ditional BCA calculations, a positive NPV indicates that the project will be profitable, and a negative NPV indicates it will not. However, when using BCA to compare two alternatives, it is the relative magnitude that matters most, not simply whether the NPV is positive or negative. If a stormwater improvement project is required by government regulation, then there is no need to consider the option of doing nothing. In this case, two alternative project designs can be compared using a BCA. Both scenarios may have negative NPVs; in this case, the smaller nega­ tive number is associated with the more cost­effective project. NPV is a useful metric for communicating the scale of the project’s value. Regardless of the scale of costs, the project with the higher NPV should be selected. For example, Project A in Table 18 has low costs and a NPV of $100,000, but Project B has high costs and a NPV of $500,000. Although Project B would be more costly to implement, its overall value would be higher than that of Project A, and Project B would be the preferred option from a value perspective. Although Project B has a higher NPV, the NPV does not indicate the scale of the up­front costs, which may be a key factor for airports depending on their access to capital. Because of this limitation of NPV, an airport must examine costs in addition to the NPV to determine whether the project is financially feasible. If a project has a positive NPV but up­front costs are too high for the airport, the airport could explore financing options that enable it to reduce its up­front costs. 6.2.1.2 Benefit–Cost Ratio As its name suggests, the B/C is equal to the present value of benefits divided by the pres­ ent value of costs over the life of the project. This metric demonstrates the project’s return on investment. A B/C of 1.2 would mean that for every dollar the airport spends on the project, it will receive a benefit equal to $1.20. The higher the B/C, the better the return on investment. Although this metric is useful in communicating the effectiveness of each invested dollar in providing a return to the airport, it does not account for the scale of the overall project value. For example, the BCA examples in Table 19 show that Project A will produce twice the return on investment as Project B. However, Project B would return a NPV five times larger than that of Project A. Project PV Costs PV Benefits NPV A $1,000,000 $1,100,000 $100,000 B $10,000,000 $10,500,000 $500,000 Table 18. NPV project examples.

50 Benefit–Cost Analyses Guidebook for Airport Stormwater This example shows that an airport should not rely solely on the results of the B/C but should use it to better understand each project. The B/C is a good measure of the return on the investment—it shows how much airports can get for each dollar invested. But it can be mis­ leading. Here, Project B has the highest NPV and is the best option, even though it does not have the highest B/C. The issue may be one of scale: if the size of Project A could be increased to $10 million, its NPV might exceed that of Project B. 6.2.1.3 Internal Rate of Return The IRR is the discount rate that sets the NPV of all cash flows over the life of the project equal to zero. In other words, it is the discount rate at which the PV of benefits equals the PV of costs. The IRR is a good measure of profitability or yield of a project or investment, but unlike NPV, it does not indicate the magnitude of value of a project. A high IRR indicates that the benefits far outweigh the costs and corresponds to a positive NPV, but it does not mean that the NPV is high in magnitude. In addition to the NPV, an airport can use the IRR to identify the project with the highest profitability. Typically, a project would be acceptable if its IRR were greater than the airport’s minimum acceptable rate of return, which may be equivalent to the discount rate or the airport’s cost of capital. However, the airport should still rely on the NPV to compare the overall value of the project alternatives. 6.2.1.4 Payback Period The payback period is the period required for the airport to recoup the funds invested in the project. This metric does not take into account the time value of money and, therefore, does not use discounted benefits and costs in its calculations. Consequently, the payback period has significant limitations and does not demonstrate the overall value of the project(s) or the opportunity cost of not investing in an alternative project. Nevertheless, the payback period can be useful to airports in understanding the timing of benefits over the life of the project. If an airport is held to payback requirements, this metric may be necessary to understand which projects meet those requirements. If the project with the largest NPV does not have an adequate payback period, the airport should examine financing options that delay costs and perhaps move up the payback period. In stormwater infrastructure projects, it is likely that the payback period will be far into the future. As a result, it is important that airports consider the full life of the project when develop­ ing these metrics. An analysis that cuts the project timeline short may omit significant long­term benefits and result in decisions that deny the airport and the larger community benefits down the line. 6.2.1.5 Cost-Effectiveness The term “cost­effectiveness” can be used broadly to describe a number of metrics, includ­ ing those described previously. However, airports can also use measures of cost­effectiveness Project PV Costs PV Benefits NPV B/C A $1,000,000 $1,100,000 $100,000 1.10 B $10,000,000 $10,500,000 $500,000 1.05 C $8,000,000 $8,480,000 $480,000 1.06 Table 19. BCA project examples.

Compare Benefits and Costs 51 to determine how efficiently a project addresses a specific concern or outcome identified as important to the airport. In this case, cost­effectiveness would be measured as the cost per unit of some outcome, as defined by the airport. For example, an airport implementing a stormwater project may want to understand how efficient the project is at: • Reducing the volume of contaminated runoff into the receiving water (cost per million gallons of water), • Removing specific contaminants of concern from discharged water (e.g., cost to reduce the concentration of a contaminant in runoff to a desired threshold for event mean concentration), • Capturing runoff water for reuse (cost per million gallons of water), and • Improving energy efficiency (cost per kW­h saved). These metrics can be useful to the airport in understanding or communicating how effective a project is at achieving a goal or target. However, this metric has limitations and should not be used in isolation when evaluating project alternatives. For example, cost­effectiveness does not communicate the magnitude of a project’s benefits. Cost­effectiveness measures should always be examined alongside a project’s overall ability to meet a target or goal. A project could be cost­effective at removing contaminants at first. However, as more and more contaminants are removed, the process could become costlier. As a result, a project that removes more contami­ nants overall may be less cost­effective than a project that removes fewer contaminants. Additionally, cost­effectiveness offers no insight into the overall value of a project. A project may result in benefits that are unrelated to the airport’s explicit goals or targets, or the cost of the project may exceed the benefit of meeting the goals or targets. In both of these cases, cost­ effectiveness may be unrelated to, or even contradictory to, the final NPV. 6.2.2 Qualitative and Nonmonetary Quantitative Measures The quantitative measures described previously are typically most influential when evaluat­ ing one or more projects because they are based on monetary benefits and costs, which are easy to understand and of high importance to the airport. However, as previously discussed, infra­ structure stormwater projects often result in moderate to substantial environmental and social benefits or costs. Often, these benefits and costs are qualitative and cannot easily be converted to dollars without cost­ or time­prohibitive research and analysis. However, major environmental or social benefits and costs cannot be ignored when trying to determine the best outcome for significant infrastructure investments. For these benefits and costs, the interests or concerns that are to be incorporated into the BCA can be evaluated using qualitative assessments. The depth of these analyses will depend on the airport’s available resources and the phase of the project. Results of a qualitative or nonmonetary quantitative analysis should be used to supplement the results of the primary quantitative analysis. They may help sway the decision when evaluat­ ing two projects that have similar quantitative results, or they can simply be used to communi­ cate additional benefits that can be achieved by the selected project. 6.2.2.1 “Stoplight” Evaluation For a high­level qualitative analysis, a “stoplight” evaluation can be sufficient. This type of qualitative analysis would determine whether the project has a positive or negative effect on elements being analyzed. The results can be demonstrated using a color­coded scale of negative (red), neutral (yellow), or positive (green), resembling a stoplight. A stoplight or similar approach will be most useful to an airport when: • Not enough data or information are available about the qualitative elements to support a more detailed analysis,

52 Benefit–Cost Analyses Guidebook for Airport Stormwater • Airport staff are conducting the analysis early in the BCA process and potential projects are still being developed or selected, or • The qualitative elements in the analysis are of relatively low priority to the airport in the decision­making process, and a high­level assessment is sufficient. This analysis would require a reasonable assessment of the effects of each qualitative element. For example, a stormwater infrastructure project could have a positive effect on water quality if the project involves the capture and treatment of water before discharge to the receiving water. Conversely, the project could have a negative effect on water quality if the project results in the discharge of pollutants to the receiving water. The airport staff would need to identify the likely outcomes for each element. If airport staff do not have enough information to determine the outcome, even at a high level, then they can conduct additional research or represent that ele­ ment as an unknown. 6.2.2.2 Scaled Evaluation For a more extensive analysis of nonmonetary elements, the airport can use a scaled approach in which important benefits or costs are scaled in their native units or, if necessary, placed on a relative scale based on professional experience. For example, the native units for the benefit of capture and reuse may be gallons of reused water that will be used for irrigation. If only gen­ eral information is available, it may be most appropriate to use a scale for approximation. For example, a BMP known to have high public visibility and popularity may be assigned a “10” on a scale of 1 to 10 for public education, and another BMP known to have a more moderate vis­ ibility may be assigned a “5.” The outcomes from project alternatives can be compared using this scaled approach to find meaningful differences in the holistic performance of each project. Although the units in this approach cannot be compared directly to dollars, the relative performance of each project on each element can be represented. It is possible to compare monetary and nonmonetary quanti­ tative elements in an analysis by normalizing monetary estimates on the same scale used for the nonmonetary elements. Keep in mind that this approach can involve values with a range of precision, subjectivity, and uncertainty. For example, anticipated BMP hydrologic performance based on engineers’ estimates for the site (e.g., runoff volumes, estimated capacity) would likely have less error than general literature information. Thus, quantitative but nonmonetary values may still have con­ siderable uncertainty depending on the information available. However, they may be less sub­ jective than those that must be initially represented by just a relative scale. (See Section 6.3 on addressing risk and uncertainty.) To begin a scaled evaluation, the team leading the effort must first determine which environ­ mental or social criteria are appropriate to scale and compare and which are not. It is best prac­ tice to document assumptions or understandings in case assumptions change. Next, a simple step­by­step process can be employed to assign weights and scores, as described in Figure 8. 6.2.3 Evaluating Results Using the TBL When conducting the quantitative and qualitative analyses, the airport should develop metrics and measures for each account of the TBL to understand how the benefits and costs break down across the three accounts. While the overall NPV may be positive for a project, the airport may find that the costs are borne primarily by one account, whereas the benefits accrue largely under a different account. For example, it is common for the airport’s financial account to bear the majority of the costs of the project due to the high costs of construction and long­term O&M. Conversely, the benefits of a stormwater infrastructure project might accrue

Compare Benefits and Costs 53 Analyze each project alternative based on the cost or benefit of interest and identify the native units of that interest. Normalize units to base 10 relative to each other. (Any scale can be used, as long as it is consistent throughout the analysis.) If there is an obvious perfection state, set that to a 10. For example, if the airport has eight sustainability goals, then a project that meets all eight goals would receive a score of 10. If there is no clear perfection state, then assign the largest benefit a value of 10 and prorate the units for the other project alternatives. For example, if the reuse project with the largest potential results in the capture and reuse of 7 million gallons, then this would be given a score of 10. Where data are not available or will take too much time to obtain, a subject-matter expert can help the airport score that element for the project alternatives on a 1 to 10 scale, with 10 being best. Step 1. Normalize (score) each benefit or cost of interest to base-10 units. Step 2. Seek input on priorities and elements being scored. Convene a meeting with the project managers or elected officials who will make the final decision on the project, lay out all the project elements that will be evaluated qualitatively, and work together to determine the airport’s priorities. No scaled values should be shared at this point. In a blind-to-each-other survey, ask the decision makers to determine the relative importance of each element (i.e., the weight of each element). This can be represented as a percentage of the total decision that each element should represent. With a percentage for each element from each person, the percentages will need to be weighted to develop one set of percentages (one percentage per qualitative element) that add up to 100% for each project. This can be achieved by averaging the percentages for each element. For example, if five people rated one element at 10%, 15%, 25%, 10%, and 12%, the average for that element would be 14.4%. Once this has been completed for all elements, it may be necessary to round up or down to ensure that all weightings add up to 100% total. Caution: The airport is advised against discussing recalibrating the weightings that have already been averaged. This may result in stronger voices adjusting the weightings to their beliefs and will discount the input of others involved in the process. This is a common decision-making trap in groups, where politeness to one person’s requests allows for the nonrepresentational weightings of certain aspects, resulting in the subversion of the multiple understandings of the leadership. Once the scores (step 1) and weights (step 3) have been established, weighting the qualitative elements is a straightforward process of multiplying the scaled score for each element by its weight. Once this step is complete, the airport will have a weighted score for each qualitative element for each project alternative. For each project, add the weighted qualitative scores for a single total score. The range is from 1 to 10, with 10 being best. Step 3. Develop weights for each qualitative element. Step 4. Weight the qualitative elements. Step 5. Calculate the final combined qualitative score for each project. Figure 8. Step-by-step process to assign weights and scores. (continued on next page)

54 Benefit–Cost Analyses Guidebook for Airport Stormwater primarily to the social or environmental account. This is likely to be the case if the stormwater project results in significant benefits to water quality beyond those required by the airport’s permit. In this case, the water quality benefits may not be realized directly by the airport, and the NPV on the airport’s financial account of the TBL could be negative even though the overall NPV on the project is positive. Presenting the quantitative and qualitative measures according to the accounts of the TBL is a fundamental benefit of using the TBL approach in a BCA. This approach demonstrates the effect of the project on the environment and larger community rather than focusing on the airport’s financials. This approach will also capture the nuances of the outcome of the project. Different measures of value may be important for different accounts of the TBL. For example, the payback period or the internal rate of return may be important for the airport’s financial account, but cost­effectiveness may be a more important measure for the environmental or social accounts. Finally, representing measures of value for each account of the TBL can help the airport build a business case for the preferred project. When seeking internal approval from decision makers, the airport can use these results to demonstrate how the project will directly affect the airport’s financials and how the project will change or maintain the airport’s role in the community. Airport staff can also use these results when communicating externally to demonstrate how the project will affect the larger community or demonstrate that the airport has considered the effect of the project on the environment and the community. To the extent that the airport is able to demonstrate favorable outcomes for the environment and community, it may be able to garner more support from the community or good will for the project. 6.3 Addressing Risk and Uncertainty Every BCA must rely on a few important assumptions. For example, a project that is expected to reduce stormwater runoff may use predicted rainfall levels based on historical averages. Variations in average annual rainfall likely will affect the actual TBL. Uncertainty about other Review the projects to see if any projects have final qualitative scores that are too close to be considered significantly different (e.g., a score of 5.8 is not significantly different from a score of 5.9). The airport staff can review the element-level weighted scores as a sensitivity analysis to determine if one aspect of the decision is disprop ortionately driving the outcome due to a heavy weighting on that element. Step 6. Review the results. Step 7. Determine if the analysis is highly sensitive to any single qualitative element. If one qualitative element swings the scores dramatically, the airport should review the alternatives to determine if that element could be addressed through design choices or another mitigation strategy. If so, rescore the alternatives with that change made. Perform this task as often as needed to see clear differences in alternatives. With the analysis complete, the airport is ready to share the scores with both the technical team and the decision makers. These results can be used to support the project selection and as a record of how the environmental and social aspects were considered. Step 8. Communicate the final results. Figure 8. (Continued).

Compare Benefits and Costs 55 important factors, such as the cost of capital, the useful life of the project, and the discount rate, will affect the estimates. The results of a BCA will depend on assumptions and uncertain­ ties. Placing too much confidence in rough estimates and assumptions can lead to incorrect or misleading results. Airports can address risk and uncertainty in BCAs by performing a series of steps that will help them define where assumptions have been made and identify how those assumptions affect the outcome of the BCA. This process should involve staff (such as facility managers, sustainability staff, and financial staff) with expertise in the areas where assumptions may have been made in the analysis. 6.3.1 Identify Elements of Risk and Uncertainty in the Analysis The first step is to identify which elements in the analysis are subject to risk and uncertainty. Risk is the possibility of an event occurring that results in loss and typically describes events of known probability. For example, airports can estimate what proportion of storm events will exceed their stormwater system’s capacity. In contrast, uncertainty is the chance of something occurring, but the probability of its occurrence is unknown. There can also be uncertainty around the magnitude of a risk event or around performance of the stormwater system. For example, a BMP type may be known to remove sediment, but the exact percentage removal is subject to some error and may also vary from storm to storm. Also, storm events vary in magni­ tude, intensity, duration, and frequency, introducing error into relevant measures. Labor rates for construction vary regionally and possibly seasonally, introducing a potential uncertainty into initial construction estimates. Airports should carefully review the inputs in a BCA to identify where assumptions have been made and where risk and uncertainty may affect the results. A complete list of risks and uncertainties should be compiled at this stage of the analysis. 6.3.2 Characterize Risks and Uncertainties in the Analysis After a complete list of risks and uncertainties has been created, the airport should evaluate and characterize each element on the list. Typically, each risk or uncertainty must be defined in terms of the likelihood of an event or the error inherent in certain measures. Airports can use a number of approaches to define these risks and uncertainties: • Use insurance underwriters to determine policy costs and adjustments as indicators of risk likelihood and intensity. This information is most likely to be available for risks and uncertainties that are common in the industry but may not be available for those that are not commonly insured (such as the effects of climate change). For example, airports may have good estimates of the risks of and costs associated with major storm events from past years; estimates of the effects of more frequent, higher­intensity storms and sea level rise may be less certain. • Use historic information to estimate possible outcomes. For example, collecting data on historic airport water use may help the airport define reasonable estimates of future demand for water in capture and reuse systems. • Identify low-, medium-, and high-potential outcomes. This may be most useful in cases where historic or industry data are not available or are unlikely to reflect future outcomes. For example, the future intensity of rainfall events could affect the performance of storm­ water infrastructure, but data predicting future intensity or frequency of rainfall events may be limited. In this case, using low­, medium­, and high­potential outcomes could be an appropriate approach to characterizing this uncertainty. • Use experience from prior construction projects to develop estimates of the likely cost of construction, operations, and maintenance of stormwater projects. Past projects can provide information about the potential range of costs (e.g., the factors that can delay the design and construction phase of the project, increasing the project’s costs).

56 Benefit–Cost Analyses Guidebook for Airport Stormwater • Use prior projects and design studies for information about stormwater project perfor- mance. For example, prior projects and engineering practices provide information about aspects of stormwater system design such as capacity and estimated pollutant removal capa­ bilities, as well as their associated uncertainties. These uncertainties in turn affect the likeli­ hood of negative environmental effects. This process is important to the overall BCA because it forces the airport to think critically about all assumptions and protects the airport from placing too much emphasis on rough estimates. 6.3.3 Conduct Analyses that Demonstrate the Effects of Risks and Uncertainties on the Outcome of the BCA Once the risks and uncertainties have been identified and characterized, the airport is ready to conduct a sensitivity analysis demonstrating the effects of those risks and uncertainties on the outcome of the BCA. A sensitivity analysis (an analysis in which key assumptions and compo­ nents of an analysis or simulation are systematically varied to determine how they affect the out­ come) can be used to identify the effect of key assumptions on the TBL. There are three primary methods airports can use in a sensitivity analysis: probabilistic evaluation, high–medium–low estimates, and conducting simulation­based analyses such as Monte Carlo simulations. To address elements of risk, the analysis can use a probabilistic approach to estimate the likely costs or benefits associated with particular elements. This approach is most useful when the probability of an event occurring is known and the magnitude of the cost or benefit is well defined. For example, if there is a 20% chance that a $1 million cost will be incurred, then the analysis can incorporate that cost as $200,000 (i.e., 20% of $1 million). This is defined as the “expected value” of that element and accounts for the risk in the BCA; no additional iterations of the analysis are needed. It often is referred to as the “certainty equivalent” amount. However, if there is any uncertainty associated with the probability or the magnitude of the outcome, the airport should conduct additional sensitivity analyses, such as those described in the following. In many cases, the likelihood or magnitude of the cost or benefit is unknown, especially when airports are exploring innovative stormwater projects. In these cases, a range of values can be used to understand the potential effect of that element on the final analysis. A fairly simple and straightforward version of this approach is to simply run the BCA with a small range of outcomes for each risk or uncertainty. The benefit of this approach is that it is a simple and low­effort way to understand how various assumptions affect the TBL and the range of potential outcomes. This approach can also help the airport identify the assumptions that have the largest effect on the outcome, and, if necessary, the airport may choose to investigate those assumptions in greater detail. This approach may require an extensive analysis when multiple uncertainties are at play. For example, if an airport develops three possible scenarios for five different uncertainties, there are 243 possible combinations of outcomes. The airport staff can reduce the number of possible combinations by eliminating combinations that are not possible or are highly unlikely. When determining which scenarios to evaluate, an airport may find it helpful to seek input from internal staff, other airports, or consulting services with knowledge of the likelihood of some options or risks associated with new or innovative projects (such as GSI). When running such an analysis, an airport may consider the following outcomes to evaluate the results: • The number of combinations that produce a positive NPV, • Combinations that change the ranking of project alternatives (in terms of NPV), and • The change in magnitude of NPV.

Compare Benefits and Costs 57 In a more detailed and exhaustive sensitivity analysis, an airport can use statistical analysis, such as a Monte Carlo simulation. Monte Carlo simulations run large numbers of scenar­ ios for multiple variables, typically producing hundreds of thousands of potential outcomes. This analysis can help an airport identify the probability of different outcomes occurring. For example, an airport could use a Monte Carlo simulation to identify the probability of a proj­ ect having a positive NPV over a range of assumptions about critical inputs. A Monte Carlo simulation relies on assumptions about the possible values these inputs can take. For example, the distribution and mean of potential input values must be defined. Using this information, a Monte Carlo simulation can provide the likelihood of each potential outcome of the BCA. In contrast, using the high–medium–low approach only produces a range of outcomes and does not provide insight into the likelihood of those outcomes occurring. While Monte Carlo simulations can be useful and informative, these analyses require sophis­ ticated data and computational software or the resources to hire consultation services. Most airports will be best informed through the simpler analyses previously described. 6.3.4 Identify Risk Mitigation Options Finally, after understanding the effect of risk and uncertainty on the outcome of the BCA, the airport should determine whether any mitigation options are available to lessen the probability of a cost occurring or reduce the magnitude of that cost should it occur. For this analysis, the airport should consider the following questions: • What mitigation options are available for the project’s risks or uncertainties? • How much do these mitigation steps cost? • How effectively would they reduce costs on the project? • Do they improve the results of the BCA enough to make a difference in the airport’s selection? 6.4 Bayside Airport’s BCA The BAY BCA team completed its research on benefits and costs and began preparing the comparison analysis. By this point, BAY’s team had made most of the difficult and time­ consuming decisions on the analysis—monetization of benefits and costs is the most nuanced stage of the BCA and has the potential to have the greatest effect on the outcome of the analysis. With the best estimate of these monetary values over the life of the project, the team was ready to complete the analysis using simple calculations. The team used the following basic assumptions: • The discount rate is equal to 3%, which is equivalent to the airport’s real cost of capital, • Planning and construction would take 2 years, and • The term of the analysis is 50 years (equivalent to the expected life of the infrastructure) fol­ lowing completion of construction. BAY’s team began by organizing the monetary values according to when they would be incurred. Tables 20 and 21 summarize the breakdown of these values for each option. Note that O&M costs anticipated to occur periodically (e.g., vegetation replacement) are included in the annual costs with the understanding that actual O&M costs will vary from year to year and will include these activities. This information helped BAY’s team apply the correct discounting to each cost and benefit and identify the final NPV. For example, up­front costs were divided between years 1 and 2 and discounted, annual benefits and costs were discounted as a stream of costs or benefits over

58 Benefit–Cost Analyses Guidebook for Airport Stormwater 50 years, and periodic benefits and costs were discounted the appropri­ ate number of years for each occurrence. Tables 22 and 23 demonstrate the analysis team’s calculation pro­ cess. They show the up­front and O&M costs associated with the dry detention basin, which is a component of Option 1. Table 24 shows the results of calculation of avoided fines for the underground harvesting and reuse system proposed as part of Options 2 and 3. Table 25 shows the total NPVs for each of the three full project options. (In these tables, the components may not sum to the total due to rounding.) Because the stream of O&M costs for the dry detention basin will not begin until after construction, BAY’s team further discounted the PV stream of costs for two additional years. The team used this approach for all annual benefits and costs. After computing similar calculations for all benefits and costs asso­ ciated with the project options, BAY’s team summed all the PVs and obtained the results presented in Table 25. The NPV is highest for Option 3, which is also the only option with a positive NPV in the air­ port’s financial account. Option 1 has the lowest overall costs but the smallest benefits. As a result, the NPV of Option 1 is negative. Table 25 also demonstrates how the benefits and costs break down across the three accounts of the TBL. Under each option, the majority of benefits and costs are under the financial account. Because BAY would fund, operate, and maintain these projects, it is expected that most or all of the costs hit the financial account of the TBL. Furthermore, the Outcome One Time Annual Periodic Planning, permitting, construction Up front O&M Decommissioning Endof life Education campaign Up front Table 20. Breakdown of costs. Is This Confusing? Here Are Two Tips • If the discount rate is 0.03, why doesn’t the discount factor equal 0.97? Because discounting compounds over time. Discounting is achieved by using the formula 1/(1 + r)t, where r = discount rate and t = year of the cost. (See Appendix A for more details.) A cost incurred in year 5 must be discounted to year 4, which then must be discounted to year 3, and so forth. • Why is the resulting discounted cost so precise? In reality, it need not be. The final PV results should be rounded to an appropriate level of precision, similar to the level of precision of the cost inputs. The calculations in Tables 22 and 23 present precise results to show how the calculations are made. The analysis for BAY rounds the final NPV results. Outcome One Time Annual Periodic Frees up land area for other uses Year 10 Avoided cost – stormwater conveyance construction Up front Avoided cost – permit delays Up front Avoided cost – annual stormwater program Avoided cost – reduction in potable water use Avoided cost – lost recreation Avoided cost – compliance fines Every 10 years Avoided cost – beach repair Every 10 years Table 21. Breakdown of benefits.

Compare Benefits and Costs 59 Option Financial Environmental Social Total NPV Costs 1 -$2.69 $0 $0 -$2.69 2 -$8.6 $0 $0 -$8.60 3 -$8.61 $0 $0 -$8.61 Benefits 1 $0.22 $0 $0.02 $0.24 2 $8.07 $0 $5.16 $13.23 3 $10.2 $0 $5.15 $15.35 Net 1 -$2.47 $0 $0.02 -$2.45 2 -$0.53 $0 $5.16 $4.63 3 $1.59 $0 $5.15 $6.74 Table 25. NPVs for each option (millions of dollars). Year Cost Discount Factor (using 3% discount rate) Discounted (PV) Cost 1 $195,000 0.970874 $189,320 2 $195,000 0.942596 $183,806 Total PV $373,127 Note: The formula for the discount factor is 1/(1 + 0.03)t, where t = year cost is incurred. Table 22. Up-front cost of construction for the dry detention basin. Number of Years Annual Cost 50-Year PV Cost First Year of Cost 50-Year PV Cost (discounted for beginning in year 3) 50 $12,300 $316,476 3 $298,309 Note: 50-year PV cost is calculated for each year using the formula ($12,300)/(1 + 0.03)t, where t = year cost is incurred. The combined 50-year PV cost is then discounted an additional 2 years using the formula ($316,476)/(1 + 0.03)2. Table 23. Annual O&M costs for the dry detention basin. Benefit Incurred in Year # Discount Factor Discounted (PV) Cost $125,000 7 0.813092 $101,636 $125,000 17 0.605016 $75,627 $125,000 27 0.450189 $56,274 $125,000 37 0.334983 $41,873 $125,000 47 0.249259 $31,157 Total PV $306,567 Note: The formula for the discount factor is 1/(1 + 0.03)t, where t = year cost is incurred. Table 24. Periodic avoided cost of compliance fines for the subsurface detention system with vaults and reuse.

60 Benefit–Cost Analyses Guidebook for Airport Stormwater analysis team represented many social costs for Option 1 as benefits in the form of avoided costs under Options 2 and 3. BAY will internalize some of the benefits associated with Options 2 and 3 (in the form of permit noncompliance fines and high water rates), but the NPV in the financial account is still negative for all three options. The additional avoided costs associated with city­funded beach repairs and nonmarket value of lost recreation attributable to the social account causes Option 3 to have a positive NPV. In addition to the NPV, BAY’s team calculated the B/C and the cost­effectiveness of metal removal from stormwater. The B/C would be a metric needed by BAY’s executive board to evaluate the return on investment for this project. Heavy metal contamination in the bay was an issue highlighted by stakeholders early in the planning phase of the analysis. The team estimated the cost­effectiveness of each option in removing heavy metals from runoff (repre­ sented as the cost per percentage point removal of the contaminant). This demonstrated to the executive board the value of each project option for managing heavy metal pollution in runoff. The results of these analyses are presented in Table 26, which demonstrates that Option 3 has the highest B/C and achieves the highest level of metal removal. Option 1 appears to be more cost­effective at removing metals, but BAY’s engineers noted that the metals removed in Option 1 would be primarily sediment­associated, while Option 3 would remove both dis­ solved and sediment­associated metals, providing greater environmental benefit. The analysis team conducted two evaluations for outcomes that were not assigned mon­ etary values. The first analysis used the stoplight approach to demonstrate whether each project option would have a negative (red), neutral (orange), or positive (green) effect on the outcomes identified by the team. The results of the analysis (Figure 9) demonstrate that Option 3 results in the most positive outcomes, and Option 1 results in mostly neutral or negative outcomes. BAY will use this evaluation to supplement the monetary results. The second evaluation was a scored analysis, as described in Steps 1 through 8 in Figure 8. The team determined that it had enough information to conduct a scored analysis for three outcomes: pollutant removal, sustainability goals, and education and outreach. The team began by rating how well each benefit ranked on a score of 1 to 100 (Step 1). The team estimated the percentage of pollution reduction by each project component based on five pollutants of concern [total suspended solids (TSS), nutrients, pathogens, metals, and organics]. The highest possible pollutant reduction was set as the goal. For example, the bio­ retention cells were expected to achieve 85% reduction of TSS, and the detention basin was expected to achieve 50% reduction in TSS. Therefore, the bioretention cells achieved a score of 85/85 = 100, and the detention basin achieved a score of 50/85 = 59. This process was completed for all pollutants in all project components. The team then calculated a score for each option by adding the scores for each contaminant in each project option and averaging them. The team identified the airport’s sustainability goals and evaluated how many of the goals would be achieved under each project option. Options 2 and 3 would meet three out of four Option Benefit (millions) Cost (millions) B/C Percentage of Metal Removal Cost-Effectiveness (cost per percentage point removal) 1 $2.69 $0.24 0.09 33% $82,615 2 $8.60 $13.23 1.54 63% $137,552 3 $8.61 $15.35 1.78 90% $95,700 Table 26. Cost-effectiveness of each option in removing metals.

Compare Benefits and Costs 61 of the airport’s main sustainability goals, while Option 1 did not meet any. Consequently, Option 1 received a score of 0/4 = 0, and Options 2 and 3 received a score of 3/4 = 0.75. (This was multiplied by 100 to adjust it to a scale of 0 through 100.) The team developed quantitative scores for education and outreach by estimating what por­ tion of customers would be informed by educational materials such as placards. The airport would develop educational materials under Options 2 and 3 for the biofiltration (which applies to Options 2 and 3) and water reuse (which applies only to Option 3) components of a project. Based on the planned location of the materials, the team assumed that 30% of passengers would see educational materials posted. The team assumed that the materials for Option 3 would reach the same number of people as for Option 2 but would achieve a 50% increase in educa­ tional value because of the reuse element under Option 3. Therefore, the score for Option 2 is (30% × 100) = 30, and for Option 3 the score is (30% × 100 × 1.5) = 45. The results of the scored analysis for pollution removal, sustainability goals, and education and outreach are summarized in Table 27. The airport’s executive board discussed the airport’s priorities for each of the three compo­ nents (Step 2) and proceeded to develop weights reflecting the importance of each of the three outcomes (Step 3). The team leader collected these weights and averaged them to calculate a final weight for each of the three benefits. The final weights were multiplied by each option’s rating to calculate the final score on each option (Step 4) and added together to develop a single final TBL Account Outcome Option 1 Option 2 Option 3 Financial Meets drainage requirements Financial Protects against small rise in sea level Environmental Pollutant removal prior to discharge Environmental Meeting airport sustainability goals Environmental Use of native vegetation Social Education and Outreach Social Aesthetics Social Positive community relations Social Advancing permitting procedures for GI Social Supports city’s focus on use of LID Notes: Darkest circle is red, lightest circles are yellow, and the rest are green. See online version of report for color graphic. Figure 9. Stoplight analysis of each option. Benefit Basis for Rating Option 1 Option 2 Option 3 Pollutant removal prior to discharge Degree to which pollutants are removed, using Option 3 as the best-case-scenario 34 67 100 Meeting airport sustainability goals The percentage of sustainability goals each option meets (out of 4) 0 75 75 Education and outreach The estimated percentage of passengers that would see education material and the extent of information that would be communicated 0 30 45 Table 27. Scored analysis.

62 Benefit–Cost Analyses Guidebook for Airport Stormwater score for each option (Step 5). Table 28 shows the outcome of this process. Option 3 returned the highest score, and Option 1 returned the lowest. BAY’s analysis team is now almost done with the BCA. The final step is to conduct a sensitivity analysis to understand how assumptions affect the outcome of each option. The analysis team identified three elements with the most uncertainty and the largest potential effect on the NPV: • Up­front costs associated with the subsurface detention system with reuse (Options 2 and 3). • Benefits of improved water quality in the bay (as measured by nonmarket value of recreation at the beach). • The degree to which Options 2 and 3 result in expedited permit approval process—the sooner the permit is approved, the higher the cost savings will be for the airport. The team estimated likely scenarios for each uncertainty. Table 29 demonstrates each scenario tested by the team. Each scenario changes only one assumption, leaving all others equal to the original assumptions in the analysis. The team also estimated the best­ and worst­case scenarios using the assumptions in Table 29. The best­case scenario would reduce costs by 20%, increase the beach value to the midpoint estimate (the high­end estimate was ultimately deemed unrealistic), and increase the permit Benefit Rating Final Weight Score Option 1 Option 2 Option 3 Option 1 Option 2 Option 3 Pollutant removal prior to discharge 34 67 100 0.37 13 25 37 Meeting airport sustainability goals 0 75 75 0.30 0 23 23 Education and Outreach 0 30 45 0.33 0 10 15 Final Score 13 58 75 Table 28. Scored analysis with weights. Incorporating Monetary Results into Scoring An airport may find that the results of the monetary and nonmonetary analyses support the selection of different options. In this case, the airport could include a score for the results of the monetary analysis to compare the nonmonetary and monetary analyses. For example, the NPV for each option could be scored from 0 to 1 based on the options’ relative values. The airport staff would then adjust the weights for all benefits and include a weight for NPV. Using this approach, the airport can develop a final score that includes the monetary results of the analysis. Because Option 3 was the preferred option in both BAY’s monetary and nonmonetary analyses, the BAY BCA team did not incorporate NPV into the final score.

Compare Benefits and Costs 63 expediting under Option 1 from 1 year to 1.5 years. The worst­case scenario would increase costs by 20%, retain the low­end estimate of beach value (original assumption), and reduce the permit expediting process to 6 months. Figure 10 demonstrates the results, showing that Option 3 is the favorable option under both scenarios. (Note that the NPVs represented for each scenario in Table 29 do not sum to the best­ or worst­case scenarios because the scenarios in the table hold all other factors equivalent to the baseline analysis, whereas Figure 10 applies more than one change to the base assumptions.) Scenario Effect on NPV Option 1 Option 2 Option 3 Increase up-front costs by 20% Negative -$2.84 $4.09 $6.33 Reduce up-front costs by 20% Positive -$2.06 $4.91 $7.15 Increase beach value to high-end estimate Positive -$2.45 $16.46 $18.71 Increase beach value to midpoint estimate Positive -$2.45 $5.84 $8.08 Reduce permit expediting time for Option 2 and Option 3 by 6 months Negative -$2.45 $3.99 $6.12 Increase permit expediting time for Option 2 and Option 3 by 6 months Positive -$2.45 $5.26 $7.36 Table 29. Accounting for uncertainty: NPV under alternative assumptions (in millions of dollars). –$4 –$2 $0 $2 $4 $6 $8 $10 Option 1 Option 2 Option 3 M ill io ns Worst Case (left-most bars) Best Case (right-most bars) Figure 10. Best- and worst-case scenarios on the NPV.