Airport Capital Improvements: A Business Planning and Decision-Making Approach (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

44 An accurate cost estimate is recognized by practically all stakeholders as being a significant con- tributor to successful airport capital improvement planning. Access to reliable cost estimates helps ensure optimal use of limited airport investment funds and reduces the risk of project cancellations or cutbacks. At the same time, there are a number of recognized risks that affect the quality of any cost estimate, no matter how sound the underlying methodology is. These include scope changes, volatility in material costs, uncertainty in mobilization costs, environmental issues, community concerns, the inherent complexity of airport systems, contractor management issues, and poor implementation of best practices. The literature review and stakeholder survey conducted for this study describe the current practices for estimating costs for airport construction projects in both the horizontal and verti- cal domains. In general, existing practices utilize well-established and proven methodologies. The methodologies draw on procedures and guidance published by a number of entities that provide relevant resources, particularly professional organizations and state agencies. Cost estimating for vertical projects has an added layer of structure through the use of standard classification schemes. The two primary methods used for estimating airport project costs are estimation through historical bid prices and cost-based estimating. All existing methods are limited in their ability to accurately account for unique project conditions. Such uncertainties can significantly affect the estimate and can result in wide variations between initial cost assumptions and the actual costs incurred on a particular project. To account for such risks, contingency analyses are often applied, but usually in a simplified manner. A typical method is the inclusion of a percentage multiplier to line item quantities and/or an overall contingency factor that is applied to the final cost estimate. There are few, if any, standards for applying such contingency factors. The stake- holder outreach effort conducted for this project indicates that the numerical values used can vary greatly. Since overall contingency factors can be applied on top of contingencies for line item quantities, the cumulative contingency can be substantial. The lack of established standards in this area results in potentially large variations. Use of computer models for cost estimating is not currently a common practice for airport construction. It is less clear whether this is due to lack of availability of suitable models or whether the challenges in airport construction cost estimating are not easily solved through computer modeling techniques. It does, however, indicate the potential for the development of an airport- specific model, provided the challenges identified previously are carefully considered and the appropriate solutions are identified. Lessons learned through the course of this study, potential solutions to some of the challenges, and recommendations for future work are discussed in the following sections. Lessons Learned C H A P T E R 6

Lessons Learned 45 Challenges to Developing an Airport Cost-Estimating Model The literature review and industry stakeholder survey conducted as part of this study addressed existing sources of cost data. The practice of storing past bid tabulations is common and a number of agencies maintain their own cost data. Nonetheless, for the purpose of developing a compre- hensive cost model, several significant challenges related to data availability exist: • Many of the most commonly used data sources are proprietary and cannot readily be distrib- uted as part of a publicly accessible model intended for delivery through the ACRP. • Data maintained by public agencies are distributed across a range of state and regional agencies and stored in inconsistent formats. • There is no standard format for data and in many cases the data is stored in formats that are notionally electronic but essentially represent digital versions of printed documents (e.g., the PDF format). This precludes automated transfer of historical cost data into a comprehensive cost database. • Even when cost data is available, data for the key cost drivers represented by the CIVs is often not. For example, for a pavement project, the amount of asphalt or concrete required is usu- ally included, but quantified as volumes. Key cost drivers such as the pavement surface area, design aircraft MTOW, landing gear configuration, and design freezing index are usually not included. • Historical grant information often contains several projects that have been bundled together in such a way that prevents costs and CIV data to be separately identified and assigned to specific project types. The main challenge in developing an effective cost model for airport projects using paramet- ric cost-estimating methodology is in fact the availability of a sufficiently large and rich set of historical data. Assembling a cost database that is sufficiently rich in both quantity and variation across geographic locations and project types would address a number of the challenges identi- fied previously. The potential benefits of expanding the cost database are many and include the following: • Each project type is represented by a unique CER, requiring its own data set. Expanding the data collection would enable cost modeling support for additional project types. • CERs incorporate independent variables that represent cost drivers and that have a causal rela- tionship with cost. Lack of data limits the number of cost drivers that can be included, reducing the explanatory power of the CER. Variables that are not included but that affect cost result in unexplained variation and less accurate models. Expanding the number of historical observa- tions would allow the inclusion of additional CIVs in the CER, thereby improving the model’s ability to predict cost. • Linear regression is based on statistical samples, which inherently have some random variation. This random variation introduces errors in the resulting cost model. Increasing the number of observations reduces the errors due to random variation in the sampling process. • Similarly, in the case of a small sample, it is more likely that the results are biased because of lack of variation. For example, if the database is small and contains a disproportionate number of observations from a particular geographic region or type of airport, the likelihood is greater that the model will be biased due to lack of variation in the data. The database should be suf- ficiently large to ensure variation across geographic locations, urban versus rural communities, and types of airports. • The larger the database, the less likely it is that user-entered inputs will fall outside the range of the historical observations used to develop the CER in question. As described in Chapter 5, when the CIV input values fall outside the range of historical CIV values used in the cost model- ing, the cost estimate is generally more uncertain.

46 Airport Capital Improvements: A Business Planning and Decision-Making Approach Future Work As described previously, future work on the development of a cost model for capital planning purposes should first and foremost focus on expanding the database. This section includes spe- cific recommendations for future data collection practices. These are based on lessons learned during the implementation of the ACCE cost model, as well as recommendations by the research team’s airport construction SMEs. Initiating an effort to expand the data collection requires addressing a number of challenges. These include establishing a framework for collecting the data, establishing support from the airport community, obtaining necessary resources, and creating standards for collection of his- torical cost and project data. While identifying solutions to some of these challenges is beyond the scope of this study, the key issues that need to be addressed include the following: • Organization: For an expanded data collection effort to be implemented, ideally a frame- work should be established that can engage a large number of airport participants across the United States. This is necessary to ensure that the resulting database has sufficient number of observations, which is currently the biggest limitation in implementing the parametric cost- estimating method. It would also provide sufficient regional variation, preventing biases due to smaller and more narrowly focused samples. While there are a number of potential options to establish an organization framework, it is not possible to predict the exact makeup. Key stakeholders would likely include trade and industry organizations, state aviation agencies and their umbrella groups, and the Airports organization of the FAA. • Resources: The resources required for this effort would depend on the framework and imple- mentation of an expanded data collection program. The effort would require development of standards, a mechanism to collect data, and management and development of the database. A potential option for an initial effort would be a voluntary pilot project. However, a full implemen- tation of an expanded data collection effort may require identifying a source of project funding. • Data collection: Prior to initiating an expanded data collection effort, standards must be estab- lished for the type of data to be collected, including definitions for each field in the database. This is required in order to ensure that the right type of data is collected and that data from dif- ferent airports, projects, and regions shares consistent definitions. One of the lessons learned in this project is that it can be very difficult and resource intensive to retroactively fill gaps in the database. For this reason, it is important to invest sufficient resources upfront, to ensure that effective and comprehensive data standards are established. These standards should balance the need for a rich data set to support the cost model development with ease of data collection. If the data requirements are too onerous, the data collection will suffer from an insufficient number of submitted projects. It is important to keep in mind that the parametric cost-estimating tech- nique requires that each record is complete. In other words, records that are missing value for one or more data fields cannot be included in the statistical analysis used to develop the CERs. The following section includes additional detail on recommended practices for establishing the data collection framework. These recommendations are based on lessons learned during the conduct of this research project, best practices identified in the literature review and stakeholder outreach effort, and SME input. Recommendations for Data Collection Practices The most important step in ensuring a successful data collection effort is the establishment of data standards. These standards should include the following: • Specifications for general data to be collected for all projects. • Specifications for project-specific data (i.e., data that varies by project type).

Lessons Learned 47 These specifications should both identify the data fields to be collected for each project, as well as provide definitions that clearly identify the intent and meaning of each field. These definitions should be sufficiently detailed so as to ensure that data are collected consistently. As an example, consider the CIV “area” for vertical projects. The definition should specify that the combined floor area across all stories should be included. The definition should also determine whether the floor space should be measured to the exterior and interior walls and address the handling of unusable space. Finally, for each data field, the units of measurements should be specified (where applicable). General Data The requirements for collecting general data are likely to be very similar to the data collected during the course of this project. However, some added specificity and improvements are possible. Likely data fields include the following: • Record identifier: Each record in the database should be assigned a unique identifier that can be used for indexing and cross-referencing purposes. • Airport identifier: A unique airport identifier is required in order to establish the location of the project. This is necessary to adjust for regional variation and can also be used to test that the database is not biased toward a specific geographic area. It also allows for follow-up queries, for example, if the data collected for the airport contains inconsistencies or missing fields. The data requirements should specify whether the FAA or International Civil Aviation Organiza- tion identifier should be used. If the identifier is linked to an airport database, no additional geographic information needs to be collected. If this is not the case, or the airport is not in the database being used, it is recommended that one or more of the following geographic identifiers be collected: zip code, county, and/or state. • Project type: The project type allows the data to be mapped to a specific CER. While this requires that the project types be static (i.e., they must be established in advance), the research conducted during this project suggests that a relatively small number of project types account for the majority of construction projects. In this study, the number of supported project types was limited to eight. However, this was primarily the result of limited data availability. In an expanded data collection effort, it is recommended that a broader range of project types be supported. The projects originally identified as candidates for inclusion can serve as the starting point for identifying the project types to be supported in a future effort: – Airfield signage – Construct ARFF facility – Construct or rehabilitate taxiway – Construct parking garage – Construct parking lot – Construct SRE building – Construct, expand, or rehabilitate apron – Construct, expand, or rehabilitate terminal building – Construct, extend, or rehabilitate runway – Improve runway safety area – Install airport visual aid – Install NAVAIDs – Install perimeter fencing – Install weather reporting equipment – Rehabilitate runway lighting – Remove obstructions – Runway pavement marking – Security access systems

48 Airport Capital Improvements: A Business Planning and Decision-Making Approach • Project description: The project description is useful for identifying project type and, espe- cially, for determining whether the project includes bundled construction types. It appears most practical to leave the project description as a free text field. However, guidelines should be established for the level of specificity desired in the description. For example, for pave- ment projects, it should be clear whether the project consists of constructing a new pavement area, expanding an existing pavement area, or rehabilitating old pavement. The type of pave- ment used (i.e., asphalt, PCC, or a hybrid) should be specified. The description should specify whether the project includes design only, construction only, or both. A table of relevant key- words may serve as a useful guide to craft clear and comprehensive project descriptions. • Year: The year of construction is required for normalizing construction costs to take inflation into account. This is a relatively straightforward input, but the guidance should specify whether calendar or fiscal year should be used, and how to treat projects that span multiple years. Also, some thought should be given as to which is most relevant to the cost modeling—the year(s) of construction activity or the budget year(s) associated with the grant funds expended on the project. • Total project cost: Project cost is the sole dependent variable in the parametric cost method- ology presented here and is the most critical variable in the model. For this reason, particular care should be taken in both defining the meaning of total project cost and in ensuring that the data is collected according to the resulting definition. In the database created for this project, cost was unavailable for some data records and had to be estimated based on the federal share for AIP-funded projects. While the federal share is theoretically established by formula allocation, in practice, the share can vary from project to project due to items ineligible for federal funding. For this reason, estimating the total project cost based on the federal share is not ideal and is likely to introduce inaccuracies in the cost database. The guidance for collecting historical project cost data should clearly specify that total costs should be considered. This total includes the federal share, the state share, and the sponsor’s share. Moreover, guidance should specify which stage in the project the historical cost should be based on. Options range from the cost provided during the bidding phase to that provided on the project close-out report. In general, the latest available cost data is preferred. Another important aspect of providing specifications for the collection of historical costs is the treatment of soft costs. Soft costs typically range from 10% to 30% of total project costs. These include design fees, permitting fees, utilities, costs associated with inspections and land acquisition, costs associated with the bidding and procurement process, and project admin- istration and management costs. The guidance should clearly specify which costs should be included, so that the historical cost data follows a consistent pattern that allows for pooling historical observations across many projects and airports. Project-Specific Data The project-specific data is the set of historical values for the CIVs that are part of the hypoth- esized CER for the project type under consideration. Since one of the major goals of any expanded data collection effort is to improve the performance and robustness of the cost model, the number of CIVs should be expanded significantly from the final list selected for the development of ACCE. The goal should be to identify and include all major variables that are measurable and that have the potential to affect the cost of a project significantly. At the same time, since the number of data points required increases with the number of CIVs included, the guidelines should not call for the inclusion of CIVs that only have a minor impact on cost. If the number of CIVs is exces- sive, the labor effort required to collect historical project data could also increase to the point that the number of records collected is substantially reduced. It is important to keep in mind that in order for a past project to be included in the model, all fields must be complete, which means a value must be collected for each CIV included in the CER.

Lessons Learned 49 In identifying which CIVs to include, the CERs hypothesized at the beginning of this project will serve as a useful starting point. This is because the original CERs included many more CIVs than contained in the final database, since the number of CIVs was reduced substantially to deal with the lack of available data. An expanded data collection effort should allow for a number of the rejected CIVs to be included in the model as originally intended. Table 13 displays a list of proposed CIVs for potential horizontal projects and Table 14 displays a similar list for vertical projects. These lists employ up to six CIVs per project type (compared to three for the cost model implemented in ACCE). Table 13. Potential cost drivers for horizontal airport construction project. Project Category CIV 1 CIV 2 CIV 3 CIV 4 CIV 5 Airfield signage No. of intersections Airplane design group Control tower Construct or rehabilitate taxiway Area MTOW Landing gear configuration Pavement type Design freezing index value Construct parking lot No. of spaces Drainage type Construct, expand, or rehabilitate apron Area MTOW Landing gear configuration Pavement type Design freezing index value Construct, extend, or rehabilitate runway Area MTOW Landing gear configuration Pavement type Design freezing index value Install airport visual aid Type of system No. of systems/ runway ends Install NAVAIDs Type of NAVAID Install perimeter fencing Length No. of automatic gates No. of manual gates No. of pedestrian gates Install or rehabilitate runway lighting Length Runway approach type Install weather reporting equipment Type of equipment Rehabilitate runway lighting Length Runway approach type Remove on-airport obstructions (vegetation) Acres Runway pavement marking Length Runway approach type Security access systems No. of pedestrian gates No. of vehicle gates Table 14. Potential cost drivers for vertical airport construction projects. Project Category CIV 1 CIV 2 CIV 3 CIV 4 CIV 5 CIV 6 Construct ARFF facility Area No. of stories No. of bays Construction type Building skin type Site conditions Construct, expand, or rehabilitate terminal building Area No. of stories No. of spaces Structural system Architectural treatment Lobby area Construct parking garage Area No. of stories Construction type Building skin type Site conditions Construct SRE building Area Annual enplanements No. of stories Building skin type Site conditions

50 Airport Capital Improvements: A Business Planning and Decision-Making Approach Conclusions The goal of this project was to develop a model and database for estimating the cost of airport construction projects during the capital planning phase. The recommend approach—parametric cost estimating—uses historical cost data to establish mathematical relationships between con- struction cost and the hypothesized cost drivers for the project type in question. The study resulted in the creation of a database that includes data on construction cost and cost drivers for eight different types of airport construction projects. The database was used to develop a statistical cost model using the parametric cost-estimating approach. Both the database and the model were implemented in Microsoft Excel. A user interface allows the user to enter airport and project-specific information and generate a cost estimate report that can then be saved, printed, or exported. The model also provides a simple what-if analysis capability that allows the user to modify the assumptions. The resulting cost estimates are adjusted for inflation and geographi- cal variations in construction cost. The cost estimate is presented as a range of estimates, with best, low, and high values. This allows the user to take into account uncertainties and unique factors that affect cost. The cost model was evaluated using statistical measures of quality of fit and subjective evalu- ations by the research team’s SMEs. The model was also validated using a case study approach. The model passes the statistical tests of significance and quality of fit and, in general, generates cost estimates that match the experience of the SMEs. The research team concludes that the parametric cost-estimating methodology is a suitable approach for cost estimating for airport construction projects. This is especially true in the capital planning phase, where cost estimates need to balance accuracy with the effort required to develop the estimates. At the same time, the validation effort showed that the performance of the model is highly variable. Depending on the project type and specific circumstances, actual costs may vary significantly from those predicted by the model. This is true even when considering the range of low and high estimates provided by the model to take uncertainty into account. For this reason, the model should be treated as a proof-of-concept tool. Estimates prepared with the current model should only be used for initial planning purposes and should not be the sole means for evaluating the cost of a proposed project. The lack of robustness and variations in performance in the model are primarily caused by the limited availability of historical cost data. Collecting data in a format that supports inclusion in a cost database was the greatest challenge identified by the research team. Data is often stored in a manner that prevents the data from being imported electronically. Also, in many cases the total project cost is available but not the values of the cost drivers that are required to perform the cost estimate. Finally, bundling of multiple projects frequently prevents historical project data from being used in the model. Because the model suffers from a lack of robustness, the guidebook contains specific and in-depth recommendations on how to interpret the results and identify specific risks. Checklists are included for evaluating the results in order to assess the uncertainty of the cost estimate report. If the check- lists identify risks that could drive the cost up or down, the airport should consider using the high or low range of the estimate. If the risk assessment reveals an unusually high level of uncertainty, an alternative cost estimate should be considered. The guidebook includes a series of recommended best practices for any future data collection intended to update and expand the model. Increasing the number of observations and incor- porating additional cost drivers are likely to substantially improve model performance. For this reason, the guidance on expanded data collection is the focus of the discussion on recommended future research.

Lessons Learned 51 Any expanded data collection would require a framework for collecting the data in a central- ized manner. Standards need to be established to ensure data consistency and that the format supports transfer into a spreadsheet or database. Consideration should also be given to collecting site plans. These drawings provide important information on project dimensions, such as the size of pavement surface areas. Analyzing such information would require analysis by an archi- tect or engineer to interpret the drawings, however. A key finding of the data collection effort is that there is no single entity that can provide the data required to expand and improve the model. Consequently, the research team suggests that a cooperative approach to data collection be considered that involves state aviation agencies, transportation departments, industry organizations, and the FAA Airports organization, espe- cially at the regional level. The research team believes that a broad-based, collaborative approach to the collection of airport project and cost data has the greatest potential for achieving the best outcome. The resulting improvements could provide substantial benefits to the airport com- munity by enabling standardized and more accurate cost estimates to be available in the capital planning phase.