Airport Capital Improvements: Developing a Cost-Estimating Model and Database (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

CHAPTER 3: FINDINGS AND APPLICATIONS This section describes key findings from the research project. The discussion is broken down into the findings from the background research, the data collection and model development, the model validation, and the user interface implementation. Background Research Prior to establishing the framework for the cost estimating model, the Research Team surveyed the existing literature to document existing best industry practices. A stakeholder survey was also conducted in order to document cost estimating practices and needs. The results of these efforts are presented in the sections below. Literature Review The Research Team conducted a review of existing literature related to cost estimating. The review is reproduced in Appendix A in the form of an annotated bibliography which documents best practices. The review covers publications on cost estimating practices, as well as existing databases and cost estimating software. The works are listed in alphabetical order within each category. Stakeholder Survey As part of the review of existing cost estimating practices and needs, the Research Team surveyed stakeholders such as airport directors, industry groups, and transportation agencies. The outreach effort was implemented as a web-based survey combined with some telephone follow- up. The goal of the survey was to evaluate current practices in airport construction cost estimating, define key issues and challenges, and identify sources of data. The questions asked were intended to provide a holistic sense for how cost estimating is currently being accomplished in the airport industry. The questions also solicited input on where the process could be improved, sources of cost data, and the types of projects that are estimated. The survey instrument is reproduced in Appendix B. Key results from the survey are summarized below. A total of 272 survey recipients were identified, of which 52 completed the survey, for a response rate of 24%. Table 1 summarizes all questions that could be answered with a “yes” or “no” answer (“N/A” indicates that the respondent answered “not applicable”; “NR” indicates no response). The computation of percentage shares of “yes” and “no” answers excludes respondents who did not respond. When asked what types of construction cost estimating issues were found to be most difficult to deal with and what respondents would like to see this model do to help solve those issues, the answers varied widely. One answer that was repeated numerous times addressed the issue estimating the cost of mobilization. It is also clear that the majority of responses spoke to the need for estimators to know more about the cost of specialty project elements and the bidding environment, rather than to challenges with quantification of standard project elements. This may suggest the need to focus more on specific parts of the cost estimating process, versus the modeling the process as a whole. 7

Table 1: Survey responses to "yes/no" questions Question Yes % Yes No % No N/A NR 4. Is construction cost estimation at your facility performed in-house? 14 27% 30 58% 8 0 5. Do you use outside consultants to perform construction cost estimating services? 45 87% 5 10% 2 0 6. Do you access online cost data for generating construction cost estimations? 9 17% 30 58% 13 0 7. Do you store historical construction cost estimations? 32 63% 19 37% 0 1 8. Do you store construction project bid tabs? 36 73% 13 27% 0 3 9. Do the quantities in your construction cost estimates include a round up or contingency? 24 51% 14 30% 9 5 10. Do you use an overall contingency factor at the end of a construction estimate? 20 43% 14 30% 12 6 12. Do you use standard construction cost estimating factors such as dollars per square feet of runway? 15 34% 20 45% 9 8 When asked “what are the most common types of construction projects that you estimate” the answers reflected a broad range of common types of airfield improvement projects. The most common project types indicated are summarized into broad categories in Table 2. Table 2: Common types of cost estimates by project type Project Type Responses Runway 24 Terminal Upgrades 24 Airfield lighting 18 Other airfield pavements 17 Taxiway 15 Equipment storage facilities 14 Roadways 7 Utilities 7 Navigation aids 5 8

Fourteen respondents indicated that they perform in-house construction cost estimating, twelve of whom use software. Nine respondents noted that they utilize Microsoft Excel, and one each uses AASHTO’s Trns•port Estimator, RSMeans, and Success Estimator. Four respondents indicated that they have in-house engineering staff to assist with or perform construction cost estimating. When asked what sources respondents use to obtain online unit cost data, the answers included the following broad categories: • Construction cost data from RSMeans or NECA. • Vendors. • Previous bids. • Tabulation of bids by state transportation or highway agencies. Fifteen respondents indicated that they use standard construction cost estimating factors such as dollars per square feet of runway. Of these, eight said they would be willing to provide those factors and, if documented, the rationale for selecting them. When asked “Do you store historical construction cost estimations and what format is the data in?” the answers were varied and ranged from Microsoft Excel to hard copy to PDF format. Of the 17 responses to this question, 11 indicated data was stored in Excel format. When asked “Do you store construction project bid tabs, for how long, in what format, and in what type of database?” the answers ranged considerably in both time span and data format. A total of 19 of 32 respondents indicate that they store construction project bid tabs in hard copy format. Of those who use an electronic format, Microsoft Excel was the most common format. When asked “Do the quantities in your construction cost estimates include a round up or contingency?” 24 respondents answered “yes” to the question. Of these, 11 provided specific values or ranges of values, whereas one responded by saying they did not use a contingency factor. Reported contingency factors ranged from 5% to 40%. When asked “Do you use an overall contingency factor at the end of a construction estimate?” 20 respondents answered “yes” to the question. Of these, 12 provided specific values or ranges of values, generally ranging from 5 to 30%. When asked “Please provide any rules of thumb for applying construction cost adjustments for your local region” 23 respondents provided answers. Of these, ten provided specific values or ranges of values, whereas four responded by saying they did not have any rules of thumb for applying construction cost adjustments. The rule of thumb percentage adjustments ranged from 5% to 25%. 9

Data Collection and CER Development Limited data availability led the Research Team to request change to Phase II of the Amplified Work Plan to incorporate supplemental rounds of data collection and scale back the user interface development and software capabilities. After the completion of the supplemental efforts, the database consisted of 251 observations. The data were examined for statistical outliers, by inspecting data visually and comparing unit costs (e.g., cost per square foot of floor space) against typical costs. After the removal of statistical outliers 183 observations remained, for a total yield of 73%. The results of the data collection are summarized in Table 3. Table 3: Results of data collection Project Type Total Data Points Collected Total Data Points Used Yield Horizontal Construction Projects Construct or rehabilitate taxiway 25 22 88.0% Construct, expand, or rehabilitate apron 29 22 75.9% Construct, extend, or rehabilitate runway 48 30 62.5% Install perimeter fencing 24 18 75.0% Install PAPI 10 5 50.0% Install weather reporting equipment 31 28 90.3% Remove on-airport obstructions (vegetation) 4 Vertical Construction Projects Construct ARFF facility 42 25 59.5% Construct SRE building 42 33 78.6% All Projects Total 183 251 72.9% The updated database was used to support the development of CERs for eight project types (compared to the five project types used for the prototype CER development. Ordinary least squares regression was used. A number of logarithmic and functional forms were investigated, as well as multiple permutations of CIVs and interactions between CIVs. In the final accepted model, all CERs used a linear-linear specification with no interactions. All but two CERs were modeled using a zero intercept. All costs were normalized to fiscal year (FY) 2014 dollars to correct for inflation and to Kansas (KS) construction cost levels to adjust for regional variations. The coefficients for the resulting CERs and results from the associated statistical tests used to evaluate the model’s quality of fit and performance are summarized in Table 4 and Table 5 respectively. 10

Table 4: Final CERs Project Type Intercept (FY2014 KS $) Coefficient 1 Coefficient 2 Horizontal Projects Construct or rehabilitate taxiway 11.9 Pavement area (SF) 6.1 MTOW (lbs.) Construct, expand, or rehabilitate apron 1.2 Pavement area (SF) 12.2 MTOW (lbs.) Construct, extend, or rehabilitate runway 2.9 Pavement area (SF) 35.4 Adj. MTOW (lbs.) Install perimeter fencing 32.2 Fencing (LF) Install PAPI 83.1 No. of Systems Install weather reporting equipment 171,700 Vertical Projects Construct ARFF facility 374.5 Floor area (SF) Construct SRE building 111,500 116.5 Floor area (SF) Table 5: Statistical tests Project Type Adj. R2 P-value 𝛃𝟏 P-value 𝛃𝟐 P-value F- statistic Horizontal Projects Construct or rehabilitate taxiway 82.5% 0.0% 0.4% 0.0% Construct, expand, or rehabilitate apron 87.4% 1.6% 0.0% 0.0% Construct, extend, or rehabilitate runway 83.7% 0.1% 0.1% 0.0% Install perimeter fencing 83.5% 0.0% 0.0% Install PAPI N/A N/A N/A N/A Install weather reporting equipment N/A N/A N/A N/A Vertical Projects Construct ARFF facility 88.2% 0.0% 0.0% Construct SRE building 88.3% 0.0% 0.0% As shown in Table 5, adjusted R2 values are generally close to the target value of 90%. This means that there is a good correlation between construction cost and the cost drivers selected for inclusion in the CERs. The P-value target of <5% is met for all coefficients. “N/A” indicates that the CER did not include any independent variables. In these cases, the CER is simply the average construction cost for the project type in question. This approach was used for project types that consist exclusively of the installation of equipment, such as a Precision Approach Path Indicator (PAPI) or Automated Weather Observing System (AWOS). 11

Note that in the initial model specification, the project type “Install airport visual aid” was used. However the data collection did not result in enough observations to support CER development for visual aids other than the installation of PAPI systems. Consequently, the project type was changed from “Install airport visual aid” to “Install PAPI.” Also, while the project type “Install weather reporting equipment” originally had the type of system as a candidate independent variable, in the final model only AWOS systems were sufficiently represented in the database to support CER development. Landing gear configuration, a CIV originally proposed for all pavement projects, was eliminated for the taxiway and apron CERs, as it was not found to be a statistically significant predictor of cost. For the runway CER, it was incorporated indirectly by adjusting the MTOW by a factor corresponding to the landing gear configuration. These adjustment factors were computed from data on maximum MTOW for runways in the FAA’s Airport Master Record database. The resulting factors are shown in. The factors were used to convert the design aircraft MTOW into a “single-wheel equivalent MTOW”, which was then used in the CER. The computation of the single-wheel equivalent MTOW is handled automatically by the cost model and is transparent to the user. Table 6: Adjustment factors for landing gear configuration Code Landing Gear Configuration Adjustment Factor SW Single wheel 1.0000 DW Dual wheel 1.6429 DTW Dual tandem 3.0267 DDTW Double dual tandem 5.7659 Model Validation In addition to the statistical metrics shown in Table 5, several other methods were used to test the validity of the cost model. One was to retroactively predict construction costs for the data points in the cost database. In the utopian case of a perfect model, the predicted costs should perfectly match the actual costs. When plotted against each other, the values would then fall on a line with a 1:1 slope Plots of predicted vs. actual costs are shown in Figure 2 through Figure 7. As can be seen, the plotted points generally follow the 1:1 reference line, but there are substantial variations about the line. As an example, of the 24 taxiway-related observations in the cost database, six (or 25%) have differences between predicted and actual costs of 10% or less. At the other extreme, seven data points (or 29%) have differences of 50% of more, while the remaining 11 data points (46%) have differences between 10% and 50%. Results are similar for the remaining project types, with some variation. 12

Figure 2: Predicted vs. actual cost – construct or rehabilitate taxiway $0 $5,000 $10,000 $15,000 $0 $5,000 $10,000 $15,000 Pr ed ic te d Co st (F Y 20 14 K S $k ) Actual Cost (FY 2014 KS $k) Construct or Rehabilitate Taxiway Predicted vs. Actual Cost Reference Line 13

Figure 3: Predicted vs. actual cost – construct, expand, or rehabilitate apron $0 $3,000 $6,000 $9,000 $0 $3,000 $6,000 $9,000 Pr ed ic te d Co st (F Y 20 14 K S $k ) Actual Cost (FY 2014 KS $k) Construct, Expand, or Rehabilitate Apron Predicted vs. Actual Cost Reference Line 14

Figure 4: Predicted vs. actual cost – construct, extend, or rehabilitate runway $0 $3,000 $6,000 $9,000 $12,000 $0 $4,000 $8,000 $12,000 Pr ed ic te d Co st (F Y 20 14 K S $k ) Actual Cost (FY 2014 KS $k) Construct, Extend, or Rehabilitate Runway Predicted vs. Actual Cost Reference Line 15

Figure 5: Predicted vs. actual cost – install perimeter fencing $0 $150 $300 $450 $0 $150 $300 $450 Pr ed ic te d Co st (F Y 20 14 K S $k ) Actual Cost (FY 2014 KS $k) Install Perimeter Fencing Predicted vs. Actual Cost Reference Line 16

Figure 6: Predicted vs. actual cost – construct ARFF facility $0 $6,000 $12,000 $18,000 $0 $6,000 $12,000 $18,000 Pr ed ic te d Co st (F Y 20 14 K S $k ) Actual Cost (FY 2014 KS $k) Construct ARFF Facility Predicted vs. Actual Cost Reference Line 17

Figure 7: Predicted vs. actual cost – construct SRE building Another validation method conducted for this study was a case study analysis. Fifteen potential case studies were collected for this purpose. After data verification, nine of the projects were deemed to be sufficiently complete to include in the case study analysis. There was relatively little variation among the case studies: They represent only three states and four project types, all in the horizontal domain. As shown in Figure 8, the case study validation effort showed that the prototype CERs exhibited mixed performance: One-third of the case studies had errors of 10% or less, one-third had errors of approximately 30%, and the remaining one-third had errors of approximately 100% (i.e. the predicted cost was twice the actual cost). The case study analysis indicates that the cost model appears more likely to overpredict cost than to underpredict cost. This is viewed as a positive characteristic, since an overly conservative cost estimate is generally viewed as a more favorable outcome than a predicted cost that turns out to have been too low. $0 $3,000 $6,000 $9,000 $0 $3,000 $6,000 $9,000 Pr ed ic te d Co st (F Y 20 14 K S $k ) Actual Cost (FY 2014 KS $k) Construct SRE Building Predicted vs. Actual Cost Reference Line 18

Figure 8: Case study validation – difference between predicted and actual cost As a final step in the validation effort, the airport construction SMEs on the Research Team were asked to test the CERs against projects that they were currently involved in. The SMEs were also asked to compare the results of the cost model with their professional experience and familiarity with cost estimating for airport construction projects. The horizontal project SME tested the model against four projects: 1. Construction of 10,750 LF security fence at a small regional airport: The model substantially underestimated the cost, by approximately 60%. Follow- up analysis indicated that the presence of tidal wetlands was a substantial cost driver in the project. 2. Construction of 45,000 SF apron at a general aviation airport: The model estimated the cost within 3%. 3. Construction of an 83,000 SF taxiway at a general aviation airport: The model significantly overestimated the cost, by a factor of 225%. No specific explanation was identified in the follow-up analysis, except that the cost database included very few data points with CIV values in the range of the project. -150% -100% -50% 0% 50% 100% 150% $0 $5,000 $10,000 $15,000 $20,000 $25,000 $30,000Di ffe re nc e Actual Construction Cost (FY14 KS$) 19

4. Construction of 78,000 SF of taxiway pavement surface at a small regional airport: The model estimated the cost within 18%. With the exception of the third case, these are all viewed as acceptable outcomes. The third case illustrates that despite the acceptable statistical measures of fit, the model is not as robust as it could be. This is most due to the relatively low number of data points in the database, even after completion of the supplemental data collection. The small size of the database and the resultant lack of variation in projects mean that a number of factors that affect construction cost are not represented in the CERs. For this reason, the final cost model should be viewed primarily as a proof-of-concept tool and a disclaimer to this effect is included with the output of the cost model. The Research Team’s architecture SME reported the following findings after testing the cost model: 1. For snow removal equipment storage buildings, the cost model resulted in unit costs slightly lower than expected. However, the high/low range of cost estimates produced by the model was found to be reasonable. 2. For aircraft rescue and fire fighting facilities, the cost model resulted in unit costs slightly higher than expected. Again, the high/low range of cost estimates produced by the model was found to be reasonable. User Interface Development The Research Team implemented the cost database and model in Microsoft Excel, incorporating a graphical user interface to provide inputs and support user interaction with the model. It consists of two key interactive elements: 1. An input window that allows for entry of contact information, airport information, and project-specific information. 2. An output window that displays the reported cost estimate, along with all relevant input data. The resulting Excel-based tool incorporates airport information for all airports in the FAA’s National Plan for Integrated Airport Systems (NPIAS) (FAA 2012). This means that entry of the three-letter airport identifier allows automatic retrieval of airport-specific information for all NPIAS airports. For non-NPIAS airports, the user is prompted to enter the information manually. This information can, as an option, be saved to the database for later retrieval. 20

The input window is shown in Figure 9 and contains four sections: (1) Contact information for the preparer; (2) airport data; (3) project input; and (4) preliminary cost estimate. The preliminary cost estimate is a running calculation of the cost estimate shown next to the values of the independent variables. In addition to the most likely cost estimate, low and high estimates are also provided. This creates a high/low range of potential construction costs. Costs are shown both in current dollars (i.e. fiscal year 2014) and dollars that have been inflation adjusted to the anticipated year of construction (fiscal year 2018 in this example). Figure 9: ACCE input window Figure 10 shows a sample cost estimate report generated in the output window. While not shown, the output window also features support for the following features: • Exporting the cost estimate to another Microsoft Excel file • Sending the report to a printer • Saving the file as a PDF file • Returning to the input window in order to modify the input values The last feature allows for the preparation of what-if analyses, in which the input values that define the project can be modified. The report includes all of the data provided in the input fields, the calculation of the cost estimate (including the high/low range), a disclaimer that highlights that the actual cost may differ substantially from the estimate, and any other warning messages generated. 21

Figure 10: ACCE cost estimate report 22