Traffic Forecasting Accuracy Assessment Research (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

II-20 Deep Dives 3.1 Introduction to the Deep Dives The Large-N analysis could be used to measure the error and could shed light on certain factors associated with forecast errors, but it did not explain why the forecasts might be in error. The project team used deep dives to fill that gap to the extent possible. The deep dive analysis focused on addressing the following questions: • What aspects of the forecasts (e.g., population forecasts, project scope) can be identified clearly as being accurate or inaccurate? • If the inaccurate aspects of a traffic forecast had been accurate, how much would the forecast have changed? In the deep dives, the goal was to attribute as much of the error as possible to known factors. The remaining error would be recorded as occurring for “unknown reasons,” and the project team would be able to say little about it beyond the fact that it was not due to the aspects the team had identified and quantified. Deep dives guided the team’s efforts to identify the reasons behind the forecast errors. The specific methods for answering these questions varied across the deep dives, depending both on the process options that were being considered and on the data that had been made available for that specific project. To provide a range of project types and a range of available data for analysis, the research team completed five deep dive analyses. A sixth deep dive was only partially completed due to lack of clarity in the forecast documents. In selecting the deep dive cases, the project team aimed to find projects for which: 1. The project was already open, and the project team expected to be able to find post-opening data. 2. The project was big enough to have a meaningful impact. 3. Detailed information was available about the forecasts. Ideally, the forecast information would be in the form of archived model runs. Lacking that, detailed forecast reports would be beneficial, and if those were unavailable, the project team would rely on the project’s EISs or other public documents. 4. The projects as a set would show some diversity of types. The project team found it to be surprisingly difficult to find suitable case studies. Often, points one and three were in direct conflict. A few agencies were doing a commendable job of archiving forecasts, but even in the best cases, the archives got thin more than about 10 years back, and the projects that had been forecast less than 10 years previously often were not yet open. In addition, for projects with longer timeframes, agency staff turnover often had occurred and institutional memory had been lost. In these cases, the most promise came from finding long-time staff who C H A P T E R 3

Deep Dives II-21 happened to be good at keeping their own records or who had seen the value in saving the information. The project team looked for big projects with the idea that, if the projects were more important to start with, they would be better documented and would show more meaningful impacts. What the project team found, though, was that many of the major projects that had opened over the last decade were for tolled roads. This was natural given funding constraints in recent years; however, because toll forecasts have been studied more extensively elsewhere, the project team wanted them to be a part of, but not the dominant part of, this study. The resulting selection of deep dive cases is listed in Table II-9. These cases provided a reasonable diversity of project types and available data. They included a new bridge, the expansion and extension of an arterial on the fringe of an urban area, a major new expressway built as a toll road, the rebuild and expansion of an urban freeway, and a state highway bypass around a small town. The remaining sections of this chapter describe the method that was adopted in conducting the deep dives, provide short descriptions of the completed deep dives, and include discussions of the findings. More detailed reports for all six deep dive projects are given in Part III, Appendix H, which is printed at the end of this report and is available for download as a PDF file from the report webpage on www.trb.org. The chapter concludes with the generalized findings from the deep dive analysis alongside a discussion. 3.2 Methodology 3.2.1 Sources of Error as Cited in Existing Literature Considerable research has been done on the traffic forecast inaccuracies. Consulting the existing literature (e.g., journal articles, DOT reports), the project team identified several key reasons for forecast inaccuracy, as summarized in Table II-10. (For the detailed literature review, see Part III, Appendix F.) The table lists the issues and items in order of their importance, that is, how many times they have been cited, with the citations column indicating how many papers mention that topic as a source of forecast error. Some of the errors also have been quantified by measuring their elasticity with travel demand. In most cases, the research found in the literature Project Name Brief Description Eastown Road Extension, Lima, Ohio Widening of a 2.5-mile segment of the arterial from 2 lanes to 5 lanesand extension of the arterial by an additional mile. Indian Street Bridge, Palm City, Florida Construction of a new 0.6-mile long bridge with 4 travel lanes running along CR-714 (Martin Highway), connecting with Indian Street, and going across the St. Lucie River. Central Artery Tunnel, Boston, Massachusetts Reconstruction of Interstate Highway 93 (I-93) in downtown Boston, the extension of I-90 to Logan International Airport, the construction of 2 new bridges over the Charles River, six interchanges, and the Rose Kennedy Greenway in the space vacated by the previous elevated I-93 central artery in Boston, Massachusetts. Cynthiana Bypass, Cynthiana, Kentucky A 2-lane state highway bypass project, to the west of the city from the southern terminus where US-62S and US-27S meet. South Bay Expressway, San Diego, California A 9.2-mile tolled highway segment of SR-125 in eastern San Diego, California, this project was funded as a Public Private Partnership (P3). It opened in 2007 and the P3 filed for bankruptcy in 2010. US 41 (later renamed as I-41), Brown County, Wisconsin A project of capacity addition involving reconstruction of 9 interchanges, construction of 24 roundabouts, adding collector-distributer lanes, and building of 2 system interchanges located in Brown County, Wisconsin. Table II-9. Projects selected for deep dive analysis.

II-22 Traffic Forecasting Accuracy Assessment Research review simply identified that the topic could be a source of error from a logical standpoint, rather than clearly showing the amount of error due to that cause. Several general observations can be made about the sources of error as cited in the literature: • Significant overlap exists in the identified sources of error. For example, GDP and economic conditions are clearly related to employment, just as land use and housing projections are related to population projections. This is not necessarily a problem, but it means that there is room to consolidate. • The list focuses largely on the assumptions and inputs that were fed into the models. This focus is useful in that these factors can generally be observed independently of the count, allowing researchers to better evaluate their effects. • Sources of error that relate to the model structure and the stability of travel behavior over time are not often cited. These factors could still be sources of error, but they are more difficult to evaluate as such, which may be why they have been identified less often. One exception is trip generation/traveler characteristics, which relates to limitations of a model component and the associated data itself. In the deep dives for this research, the project team investigated these issues for each project. The goal of the deep dives was to quantify each of the errors and find out how they influence the forecast and travel demand. By doing so, the project team aimed to quantify the relative importance of these factors, at least for the small sample of projects that had been chosen. Items Notes Citations Quantified? GDP/GDP Growth/GDP Per Capita/Economic Condition GDP (particularly GDP per capita for the traffic analysis zone as well as for the entire state) is particularly prominent since GDP has bearing on car ownership, employment, and even toll culture. From Andersson et al. (2017), “In forecasts since 2005, GDP has been the largest source of error due to the sluggish economic growth since 2008.” 8 Yes Employment Kain (1990) found changes in employment and population growth to account for most of the changes in transit ridership. 7 Yes Recession/Short-term Economic Fluctuation Miller et al. (2016) quantified the effect of a number of economic recessions on forecast accuracy. It can be argued that recessions/short-term economic fluctuations belong in the same category as GDP growth. 7 Yes Trip Generation/Travel Characteristics The availability of appropriate data and their quality, in particular traffic counts, network characteristics, travel costs, and so forth. 5 No Land Use Changes/ Housing Prediction/ Location of the Project Changes in the building environment that are not specific to the project (Andersson et al. 2017). Flyvbjerg et al. (2006) found that 26% of projects experience problems regarding a change in land use. 5 No Population Projection/ Household Survey Also a projection of population distribution. Relevant for the TAZ. Can be linked with employment and car ownership. 4 Yes Fuel Price/Efficiency Base price, tax, and fuel economy. 4 Yes Car Ownership Car ownership in each household in the TAZ. Affects the VMT and travel characteristics. Andersson et al. (2017) terms errors in car ownership calculations as a model-specification error. 3 Yes Time Savings from Traveling in the Proposed Route or Value of Time/ Willingness to Pay The monetary value given by travelers to travel time. Alternate route choices may have an effect. 3 No Toll Culture Better performance for countries that had a “history” of toll roads, compared with countries for which road tolling was new (Bain 2011b). 3 No Forecast Duration Number of years between the forecast year and the base year.* According to Miller et al. (2016), as the difference decreases (i.e., as the forecast duration shrinks), accuracy increases. 3 Yes *The base year is the year in which data are collected to serve as the baseline for forecasts. The base year may be up to 5 years before the year in which the forecast is made. Table II-10. Sources of forecast error cited in existing literature.

Deep Dives II-23 The deep dives would provide readers with information about where they should focus their efforts to improve forecasts. 3.2.2 Procedure for Analysis The UK Post-Opening Project Evaluations (POPEs) take an approach of evaluating both a do-minimum and a do-something scenario, as illustrated by their assessment of M6 improve- ments (Highways England 2015). Figure II-4 shows the count comparisons that were included in the research for this report. The do-minimum forecast scenario is evaluated against a count year prior to construction, and the do-something scenario is evaluated against a count year after construction. As shown in the “Do Something” part of the figure, this approach helps to decipher whether any differences are due to the net project effect or due to differences in back- ground growth. As for how one could quantify the sources of error, Andersson et al. (2017) provides a good start. In the Andersson study, the actual and forecast passenger traffic (measured in vehicle- kilometers traveled, or VKT) were compared for eight national reference forecasts spanning several decades. The input assumptions (e.g., GDP, fuel price, and so forth) were obtained from the forecast reports, and updated figures for those same assumptions (the actual GDP, fuel price, and so forth at time of project opening) were found from national statistics. The forecasts were then adjusted for the errors in input assumption by replacing the original assumptions with the corrected (actual) numbers. The primary approach in this regard was to calculate elasticities of traffic with respect to the input variables (income, fuel price, car ownership, and population). Cross-sectional elasticities were calculated by increasing the variables by 10% at a time and calculating the resulting change in VKT (d): ln 1 ln 1 10% , (II-7) ( ) ( ) ε = + δ + x where e = the elasticity, x = the change in the variable of interest, and d = the resulting change in VKT. Figure II-4. Example of post-opening project evaluation count comparisons (Highways England 2015).

II-24 Traffic Forecasting Accuracy Assessment Research A reference model was used as a comparison forecast to see how much better the actual fore- casts did relative to pre-existing trends. The reference model was estimated as a time-series model regressing VKT on GDP/capita and fuel price: eVKT GDP cap fuel priceln ln ln , (II-8)( )( ) = α + β     + γ + where α, β, and γ = estimated model parameters, and  = a random error term. The interesting observations here are that (1) the researchers used elasticities from the existing transportation model (older models were not available) to adjust forecasts and (2) the elasticities (cross-sectional and time series) have remained remarkably close to each other over time. Elasticities could alternatively be taken from the literature or from another source. The changes in the forecast were documented for the cumulative adjustments in input variables (i.e., each adjustment built on the preceding one and adjusted one more variable). The overall accuracy was assessed by comparing the root mean squared errors of the original and adjusted forecasts to the actual outcomes. Table II-11 reproduces the final table from the research by Andersson et al. (2017) and is taken directly from that paper. It is reproduced in this report as a reference point. The numbers are the percent growth in VKT in the region from the base year. Each row in the table shows a different forecast. The first column shows the expected percent growth based on the trend line. The second column shows the forecast percent growth in VKT, and the last column shows the actual percent growth. The intermediate columns show what the forecast would be if it had the correct population growth, fuel price, fuel economy, car ownership, and GDP, with each Trend Forecast Adj. for Pop. Growth & Adj. for Fuel Price & Adj. for Fuel Economy & Adj. for Car Ownership & Adj. for GDP Actual TPR 1980 (1980–1990) (%) 46 5 6 11 11 17 16 25 TPR 1990 (1990–2000) (%) 25 20 20 16 15 9 9 8 TPR 1990 (1990–2010) (%) 50 31 31 26 25 18 16 19 VTI 1992 (1991–2005) (%) 38 26 26 17 18 16 17 15 VTI 1992 (1991–2013) (%) 59 41 41 26 32 30 24 20 Samplan 1996 (1993–2010) (%) 38 30 30 21 24 20 24 20 Samplan 1999 (1997–2010) (%) 19 20 24 12 11 10 12 16 SIKA 2005 LU (2001–2013) (%) 10 18 20 11 12 10 9 11 Trp Adm 2009 (2006–2013) (%) 10 6 8 3 6 7 5 3 Root mean square error 0.64 0.38 0.39 0.18 0.21 0.14 0.12 Table II-11. Decomposition of forecast errors from Andersson et al. (2017).

Deep Dives II-25 adjustment building incrementally upon the others that preceded it. The analysis shows that correcting for these five assumptions would have reduced the root mean square error (RMSE) of the forecasts from 0.38 to 0.12. Modifications to the analysis approach used by Andersson et al. were warranted because the continuous adjustment and subsequent changes in traffic growth do not, by themselves, quantify the effect of each variable on traffic forecast accuracy. Also, the existing approach did not address the issue of uncertainty and inherent variation in each of the components. Another, similar approach is described in the FHWA’s guidance for road traffic forecasting, which recommends an “incremental buildup” of the forecast variables (FHWA 2010, p. 23). The effect on the forecast can be quantified in this way: First, the change in forecast value— a delta between the opening year forecast and the actual observed traffic count in the opening year—is calculated: Actual Value Forecast Value Forecast ValueChange in Forecast Value (II-9) ( ) ( )= − Second, a factor of the effect on the forecast is calculated by exponentiating an elasticity of the common source errors and natural-log of the change rate in forecast value, as seen in Equa- tion II-10. This factor is applied to the actual forecast volume to generate an adjusted forecast, as shown in Equation II-11. Elasticity Change in ValueEffect on Forecast exp 1. (II-10)ln 1= −( )( )∗ + Effect on Forecast Actual Forecast VolumeAdjusted Forecast 1 . (II-11)( )= + ∗ To quantify the effects of individual variables on forecast accuracy, a sensitivity analysis also could be performed. Kain (1990) evaluated the effect of various explanatory variables by comparing their elasticities with transit ridership across several scenarios (traditional, moderate, aggressive, and conservative) to base-year statistics. The conservative scenario was the one devised by the author, whereas the other scenarios were used in the model to project transit ridership into 2010 from the base year of 1986. The modus operandi for the sensitivity analysis is shown in Figure II-5. Lemp and Kockelman (2009) have suggested that, whereas a sensitivity test allows for greater understanding of the magnitude of uncertainty in the model, it does not provide a probability • Received from forecasting agency • Not the ones used in the original modeling process • Obtained from updated reports provided by the forecasting agency • Compared with base-year statistics • Multiplying elasticities with percentage change in variables Percentage Changes in Projection Percentage Changes in Explanatory Variables Updated Elasticities of Explanatory Variables Figure II-5. Method for testing effects of errors in forecast inputs.

II-26 Traffic Forecasting Accuracy Assessment Research of a particular outcome occurring. The authors recommend using Monte Carlo simulations to generate possible scenarios; however, such an approach went beyond the analysis conducted in this project. The deep dives began with a comparison of the actual and forecast ADT, using worksheets similar to what is shown in Table II-12. If pre-construction counts and forecasts were available, they could be added to the table as had been done for the POPE cases. For each cell, the values and the PDFFs would be reported. Given the review and the project team’s own assessment of the important factors associated with forecast error, the deep dives focused on evaluating each of the items listed in Table II-13. Items Definition Quantifiable Employment The actual employment (or GDP) differs from what was projected. Yes Population/Household The actual population or households differ from what was projected. Yes Car Ownership Actual car ownership differs from projection. Should note whether car ownership is endogenous or exogenous to the forecast. Yes Fuel Price/Efficiency The average fuel price or fuel efficiency is different from expectations. Yes Travel Time/Speed Travel time comparison of the facility itself and alternative routes. Yes Toll Sensitivity/Value of Time The sensitivity to tolls, or the value of the tolls themselves, is in error. For example, the study by Anam (2016) on Coleman Bridge found that the project considered two toll amounts ($1.00 and $0.75); however, by the time of opening/horizon year these values had changed to $0.85 and $2. Yes Project Scope The project was built to different specifications than were assumed at the time of the forecast. For example, budget constraints meant that only 4 lanes were built instead of 6 lanes. Yes Rest of Network Assumptions There were assumptions about related projects that would be constructed that differed from what was actually built. Yes Model Deficiency/Issues Limitations of the model itself. These could include possible errors, or they could be limitations of the method. For example, the project was built in a tourist area, but the model was not able to account for tourism. No Data Deficiency/Issues Limitations of the data available at the time of the forecast. For example, erroneous or outdated counts were used as the basis for pivoting. No Unexpected Changes During the latter third of the 20th century, this could include changes in the number of workers per household or other broad social trends. During the 21st century, this could include technology changes, such as self-driving cars. No Other Other issues that are not articulated above. No Project Segment & Direction Base-Year Count Base-Year Forecast (if Different) Opening-Year Count Opening-Year Forecast % Growth in Count % Growth in Forecast Table II-12. Deep dive count worksheet. Table II-13. Sources of forecast error to be considered by deep dives.

Deep Dives II-27 Each deep dive followed a similar structure: working through the list of factors, attempting to identify whether each item was an important source of error for the forecast, and, if so, attempting to quantify how much it would have changed the forecast if the forecasters had gotten it right. The last column in Table II-13 also identifies whether the project team expected to be able to quantify the effect of that item on the resulting forecast. As seen in the table, the top seven factors are generally model inputs, and it was reasonable to expect that the project team could observe the actual outcomes and apply an elasticity or updated model run to evaluate the effect of having the correct input on the forecast. The project team expected the remaining factors to be more difficult to quantify and planned to address them qualitatively if they were identified as being important. For those inputs that could be quantified, the deep dives were structured using worksheets (see Table II-14). The first row of the worksheet table shows the original traffic forecast, the actual count, and the forecast error. The subsequent rows show the effects of the different inputs. For example, the second row shows the forecast and actual employment. An elasticity is shown, as well as the effect on the forecast of correcting for errors in that input. Then the remaining PDFF is shown if the forecast was adjusted to account for errors in that input. This structure is similar to the structure that was used by Andersson et al. (2017), with the exception that the method for filling in the table was left open-ended. This choice was deliberate, and it allowed the project team to adapt the analysis based on what information was available for each deep dive. For example, when only the public documents (e.g., an EIS) were available, the project team could look at the total regional employment and apply an elasticity to adjust the forecast. If the project team had the model run available, however, it was possible to scale the employment in the TAZ file and rerun the model. In some of the deep dive cases, the project team was not able to evaluate the effect of a certain factor. In those cases, a notation was made and the project team identified what information would have been needed in order to complete that evaluation. The diversity of the deep dive projects allowed the project team to comment both on what was found and on the effectiveness of attempting such evaluation with different levels of data. Each deep dive was documented as a case study following a semi-standard format that included the following sections: • Project Description. Each case study opens with a brief description of the project itself. • Forecasts. This section captures notes about the forecast, including the method, the year the forecast was developed, the expected opening year, the design year, and the sources of information that were available about the forecast. Items Forecast Value Actual Value Elasticity Effect on Forecast Remaining PDFF for Adjusted Forecast Original Traffic Forecast N/A N/A Employment Population/Household Car Ownership Fuel Price/Efficiency Travel Time/Speed Toll Sensitivity/Value of Time Project Scope Rest of Network Assumptions Adjusted Traffic Forecast N/A N/A N/A Table II-14. Deep dive worksheet.

II-28 Traffic Forecasting Accuracy Assessment Research • Comparison to Actual Outcomes. Notes on the actual project outcomes and associated data sources are placed in this section. It also records comparisons between the forecast and actual outcomes using a structure equivalent to that shown in Table II-12. • Evaluation of Sources of Forecast Error. This section presents an evaluation of the factors identified as having contributed to forecast error for this particular project. For each factor, the evaluation may include a quantitative assessment, a qualitative assessment, or both. Recogniz- ing that it might not be possible to evaluate all factors, the project team structured the section to acknowledge that they should be considered. For each deep dive, this section includes a table similar to Table II-14 • Discussion. Each deep dive case study concludes with a discussion of the findings from the analysis. 3.3 Results 3.3.1 Eastown Road Extension, Lima, Ohio The Eastown Road Extension is a project in the city of Lima, Ohio, that widened a 2.5-mile segment of the arterial from 2 lanes to 5 lanes and extended the arterial by an additional mile. This north-south arterial is located on the western edge of the city of Lima in Allen County, Ohio. The project extended Eastown Road from just north of Elida Road in the north to Spencerville Road in the south. The project included a 2.5-mile expansion from 2 lanes to 5 lanes on the segment between Elida Road and West Elm Street, and a 1-mile extension further south to Spencerville Road (see Figure II-6). The traffic forecasts on Eastown Road were generally overestimated by about 20%, with the extension segment overestimated by 43%. It should be noted that there is a possible error in the observed counts on the extension segment. The project opened in 2009, which was the time of peak economic recession and high gas prices in the country. As a result, overestimation of employment and underestimation of fuel price in the opening year were two key contributors to the forecasting errors in this project. Additionally, the modeled travel speeds on certain segments of the project were overestimated by up to 13%. This was the third key contributor to the forecasting error in this project. Population and car ownership forecasts were very similar to the observed values and contributed a tiny portion to the forecasting error. Adjustments to the forecasts using elasticities and model reruns confirmed that significant errors in opening-year forecasts of employment, fuel price, and travel speed had a major role in the overestimation of traffic volumes on Eastown Road. After accounting for the corrected exogenous forecasts and project assumptions, the traffic forecasts on the project segments that were widened from 2 lanes to 5 lanes improved from an average overestimation of 20% to 3%. The forecasts on the extension segment improved from 43% to 39% overestimation. Overall, the prevailing macroeconomic conditions around the opening year played a major part in the accuracy of the forecasts for the Eastown Road Extension project. This variable represents a major uncertainty that is extremely difficult to directly consider at the time of preparation of traffic forecasts given the various modeling parameters that could change in an economic downturn. One way to account for this variable is to evaluate and document the changes in the traffic forecasts using reduced employment and higher fuel prices. It is unknown whether risk and uncertainty were considered in the traffic forecasts due to the absence of project documentation. For future forecasting efforts, it is suggested that a copy of the project and traffic forecasting documentation be saved along with the actual model used to generate the forecasts.

Deep Dives II-29 Source: Map data: Google Earth, annotated by NCHRP 08-110 project team W Elm St 5 6 8 9 10 11 1 2 3 4 7 These are the 11 segments identified for traffic volume accuracy assessment. Figure II-6. Project corridor for Eastown Road Extension.

II-30 Traffic Forecasting Accuracy Assessment Research 3.3.2 Indian Street Bridge, Palm City, Florida The Indian Street Bridge is a new bridge construction project located in Palm City, Florida (Martin County). The bridge is 0.6 miles long with four travel lanes in total (2 lanes in each direction). This bridge runs along CR-714 (Martin Highway), connecting with Indian Street and going across the St. Lucie River. The Indian Street Bridge acts as a reliever bridge for the Palm City Bridge (old bridge), which is located approximately 1 mile north of the new bridge. It is also expected to provide relief to the existing SR-714 corridor, which is connected with the Palm City Bridge. The study area boundaries extend from Florida’s Turnpike to the west, the Federal Highway (US-1) to the east, I-95 crossing of the St. Lucie Canal to the south, and the Martin County/St. Lucie county line to the north. Figure II-7 shows the study area for this project. This project concentrated on multiple alternatives and was later finalized in the construction of a new 4-lane bridge. The updated study was reported in 2003. The construction was started in 2009 and was completed in 2014. The estimated construction cost of the project is $63.9 million. This project was interesting because it provided an opportunity to examine a new bridge project crossing over a river, with clear diversion effects and detailed modeling information available. The model was built using the Transportation Planning (TRANPLAN) program. Florida DOT’s District 4 (D4) provided archived model runs and detailed project reports to support this deep dive analysis. The model forecast for the Indian Street Bridge (new construction) was generally over- estimated by about 60%. For the Palm City Bridge (competing route) the forecasts were over- estimated by 36%. After applying corrections through elasticity, the PDFF on the new bridge was reduced to 56%, and on the competing bridge it was reduced to 29%. Model alterations resulted in new forecast volumes that were 59% off for the new bridge and 34% off for the old bridge. Adjustments were made in the model forecast based on the elasticity and the model reruns. The elasticity study showed more promising results as compared to the model adjustments. Fuel price was an influencing factor during the corrections by elasticity. Inclusion of fuel price effects in the model could have been beneficial in reducing error; however, both methods could only explain part of the forecasting error. Clearly, other factors that are not accounted for in the model caused overall underestimation of the traffic in the study area and especially on the Indian Street Bridge. One source of error might have been the forecasting method. The opening-year traffic was forecast by scaling the design-year model volumes in accordance with the existing counts. Because the new bridge had no existing count information, such a procedure might have given rise to inaccurate forecasts; however, it is challenging to develop a more robust forecasting method for projects for which no existing counts are available. Moreover, the bridge may repre- sent too intense a change in infrastructure because it connects two different land areas through a single link, and few comparable alternative paths are available. The effects of the economic downturn might impact the travel behavior of a particular region for years following the recession. For example, Figure II-8 shows the clear impact of the 2008 recession on Martin County unemployment. The years 2010 to 2012 show peak unemployment. Job losses affect not only work trips but also leisure trips. Moreover, the recession is assumed to cause a change in the value of time, which also results in an updated coefficient for highway assignment purposes. Changes in job locations, even while maintaining the same housing locations, would alter individuals’ route selections and would clearly change travel patterns for the following years. These effects could be better studied by comparing the information

Source: Map data: Indian Street Bridge PD&E, Design Traffic Technical Memorandum, Florida DOT (January 23, 2003) Figure II-7. Project corridor for Indian Street Bridge.

II-32 Traffic Forecasting Accuracy Assessment Research available on trips from Big Data sources (e.g., Streetlight or AirSage data) before and after the recession years. As seen in Table II-15, external trips accounted for 9% of the traffic on the new Indian Street Bridge and only 2% of the traffic on the Palm City Bridge. This information supports the assumption that both the new and the old bridge are heavily used by the internal population. Further analysis comparing the modeled trip patterns to the information available from Big Data sources might reveal travel patterns that were insufficiently represented in the model. Another possible factor is that Martin County, St. Lucie County, and Indian River County show the steepest increases in the median age of the population (see Figure II-9). This informa- tion suggests that a lot of retirees moved into this region. Retirees tend to travel less than working families do, which may help explain why the population of St. Lucie County was underestimated by 22% whereas the traffic forecasts for all links in the study area were overestimated. The travel model did not have a component that adjusted the travel rates based on the number of workers in the household, which may have contributed to the overestimation. Source: Federal Reserve Economic Data (FRED) database (https://fred.stlouisfed.org) Figure II-8. Martin County unemployment rate chart. 2025 Original Run 2025 New Run External Trips New Bridge Old Bridge Total New Bridge Old Bridge Total I-95 1,563 - 1,563 1,523 6 1,529 Turnpike 1,227 936 2,163 1,298 968 2,266 US-1 1,095 174 1,269 1,080 286 1,366 Total 3,885 1,110 4,995 3,901 1,260 5,161 Table II-15. External trip distribution using both competing bridges.

Deep Dives II-33 Overall, the prevailing macroeconomic conditions around the opening year played a major part in the accuracy of the forecasts for this project. Other exogenous factors causing the overestimate may be the increase in fuel prices and an increase in retirees. Both factors could not be replicated precisely in the travel model used for the Indian Street Bridge. Further analysis using Big Data sources could add more insight on the overestimation of traffic. This study highlights importance of archiving not only the model runs and forecast reports, but also the validation approach used during model development. 3.3.3 Central Artery Tunnel, Boston, Massachusetts The I-93 Central Artery/Tunnel (CA/T) project, popularly known as the “Big Dig,” was a megaproject that included the reconstruction of Interstate Highway 93 (I-93) in downtown Boston, Massachusetts; the extension of I-90 to the General Edward Lawrence Logan Inter- national Airport (Logan Airport); the construction of two new bridges over the Charles River; six interchanges; and the Rose Kennedy Greenway in the space vacated by the previous elevated I-93 central artery. The project involved 7.8 miles of highway construction, about half of which took place in tunnels (Figure II-10). The study area for this deep dive consisted of the I-93 in downtown Boston and the I-90 near Ted Williams Tunnel that connects to Logan Airport under Boston Harbor. A highlight of the CA/T Project was the replacement of the elevated I-93 central artery with the underground expressway. It was built to reduce traffic congestion and improve mobility and environment in one of the most congested parts of Boston and the United States, and to establish the groundwork for economic growth. The Central Transportation Planning Staff (CTPS) backcasting report showed that roadways in the CA/T project generally were overestimated, by amounts ranging from 1% to 22%, with one roadway segment underestimated by 6%. Overall, traffic forecasting accuracy improved after correcting the exogenous forecasts and project assumptions. Nine of 12 roadway segments experienced a decrease in the level of forecast error as a result of such corrections. Source: BEBR 20 25 30 35 40 45 50 55 1970 1980 1990 2000 2010 M ed ia n A ge Year U.S. FLORIDA Indian River Martin Miami-Dade Palm Beach St. Lucie Figure II-9. Median age (in years) in southeastern Florida counties.

II-34 Traffic Forecasting Accuracy Assessment Research It should be noted that, although abundant documentation exists on the CA/T projects, virtually all of it is associated with project management, construction, project finance, and economic impacts. It is unknown whether risk and uncertainty were considered during the project due to the absence of documentation on the subject. For future forecasting efforts, it is suggested that a copy of the forecasting documentation and assumptions be archived along with the travel model files used to generate the forecasts. 3.3.4 Cynthiana Bypass, Cynthiana, Kentucky The Cynthiana Bypass is a 2-lane state highway bypass project located in Cynthiana, Kentucky. The study area included the Cynthiana city limits and immediate environs in Harrison County, Kentucky. The project created a bypass to the west of the city, starting at a southern terminus where US-62S and US-27S meet, and extending northward to a point of the city along Main Street/US-27N. The length of the bypass is 3.6 miles, and it includes a new bridge across the south fork of the Licking River, north of the city (Figure II-11). The traffic forecasts on the Cynthiana Bypass were generally overestimated by about 45%, with the notable exception of the northernmost section, which was estimated to within 4% of the observed values. As would be expected for a bypass project, the biggest source of error in the model forecast was the overestimated growth factor (2.5% per year) in external traffic volumes. Three out of four segments of the project showed a significant improvement after accounting for the corrected external forecasts. Source: Map data: Transportation impacts of the Massachusetts Turnpike Authority and the Central Artery/Third Harbor Tunnel Project, Economic Development Research Group, Inc. February 2006). Figure II-10. CA/T project locations.

Deep Dives II-35 The project opened in 2012, which was shortly after the peak of the economic recession and during a time of high gas prices. As a result, overestimation of employment in the opening year was a contributor to the forecasting errors in this project. Population forecasts were very similar to the observed values and did not contribute to the forecasting error (in fact, correcting for actual population alone made the forecasts a bit worse). Risk and uncertainty were not explicitly considered in the traffic forecasts. Project documen- tation was not archived by the project owners. Fortunately, a copy of the documentation was obtained from the consultant who happened to keep a paper copy in her personal files. (This consultant had long since left employment at the consulting company that was contracted Source: Map data: Google Maps, annotated by the NCHRP 08-110 project team Figure II-11. Project corridor (Cynthiana Bypass).

II-36 Traffic Forecasting Accuracy Assessment Research to do the study). For future forecasting efforts, it is suggested that copies of the project and traffic forecasting documentation be saved by the project owners—in this case, the state highway authority—along with the actual models used to generate the forecasts. 3.3.5 South Bay Expressway, San Diego, California The South Bay Expressway (SBX) is a 9.2-mile tolled highway segment of SR-125 in eastern San Diego, California. The SBX generally runs north-south from SR-54 near the Sweetwater Reservoir to SR-905/SR-11 in Otay Mesa, California, near the U.S.–Mexico border. A 3.2-mile untolled link to the existing freeway network at the northern end was publicly funded and built with the construction of the private toll road. Originally developed as a public-private partnership (P3), the SBX opened in November 2007. Initial traffic and revenue were below expectations, and the company was involved in ongoing litigation with contractors. In March 2010, the operator filed for bankruptcy. In July 2011, according to a website about the project, the San Diego Association of Governments (SANDAG) agreed to purchase the lease from the operator, taking control of the remainder of the 35-year lease in November 2011 (https:// www.transportation.gov/tifia/financed-projects/south-bay-expressway). The original study area boundary was essentially the entire San Diego region. The SBX is the easternmost north-south expressway in San Diego. It was originally developed to accommodate the rapidly growing residential and industrial South Bay area and to provide improved access to the U.S.–Mexico border crossing facility at Otay Mesa. The original SBX analysis was for the first toll facility in San Diego. The SBX was developed as permitted by California AB 680, passed by the California legislature in 1989. Under the agreement, the concessionaire developed the project and constructed the road in return for operating and maintaining the facility and collecting toll revenue for 35 years, until 2042. As per the agreement, the state of California owns the facility but leases it to the concessionaire. After the original concessionaire declared bankruptcy, SANDAG purchased the concession in December 2011 and will retain tolling control until the facility reverts back to Caltrans in 2042. As opposed to maximizing revenue on the facility, SANDAG sets the toll prices to relieve congestion on the I-5 and I-805. A map of the corridor and current toll rates are shown in Figure II-12. This deep dive is not intended as a criticism of the forecasts that were developed in 2003. Hindsight can allow us to perceive the warning signs, but very few people who saw the weakening of the housing bubble in 2007 recognized that a global financial crisis would follow in 2008. The SBX’s Transportation Infrastructure Finance and Innovation Act (TIFIA) program risk analysis report showed that the early-year project development forecasts had a probability of less than 5%, and that this included only risks associated with toll revenues (projections of construction costs and operating costs were held constant)—but the U.S. DOT certainly did not forecast the impending global financial crisis. If a more conservative approach were taken in the development of the project, it is unlikely that a P3 would have found this an appropriate project. At the least the concessionaire would have structured the deal differently. Researching through forecasts and comparable data for this project, one clear recommendation emerged: during the forecast period, it is important for every project to develop clear model performance metrics that can be checked against observed data. Much like the data collected for transit before-and-after studies, these data could provide clear insight to the forecasting process and could be used in each region (and collectively in the United States) to understand common forecasting errors. These metrics may include:

Deep Dives II-37 • Socioeconomic variables such as population and employment at sub-regional levels (focusing on the project corridors); • Regional VMT and vehicle-hours traveled (VHT) values; • Consistent ADT measures at specific points in the corridor (e.g., a plan to collect annual traffic counts on the facility for the first 5–10 years after opening); and • Consistent definitions of other measures to be collected and maintained. For toll facilities this could be annual or daily transactions, revenue-miles traveled, daily or annual revenue, average toll rates, and so forth. 3.3.6 US-41, Brown County, Wisconsin The largest reconstruction project in the history of the Northeast Region in Wisconsin, the US-41 project in Brown County and Winnebago County, Wisconsin, was primarily a project of capacity addition involving the reconstruction of nine interchanges, construction of 24 round- abouts, addition of collector-distributer lanes, and building of two system interchanges. The US-41 deep dive focused on a portion of this 31-mile US-41 highway reconstruction project. The project areas in the two counties were not all connected along US-41; rather, they were located on portions of US-41 that were adjacent to two major cities—Green Bay, in Brown County, and Oshkosh, in Winnebago County. The project mainly increased highway lanes from 4 lanes to 6 lanes (or 8 lanes, in the cases of some segments that included auxiliary lanes). It aimed to improve safety and road capacity by replacing old and deteriorating pavement and outdated design infrastructure with new pavement and infrastructure meeting current standards. The project also was intended to upgrade a transportation link that supported important economic vitality in the region between southeastern and northeastern Wisconsin. In Brown County, the project area was an approximately 14-mile portion of US-41 from Orange Lane near the County Road F interchange to the County Road M interchange. Figure II-13 shows five roadway segments in this area of the project. For the deep dive, the study Source: (Left) Google Maps; (Right) http://www.sr125.com Figure II-12. Project study area (SBX).

II-38 Traffic Forecasting Accuracy Assessment Research area concentrated on Segment 5, the 3.3-mile part of the roadway that was covered by the FEIS (i.e., the Memorial Drive to County M segment). The Memorial Drive to County M segment required an EIS because of the potential environ- mental impacts of building two system interchanges at WIS-29 and I-43 with tall flyover-type ramps built on top of swampland. The other four segments in the US-41 project underwent increases from 4 lanes to 6 lanes (or 8 lanes, including some auxiliary lanes). For these segments, it was necessary to either re-evaluate the original Environmental Assessment (EA) that had been completed in 2002 or complete a new EA, but a full EIS was not required. There are four links/roadways in Segment 5 (the deep dive study area). For the four links, the FEIS for the US-41 Memorial Drive to County M segment provided an existing-year traffic count, a 2005 count, and two future-year forecasts (for 2015 and 2035). The traffic forecasts, model outputs from a regional travel demand model, expressed traffic volumes as ADT volumes, which reflect average travel conditions rather than daily or seasonal fluctuations. Of the four links, three links had ADT traffic counts that were publicly available as of June 2018. The original traffic forecasts were slightly overestimated (by 3% to 10%) for these three study sites, but they were generally close. It should be noted that the traffic count for Site 3 was the preliminary ADT, not the final ADT. The highest delta between the traffic forecast and the opening-year count for Site 3 may derive from the usage of the preliminary estimate. Source: Final EIS, US-41 Memorial Drive to County M, Brown County, Wisconsin (Wisconsin DOT Project ID 1133-10-01), ftp://ftp.dot.wi.gov/dtsd/bts/environment/library/1133-10-01-F.pdf Figure II-13. Project study area (U.S.-41, Brown County).

Deep Dives II-39 The traffic forecasting accuracy improved after correcting the exogenous population forecast. However, the fuel price adjustment increased the PDFF. This change could have been accounted for in that the change in fuel price had little effect on the traffic volumes in the study area where public transportation is not a reasonable alternative mode. This interpretation could be wrong, however, because of the uncertainty in how the fuel price impact was implemented in the traffic forecast model. Availability of the archived model and its inputs would have provided deeper understanding of the parameters and methods used for forecasting traffic for the US-41 project. A small number of documents and data were available for the US-41 project. It is unknown whether risk and uncertainty were considered during the project due to the inaccessibility of the documentation on this project. For future forecasting efforts, it is suggested that a copy of the forecasting documentation and assumption be archived along with the travel model files used to generate the forecasts.