Transferability of Activity-Based Model Parameters (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3Motivation for the Study The motivation for this extension to the C10A report, Dynamic, Integrated Model System: Jacksonville-Area Application, was to demonstrate the transferability of activity-based model speci- fications between regions. The model that was developed origi- nally for Jacksonville, Florida, was intended to demonstrate the integration of an activity-based travel demand model, DaySim, with a time-dependent network supply model, TRANSIMS. The DaySim model components were specified using param- eters from a version of the DaySim model originally developed for the Sacramento Council of Governments (SACOG), cali- brated to local survey data from the National Household Travel Survey (NHTS) Florida Add-On Survey for the Jacksonville region. It was beyond the scope of the original contract, however, to formally test whether the transferred model parameters produced meaningful differences from what could have been obtained using a parameter set developed from local data. Using the Sacramento model parameters with Jacksonville socioeconomic and network data inputs, the transferred model produced plausible results when compared with calibration target values. This was an important finding in the original study. The study team theorized that this was because of the activity-based model’s more detailed representation of travel behavior, linking trips into tours and daily patterns while pro- viding greater spatial and temporal resolution. Recognizing that there could be significant cost savings for agencies that would like to develop advanced models and were willing to transfer parameters from another region, this study focused on the question of study model transferability. Overview of the Study The work in this study involved creating two new versions of the DaySim model, one for Jacksonville and a parallel version for the Tampa Bay region. New sets of parameters were devel- oped using the NHTS Florida Add-On Survey, which was used for both model estimation and calibration, as described later in this report. For model estimation, the original plan for the study was to pool the NHTS data for estimation. However, concurrent work on the Federal Highway Administration’s Surface Trans- portation Environment Planning and Cooperative Research Program (FHWA STEP) project suggested that Jacksonville was actually more similar to Sacramento than it was to Tampa in terms of travel patterns and sociodemographics, which was revealed through preliminary model estimation exercises. With that knowledge, it did not make sense to pool the data for estimation. That would have produced results that were a blend of two dissimilar regions. This realization led the project team to estimate and test Jacksonville and Tampa specifications independently against the Sacramento model and against each other. The sample sizes for the North Florida Transportation Plan- ning Organization (NFTPO) and for District 7 of the Florida Department of Transportation (FDOT-D7) were individually too small to estimate parameters for all of the models and vari- ables in the Sacramento DaySim specification. However, the study team felt they were adequate, individually, for calibrating to regional target values for all of the original Sacramento model components. This led the study team to undertake a research exercise in which estimation of new parameters from the NHTS sample data would allow the team to draw inferences from the outcomes about similarities between regions. The pro- duction version of each regional model system started from the original Sacramento model system, which has been used by SACOG for several years and is known to be a behaviorally robust model. The NHTS sample data were then used only to develop Jacksonville and Tampa regional calibration target values for various DaySim model components. Calibration of activity-based model components typically means adjusting constants in choice models or, in the case of destination choice models, coefficients on impedance terms to match trip lengths. To encourage the regions to participate, the models were created as updates conforming to each region’s current base C h a p T E R 1 Background

4model year, networks, and zone systems. Thus, NFTPO in Jacksonville and FDOT-D7 in Tampa Bay received new regional activity-based modeling systems, using the DaySim design ready for operational use. Both agencies expressed interest in using the models in upcoming long-range transportation plan (LRTP) updates. The initial SHRP 2 C10 study funded the development of a Jacksonville-area model using existing 2005 inputs and, for a limited geographic scope, covering four counties. The 2010 regional modeling area actually encompasses six counties. For NFTPO, this meant expanding the previous Jacksonville model from four counties to six and developing 2010 land-use and socioeconomic data and network models. Similarly, for FDOT-D7, this meant updating its model to a 2010 base year, which included five counties and a small portion of a sixth. For greater compatibility with DaySim, both regions devel- oped multiperiod highway assignment methods rather than maintaining their current all-day assignments. In addition, both regions updated auxiliary demand models related to external flows, truck traffic, and visitors. Like the Sacramento model, both NFTPO and FDOT-D7 models utilized the Cube Voyager travel demand modeling package, minimizing the need to con- sider differences in assignment algorithms or other software specifics when making comparisons. This enabled the study team to estimate and calibrate parameters using common path-building methods. assessing Transferability The goals of this project were to assess the transferability of parameters between regions and to provide guidance to agency modelers who might want to do the same in order to avoid the costs of a potentially expensive household interview survey (HIS) and parameter estimation work. The availability of the NHTS data and Florida Add-On Survey was viewed as an economical alternative to larger-scale survey collection efforts; and a secondary objective of this study was to deter- mine to what extent the NHTS data would be useful for this purpose. To determine the practicality of transferability, the study team sought to assess transferability from the standpoint of both statistical differences in parameter values and model performance in validation tests. In doing so, the team focused on answering the following questions: • What are the statistically significant differences between the DaySim model parameter estimated for the Sacramento model specification and the same parameters estimated using the Jacksonville and Tampa models? • Will there be any model components that cannot be esti- mated due to smaller sample sizes? • Will there be significant differences between Jacksonville and Tampa parameters, when the data are sufficient to estimate those parameters? • Can a DaySim model system, using parameters transferred from Sacramento, produce credible results such that calibra- tion to Jacksonville or Tampa regional target values can pass validation tests? • How much calibration is required to provide acceptable results? • What are the barriers to transferring a model and how might they be overcome? The answers to these questions have implications for the validity of transferring a model as a general strategy. The study team expected to find that some parameter values for certain model components could be significantly different from one region to the next due to different levels of transit availability. This is especially true in the case of values derived from region-specific transportation system supply variables, such as transit skims. In addition, the spatial distribution of activity centers is region specific, which may lead to signifi- cant differences in mean trip lengths and propensities toward nonmotorized travel. At the same time, the team expected that long-term choices (such as auto ownership, basic household structures, and indi- vidual daily activity-travel patterns—including time spent in various activities and the times of day when individuals engage in these activities) should be very similar between two regions and should lead to very similar parameter estimates. This assumes that relevant cultural norms and business hours are more or less the same for people living in cities of similar size and density in the United States. To be transferable, a model should not have too many statistically significant differences; the similarities should dominate the predicted behavioral pat- terns such that only a modest amount of calibration is needed to overcome supply-side differences in transportation system level of service and differences in the spatial distribution of activity centers. However, if the level of effort required to achieve a calibrated and validated model approaches the level of effort required to estimate parameters from scratch, then it may be concluded that transferring a model is not an eco- nomical model development strategy. Confounding Factors A number of important factors must be taken into consider- ation when making an assessment of model transferability because they confound a scientific comparison. 1. The survey and count data used to estimate and calibrate the transferred model, the source of the variable specification, will be different from the survey data used to calibrate and

5 validate the recipient models, varying in both quality and sample validity. For example, the transferred Sacramento model was developed from a 2000 survey of 3,942 house- holds. The 2008–2009 NHTS Florida Add-On Survey cap- tured responses from 1,335 households in the Jacksonville region and 2,517 households in the Tampa Bay region and did not include separate records of household members younger than 5 years of age (FHWA 2009). Moreover, many of the household diary days in the Jacksonville and Tampa Bay samples were weekends, which could not be used for estimating models of daily patterns, tours, or trips due to the weekday focus of the model specification. 2. The effort put into the development of the source model specification will play a large role in its transferability. The Sacramento DaySim model has a very rich variable specification in nearly every model component. On one hand, this is generally viewed as beneficial, because it explains more variation in behavior across the population. To the extent that household and person-level decision making with respect to activities and travel is similar from one region to the next, this should enable the source model specifi- cations to produce results in the recipient model system that demonstrate similar behavioral patterns and levels of variation among households and persons with the same attributes. A specification that was too sparse or simplistic would not be expected to transfer well, because it would leave out important explanatory variables. On the other hand, some model components may have been over-specified in the Sacramento model, meaning they unintentionally capture idiosyncrasies of the survey sample or supply-side variable inputs that are not true measures of activity-travel behavior decision making. This might lead to distortions in travel patterns when applied in another region. 3. The first two factors—when combined—can lead to a situ- ation in which the survey data used to calibrate and validate the recipient model may not be adequate in every dimension to handle the specification provided by the source model. The study team recognized this as a minor concern in calibrating and validating the Jacksonville model and, to a lesser extent, the Tampa Bay regional model; the relatively small number of observations in the NHTS Add-On samples and the missing observations on very young household members may not provide adequate benchmarks across the market segments specified in the Sacramento model. This mismatch in survey sample quality and quantity was an even greater concern to the team for the purposes of estimating Jacksonville- and Tampa-specific model param- eters corresponding to the Sacramento model variable spec- ifications. The team anticipated that many model variables would not be estimable due to insufficient variation in observations and that significance levels would be low. 4. An additional factor with a large influence on model performance is the network-based supply models, both highway and transit, and the quality of the land-use and socioeconomic data inputs. The presumption is that the agency, and external consultants who work on these model components, are faithfully modeling existing conditions to the extent possible, thus minimizing errors in routing, travel times, and the locations of households and activity centers; however, the possibility of errors in these processes must be acknowledged. Further, since traffic counts and transit boarding counts are central to the validation of travel models, it is important to minimize logical inconsistencies that are often identified when, for example, counts on adja- cent links are in sharp disagreement. It is also important to recognize that the counts themselves are subject to error, particularly when collected over a limited time span. To synthesize, the principal challenge in assessing the validity of a model transfer exercise is to determine • How much the differences we observe in model behavior and statistical fit are due to true differences in the underlying behavior between regions; • How much is due to differences in transportation systems and urban spatial structure; and • How much is due to confounding factors related to estima- tion and calibration approaches, survey sample quality, and supply-side models and data. The remainder of this report attempts to untangle these factors in the context of this study and to provide guidance to modeling practitioners who may be contemplating an activity- based model transfer for their region. To begin, a summary is provided of the research approach used to develop the two model systems in preparation for the analysis.