Transferability of Activity-Based Model Parameters (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

29 A broader understanding of transferability includes under- standing how model specifications and available data should be appropriately matched. Typically, transferring a model means borrowing specifications (such as model structures, variables, and parameters) from a source model so that the recipient agency does not have to develop these specifications itself, usually for lack of funding and available local data. If the recipient agency had good survey data and funding to estimate its own model parameters, then it would do so, and this process would not be considered a transfer. This project followed the usual process of transferring a model specification from another region but then went one step further by attempt- ing to estimate new model coefficients for the same source specification. Two different questions of transferability have been researched in this study: • Does the model represent local travel behavior? This question has been researched by reestimating each model component of the transferred model using local data and comparing it with the original model components. • Can the model reproduce observed local data? This question has been researched by calibrating each component of the transferred model to local data and comparing it with the original model components. There are no standard practice tests for transferability; most transferred models have relied on the second question to deter- mine credibility for a transferred model. Both questions are important to providing credibility for travel forecasting. Reestimation Tests The reestimation transferability tests provided some impor- tant lessons learned regarding local travel behavior (and the coefficient values that represent this behavior): 1. Models estimated with small sample surveys may not pro- duce reliable coefficient estimates. 2. Models estimated with small samples for rare household types or travel behaviors may not adequately represent these behaviors. 3. Models estimated without key data available will be limited in forecasting these behaviors. 4. Key differences in coefficient values are more difficult to discern with smaller sample size surveys. Despite these problems with small sample sizes, this study was able to identify statistically significant differences in enough model components to begin to characterize the travel patterns in the Tampa region as being heavily influenced by lifestyles that are significantly different from those of Sacramento and Jacksonville. Looking across pairings of regional models in which the same parameter was significant in both regions, there were proportionally far more differences in the Tampa- Sacramento pairing than either Jacksonville-Sacramento or Tampa-Jacksonville. These differences pointed to the influence of that region’s large population of retirees as evidenced by significant effects of retiree-household and single-driver- household variables, single-auto households, and a lower consideration of the presence of children on escort tour des- tination choices. In addition, the models estimated for the Tampa region had significantly higher propensities toward leisure tours as well as lower propensities toward work tours and shared rides involving more than two persons. This study also illustrated how certain types of activity- based model components need to be treated differently when transferring a model system. Models representing household- level and person-day pattern choices tend to be specified using many alternative-specific constants, or groups thereof, interacted with various household attributes requiring esti- mation of separate coefficients. In such models, reference cases influence parameter significance (difference from zero), which can be tricky when comparing regions. Such models are challenging to estimate even with larger sample sizes for complex multidimensional choices; this is often the case with C h a p T E R 4 Conclusions

30 day-pattern models and departure and arrival time choice models, which may include many nonsignificant parameters that are retained to maintain theoretical continuity between household and person market segments. If small sample sizes are a concern as they were in this study, it would be better to borrow these models from another region for which param- eters were estimated using a sufficiently large sample. Such models could be calibrated using local data that can be aggre- gated to provide target values. With day-pattern models, including tour and stop frequency models, the analyst should look for important market segments that may have been left out of the original specification or underrepresented, such as the retiree market segment in the Tampa region. Time-of-day choice models, such as tour and trip departure and arrival times, should be fairly stable from one region to the next and can probably be transferred with little extra calibration required, other than adjustments to meet time-period-specific demand during validation. In contrast, tour- and trip-level decisions (such as destina- tion and mode choices) make greater use of transportation level-of-service variables; these vary over alternatives and can be estimated using fewer generic coefficients, which makes it easier to obtain statistically significant outcomes. Mode choice models are often easier to calibrate, as they typically utilize a small number of alternative-specific constants. Extra care should be taken if there are nesting parameters, such as found in the work-tour-mode choice model here, because regional differences might go undetected. Sensitivity testing is recom- mended postcalibration to determine whether nesting coeffi- cients, which rescale mode utilities, are appropriately sensitive to changes in level-of-service variables. Although relatively easy to estimate, destination and loca- tion choice models can be tricky because they do not utilize alternative-specific constants and are typically calibrated to trip-length distributions. When applied in forecasting, work and school location choice models can be doubly constrained using shadow pricing or similar methods, allowing for a greater degree of control over trip tables. Other/nonmandatory tour and trip purposes are singly constrained, however, which provides less control when forecasting trip tables. Origin–destination patterns are inherently region specific. If it is suspected that there are significant differences in desti- nation choice model specifications, then it would be better to reestimate nonmandatory-tour and intermediate-stop desti- nation choice models using local data, provided there is a sufficient local sample. In this research, the study team found that the NHTS sample size was insufficient to reestimate many of the model com- ponents found in the original Sacramento specification. For purposes of delivering production-ready versions of model systems to both regions, the study team concluded that it would be better to start with the Sacramento specification, which was at least a holistic description of variation in regional travel behavior across a representative population, rather than to piece together versions of models that were a partial blend of estimated parameters from multiple regions. In particular, the team found that the NHTS samples lacked adequate representation of certain submarkets, such as young children, and also underreported evening and non-work travel and non-auto modes. Calibration Tests The work conducted during the model calibration tests pre- sents a systematic and statistical approach to determining the transferability results from a calibration effort to match observed local data. The calibration effort succeeded in trans- ferring a behaviorally rich model that was estimated elsewhere and calibrating its behavior to local data. Despite some sam- pling deficiencies in the NHTS survey, the study team was able to identify shortcomings in the model outputs and supplement the NHTS data with other data sources in what can best be described as an iterative calibration and validation exercise. In general, the upper-level models require more calibration than lower-level models, because they are more sensitive to regional differences and because the lower-level models are affected by the improved results of the upper-level models. The process of working from the top down in a model system in which the upper-level models condition lower-level model outcomes would seem to be appropriate. That process places more emphasis on the model components that have the great- est effect on overall system performance, and these upper- level com ponents tend to be better supported in terms of available data. Socioeconomic variations in the population can have a sig- nificant impact on the ability to transfer a model. For example, the Tampa region has a large percentage of retired persons that are not as evident in Sacramento and are therefore not adequately represented by the variables in the Sacramento model. Work and school travel tends to be more transferable than other travel, because the differences in socioeconomic factors are more important for non-work/school travel than they are for work and school travel. For these reasons, some socioeconomic interaction effects that were left out of the Sacramento specification were inserted and calibrated in both the Tampa and Jacksonville models. Accordingly, it will be important for agencies that transfer a model from another region and then calibrate it to match local targets to study the specification of the source model to see if there are any impor- tant demographic or travel context—variables that seem to be missing or that may be important in their region. Figuring out what may have been left out of the model may reveal defi- ciencies and opportunities to improve forecasting.

31 Final Recommendations This study also provided useful information for future house- hold interview survey (HIS) data collection. The smaller sample sizes and more limited data in the NHTS data did not produce reliable reestimation of the Sacramento models. Many variables were constrained due to the smaller sample sizes, and the results were therefore not conclusive regarding transferability. Using a household survey sample that is appro- priately sized to the task of estimating an activity-based model is an important lesson of this study. This research pushed the limits of how far one can go with a limited sample size and a complex model specification. It is not always clear how large a sample is large enough until one gets into the analysis. When this project was conceptual- ized, the study team expected that the NHTS data would not be sufficient to estimate some of the model parameters in the original specification but expected greater concurrence than was obtained. There were many models to consider and many parameters representing different behavioral proclivities to explore. Moreover, the data source was an important context for this study. The NHTS data, which included add-on samples for Florida, have been offered to the public as a source of HIS data that could be used for model estimation. Various parties involved in this project were interested in testing the data for the development of an activity-based model. Moreover, NHTS is generally viewed as an economical alternative for agencies that lack the means to conduct their own HIS. This project showed that for a more complex activity-based travel model specification, the NHTS sample did not provide suffi- cient variation across enough model dimensions to make it useful for parameter estimation. Despite this limitation in the NHTS data, there would seem to be more to gain than to lose by transferring a model from another region, followed by calibration to local target values, provided the regions are similar enough in terms of their lifestyles. A great deal may be learned just by running an activity/tour-based model and going through the calibration and validation exercises. Even regions that develop an activity- based model from scratch using a large local survey sample will spend many months, if not a couple of years, getting to know their model system and fine tuning its behavior. There are two remedies for the mismatch between model specification and the sample data used to develop it: larger and more robust survey samples or simpler and more parsimonious model specifications. When models are specified and estimated from scratch, the ability to estimate statistically significant parameters governs the richness of the model specification. With a transferred model, however, the specification of the source model may or may not be supported by the data avail- able to reestimate or calibrate the recipient region’s model. There can be some deficiencies in a traditional HIS in how questions are asked, how responses are coded, and consistency checking. For this study, the lack of data on young household members was problematic, which is a sampling design issue. The most profound issue, however, was simply the lack of enough observations to represent enough of the variation in travel patterns that were desired for the model design. The necessary variation can be found in a traditional HIS, as was used in Sacramento and other places. When designing a sampling plan for a traditional HIS that will be used for a trip-based model, there is careful delineation of targeted numbers of households of various demographic levels. These attributes and stratification often correspond to traditional trip-based model trip-generation segmentation. For activity-based surveys, stratified random sampling is also needed; but stratifying by all of the household and person attributes that one would like to have for, say, a day-pattern model, can be challenging. Given the limited resources that most agencies face, more strategy is required in designing the sampling plan due to the effort and cost needed to ensure adequate representation across these segments. In addition, a greater emphasis on non-auto modes sometimes requires targeted oversampling or choice-based sampling. Finally, when transferring a model from one region to another, it is a good idea to summarize the HIS and supporting data that were developed to estimate the source model along key demographic and travel dimensions (e.g., household size, age, income, auto ownership, race; trip purposes, lengths, time- of-day distributions, and mode shares) to determine how simi- lar these basic measures are to the recipient region. Assuming that the transfer involves a borrowed specification that is cali- brated to local target values, the recipient agency may want to consider borrowing models from regions that are more similar to its own and that have sufficiently parsimonious specifica- tions to avoid using a model that was perhaps overspecified or too region specific. In addition, the recipient agency should consider what sources of calibration data are available and to what extent they can produce a set of calibration target values that is up to the task presented by the transferred model specification.