Transferability of Activity-Based Model Parameters (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

133 Concurrent to this project, members of the study team were involved in other, similar transferability tests as part of the FHWA STEP project, Making Advanced Travel Fore- casting Methods Affordable Through Model Transferabil- ity (Bowman et al. 2014). The overall approach used for estimation-based transferability tests in that project was similar to what was used for this project and reported in this document and in Appendix A. There were, however, some important differences: • The FHWA STEP project used 2009 NHTS data from six different regions, including the Tampa and Jacksonville regions in Florida, as well as the Sacramento, San Diego, San Joaquin, and Fresno regions in California. This pro- vided a more consistent comparison across regions, because the survey data were collected at the same time with the same survey instrument. • For the Tampa and Jacksonville regions (as well as Sacra- mento and Fresno), the parcel data for land use was aggre- gated to Census block-sized “microzones,” so that all six regions in the study would be using data defined at the same level of aggregation. (The San Diego and San Joaquin regions did not have parcel data available.) • For Jacksonville, the 2005 land-use data and skims were used rather than the new 2010 base year data. (That same 2005 data was used as the basis for the main C10A study.) • For Tampa, the 2010 land-use data and skims were used for the FHWA study, but at an earlier stage of development, so some refinement of the Tampa data has been done for this study since the time of the FHWA analysis. • The specifications of the original Sacramento models were simplified in some cases to make them more “estimable” on the data across the regions. Given those caveats regarding the differences between the FHWA STEP analysis and the analysis done for this project, it is useful to look at the summary of results from that study with regard to the transferability of the Tampa and Jackson- ville NHTS data versus the corresponding travel data from the other regions. Figure C.1 plots the differences between the Tampa and Jacksonville model estimates by model type in terms of the percentage of coefficients that are significantly different, insig- nificantly different, or not estimable in one region and/or the other. Note that the list of 14 models is slightly different from the list of 17 models tested for this project; and it includes two models (usual work location, person-day tour generation) that were not included in this project. As was found here and described in this report, the mode choice models have the most inestimable parameters, particularly the parameters for the bike and transit modes for work and school tours, which are generally not observed in the NHTS data. The largest numbers of significant differences are in the destination choice models (usual work location, other tour destination, and intermediate- stop location). That result suggests that in transferring and calibrating a model from another region, a good deal of atten- tion should be paid to the destination choice models and how well they can predict observed origin–destination patterns and replicate screen line data. The work-based sub-tour generation model has no signifi- cant differences because there are so few work-based sub- tours observed in the NHTS data, and no significant statistical relationships can be estimated (as already mentioned). For that reason, any other models related to work-based sub- tours were excluded from the FHWA analysis. In general, the tour generation models and tour time-of- day models transfer relatively well between the regions, with few inestimable parameters or significant differences. These types of models, which are generally focused on household and individual social organization more than on land use and accessibility, seem to transfer more readily across regions. a p p E N D I x C Transferability Tests for Six Regions

134 0% 20% 40% 60% 80% 100% Usual work location Auto ownership Person-day tour generation Exact number of tours Work tour time of day Work tour mode WB subtour generation School tour mode Other tour destination Other HB tour time of day Other HB tour mode Intermediate stop generation Intermediate stop location Trip time of day significant difference insignificant difference not estimable Figure C.1. Estimated differences between Tampa and Jacksonville coefficient estimates, by type of choice model. 0% 20% 40% 60% 80% 100%10% 30% 50% 70% 90% alt-specific constant person characteristic household characteristic day-pattern characteristic tour/trip characteristic impedance measure land use measure time schedule measure logsum from lower model significant difference insignificant difference not estimable Figure C.2. Estimated differences between Tampa and Jacksonville coefficient estimates, by type of variable. Figure C.2 is the same type of graph as Figure C.1, but the coefficients are classified in terms of the type of variable rather than the type of model. The land-use and impedance mea- sures have the greatest number of inestimable parameters, and these are mainly associated with the transit and bike modes in the mode choice models. The log-sum coefficients and land-use measures tend to show the highest percentage of significant differences—many of these are the mode choice log-sum effects and size variables in the location choice models. There are relatively few significant differences for person and household characteristics, or for those impedance variables (time and cost) that could be estimated.

135 Conclusions from the Initial FhWa STEp Model Transferability Research The paragraphs and charts that follow are from the Conclu- sions section of Bowman et al. (2014). In the FHWA study, although small sample sizes limited the abil- ity to draw strong conclusions about comparability among the four California regions and two Florida regions included in this study, there is some substantial evidence of comparability among them. This is shown in Figure C.3, where it can be seen that, for all regions, the differences from the two-state model (where the data were pooled across all six regions in both states) are insig- nificant for over 80 percent of the coefficients; however, Tampa stands out as less comparable than the others. This study did not identify the exact reason, although the socioeconomic data show a much higher presence of all-senior households in Tampa. The California regions are more comparable within state than across states, perhaps because of the presence of Tampa in the two-state comparison. The issue with Tampa draws attention to the likeli- hood that there may be factors that would cause two regions, even two regions within the same state, to be bad candidates for a model transfer. (Bowman et al. 2013). The FHWA study did not explore comparability for regions in states other than California and Florida; estimability and comparability for a full spectrum of sample sizes, especially samples with more than 2,500 households; or comparability in categories other than state boundaries, such as urban density, size, or socioeconomic make-up. For example, university towns or cities with a large seasonal retirement population may be distinctly different in ways that make transferring from other regions inadvisable, and this study lacks evidence to draw con- clusions one way or the other. These remain important avenues for further research. In some cases, there may be good reasons for transferring a model from a region that is not currently comparable if there is reason to believe that it will be comparable in the future. For example, a region may be growing rapidly and/or adding new travel options but lacks the data to develop a model that would serve it well even if it could conduct a very large household interview survey. The diversity of conditions needed to estimate the coefficients of the model simply may not exist within the region. In a case such as that, perhaps a model transfer should be considered. The FHWA study is also limited in its ability to determine what sample size is large enough for local estimation, because the largest sample in this study includes only 6,000 households and the rest are 2,500 or less. However, as shown in Figure C.4, where estimation results for each region are compared to the two-state combined models, the results show that a sample of 6,000 house- holds provides much better information for estimating activity- based model coefficients than samples of size 2,500 or less. It is also likely that sample sizes considerably larger than 6,000 would substantially improve estimation results, enabling significant coefficient estimates for important small population segments. Although it is not possible to make a definitive statement about the transferability of activity-based models based in the FHWA study, the study provides some new and unique evi- dence. Overall, although the strictest statistical tests (chi- squared test of model equality) usually rejected the hypothesis that models based on data from different regions are statisti- cally indistinguishable, it is also true that most of the individual coefficients are not significantly different from one region to the next. In addition, this study shows the substantial improve- ment of estimability that occurs with large survey samples. Based on these findings, the most important conclusion of this study is that, although estimation of models using a large local sample is best, it is better to transfer models that are based on a large sample from a comparable region than it is to estimate new models using a much smaller local sample. Figure C.3. Significance of parameter estimate differences between regional model and two-state model, by region (excluding inestimable parameters).

136 This conclusion does not mean, however, that metropolitan regions can relegate survey data collection and model devel- opment to the past and simply borrow a model from others who have gone before. Even if a comparable region and its model can be found, survey data should be collected for pur- poses of calibrating components of the model, such as activity and tour generation, which cannot be calibrated using traffic count data. References Bowman, J. L., M. Bradley, and J. Castiglione. 2013. Making Advanced Travel Forecasting Models Affordable Through Model Transferabil- ity. Final Report. Prepared for Federal Highway Administration, U.S. Department of Transportation, Washington, D.C. Bowman, J. L., M. Bradley, J. Castiglione, and S. L. Yoder. 2014. Making Advanced Travel Forecasting Models Affordable Through Model Transferability. Presented at the 93rd Annual Meeting of the Trans- portation Research Board, Washington, D.C., January. Figure C.4. Significance and sign of parameter estimates, by number of survey households.