Dynamic, Integrated Model System: Sacramento-Area Application, Volume 1: Summary Report (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

93 Conclusions and Lessons Learned The SHRP 2 C10B project developed and performed a limited set of tests for a completely dynamic, disaggregate travel demand and traffic and transit simulation model. The model was imple- mented using available software, mainly open source, and soft- ware developed for the project that is available through the National Academy of Sciences. The model was implemented and tested for the Sacramento, California, metropolitan area. The new integrated model uses available data as inputs. The data needs are similar to those used in existing planning and operational models. The socioeconomic and land use data inputs are the same as those used in the existing activity-based travel demand model used by SACOG, the Sacramento MPO. The highway network data requirements are consistent with those needed for traffic simulation. Note that those require- ments can be substantial at the regional level, and detailed actual data may have to be replaced with default data in some cases. The transit network data are generated directly from Google’s Gen- eral Transit Feed Specification (GTFS), which includes infor- mation for most major metropolitan areas in the United States. The model was designed to run on software and hardware configurations that would typically be available at most larger MPOs and state planning agencies. The software only runs on a typical Windows Server configuration. The testing of the new model was limited. A complete vali- dation of the new model was not performed so that project resources could be reserved for a series of tests in which the model was used to estimate the effects of various policy and transportation improvement scenarios. This means that the model will need further calibration and improvements to be used in realistic planning applications. What Users Need to Know to Run the C10B Integrated Model Users of the C10B integrated model should be familiar with the following: • Travel demand modeling concepts and procedures, and interpretation of model validation and outputs; • Traffic simulation modeling, particularly using the DynusT model and software, and interpretation of model valida- tion and outputs; and • The General Transit Feed Specification (GTFS). If the model run is to include MOVES, then familiarity with the MOVES model is also important. It is important to recognize that, as with any advanced travel model, users should have a solid understanding of travel demand modeling to be able to understand the operation of the model and to understand and interpret the model’s out- puts. The exogenous travel components of the model are rep- resented by conventional trip tables, and so an understanding of how trip tables are created is also necessary. DaySim is an activity-based model; since DaySim is a com- ponent of the C10B integrated model, users should have a fundamental understanding of the concepts of activity-based modeling. It is not necessary to be facile with all the details of the estimation of each model component; but the way in which individuals’ activity patterns and choices of destina- tion, mode, and time of day are realized in the model should be fully understood. Because the highway network in the C10B integrated model is maintained in DynusT, users need a thorough understand- ing of this simulation model as well. While most members of the project team had substantial expertise in travel demand modeling, only a few had significant experience using DynusT. Team members who were to perform model runs, particularly at SACOG, underwent multiday training sessions by Univer- sity of Arizona team members, who are among those who typically perform such training for other DynusT users. Even with the training, because DynusT was new to many team members, it took a substantial amount of time for them to become proficient enough to perform the network coding required to create model scenarios, and to examine and inter- pret DynusT outputs. New users of the C10B model who are not familiar with DynusT should be prepared to spend some time becoming familiar with it before proceeding and may C H A P T E R 5

94 want to consider getting the formal DynusT training offered by the University of Arizona. The original FAST-TrIPs transit network was developed by University of Arizona team members using the GTFS infor- mation for Sacramento. Since these team members were also the developers of FAST-TrIPs, the other project team mem- bers do not have a specific estimate for the level of effort to develop a complete FAST-TrIPs network. SACOG staff who performed the policy testing discussed in Chapter 4 were able to make the relatively simple edits required for Scenarios 1, 4, and 5. These edits, however, did not involve coding new routes; rather a route was deleted, hours of service were extended, and frequencies were changed. It should be noted that a frequency change is somewhat more involved than it would be in a transit network used in a conventional model with static assignment. Since each transit vehicle run is coded separately, an increase in fre- quency means adding a number of runs. For example, if average headways on a bus route were reduced from 20 min to 10 min over a 3-h period, nine new bus runs would have to be added to the existing nine runs. Since transit vehicle runs are not necessarily evenly spaced (for example, runs could be scheduled to start at 8:00, 8:16, 8:38, and 9:00), the user would have to decide when to start each new run. One might, say, add a new run beginning at the midpoint between existing runs (e.g., 8:08, 8:27, and 8:49). One task that was not done as part of the C10B project was coding a future year transit network. Since GTFS data would not be available for a future year, the transit vehicle runs would have to be generated, perhaps starting from an existing year network and adding, deleting, or revising routes and stops as needed. It might be necessary to examine future year highway speeds to help estimate the times for buses to travel from one stop to another. It should be noted that beyond the modeling terminology that is part of the C10B model user interface (UI), there is no specialized computing knowledge or experience necessary to run the model. The UI is similar to many other Windows- based software programs in that users create and modify sce- narios and examine the model’s reports through familiar concepts such as radio buttons, tabs, and drop down menus. Lessons Learned and Improvements Needed As previously mentioned, the testing of the new model was limited, and a complete model validation was not performed. Additionally, a number of challenges were experienced dur- ing the development, implementation, and testing of the C10B integrated model. Some of these issues were addressed fully or in part, while others could not be addressed within the schedule and resources available for the C10B project. These issues need to be addressed to make the model ready for real-world applications. This section discusses some of these challenges, how they were addressed during the project, and how they affect the model results. Some of them relate to areas of further devel- opment and research. Where applicable, recommendations for C10B model users in response to these challenges are presented. Model Validation In the early stages of the project, consideration was given to performing a full validation of the C10B model, similar to what might be done for a travel model that would be used by an MPO for transportation planning. This full validation would have included comparisons to observed data for the base year of 2005 as well as SACSIM model results, and sen- sitivity testing using a forecast or backcast year. This concept was abandoned because other delays left too little time at the end of the project to perform both a conventional model vali- dation and sensitivity testing and the planned policy testing. It was decided that the policy testing would proceed, and con- ventional model validation and sensitivity testing would not be performed. The model testing that was performed, as discussed in Chapter 3, focused on “proof of concept,” meaning that the results were examined mainly using aggregate measures, and extensive calibration of the model was not performed. It was obvious that there were some issues in the C10B model results that would have required further work on the model had it been intended for use in an actual transportation planning setting. These included the following. An underestimation of transit ridership. For 2005 the C10B model estimated fewer transit riders than SACSIM and fewer than the observed ridership for that year. It is unclear why this would happen, given that the inputs to DaySim were the same as for the original 2005 SACSIM model and the revisions made to DaySim as part of the C10B project were not focused on transit. FAST-TrIPs uses the transit tour and trip outputs of DaySim, and so that component of the integrated model would not result in any change in the number of transit tours. Given that some of the policy tests did focus on transit, it would be desirable to determine the cause of this difference before doing any additional work with the model. Lower highway speeds. The C10B model resulted in lower travel average speeds (about 8 to 10 mph) for all roadway types at all times of day. This may be a result of the more realistic representation of traffic dynamics than in the static traffic assignment process used in SACSIM. The output speeds in SACSIM do not seem particularly high (though the original model validation did not include speed comparisons to observed data). More examination (i.e., comparisons with

95 observed speeds) is warranted. It should be noted that traffic simulation models such as DynusT have various parameters that represent driver behavior which were not calibrated for the C10B project. Temporal distribution of travel. Despite the differences in output speeds, the C10B model did not show substantial differences from SACSIM in terms of total travel (for exam- ple, as measured by vehicle-miles) or trip lengths. The dis- tribution of travel by time of day, however, did differ noticeably from the SACSIM results. In particular, there was more travel in the evening period in the C10B model and less travel at other times of day, especially the midday period. It is possible that the lower speeds being fed back from DynusT resulted in a shifting of travel (in DaySim’s time-of-day choice models) to less congested periods. The finer temporal resolution of the C10B integrated model points out an area where additional validation and calibra- tion are necessary. Convergence As discussed in Chapter 3, it was found that after running three big loop iterations, each of which includes 10 DynusT iterations, the systemwide model convergence reached a plateau that did not improve with more iterations. It was found that three big loop iterations result in a systemwide convergence level between 10% and 15%, meaning that— on average—the number of trips between each zone pair changes by no more than 10% to 15% between successive big loop iterations; that is approximately what can be achieved by DynusT in 10 iterations in the Sacramento implementation. This is not a particularly stringent convergence level for either static or dynamic traffic assignment models. The rela- tively high convergence level may well have affected the results of the policy tests described in Chapter 4. Test 3 was rerun using more iterations of DynusT because the conver- gence level for Scenario 3 was even higher than the level achieved in the baseline tests; but this did not substantially improve the convergence level. It would make sense to perform more tests to see if better convergence can be achieved in the simulations and what types of model changes might be considered beyond simply running more loops or iterations to improve convergence. Noise in Model Results As noted in Chapter 4, it appears that the “noise” in the C10B integrated model made it difficult to identify some of the changes in travel behavior related to the tested scenarios. All simulation models, of course, are noisy since they are proba- bilistic in nature, and model results vary from one run to another. But there are two components to the simulation involved in the C10B integrated model: the activity-based demand model (DaySim) and the traffic and transit simula- tion (DynusT/FAST-TrIPs). There has not been an examina- tion of the propagation of the noise due to this double simulation approach. Since SACOG is using an activity-based demand model for its planning purposes, the modelers are familiar with the issues of simulation noise. Before the C10B project, they had estimated the noise level in SACSIM/DaySim and used this information in their planning process. Such an assessment should be made with the C10B integrated model before it is used in a practical setting. In theory, a simulation model should be run multiple times with the results averaged to get the noise to an accept- able level. This seldom happens in practice with current activity-based models in the United States, even with static highway and transit assignment procedures. It may be neces- sary to consider doing this for integrated models. Run Time The run time for the model as used in the policy tests by SACOG was about 70 h, for three big loops with 10 iterations of dynamic traffic assignment with DynusT within each loop. While this is a bit longer than advanced models using static assignment in larger metropolitan areas, it is quite reasonable given the limited time and resources available for making the model more efficient. A model with runs times such as this would be practical in most settings. It is important to point out that run times could be longer if some of the other issues already discussed were addressed. For example, the number of big loops and DynusT iterations was decided based on tests that showed a lack of improvement in convergence with additional iterations and loops. Running the model for more iterations and loops might be expected to pro- duce a tighter convergence, and perhaps if some of the valida- tion issues were addressed, this could be achieved. However, this could not be tested within Project C10B. It is also important to note that run times would be greater, of course, in regions larger than Sacramento. Even in Sacra- mento, run times would be longer for future year scenarios in which the number of persons to be simulated would be greater and the higher levels of congestion might require additional loops and iterations to converge. Further improvements to the run time of the C10B integrated model should be investigated. The discussion of run times, however, should note that the C10B integrated model has been designed to run on a server that a U.S. MPO might typically have available. Another way to reduce run times is to run the model on bigger, faster computers, as has been done with other complex transpor- tation models.

96 Future Applications and Additional Research There are a number of areas for future research that follow from the work on SHRP 2 Project C10B. Further work could be performed related to the challenges cited in the previous section: • Model validation. Further work is needed to determine the level of effort required to achieve a full model validation consistent with industry standards. Additionally, further discussion is warranted about what specifically should constitute the validation of an integrated model such as this. The effects of using a fully validated model in policy testing should also be examined. • Convergence. A tighter level of convergence than could be achieved during Project C10B is highly desirable. It is unknown whether the ability to achieve better convergence was limited by the nature of the integrated model, the way in which DynusT works, the characteristics of the trans- portation system and travel demand in the Sacramento region, or some other factors. It would be valuable to examine what level of convergence can be achieved in the C10B model and what types of model changes might be considered beyond simply running more loops or itera- tions to improve convergence. • Noise in model results. Performing multiple model runs would provide useful information on measuring the mag- nitude of the noise related to the simulations in the C10B integrated model. It would be worthwhile to compare esti- mates of the noise to those associated with the activity- based model alone, to get a handle on the propagation of noise related to the multiple simulations that are part of the integrated model. Another area of valuable research would be tests to determine the number of model runs required to achieve stable results for a variety of types of planning analyses. • Run times. Several areas of further work would provide useful information regarding run times. A detailed examination of the run times for various model components could help determine where the bottlenecks in the model stream are; then ways of making those areas more efficient could be examined. The effects of different convergence levels on run times could be tested. The effects of greater demand and higher congestion levels on run times would be useful to examine. Additionally, the effects of more powerful hardware configurations on run time could be examined. There are other areas where further research could help make models like the C10B integrated model more useful and practical. These include the following: • Decreasing the learning curve. As discussed previously, it took substantial time and effort for project team mem- bers—especially those at SACOG, who performed most of the work on the policy testing of the model—to become familiar enough with the workings of the model (particu- larly DynusT and FAST-TrIPs since they were already familiar with SACSIM) to be able to efficiently and effec- tively perform the policy tests. While there are many prac- titioners familiar with traffic simulation, a greater number of transportation professionals need to be proficient in demand modeling and traffic simulation if models such as these are to become more widely used. There will need to be more organized training opportunities available for plan- ners, such as those currently provided by government and educational organizations for travel demand modeling. • Testing the model in other geographic areas. Now that the effort to develop the integrated model and the software to run it is complete, it is important to gather information on how well the model would perform in other areas. It would be particularly useful to test the model in places that are larger or notably different from Sacramento. It would be interesting to know how long such a test would take and the level of effort required to get the model up and run- ning. Developing the regional highway network for dynamic assignment is one area known to require significant effort; staff training is another. Determining what other areas require substantial effort and what differences might arise in other areas may point to requirements that were not rel- evant in Sacramento.