Dynamic, Integrated Model System: Sacramento-Area Application, Volume 1: Summary Report (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1The second Strategic Highway Research Program (SHRP 2) Project C10B, Partnership to Develop an Integrated, Advanced Travel Demand Model with Fine-Grained, Time-Sensitive Networks: Sacramento-Area Application, is an important step in the evolution of travel modeling from an aggregate, trip-based approach to a completely dynamic, disaggregate methodology. In this project, an existing disaggregate activity-based model was integrated with an existing traffic simulation model to create a new, completely disaggregate model. At the same time that travel demand models have been evolving, traffic simulation models— which simulate the movements of vehicles through a highway network—have become more sophis- ticated due to improvements in computing. The product of SHRP 2 Project C10B is an integrated model that simulates individuals’ activity patterns and travel and their vehicle and transit trips as they move on a real-time basis through the transportation system. It produces a true regional simulation of the travel within a region, for the first time using individually simulated travel patterns as input rather than aggregate trip tables to which temporal and spatial distributions have been applied to create synthetic patterns. A unique feature of this model is the simulation of transit vehicles as well as individual person tours using transit. The new integrated model has been developed and implemented for the entire Sacramento, California, region. The integrated model components include (1) SACSIM, the regional travel model maintained by the Sacramento Area Council of Governments (SACOG), the regional met- ropolitan planning organization (MPO), and (2) DynusT, a mesoscopic traffic simulation model developed by the University of Arizona. SACSIM includes an activity-based demand model, Day- Sim. The transit simulation is performed by FAST-TrIPs (Flexible Assignment and Simulation Tool for Transit and Intermodal Passengers), also developed by the University of Arizona. The integrated model also includes the ability to run MOVES (Motor Vehicle Emission Simulator), the air-quality analysis program developed by the U.S. Environmental Protection Agency (EPA). The testing of the new model was limited. A complete validation of the new model was not performed so that project resources could be reserved for a series of tests in which the model was used to estimate the effects of various policy and transportation improvement scenarios. This means that the model will need further calibration and improvements to be used in realistic planning applications. While the C10B integrated model produces results that are reasonable for regional travel pat- terns and behavior within the limits of the testing that was done, the true value of the model is its ability to provide analysis results that demonstrate sensitivity to policy variables more accu- rately than models that use aggregate demand or assignment procedures. This sensitivity was tested through a series of policy and project tests conducted by SACOG, using the new integrated model and the existing SACSIM model with aggregate assignment. The objective is to address key policy and investment questions by implementing an inte- grated, advanced travel demand model with fine-grained, time-dependent networks; the ideal Executive Summary

2approach is to combine the capabilities of an activity-based travel demand model with a traffic simulation model, adding enhancements to achieve goals such as the consideration of reliability in travel choices. Project C10B implemented this approach by using the SACSIM travel demand model for the Sacramento area; SACSIM includes the original DaySim activity-based model and the DynusT mesoscopic traffic microsimulation model. The integrated model was tested in the Sacramento metropolitan area, which is the 27th largest in the United States and has all of the desirable characteristics for testing the new model. The Sacramento Area Council of Governments (SACOG) served as the public agency partner for this project. SACOG is the designated MPO for the metropolitan area and is responsible for all transportation planning in the region. Model Components The components of the integrated model developed in the SHRP 2 C10B project include SACSIM, DynusT, FAST-TrIPs, and MOVES. SACSIM is a complete travel demand model that SACOG uses for planning in the Sacramento region. The demand for personal travel within the region is modeled by DaySim, an activity-based demand model. DaySim incorporates a variety of model features, including • The ability to model each person in the Sacramento region separately through the use of a popu- lation synthesizer that creates a synthetic population representing each person and household in the region; • The ability to model the complete daily activity pattern for each individual, including the number and sequencing of activities defined by seven purposes; • A series of logit destination, mode, and time-of-day choice models at the tour and trip levels to simulate the choices for each individual; • Estimation of the start and end times of all activities and trips to the half-hour level of resolu- tion; and • Parcel-level spatial resolution for home and activity locations. Other components of SACSIM are used to model, at an aggregate level, the remaining com- ponents of regional travel—including travel into, out of, and through the region (external travel); truck travel; and travel to and from Sacramento International Airport. DynusT is a traffic simulation model that is used in a number of areas and lends itself well to integration with both SACSIM and MOVES. DynusT is a true disaggregate simulation model that can track individual vehicles through the network—consistent with tracking traveler activi- ties in a travel demand model. Furthermore, DynusT is a true dynamic traffic assignment (DTA) model that takes into account both the spatial and temporal effects of congestion. Travelers departing at different times are assigned to routes calculated on the basis of the traveler’s actual experienced travel time, which is a critical capability for establishing a consistent and reliable traffic assignment outcome. FAST-TrIPs is a model that assigns transit passengers within the transportation network and loads those passengers in a dynamic (time-sensitive) simulation of actual travel. FAST-TrIPs is a regionwide dynamic transit assignment model that determines an individual-specific transit route for each transit traveler in the system; it takes into account published transit schedules and transit vehicle run times that are congestion responsive and are provided by the traffic simulation component of DynusT. FAST-TrIPs deals with both transit-only and park-and-ride trips and is able to maintain multiple constraints associated with activity time-windows and the choice of modes in multimodal travel tours. MOVES is the next-generation mobile source emission model developed by the EPA. MOVES serves as a single comprehensive system for estimating emissions from both on-road and nonroad mobile sources. It replaces EPA’s MOBILE model as the approved model for state implementation

3 plans (SIPs) and regional or project-level transportation conformity analyses. MOVES is designed to estimate emissions at scales ranging from individual roads and intersections to large regions. MOVES is a database-driven model—inputs, outputs, default activities, base modal emission rates, and all intermediate calculation data are stored and managed in MySQL database. Software Approach The software architecture for the integrated model allows users to access the modeling software using a web browser, with the major model components running on one or more shared servers. This allows for the efficient sharing of large data files, alleviates the need for every modeler to have a powerful desktop computer, and enables analysts to use parallel processing or other tech- niques as necessary to ensure adequate performance. The software architecture is efficient, mod- ular, and maintainable, and it reduces the risk of changes to one model component affecting the operation of the model as a whole. The software was developed using an iterative, incremental methodology that reduced risk, ensured continuous testing, and makes progress more transparent and predictable. The devel- opment approach has made virtually the entire suite of C10B products available to the trans- portation community. SACSIM and DynusT are available under open-source licenses, and the National Academy of Sciences (NAS) is the owner of all new software. While the tests of the model used some input data from SACSIM that were developed using a proprietary modeling software package licensed to SACOG and Cambridge Systematics, Inc., the operation of the integrated model does not require any commercial travel demand modeling or simulation software. Model Component Revisions To meet the objectives of the SHRP 2 C10B project, some revisions were made to the original models that make up the integrated model. Incorporation of Variable Value of Time One of the shortcomings of aggregately applied models is the need to use average values across broad market segments for key parameters. One example is the use of a single value of travel time for each segment for decisions about mode and route choice. The value of time determines the extent to which travelers will pay to save travel time. This is an important factor in estimating how many and which travelers might use toll roads or managed lanes. Because an activity-based model simulates each individual, an individualized value of time for each individual can be simulated from a probability distribution. Since it incorporates both activity-based demand modeling and traffic simulation, the C10B integrated model provides an opportunity to use individual values of time throughout the modeling process. The approach used was to revise the tour mode choice model in DaySim, preserving as much of the existing model as possible. Variable value of time is achieved in the new mode choice models by specifying a distribution for the in-vehicle time coefficient, in this case a lognormal distribution. Due to the lack of travel cost variations in the survey data set used to estimate the original tour mode choice models (there are no toll roads in the Sacramento region), value-of-time distributions were transferred from the San Francisco region. Incorporation of Reliability A method was developed for including reliability in the C10B analysis framework. As an input, reliability affects travelers’ decisions about trip making and the choice of destination, mode, and

4route. It can be thought of as an extra impedance to travel over and above the average travel time generally used in demand models. Note that the original model’s definition of average travel time is based solely on recurring (demand and capacity) conditions. Considering reliability means that nonrecurring sources of congestion factor into the process. The concept of “extra impedance due to unreliable travel” is probably the best way to incor- porate reliability into the modeling structure as an input. SHRP 2 Project L04, Incorporating Reliability Performance Measures in Operations and Planning Modeling Tools, used this approach: the impedance on a link can be captured as a generalized cost function that includes both the average travel time and its standard deviation (which is used as the indictor of reliabil- ity). Because Project L04 was not complete at the time of the relevant work in Project C10B, travel time equivalents were used, as discussed in Chapter 2. The Project C10B method uses the DynusT output to estimate recurring delay and a sketch plan- ning method to estimate incident delay, then combines the two. The steps are (1) compute the recur- ring delay for each link in hours per mile from the simulation model; (2) compute the delay due to incidents in hours per mile using the lookup table for a 1-h period; and (3) compute the overall mean Travel Time Index, which includes the effects of recurring and incident delay. Model Integration Integration of SACSIM and DynusT/FAST-TrIPs The outputs from SACSIM that are inputs to DynusT/FAST-TrIPs in the C10B integrated model are • The tour and trip rosters from DaySim; and • The trip tables representing exogenous travel. The tour and trip rosters already include most of the information required as inputs by DynusT, including the origin and destination of each trip and relevant traveler information such as the simulated value of time. The time of day is also provided, but only at the half-hour level for trips. In the C10B model, a random start time for each trip is simulated within the appropriate half- hour period. The conversion of the rosters to the input format required by DynusT is performed within the integrated model software. The exogenous travel trip tables must be converted to trip rosters for input to DynusT. This is done using existing procedures for processing trip tables in DynusT. SACSIM has trip tables for each of four aggregate time periods (a.m. peak, midday, p.m. peak, and night). Departure time profiles from traffic count data were used to define start times for each trip in the roster. The C10B integrated model is run in an iterative manner until convergence is achieved. Before each “big loop” after the first, the travel time information from DynusT and FAST-TrIPs from the previous big loop is fed back as input to SACSIM. The feedback process is somewhat complex because the traffic and transit simulation in DynusT/FAST-TrIPs represents nearly continuous time while the inputs to DaySim are in half-hour intervals, and the inputs to the exogenous travel components of SACSIM are for the four broad time periods. Furthermore, each trip in DynusT/ FAST-TrIPs has its own trajectory through the network, with its travel time based on the condi- tions confronted continuously through its journey. There is no single “travel time” from one point in the network to another in DynusT. A specialized process to compute the travel times to be fed from one big loop to the next was developed for the C10B integrated model. Stated simply, the feedback process employed in the C10B model combines information from all relevant trajectories within a time period (half hour or broad period) to estimate an average time to use for input to SACSIM. The integrated model software executes this process.

5 Integration of DynusT and MOVES A significant feature available in MOVES is the ability to support quantitative project-level emis- sions assessments using detailed vehicle activity data. The MOVES project-scale analysis func- tion is the most spatially resolved modeling level in MOVES; it calculates emissions from a single roadway link, a group of specific roadway links, and/or an off-network common area (e.g., a transit terminal or park-and-ride lot). To take advantage of these detailed activity data with improved temporal and spatial resolution, the project team developed a fine-grained integrated method that links MOVES to DynusT at the individual roadway link level. The MOVES-DynusT integration is realized through data conver- sion functions that use DynusT activity outputs to generate MOVES project-scale inputs. This integration method ensures transition of data flow from DynusT to MOVES without manual intervention or additional data preparation. Model Application When applying the C10B model, there are a few key points to consider: • The DynusT application is resource intensive on all fronts: CPU, memory, and disk space. • In addition to the DaySim and DynusT applications, there are a number of scripts that run to perform various data management functions. • The MOVES application is somewhat independent of the more tightly coupled loop between DaySim and DynusT. It runs separately on data processed from the final output of DynusT and does not necessarily have to be installed at the same time as DaySim and DynusT. The MOVES installer will also install MySQL. The model software only runs on 64-bit Windows (e.g., Windows 7, Windows Server 2008). Python and the DBF Python library should be installed before installing DaySim and DynusT. The model was designed to run on hardware configurations that are typically available at most larger MPOs and state planning agencies. The specific requirements for the C10B integrated model are as follows: • Memory: minimum 8 GB, 16 GB preferred. The configuration on which SACOG ran the policy tests included 32 GB. • CPU: minimum four cores Intel Core i5 or better. Up to 16 cores will significantly improve performance. SACOG’s configuration included Intel Core i7-3770 CPU @ 3.40 GHz. • Hard drive: 15 GB of data are generated per run. Data are written and read back in for each iteration of DynusT, so a solid state drive is recommended to improve performance. SACOG’s configuration included a solid state drive. All software can be installed and run from the same server. However, the MOVES application and support software (MySQL) could be installed and run on a separate server from the server running DaySim and DynusT if desired. Using the development configuration of Windows 7 Professional running on Intel Core i7-2600 CPU @ 3.40 GHz with 16 GB random access memory (RAM) and a 128 GB solid state drive (SSD), run times averaged a bit over 25 h per big loop. SACOG reported run times of 70 h for the policy test runs with three full feedback loops, with its slightly larger/faster configuration. Model Testing SACSIM had been validated by SACOG for a base year of 2005, prior to the beginning of the C10B project. While SACOG has continued to update SACSIM as part of its regional trans- portation planning process, it was not necessary for the purposes of the C10B project to

6implement any updates to SACSIM that took place after C10B began. The SACSIM compo- nent of the integrated model was therefore considered already validated when the project commenced. The main difference between the integrated model and SACSIM was the replacement of the static highway and transit assignment processes with the dynamic simulation processes, DynusT and FAST-TrIPs, respectively. To demonstrate that the C10B integrated model was suitable for testing the policy/planning alternatives, the project team identified a proof-of-concept plan to test the integrated model, consistent with the overall focus of the project. The testing conducted under this refined plan was designed to • Identify and measure the impact of the integration of SACSIM and DynusT on SACSIM results. The integrated model does not change the basic design or structure of the demand components of SACSIM/DaySim. Thus, under the proof-of-concept plan, it is sufficient to identify changes in SACSIM results that stem from integration of SACSIM with the DynusT assignment procedures. • Determine whether or not the SACSIM/DynusT procedure is iterating to closure. Is it getting closer to or further away from observed transit volumes, traffic volumes, and traffic speeds? • Measure the reasonableness of the traffic and transit assignment results. Of course, the rea- sonableness of the assignment results depends somewhat on the impact that the SACSIM/ DynusT procedure has on SACSIM. All testing was conducted for the 2005 base year using the same socioeconomic, land use, and network data used by SACOG for SACSIM. In addition, observed traffic and transit data used for the validation of SACSIM were available. As previously discussed, a complete validation of the C10B integrated model was not conducted. Analysis of Policies and Alternatives of Interest to Planning Agencies A set of five policy tests was conducted to demonstrate that the C10B integrated model is capable of analyzing the types of policies and alternatives that are part of typical urban transportation planning processes. The objective was to produce reasonable results in a real-world environment for typical transportation planning policies. With this in mind, it was decided that SACOG would perform the analyses at its offices, using its hardware and staff, with assistance from other team members. The idea was to get a sense of the type of effort that would be required for a plan- ning agency to perform these types of analyses using the integrated model. For each of the five tests, the results of a particular scenario related to a change in the trans- portation system were compared with the results from a baseline scenario, which was the same for all tests. The baseline represented year 2005 conditions in the Sacramento region. All sce- narios were run using the C10B integrated model; most scenarios were also run using the origi- nal SACSIM model validated for the region. The set of five policy and investment alternative scenarios analyzed were defined by SACOG. Note that while the scenarios are realistic and typical of the types of policies and scenarios that SACOG analyzes in its transportation planning function, they are not actual projects under consideration in the Sacramento region. The policy test scenarios were as follows: 1. Extending transit service coverage—extending the end of transit service for a bus route from 6:00 p.m. to midnight; 2. Improving interchange design—an operations-oriented interchange improvement project; 3. Providing freeway bottleneck relief—adding a fourth general purpose lane to a heavily con- gested freeway river crossing connecting to downtown Sacramento;

7 4. Increasing transit frequency—reducing service headways from 30 min to 10 min on a well- used bus line; and 5. Deleting bus line—deleting a well-used bus line. Limitations on project resources resulted in taking some shortcuts in the analyses and in the preparation of the C10B integrated model. These issues are described in the following bullet points. It is hoped that further research with this type of integrated model can assist in assessing the effects of these issues and their practical implications. The issues include the following: • Perhaps the most significant issue was the limited validation of the C10B integrated model. This resulted in some significant differences in the baseline scenario results between the C10B model and SACSIM, some of which were in the vicinity of the transportation system changes under study. This made comparison of the model results difficult in some cases. • Another limitation was the level of convergence achieved in DynusT. The test results implied that there was still substantial “noise” in some C10B integrated model results, which affected the ability to fully evaluate the test results. There is also, of course, noise in SACSIM because it includes an activity-based demand model that simulates individual travel behavior. But there is more noise in the C10B integrated model because it includes SACSIM as well as the traffic and transit simulation components in DynusT and FAST-TrIPs. • Another issue was that each test was run only once with each model. Ideally, simulation models should be run multiple times to get a handle on the level of noise in the results. SACOG has done this with its own validated version of SACSIM, but multiple runs were not possible within the project schedule. As the results show, some of the results appear to be questionable due to the noise level in the C10B integrated model, which is greater than in SACSIM because of the additional traffic and transit simulation components. The test results are summarized in the following sections. Summary of Test 1 In Test 1, the C10B integrated model behaved plausibly in an aggregate sense, shifting trips to the transit and walk modes from the auto modes and showing reasonable sensitivity and magnitude of response while maintaining a relatively constant level of demand. Boardings on the route for which service was extended increased while boardings on nearby routes declined. A significant part of the added boardings occurred in the extended service period between 6:00 p.m. and midnight. Even with the level of noise in the C10B model, it seems unlikely that the entirety of the model response is indistinguishable from random noise because the mode shifts and changes in boardings on individual routes are nearly all in the correct direction. In terms of localized effects, however, the C10B integrated model showed only a minor impact on transit trips. The temporal shifts are also counterintuitive because trips shift from the period when the service is extended. Summary of Test 2 In this scenario, the highway system reverted to an earlier state when a key interchange improve- ment was removed. Highway capacity was therefore lower, resulting—as expected—in a higher level of congestion in the affected area in both models. The higher level of congestion apparently caused some travelers to shift to transit. Overall, the C10B integrated model was more sensitive to congestion than SACSIM, shifting a significantly greater number of travelers from peak peri- ods to adjacent time periods. The SACSIM results showed reductions in all time periods (though very small reductions) rather than any noticeable peak spreading. Interestingly, the C10B model showed a smaller reduction in trips on work tours than on nonwork tours, which is consistent

8with the notion that the work tours are more inelastic. SACSIM, on the other hand, showed a greater reduction in trips on work tours. It is unclear whether the magnitude of the sensitivity of the C10B integrated model is reason- able; the SACSIM results seem too inelastic. The C10B model seems very sensitive in terms of shifts in demand from peak periods, although the relative inelasticity of the SACSIM results does not provide a worthwhile basis for comparison. The assignment results for both SACSIM and the C10B integrated model for five key roadways show changes in the expected direction, although the predicted volume levels and the magnitude of the impacts vary among roadways. Summary of Test 3 In Scenario 3, an additional general purpose lane was incorporated on a congested segment of the Capital City Freeway, which is the most congested freeway in the region. Both models showed a small increase in the total amount of regional travel, with the C10B integrated model showing a larger increase. However, in the SACSIM model, this increase was mainly concentrated near the vicinity of the improved highway; in the C10B model, destinations near the improvement decreased while they increased farther from the improvement. The C10B integrated model results are different from SACSIM for this segment. Both the baseline and Scenario 3 show slower speeds and higher travel times than SACSIM. It is unknown in which model’s results the speeds and volumes are more accurate. Both SACSIM and the C10B model show higher volumes on the widened highway for the test, and both show added delay in the downstream segments. The C10B model shows that the impact of higher volumes on the downstream segments is much greater than the impact shown in SACSIM, however. In other words, by widening the crossing segment, delay is reduced on that segment, but that improvement is offset by much higher delay downstream. Thus, according to the C10B model results, widening the bridge segment alone would be nearly net-zero in delay reduction. This scenario anticipated that the increased capacity would result in a higher number of trips to the affected area, both spatially and temporally. However, such an impact is seen only in SACSIM and not in the C10B integrated model. For this particular scenario, perhaps less than ideal convergence in the C10B model may have left the model with too many localized sources of instability and congestion, which have distorted the final outcome. In the last overall iteration, the study team used a higher number of DynusT iterations; this improved convergence and reduced excessive congestion but did not eliminate the unexpected results. Summary of Test 4 In this test, SACSIM produced an unexpectedly large shift in ridership on the route with the increased frequency. It is not clear why this occurred in SACSIM because the mode choice model should not be overly sensitive to headway assumptions, and the same mode choice model was used in the C10B integrated model. Nor should the static transit assignment process be overly sensitive to headway assumptions. This result is particularly puzzling given that the C10B integrated model had a larger overall increase in transit demand (5% compared with 3% for SACSIM). Exam- ining the reasons behind the unusual SACSIM results was beyond the scope of this project, but the C10B integrated model results were, for whatever reason, more reasonable. Both models showed about the same (reasonable) shifts in ridership from nearby routes. Summary of Test 5 In contrast to the results of Test 4, which used the same transit route as its basis, the results of Test 5 were more reasonable for SACSIM than for the C10B integrated model. The deletion of the route should have resulted in a decrease in overall transit ridership, but in the C10B model,

9 the opposite occurred. Both models did show increases in ridership on nearby routes, as expected. There were some unusual results in SACSIM away from the deleted route, making some direct comparisons difficult. Conclusions The SHRP 2 C10B project developed and performed a limited set of tests for a completely dynamic, disaggregate travel demand and traffic and transit simulation model. The model was implemented using available software, mainly open source, as well as software developed for the project that is available through the National Academy of Sciences. The model was implemented and tested for the Sacramento, California, metropolitan area. The new integrated model uses available data as inputs. The data needs are similar to those used in existing planning and operational models. The socioeconomic and land use data inputs are the same as those used in the existing activity-based travel demand model used by SACOG, the Sacramento MPO. The highway network data requirements are consistent with those needed for traffic simulation, although those requirements can be substantial at the regional level, and detailed actual data may have to be replaced by default data in some cases. The transit network data are generated directly from Google’s General Transit Feed Specification (GTFS), which includes information for most major metropolitan areas in the United States. The model was designed to run on software and hardware configurations that would typically be available at most larger MPOs and state planning agencies. The software only runs on a typical Windows Server configuration. A user of the C10B integrated model should be familiar with the following: • Travel demand modeling concepts and procedures, and interpretation of model validation and outputs; • Traffic simulation modeling, particularly using the DynusT model and software, and interpreta- tion of model validation and outputs; and • The GTFS. If the model run is to include MOVES, then familiarity with the MOVES model is also important. DaySim is an activity-based model, and since it is a component of the C10B integrated model, users should have a fundamental understanding of the concepts of activity-based modeling. It is not necessary for a user to be facile with all of the details of the estimation of each model com- ponent, but the user should fully understand the way in which individuals’ activity patterns and choices of destination, mode, and time of day are realized in the model. Because the highway network in the C10B integrated model is maintained in DynusT, the user needs a thorough understanding of this simulation model. While most members of the project team had substantial expertise in travel demand modeling, only a few had significant experience using DynusT. Team members who would perform model runs, particularly at SACOG, under- went multiday training sessions by University of Arizona team members. Even with the training, it took a substantial amount of time for the new users to become proficient enough to perform the network coding required to create model scenarios, and to examine and interpret DynusT outputs. New users of the C10B model who are not familiar with DynusT should be prepared to spend some time becoming familiar with it. University of Arizona team members developed the original FAST-TrIPs transit network using the GTFS information for Sacramento. Since these team members were also the developers of FAST-TrIPs, the other project team members do not have a specific estimate for the level of effort to develop a complete FAST-TrIPs network. SACOG staff who performed the policy testing were able to make the relatively simple edits required for Scenarios 1, 4, and 5. These edits, however, did not involve coding new routes; rather, a route was deleted, hours of service were extended, and frequencies were changed.

10 It should be noted that beyond the modeling terminology that is part of the C10B model user interface (UI), no specialized computing knowledge or experience is necessary to run the model. The UI is similar to many other Windows-based software in that users create and modify sce- narios and examine the model’s reports through familiar concepts such as radio buttons, tabs, and drop-down menus. Lessons Learned and Model Improvements Needed As previously noted, the testing of the new model was limited, and a complete model validation was not performed. Additionally, a number of challenges were experienced during the develop- ment, implementation, and testing of the C10B integrated model. Some of these issues were addressed fully or in part, while others could not be addressed within the schedule and resources available for the C10B project. These issues would need to be addressed to make the model ready for real-world applications. Model Validation In the early stages of the project, consideration was given to performing a full validation of the C10B model, similar to what might be done for a travel model that would be used by an MPO for transportation planning. This full validation would have included comparisons to observed data for the base year of 2005, as well as SACSIM model results and sensitivity testing using a forecast or backcast year. This concept was abandoned because other delays left too little time at the end of the project to perform both a conventional model validation and sensitivity testing, and the planned policy testing. It was decided that the policy test- ing would proceed, and conventional model validation and sensitivity testing would not be performed. The model testing that was performed focused on “proof of concept,” meaning that the results were examined mainly using aggregate measures, and extensive calibration of the model was not performed. It was obvious that some issues in the C10B model results would have required fur- ther work on the model had it been intended for use in an actual transportation planning setting. These issues included the following: • An underestimation of transit ridership. For 2005 the C10B model estimated fewer transit riders than SACSIM and fewer than the observed ridership for that year. • Lower highway speeds. The C10B model resulted in lower average travel speeds (about 8 to 10 mph) for all roadway types at all times of day. • Temporal distribution of travel. The distribution of travel by time of day in the C10B integrated model results differed noticeably from the SACSIM results. Convergence It was found that after running three big loop iterations, each of which included 10 DynusT itera- tions, the systemwide model convergence reached a plateau that did not improve with more iterations. Three big loop iterations resulted in a systemwide convergence level between 10% and 15%, meaning that on average the number of trips between each zone pair changes by no more than 10% to 15% between successive big loop iterations. That is approximately what can be achieved by DynusT in 10 iterations in the Sacramento implementation. This is not a particularly stringent convergence level for either static or dynamic traffic assign- ment models. The relatively high convergence level may well have affected the results of the policy tests. It would make sense to perform more tests to see if better convergence can be achieved in the simulations and what types of model changes might be considered beyond sim- ply running more loops or iterations to improve convergence.

11 Noise in Model Results It appears that the “noise” in the C10B integrated model made it difficult to identify some of the changes in travel behavior related to the tested scenarios. All simulation models, of course, are noisy since they are probabilistic in nature, and model results vary from one run to another. But there are two components to the simulation involved in the C10B integrated model: the activity- based demand model (DaySim) and the traffic and transit simulation (DynusT/FAST-TrIPs). The propagation of noise due to this double simulation approach has not been examined. Since SACOG is using an activity-based demand model for its planning purposes, they are familiar with the issues of simulation noise. Before the C10B project, SACOG had estimated the noise level in SACSIM/DaySim and used this information in the planning process. Such an assessment should be made with the C10B integrated model before it is used in a practical setting. In theory, a simulation model should be run multiple times with the results averaged to get the noise to an acceptable level. This seldom happens in practice with current activity-based models in the United States, even with static highway and transit assignment procedures. It may be necessary to consider doing this for integrated models. Run Time The run time for the model as used in the policy tests by SACOG was about 70 h, for three big loops with 10 iterations of dynamic traffic assignment with DynusT within each loop. While this is a bit longer than advanced models using static assignment in larger metropolitan areas, it is quite reasonable given that limited time and resources were available for making the model more efficient. A model with runs times such as this would be practical in most settings. It is important to point out that run times could be longer if some of the other issues already discussed were addressed. For example, the number of big loops and DynusT iterations was chosen on the basis of tests that showed a lack of improvement in convergence with additional iterations and loops. More iterations and loops might be expected to produce a tighter conver- gence, and perhaps if some of the validation issues were addressed, this could be achieved. How- ever, this could not be tested within Project C10B. It is also important to note that run times would be greater in regions larger than Sacramento. Even in Sacramento, run times would be longer for future-year scenarios in which the number of persons simulated would be greater, and higher levels of congestion might require additional loops and iterations to converge. Further improvements to the run time of the C10B integrated model should be investigated. Future Applications and Additional Research There are a number of areas for future research that follow from the work on SHRP 2 C10B: • Model validation. Further work is needed to determine the level of effort required to achieve a full model validation consistent with industry standards. Additionally, further discussion is warranted about what specifically should comprise the validation of an integrated model such as this. The effects of using a fully validated model in policy testing should also be examined. • Convergence. A tighter level of convergence than could be achieved during Project C10B is highly desirable. It is unknown whether the ability to achieve better convergence was limited by the nature of the integrated model, the way in which DynusT works, the characteristics of the transportation system and travel demand in the Sacramento region, or some other factors. It would be valuable to examine what level of convergence can be achieved in the C10B model and what types of model changes might be considered beyond simply running more loops or iterations to improve convergence.

12 • Noise in model results. Performing multiple model runs would provide useful information in measuring the magnitude of the noise related to the simulations in the C10B integrated model. It would be worthwhile to compare estimates of the noise with those associated with the activity-based model alone, to get a handle on the propagation of noise related to the multiple simulations that are part of the integrated model. Another area of valuable research would be tests to determine the number of model runs required to achieve stable results for a variety of types of planning analyses. • Run times. Several areas of further work would provide useful information regarding run times. A detailed examination of the run times for various model components could help determine where the bottlenecks in the model stream are; then, ways of making those areas more efficient could be examined. The effects of different convergence levels on run times could be tested. The effects of greater demand and higher congestion levels on run times would be useful to examine. Additionally, the effects of more powerful hardware configura- tions on run time could be examined. There are other areas where further research could help make models like the C10B integrated model more useful and practical. These include the following: • Decreasing the learning curve. As discussed previously, it took substantial time and effort for project team members, especially those from SACOG (who performed most of the work on the policy testing of the model), to become familiar enough with the workings of the model— particularly DynusT and FAST-TrIPs (they were already familiar with SACSIM)—to be able to efficiently and effectively perform the policy tests. While many practitioners are familiar with traffic simulation, more transportation professionals need to be proficient in demand model- ing and traffic simulation if models such as these are to become more widely used. There will need to be more organized training opportunities available for planners, such as those currently provided by government and educational organizations for travel demand modeling. • Testing the model in other geographic areas. Now that the effort to develop the integrated model and the software to run it is complete, it is important to gather information on how well the model would perform in other areas. It would be particularly useful to test the model in places that are larger or notably different from Sacramento. It would be interesting to know how long such tests would take and the level of effort required to get the model up and running. Devel- oping the regional highway network for dynamic assignment is one area known to require significant effort; staff training is another. Determining what other areas require substantial effort and what differences might arise in other regions may point to requirements that were not relevant in Sacramento.