Integrated Transportation and Land Use Models (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

45 The third common model type (spatial input-output models) is based on economic theory and assumes that demand and supply are at equilibrium. Microsimulation discrete choice models (Chapter 5) also move toward equilibrium (households will move toward less expen- sive dwellings, everything else being equal, and landlords will increase prices for dwellings in high demand in the long run), but adjustments in microsimulation discrete choice models are slower and more time-lagged than in spatial input-output models. Although input-output models have been criticized for assuming equilibrium conditions, a fundamental benefit of the input-output model approach is that prices for housing and non-residential floorspace are modeled endogenously. Although plausible price models are a noteworthy challenge for microsimulation discrete choice models, input-output model approaches endogenously use prices for market clearance. In input-output models, prices are in equilibrium with demand and supply. 6.1 Model Concept The input-output model approach was first introduced by Leontief (1936) who received the Nobel Prize for this work. Alonso (1960) combined this concept with the bid-rent theory, which assumes that interested users are bidding for a site similar to an auction, and the highest bidder will buy the land. This way, Alonso developed a cohesive economic theory explaining the rela- tionship between land value and land use. Alonso’s model assumes a featureless plain with all land being of equal quality that is freely bought and sold. Buyers and sellers have a perfect knowledge about the land market and try to maximize their profits. There is one city center only where all market activity occurs. The land rent at any location is a function of selling price minus production costs minus transport costs. The model works equally for population and employment (an example is given below for firm location choice). A different bid-rent function evolves for every firm (straight lines in the far- left diagram in Figure 6-1). The further the business has to move from the market center, the less it would be willing to pay for land because of increasing transport costs. Different firms may trade off land prices and transportation costs differently, which leads to different slopes for Firms 1, 2, and 3 shown in Figure 6-1. At distance d1, d2, and d3, the maximum distance is reached for Firms 1 through 3, respectively. The center diagram shows how each business would start bidding for locations. As soon as this bidding line hits the red line of the equilib- rium land price (far-right diagram), the firm has found a location at dc. The red line shows the equilibrium land rent p(d) resulting after all firms are located. This bid-rent approach has not only been used to allocate firms, but to select housing loca- tions and workplaces. For the housing market, households “bid” on certain dwellings with a certain price, and the bidder with the highest offer will be selected. The market clearing process C H A P T E R 6 Spatial Input-Output Land Use Models

46 Integrated Transportation and Land Use Models maximizes the utility of all households, given a certain price structure. At the same time, prices are maximized for sellers (or landlords), given the current demand structure. In input-output models, the location chosen for each firm balances land costs with transport costs necessary to obtain inputs (that is, raw materials and labor) and reach markets (that is, buyers of goods or services). Similarly, the location choice of households balances housing costs with travel time to work and other location factors. As a result, highly accessible locations obtain higher market clearing prices. Competing demand bids up land prices and captures the market pressures that increase development densities. For several spatial input-output models, part of the input-output flow rates can be interpreted as labor flows and goods flows. These models explicitly define how much labor is necessary to produce one unit of output, and thus, calculated labor flows from population to employment may be interpreted as work trips. Similarly, flows between industries represent the flow of goods by different commodities. These flows may be translated into goods flows to model freight. Some of the most relevant operational land use models that use input-output and bide-rent approaches include MEPLAN (Echenique, Crowther, and Lindsay 1969), MUSSA (Martínez 1996), PECAS (Hunt and Abraham 2003), SEAM/SLUM (Moeckel, Costinett, and Weidner 2008), and TRANUS (de la Barra and Rickaby 1982). 6.2 Interviews Three in-depth interviews were conducted for this approach to model land use. The Atlanta Regional Council (ARC) has implemented a PECAS land use model and fully integrated this model with an activity-based transport model. Ohio DOT initially implemented a PECAS model, but later moved toward a modeling suite called SEAM/SLUM, which also followed the basic input-output model approach. The Metropolitan Council in Minneapolis/St. Paul has implemented one of the CUBE Land models. CUBE Land was built using the MUSSA model. MEPLAN and TRANUS are two further models based on the input-output model approach; however, most of these models were implemented outside of the United States and Canada, and therefore, are outside the focus for this synthesis report. 6.3 Model History ARC ARC has many years of experience running integrated land use/transport models. In the 1990s and early 2000s, ARC used the gravity-based model DRAM/EMPAL (Putman 1967) to account Figure 6-1. Bid prices and price structure equilibrium (Alonso 1964).

Spatial Input-Output Land Use Models 47 for the land use/transport interactions. DRAM/EMPAL was a useful tool during those years, but today’s models are more theoretically rigorous and powerful. Hence, ARC sought to explore alternative land use models suiting their needs. Ultimately, ARC evaluated and selected PECAS as the land use model. Implementing PECAS and integrating it with ARC’s transportation model took about 3 years. After 3 years, ARC was able to run a scenario to analyze transit-oriented development (TOD) with the integrated land use/transport model. More recently, a graduate student at Georgia Tech has worked on an UrbanSim implementation for the ARC region. Having a second model implemented in the Atlanta Region should be very interesting for model result comparisons. Ohio DOT Ohio DOT started their integrated land use/transport model at the statewide level in the early 2000s. Initially, they were looking for an automated method to break down county control totals to zones. A users’ needs study was conducted, which revealed strong interest in how transporta- tion might affect the economy. Ohio DOT decided to implement a PECAS model because PECAS is built on strong economic theory. PECAS was not, however, intended to run land use scenarios. When the PECAS model was implemented, the modeling software was still in its infancy. For a long time, Ohio and Oregon and later the Baltimore Metropolitan Council were the only agencies outside of academia that tried to implement the PECAS model. It turned out to be more complex than anticipated, and data collection, calibration, and runtime were issues affecting integration. In 2007, Ohio DOT decided to halt implementation of PECAS. Instead, a newly developed model, SEAM/SLUM (Moeckel, Costinett, and Weidner 2008), was implemented. SEAM/SLUM was built to generate the same output files as PECAS, but the model design was considered simpler given that it was designed, implemented, and calibrated within 2 years. Originally written in Java, this land use model later was translated into Citilabs’ scripting language for integration with Ohio’s CUBE-based transport model (Figure 6-2). SEAM/SLUM has been used by Ohio DOT ever since. Metropolitan Council The Metropolitan Council did systematic research before selecting a land use model. Several model developers, academics, and consultants were invited to report on various models and dis- cuss the pros and cons of many models. The staff at the Metropolitan Council also did a compre- hensive needs assessment. Available data, staff, and resources were considered carefully before selecting a model. Given the short timeframe, CUBE Land seemed to offer the best combination in terms of their wish list and available resources. CUBE Land offered the benefit of working at the TAZ level. Although UrbanSim can be implemented at the TAZ level, its developer gener- ally has advised using parcels. Furthermore, the transport model of the Metropolitan Council is implemented in CUBE Voyager, which offered fairly seamless integration with CUBE Land. The Metropolitan Council started with model implementation in summer 2010. At the begin- ning of 2013, they were able to start forecasting a regional policy scenario. A preliminary forecast was developed and shared with the communities in the MPO. Communities reviewed the model results and, based on their feedback, some revisions to input data were made. Council staff also made manual adjustments to some forecasts based on community comments. In a few cases, the forecast changed quite a bit. The so-adjusted forecasts were used as the final forecast and fed into the transport model for the Long-Range Transportation Plan. Land use forecasts were not only provided for the transport model, but for wastewater planning, parks planning, and land use planning. For infrastructure improvements planned by the communities, the effects on population and employment were modeled with CUBE

48 Integrated Transportation and Land Use Models Land by the Metropolitan Council. These forecasts were adjusted based on community com- ments and were then used by the communities in their comprehensive plans. 6.4 Model Implementation and Application Effort ARC ARC has worked closely with the model developer to implement and maintain PECAS in the Atlanta Region. About four times a year, the model developer comes to Atlanta to review model implementations and to work with ARC staff on model enhancements. Staff members see these regular visits of the model developer as key for successful model applications. To independently inform model applications, ARC formed a technical advisory panel to review implementation of the PECAS model. The 12 panel members consist of bankers, real estate experts, economists, academics from Georgia Tech and the University of Georgia, utility providers, and representatives from Georgia DOT and the transit agency (MARTA). The panel had two important purposes. First, group oversight helped generate higher quality model out- put. ARC was able to identify unrealistic model results early and made corrections in the model before using land use forecasts for planning purposes. Second, the panel substantially increased regional knowledge about and acceptance of the PECAS model. ARC implemented PECAS for the 20-county area. The model may provide annual model results, and ARC updates forecasts about twice a year. The base year is, in line with ARC’s activity-based model, 2015, and forecasts are modeled through the year 2040. ARC discovered through several iterations of model data improvements that PECAS, as is common for land use models, is sensitive to the availability of developable land, the accuracy of parcel data, and the zoning applied at the parcel level. ARC gave special attention to these constraints and refined Figure 6-2. Interface to run the Ohio integrated land use/transport model (Source: Ohio DOT).

Spatial Input-Output Land Use Models 49 allocations have improved the reasonability of model output. Figure 6-3 shows the flow diagram of the PECAS model implementation for ARC. Parcel-level data provided by local jurisdictions are the main input data of the model. ARC also obtained the Quarterly Census of Employment and Wages (QCEW) employment dataset for the base year. At ARC, one person works on the economic forecast model, REMI, which provides regionwide forecasts of employment and population for PECAS. Two staff members are available to work on the PECAS model, and provide some managerial guidance. Overall, the implementation of the land use model cost roughly $1M, including consulting fees. Ohio DOT The initial attempt to implement PECAS in Ohio was a several million-dollar program, but this number is not representative, given that PECAS was not fully developed yet. Under the same consulting contract, SEAM/SLUM was developed as a PECAS replacement at no additional cost for Ohio DOT. The model input data update for 2010 required about ¼ year of one full-time equivalent staff member. In addition, US$ 50,000 to 80,000 were spent in consultant fees for this model update. Today, the model is run in house, and no consulting support is necessary to use the integrated land use/transport model. The model is not used to analyze specific land use scenarios at this point. SEAM/SLUM is run in a fully integrated mode whenever the transport model is started. Therefore, almost no staff time is allocated to support the land use model. Metropolitan Council At the Metropolitan Council, the core team for land use modeling consists of three staff members plus management. For model development, one person worked full time on this project, one person worked 50%, and a GIS expert worked one-third of the time on this CUBE Land implementation. Citilabs, the provider of the CUBE software, supported model devel- opment from summer 2010 through March 2013 through consulting contracts. The total Figure 6-3. Flow diagram for PECAS implementation for ARC (Source: ARC).

50 Integrated Transportation and Land Use Models consulting fees for almost 3 years of support were US$ 247,000. At the Metropolitan Council, the land use model is not hosted by the transport department but by researchers in Community Development. Therefore, some additional staff time had to be provided by transport planners to realize the integration with the transport model. Most input data were developed in house by staff members, which helped keep consulting fees comparatively low. A few additional datasets were purchased, including the following: • The local association of realtors provided parcel-level data on characteristics and sales prices. • A sample of anonymized data from the Department of Motor Vehicles was combined with parcel data and tied to demographic and housing data. • A database of past and present commercial real estate listings. No consulting support is needed at the Metropolitan Council to run the integrated land use/ transport model or to analyze results. 6.5 Land Use/Transport Model Integration ARC ARC maintains an activity-based transportation model that runs in 5-year increments. PECAS, which runs on a year-by-year basis, is fully integrated with the activity-based model. The activity-based model provides zone-to-zone travel time skims to PECAS that are kept unchanged for every 5-year interval. Vice versa, PECAS provides forecasts of socioeconomic data, both population by household size and income categories and employment by sector, to the activity-based model. In the activity-based model, PECAS forecasts inform the synthetic population generator to synthesize micro data that respects PECAS output as zonal control totals. PECAS also provides commodity flow data used to model truck traffic. In addition, the econometric regional model (REMI) provides control totals of population and employment for the entire study area. PECAS can freely reallocate sociodemographic data across the entire study area according to scenario settings. In contrast to models that implement sublevel control totals (e.g., at the county level), the study areawide control totals enable full policy sensitivity. Ohio DOT The SEAM/SLUM land use model is fully integrated with the Ohio Statewide Transportation Model. For every year in which the transportation model is run, usually in 5-year increments, the land use model is used to disaggregate socioeconomic forecasts. The transportation model provides a switch to turn off the land use model and use static forecasts, but more commonly, the land use model is run and modeled forecasts are used by the transport model. Population forecasts are provided at the county level by the Department of Development, which forecasts population using a cohort-survival model. SEAM/SLUM has to respect county-level population forecasts and allocates those forecasts to TAZs within each county. Employment forecasts are given for the entire state and can be freely allocated across all zones within the state. Initially, SEAM/ SLUM provided commodity flows as well. This specific output has been replaced by data from the Freight Analysis Framework (FAF), which can be downloaded free of charge at http://faf.ornl.gov. In contrast to ARC, Ohio DOT constrains their land use model to county-level control totals. Although the ARC setup without county-level controls provides added model sensitivity and likely more realistic responses to scenario analysis, Ohio DOT uses the land use model with a different motivation. Ohio DOT uses the model primarily to disaggregate socioeconomic data to zones, not to analyze land use scenarios.

Spatial Input-Output Land Use Models 51 Metropolitan Council The Metropolitan Council has fully integrated their CUBE Land model with their four- step transportation model. For integrated model runs, the transportation model was simpli- fied (called internally the abridged transportation model) to save on runtime. In this abridged transportation model, no time-of-day choice is modeled—only traffic for peak periods is modeled. CUBE Land provides zone-level population and employment forecasts to the trans- portation model. Two measures of accessibility are fed back from the transport model to the land use model: • How many jobs can be reached within 20 min in a single-occupancy vehicle • How many jobs can be reached within 20 min on high-frequency transit One detail about the land use and transport models is that they iterate until they reach equi- librium. Commonly, two to three iterations are necessary to reach equilibrium. Although these iterations help allocate population and employment to the most expected locations, the itera- tions add runtime and suggest that there is a perfect level of information. Discrete choice models explicitly assume some lag time between changes in infrastructure or land supply and reaction of population and employment (compare with discussion of equilibrium in Section 1.1). Spatial input-output approaches iterate internally to reach equilibrium between land uses and prices, and thereby assume much more information about the real estate market than users typically have. Given the benefit of equilibrium prices, this improbable assumption of perfect informa- tion is typically accepted in input-output models. At the Metropolitan Council, this equilibrium is taken one step further by updating the location of population and employment to current traffic conditions that would result under that allocation of population and employment. As discussed in the next chapter, the land use model is only run in 2010 and 2040, and intermediate years are interpolated. For this reason, the Metropolitan Council chose to iterate between land use and transportation to make sure that these 2 anchor years are in equilibrium. Models that work incrementally avoid this perfect equilibrium between land use and transportation. 6.6 Model Application ARC Model runtime has been a concern for ARC. Although the PECAS model runs overnight, the activity-based transportation model needs 24 hours to complete one model year. It is possible to run the transportation model every 10 years only to save on runtime, but updating travel times every 5 years appears to generate more reasonable results. Staff at ARC explores the option to optimize code and parallelize model runs. Occasionally, ARC staff implements manual adjustments based on local knowledge. Those adjustments are added to the base scenario, and they tend to be smaller adjustments only. Often, reviewing and correcting input data has resolved implausible model results. ARC has done some limited scenario testing with land use policies for transit-oriented development (TOD) projects, and further scenario analyses are planned. In particular, ARC is interested in understanding how PECAS will react to the introduction of autonomous vehicles. Various alternative transit investments are other likely scenarios. Finally, on-demand transit options, such as Uber or Lyft, are of interest for testing with the integrated land use/transport model as well. For decisionmakers at the ARC board level, PECAS results represent another piece of infor- mation when evaluating alternative policies. The integrated land use/transport model provided insight that TOD policies make sense when evaluating the effect at the corridor or subarea level,

52 Integrated Transportation and Land Use Models even though TOD policies may have limited effect systemwide. Decisionmakers generally found results of the integrated model more realistic than output from the transportation model only. The region’s transit agency MARTA is a proponent of the integrated land use/transport model, given that model results have shown that integrating land use and transport policies improves the effect of new transit projects significantly. Upper management at ARC is supportive of this integrated land use/transport model as well. They tend to give some flexibility on how to perform the analyses. Upper management asks for specific analyses, and staff is generally trusted to decide how to analyze the questions. Although upper management sets clear goals, qualified modelers must decide on the path to get there. This setup has allowed staff to try innovative model approaches, which might be one key factor for the successful model implementation. Ohio DOT Ohio DOT’s main purpose for implementing a land use model was the disaggregation of socioeconomic data to TAZ and not so much to model alternative land use scenarios. This dis- aggregation model run is called “Model of Record,” which is the official model setup used in all transport model runs. This Model of Record includes all projects of the Statewide Transporta- tion Improvement Program (STIP). The model runs from 2010 to 2040 in 5-year increments. For each year, the runtimes are shown in Table 6-1. In just over 12 hours, the model can run over night, if only one model year is of interest. Running all seven model years, however, takes almost 3½ days. Ohio DOT can use an override file that either (1) sets sociodemographic data for a given zone and a given year to a certain value or (2) adds or subtracts to the modeled value a certain number of households and/or employees. This file is used to account for changes due to factory closures or other known development patterns. This override file may also be used if traffic volumes are off by 1,000 vehicles or more to “fix” sociodemographic data. Eventually, Ohio DOT also plans to compare MPO sociodemographic data with SEAM/SLUM output and use the override factors to fix deviations in the model. Given that SEAM/SLUM was never intended to be used for scenario analyses, staff at Ohio DOT is fairly generous with using override files to match expected developments. If the model was used for scenario analyses, such override set- tings would limit the model’s sensitivities. In the future, it is planned to use the traffic volumes of the statewide model as external vol- umes for Ohio’s urban models. For the smallest MPOs in Ohio, it is also under consideration whether the statewide model can be used in lieu of urban models for scenarios relevant for Ohio DOT project forecasts. Should this be pursued, those MPOs would get the fully inte- grated statewide model, which technically would allow them to test land use scenarios as well. Modeling step Runtime in hours Land use model 0.75 Calculation of skims 0.75 Passenger models 2.50 Freight models 4.50 Assignment preparation 0.50 Assignment 3.25 Total runtime per model year 12.25 Table 6-1. Model runtimes for the integrated Ohio Statewide Model.

Spatial Input-Output Land Use Models 53 For those MPOs maintaining their own models, the socioeconomic data match the data of the statewide model at the county level, but not necessarily at the zonal level. Upper management at Ohio DOT trusts the modeling group to the same extent as reported by ARC. As long as analysis goals are met, the modeling team is given the space to develop solu- tions within budget and schedule. Metropolitan Council The integrated land use/transport model for the Metropolitan Council was calibrated for the base year 2010. The long-term planning horizon is the year 2040. Intermediate years for 2020 and 2030 were interpolated between 2010 and 2040. After the base year 2010 was calibrated, the future year 2040 was developed using the follow- ing assumptions: • Forecast was based on the anticipated 2040 transport system (including light rail and bus rapid transit). • Forecast also considered increases in transit frequency and some smaller network extensions. • The metropolitan urban service area, which is provided with a public sewage system, was extended for 2040 as planned. • Demographic changes were implemented. • Future land use plans were respected. So far, only one future forecast has been developed. Due to a tight timeline, no scenario analy- ses were conducted to test alternative policies or the specific effects of single policies versus the set of policies implemented right now. In the future, staff members are interested in testing scenarios to identify policies that will help reach development goals. Overall, model runs have been found to be stable and reproducible. Sensitivities, however, are fairly small. Although staff at the Metropolitan Council had expected more sensitivities to sce- narios initially, this level of model reaction to scenarios may be reasonable. Partly, this may result from the model design that requires fairly strict calibration to base year conditions. Observed conditions are not perfectly in equilibrium (i.e., households do not immediately move when utilities change slightly), but the model forces observed conditions into equilibrium. Hence, this construct may result in larger constants and, therefore, less sensitivity than desired. A sec- ond reason for limited sensitivities to the future scenario is that most of the developed area in 2040 was already developed before 2010. Thus, scenarios can only change land use development marginally. The forecast was released in the midst of a housing crisis, which meant there was no growth. Therefore, the 2040 metropolitan urban service area for sewage service was, by and large, the same area that was planned for 2030. No major extensions were introduced due to the housing crisis and an uncertain growth future. Finally, there was limited funding for highway projects— no big projects were planned through 2040. These facts also explain why the future model run did not provide dramatically different patterns from the base year. The runtime depends on the degree of land use restriction. The more restrictive a scenario is set, the longer the model needs to find an equilibrium distribution of population, employment, and traffic flows. The common runtime is between 8 and 12 hours for one model year, which includes both the land use and the transportation model. With strict restrictions, however, the model may not converge at all. Upper management has always been supportive of the integrated land use/transport model. The staff at the Metropolitan Council was cautious in assessing the effect of the integrated land use/transport model on decisionmakers. The model probably helped discussions on how

54 Integrated Transportation and Land Use Models transport projects affected growth and urban development. Staff members were hesitant to specify how much the forecasts actually influenced policy decisions. At the same time, model results often are the beginning of a conversation with the member cities about growth. The model is a tool that helps prompt discussion of changing trends, but it is only one piece in a larger discussion. The model was always presented as a starting point for this discussion, rather than a definitive answer. The model also helps users to visualize the future. Results may open eyes about cause-and-effect relationships—increased awareness as an effect of the model may be difficult to measure. The modeling effort also helped the MPO to have a better understand- ing of cities’ growth expectations. For the most part, communities of the Metropolitan Council reviewed model results and found them to be plausible. Several opportunities to provide comments and feedback were given. It was key to provide lots of opportunities for comments. Generally, the feedback was encouraging. Still, some communities expected more growth in the future than predicted by the model. Some inner ring suburbs believed their forecasts were too high, which led to some revi- sions in the model. This improved the acceptance of the model, but it required several iterations to work out a forecast that communities would agree with. This iterative procedure to agree on plausible future development trends was helpful in two regards (1) it helped in ironing out input data inconsistencies and improving the model overall and (2) discussing model results with communities helped foster conversation about future growth. The Metropolitan Council raised “catastrophic reallocation” as a potential issue of CUBE Land: The model reallocates all socioeconomic data from scratch every simulation period. The model calibration for the base year assumes that people are in their current locations because they are the preferred locations. In reality, people might prefer living somewhere else but still live in a specific apartment because of transaction costs. The Metropolitan Council staff members are interested in alternative allocation methods in the supply model. Incre- mental forecasts might be interesting for short-term forecasts. The model was used for the Long-Range Transportation Plan in 2013 and revisions to this plan in 2014 and 2015. Since then, the staff at the Metropolitan Council has not conducted any land use modeling. Forecasts provided since then have not been based on CUBE Land. In the future, staff members are interested in bringing land use forecasting earlier into the planning process and developing and testing various scenarios. Possibly, this might need a revision of the existing tool, and maybe tools need to be added, or even another model will need to be applied. The agency is reviewing these alternatives. Of particular interest are the representation of path dependency, a finer spatial resolution, and added sensitivities. The staff at the Metropolitan Council wants to ensure that policies show the appropriate effect. After the next round of census data is released in 2020, the agency plans the next round of integrated land use/transport modeling. 6.7 Lessons Learned ARC ARC staff reached out to other agencies that have done this work before and learned from their experiences. Before engaging in integrated land use/transport modeling, it was helpful to learn from the successes and mistakes of other agencies with similar tools so as not to “reinvent the wheel.” Most agencies were supportive and willing to share tools, data, and expertise. ARC also understands that a good relationship with the model developer is key. This has allowed ARC to draw on expertise to implement the model in the Atlanta Region and adjust the model as necessary. ARC staff also believes that an agency should not expect to have an operational model up and running in a few months. An integrated land use/transport model

Spatial Input-Output Land Use Models 55 takes time to develop. For ARC, 3 years seemed reasonable to budget for the initial model implementation. Ohio DOT Ohio DOT found their integrated model worked satisfactorily. The model does the job, even though it is not always accurate. Ohio DOT staff members noted that if they had to do it all over again, they would rather use a commercial product and not develop their own model. The main benefit of a commercial product would be the model user community that allows sharing experiences and knowledge on how to overcome model issues. A staff member said “We were ten years too early, FAF was not there, CUBE Land was not there, but it is also great to be at the bleeding edge.” Metropolitan Council For the Metropolitan Council, it was important to have an operational model early on, even if the model did not provide perfect results. Staff members recommended starting small and getting results early on. Improvements can be made later when the model is operational and the most relevant shortcomings of the model are well-understood. The Metropolitan Council believed that more piloting should be done, and models should start with less complex frame- works. It may also be beneficial to start with a simpler tool. Comparing model results of a more complex model with simpler model results can be a valuable exercise that helps improv- ing both the simple and the complex model. Two models may also help in building confidence that the model is generating reasonable results. If two models produce similar responses to a simple scenario, model users may be assured that this is a plausible and likely development. For more complex scenarios an advanced model is necessary, because the simpler models will have limited capabilities in scenario analysis. For simple scenarios, on the other hand, the less complex model might be entirely sufficient and could save on runtime. The Metropolitan Council also emphasized the time needed to familiarize people who will work with the forecasts. Staff members should not be expected to simply trust the model and work with model results as is, but rather should be given the opportunity to understand the model’s functioning and assess how reasonable model results are. Building this familiarity with the model requires time, but will pay off when staff members embrace model output or discover inconsistencies that can be solved during the next iteration of model updating.