Validation of Urban Freeway Models (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

48 Purpose The purpose of this document is to detail the data collection and analysis plans for performing the validation of the L03 data-rich and data-poor models. As such, this document contains three sections. The data collection plan section describes the data used in the L03 project, lists the data requirements for model validation, and details the metro- politan areas and data sets available for validation and enhancement activities. The analysis plan lists the steps needed to transform the raw collected data into the final analysis data sets for model validations. Finally, the conclu- sions section synthesizes the validation design. Data Collection Plan Overview The L03 team calibrated and validated its models using urban freeway data collected in the following metropolitan areas: • Atlanta, Georgia (calibration); • Houston, Texas (calibration); • Jacksonville, Florida (calibration); • Los Angeles, California (calibration); • Minneapolis–St. Paul, Minnesota (calibration); • San Diego, California (calibration); • San Francisco, California (calibration); and • Seattle, Washington (validation). The L03 team selected these sites in part because their agencies collect and archive continuous, high-quality traffic data. These characteristics are also critical for L33 validation and enhancement activities. As such, this validation plan pro- poses to use data collected in many of the same locations. The L33 project team will ensure that the model validation per- formed in L33 does not use the same data collected during the same time frame on the same freeways segments as were used to calibrate or validate the models in L03. Because this is a critical requirement, this data collection plan reviews the regions and data sets used in L03. Validation Data Characteristics This section lists the metropolitan area and data sets require- ments, as well as the optimal data set features, for performing the data-rich and data-poor model validation. These require- ments and features have been developed by evaluating the L03 data processing methodologies and application guidelines. Metropolitan Area Requirements • Region must have continuous archived data for at least one urban freeway; • Selected regions must offer a wide variety of seasonal weather conditions; and • At least five regions must be selected for validation. Data Set Requirements • At least a year of traffic data must be available; • At least a year of hourly precipitation data must be avail- able (data-rich only); • At least a year of incident data must be available (data-rich only); and • Urban freeway sections must offer high-quality traffic data collected from detectors at a dense spacing. Optimal Data Set Features • At least a year of work zone data are available; • At least a year of travel time data collected from technologies other than point detectors are available; and • Incident data contain detailed and accurate information on the lane blockages and duration. A P P e n D i x B Validation Plan

49 Proposed Regions Overview This section details the characteristics of the metropolitan areas and data sets that the L33 team has identified as suitable for use in the model validation and/or enhancement stages of this project. The L33 team has identified 10 metropolitan areas from which to acquire data for validation and enhancement purposes. These metropolitan regions, which are listed with key character- istics in Table B.1 and further detailed in the rest of this section, were selected based on the criteria defined in the validation data characteristics. Five of the sites were also used in the L03 project. For L33, the project team has made efforts to acquire more recent data than were used for calibration and validation in L03. A chart comparing the data by region used in L03 with the traf- fic data available to the L33 team is shown in Figure B.1. Atlanta The L33 team possesses 6 months of archived traffic data and 3 months of archived incident and work zone data collected through the SHRP 2 L02 project. L02 collected two types of traffic data in Atlanta: (1) Data from camera and radar detectors collected in real time from the Georgia Department of Transportation’s (GDOT) Navigator Advanced Transportation Manage- ment System (ATMS) and speed; and (2) Travel time data acquired from data-reseller Navteq (now NOKIA). The point detector data has been quality controlled using L02 methodologies and is aggregated to the 5-min level, which is consistent with the L03 model data needs. The Navteq data have also been aggregated to the 5-min level. The L33 team also has 3 months of incident and work zone data, collected in L02 from Atlanta traffic management cen- ters through the Navigator system. The incident and work zone data are highly detailed: they contain information on the type of incident, the number of lanes blocked, and the incident duration. This information is sufficient to directly calculate incident-lane-hours-lost by time period. Archived hourly weather data are available from the National Climatic Data Center for multiple sensors in the metropolitan area. The L03 data-rich and data-poor models were calibrated using Atlanta traffic and incident data collected from Naviga- tor during the 2006 to 2008 timeframe. Maps of L02 data cov- erage and L03 study corridors are shown in Figure B.2. L03 does not have any Atlanta data from these years, so any L33 analysis will not temporally overlap with that of L03. The L02 data set does not contain the full year of data required for L03 model validation. The incident data may be explored during validation to test the L03 relationships devel- oped between the average number of lanes blocked per inci- dent and the roadway geometry and incident clearance policies, as well as the ratio of collisions to incidents. This Table B.1. Site Selection Matrix Region Traffic Data L03 Site Other Sources Use in L33 Incidents Work Zones Weather Data-Rich Validation Data-Poor Validation Enhancement Atlanta Detectors X X X X X Navteq X Las Vegas Detectors X X X X X X Los Angeles Detectors X X X X X X X Minneapolis Detectors X X X X X X Sacramento Detectors X X X X X X Bluetooth X San Diego Detectors X X X X X X X Salt Lake City Detectors X X X X X X San Francisco Detectors X X X X X X X Toll Tag X X X X Spokane Detectors X X X X X X Washington, D.C. Detectors X X X X X

50 Figure B.1. L33 data availability and L03 data coverage. Note that validation will not be performed on the same freeway sections and years that were used to calibrate the L03 data-rich and data-poor models. In other words, no L03 data will be used in the L33 validation.

51 investigation is further detailed in the analysis plan section of this document. This data set may also be further explored in the enhancement phase of this project as needed. Las Vegas The L33 team has a year (March 2012 to March 2013) of traf- fic, incident, and work zone data collected from the Regional Transportation Commission of Southern Nevada (RTC) Freeway and Arterial System of Transportation (FAST) via Iteris’ Performance Measurement System (PeMS). The traffic data were collected from point detectors; data coverage is shown in the speed map in Figure B.3. The traffic data have been quality controlled per L02 methodologies and are available at a 5-min granularity. The incident and work zone data were collected from the FAST Traffic Management Center (TMC) and contain highly detailed information on the duration, lane blockage, and type of activity. This information is sufficient to directly calculate incident-lane-hours-lost by time period. Archived hourly weather data are available from the National Climatic Data Center for multiple sensors in the metropolitan area. No Las Vegas data were used in the L03 project, so this site adds regional diversity to the validation activities. Los Angeles Multiple years of traffic, incident, and work zone data are available in Los Angeles County and Orange County through the Caltrans PeMS. The traffic data are collected from loop and radar detec- tors, are quality controlled per L02 methodologies, and are aggregated to the 5-min level. The spacing of detectors in Los Angeles County is very dense, but the percentage of working detectors has always hovered around 60%. The spacing in neighboring Orange County is of comparable density with much higher quality data (70% to 90% working detectors since 2009). L02 Data Coverage L03 Study Corridors Figure B.2. Atlanta data availability and L03 usage. Figure B.3. Las Vegas data availability.

52 The L33 team has possession of two sources of incident data in the Los Angeles region, and throughout California, collected via PeMS: (1) Incidents from the California Highway Patrol (CHP) collected in real time from the CHP media feed; and (2) Traffic accidents from the Caltrans Traffic Accident Sur- veillance and Analysis System (TASAS), which contains severity and lane blockages, but no duration. The CHP data contain the type of incident and duration but no standardized indication of the number of lanes blocked. The L33 team is currently exploring ways to parse the CHP log details, which often indicate lane blockages in free-text form, to estimate the number of lanes blocked. The TASAS accidents are compiled by Caltrans through accident reports. They report the accident severity and lane blockage but only include the incident start time (no duration infor- mation). As such, these data may not be usable in the data- rich model but may be explored in conjunction with the CHP data to obtain a ratio between the number of collisions and the number of overall incidents. Work zone information is also available in this region, and throughout California, via PeMS. PeMS continuously collects and archives work zone information from the Caltrans Lane Closure System, which is used by Caltrans staff to plan, approve, and manage lane closures. PeMS data contain detailed infor- mation on the work zone start and end times, type of work, number of lanes closed, and estimated traffic impact. This information is sufficient to calculate the lane-hours-lost caused by work zones. Archived hourly weather data are available from the National Climatic Data Center for multiple sensors in the metropolitan area. In Los Angeles, the L03 team used traffic data collected from PeMS and incident data collected from traffic.com on two study segments in 2001, 2002, 2004, 2005, and 2006. The usage of Los Angeles data in L33 will not overlap in time and space with the usage in L03. Maps of Los Angeles data cover- age and L03 study corridors are shown in Figure B.4. Minneapolis The L33 project team currently has 10 years of traffic data in the Minneapolis–St. Paul region archived in the Minnesota Department of Transportation’s (MnDOT) PeMS. The data have been quality controlled per L02 methodologies and are available at a 5-min granularity. The MnDOT PeMS also has slightly more than a year of traffic incident data archived from MnDOT’s Intelligent Roadway Information System (IRIS) ATMS. To supplement this, the L03 project team has acquired an additional 4 years (2008 to 2012) of archived incident data from MnDOT. This incident data contain incident duration, type, and an indica- tion of the traffic impact. This information is sufficient to calculate incident-lane-hours-lost by time period for input into the data-rich model. No work zone data are available in Minneapolis. In Minneapolis, the L03 team used traffic data collected from MnDOT and incident data from traffic.com in 2001 to 2007 to calibrate the data-rich and data-poor models. The usage of Minneapolis data in L33 will not overlap in time and space with L03 Study Corridors Data Availability Figure B.4. Los Angeles data availability and L03 usage.

53 the usage in L03. Maps of Minneapolis data coverage and L03 study corridors are shown in Figure B.5. Even though Minne- apolis was studied fairly extensively in L03, it is a critical valida- tion region for L33 given its severe winter weather conditions. Sacramento Multiple years of traffic, incident, and work zone data are avail- able in the Sacramento region through the Caltrans PeMS. The types of data available are the same as those described for Los Angeles. A map of the available traffic detection network is shown in Figure B.6. In addition, the L02 project’s case study efforts in Lake Tahoe included the collection of 4 months of Bluetooth data on a segment of Interstate 5 near downtown Sacra- mento. The remainder of the L02 data collected to support this case study was on rural freeways, so it is not useful for the L33 project. This quantity of Bluetooth data is not sufficient for validation purposes, but it will be critical for exploring Bayesian approaches in later phases of the project. Archived hourly weather data are available from the National Climatic Data Center for multiple sensors in the metropolitan area. No Sacramento data were used in the L03 project, so this site adds regional diversity to the validation activities. Addi- tionally, the extreme fog conditions that this region can expe- rience present an added vector into the weather analysis portion of the L33 project. Salt Lake City Multiple years of traffic data are available in the Salt Lake City region through the Utah Department of Transportation’s (UDOT) PeMS. The data have been quality controlled and are available at a 5-min granularity. A map of the detection coverage is shown in Figure B.7. UDOT’s PeMS system does not contain any archived inci- dent or work zone data. The L33 team is currently working with staff at the UDOT Traffic Operations Center to acquire archived incident and work zone data collected by the center’s ATMS, with an anticipated delivery date of July 8, 2013. The L03 Study CorridorsData Availability Figure B.5. Minneapolis–St. Paul data availability and L03 usage. Figure B.6. Sacramento data availability.

54 team has been guaranteed the availability of these data, but has not yet acquired them. As such, the exact details and for- mat of the data are currently unknown. Archived hourly weather data are available from the National Climatic Data Center for multiple sensors in the metropolitan area. No Salt Lake City data were used in the L03 project, so this area contributes regional diversity as well as another severe winter weather site to the validation activities. San Diego Multiple years of traffic, incident, and work zone data are available in the San Diego region through the Caltrans PeMS. The types of data available are the same as those described for Los Angeles. Archived hourly weather data are available from the National Climatic Data Center for multiple sensors in the metropolitan area. San Diego was a multimodal case study site for the L02 project. The L02 project leveraged all of the data sources described above to develop a framework for linking travel time variability with the sources of nonrecurrent congestion. In San Diego, the L03 team used traffic data collected from PeMS and incident data from traffic.com in 2001 to 2006. The usage of San Diego data in L33 will not overlap in time and space with the usage in L03. Maps of San Diego PeMS data coverage and L03 study corridors are shown in Figure B.8. San Francisco Multiple years of traffic, incident, and work zone data are available in the San Francisco Bay Area through the Caltrans PeMS. The types of data available are the same as those described for Los Angeles. In the past few years, the quality of detector data in the Bay Area has decreased, so L33 study cor- ridors will have to be carefully selected and may have to rely on older data. In addition to the point detector data, PeMS also contains matched toll tag travel times. Archived hourly weather data are available from the National Climatic Data Center. In the Bay Area, the L03 team used point detector and toll tag data collected from PeMS and incident data from Figure B.7. Salt Lake City data availability. L03 Study Corridors Data Availability Figure B.8. San Diego data availability and L03 usage.

55 traffic.com in 2002. The usage of Bay Area data in L33 will not overlap in time and space with its usage in L03. Maps of Bay Area PeMS data coverage and L03 study corridors are shown in Figure B.9. Spokane Eight years of traffic data are available in the Spokane region via the Spokane Regional Transportation Management Cen- ter (SRTMC) PeMS. In Spokane, PeMS collects data from radar detectors that report data to the TMC. Figure B.10 shows the available detection network, which monitors the I-90 freeway through the city. The other available detection is on conventional highways. The traffic data have been quality controlled as recommended by the L02 reliability monitoring guidebook and are available at 5-min granularities as required by the L03 model validation. The project team also acquired incident and work zone data on I-90 in Spokane from the SRTMC. This data set has two pieces: 1. Incidents logged by TMC operators, and 2. Incident and work zone traveler alerts issued by the TMC. The TMC log incidents contain the start time, free-text description of the incident, and free-text additional remarks. No incident end time is given. The traveler alerts contain start and end times and a description of the alert. Initial analysis of the data received from the SRTMC suggests that the traveler alert information is the better data set for calculating incident- lane-hours-lost for the data-rich model. To supplement the TMC incidents, the L33 team acquired collision records from 2005 to 2012 for I-90 through Spokane from the Washington State Department of Transportation. Archived hourly weather data are available from the National Climatic Data Center. No data from Spokane were used in the L03 project, so this area contributes regional diversity as well as another severe winter weather site to the validation activities. Washington, D.C. The L33 team is in possession of 2 years of traffic data col- lected in Northern Virginia near Washington, D.C., during L03 Study CorridorsData Availability Figure B.9. San Francisco Bay Area data availability and L03 usage. Figure B.10. Spokane data availability.

56 the L02 project. In L02, data from 2009 were obtained from the Regional Integrated Transportation Information System (RITIS), developed and maintained by the CATT Laboratory at the University of Maryland. Data from 2011 and 2012 were col- lected by the L02 monitoring system in real time from RITIS. The traffic data have been quality controlled as recommended by the L02 reliability monitoring guidebook and are available at 5-min granularities as required by the L03 model validation. A map of the traffic detection coverage is shown in Figure B.11. Data are available on US-66 and I-95 near Washington, D.C. The main issue with this data source is that the data were judged to be low quality in 2011, with an average of 70% of detectors not meeting the data-quality requirements. Data in 2009 are superior, so L33 analysis will likely focus on 2009. The L33 project team has also acquired incident data for 2009 to 2011 from the University of Maryland. The incident data consist of detailed records on incident type, duration, and lane blockages over the duration of the incident. The availability and format of the work zone data in 2009 have not yet been confirmed. Archived hourly weather data are available from the National Climatic Data Center. No data from Washington, D.C., were used in L03, so this area contributes regional diversity as well as another severe winter weather site to the validation activities. Summary The L33 team proposes to perform validation and enhance- ment activities in 10 metropolitan areas around the United States. Together, these sites have the following characteristics: • 9 sites have guaranteed access to traffic data over at least a year. • 9 sites have guaranteed access to incident data over at least a year. • 10 sites have guaranteed access to weather data of at least a year. • 7 sites have guaranteed access to work zone data of at least a year. • 7 sites have guaranteed access to at least 5 years of traffic, incident, and weather data. • 3 sites have guaranteed access to traffic data collected by technologies other than point detectors. The charts in Figure B.12 summarize the availability of the needed types of data in each of the 10 proposed metro- politan areas. Analysis Plan This analysis plan outlines the L33 team’s proposal for turn- ing the data sets described in the previous section into the final data sets required for validation of the data-rich and data-poor models. The plan consists of detailed instructions for nine tasks: 1. Select validation freeway sections, 2. Quality-control and aggregate traffic data, 3. Calculate the peak hour and peak period (data-rich only), 4. Calculate travel time reliability measures, 5. Calculate demand-to-capacity ratio variables (data-rich only), 6. Calculate incident-lane-hours-lost (data-rich only), 7. Calculate hours of precipitation exceeding 0.05 in. (data- rich only), 8. Validate data-rich and data-poor equations, and 9. Sensitivity testing on alternative data processing approaches. In structuring this analysis plan, it was important to bal- ance conducting validation in the same way as L03 with the desire to gauge model performance under different data Figure B.11. Washington, D.C., data availability.

57 Figure B.12. Availability of the needed types of data in each of the 10 proposed metropolitan areas. (Continued on next page.)

58 Figure B.12. Availability of the needed types of data in each of the 10 proposed metropolitan areas. (Continued from previous page.)

59 methodologies. The fact that the data-rich and data-poor model validation was only performed in one region is a major motivation for this validation task. Performing the validation using the same data processing and estimation methodologies as L03 tests the validity of the developed models for different regions and time periods. On the other hand, performing the validation using different processing methodologies and assumptions tests the model’s sensitivity to the way the inputs are generated. To address both forms of validation, this section is centered on a core validation analysis plan, which adheres as closely as possible to the L03 process, but also contains a set of supple- mental experiments. Each of the 9 tasks contained in this sec- tion lists the inputs, outputs, and steps that will be used to perform the model validation as it was done in L03. Where rel- evant, supplemental experiments that generate separate data sets or perform other validation activities are listed. The goal of this structure is to assess the strengths and weaknesses of the L03 data-rich and data-poor models and the degree to which the validation errors are a function of the independent variable estimation process. These findings will guide the enhancement tasks of the L33 project. Task 1: Select Validation Freeway Sections The application guidelines of the L03 data-rich and data-poor models specify the freeway section characteristics required to achieve valid model results. The freeway sections used for vali- dation in L33 will meet the following criteria: • Length of around 5 mi (range from 2 to 10 mi); • Good data quality over a year; • Monitored by point detectors with no more than an aver- age spacing of 3/4 mi, or monitored by automated vehicle identification (AVI) technologies at the section origin and destination; • No mid-section freeway-to-freeway interchanges or bottle- necks; and • Relative homogeneity in terms of traffic and geometric conditions. Steps 1. Identify freeway segments with dense detector spacing. 2. Identify sections along the densely monitored segments that have a consistent number of lanes, have no mid-section freeway-to-freeway interchanges, and are approximately 5-mi long. 3. Identify yearlong periods with good data quality (average percent observed exceeding 75%). 4. Visually assess detector data to identify calibration issues. Task 2: Quality-Control and Aggregate Traffic Data In this step, the 5-min level traffic data available at the pro- posed study sites are aggregated to the section level and fil- tered to exclude data samples with poor quality. The required steps depend on the technology used to collect the traffic data; as such, separate steps are presented for point detector data and the toll tag travel times proposed for validation in the San Francisco Bay Area. The quality-control and aggre- gation plan for the point detector data is nearly identical to that used by L03, with the exception that, in L33, quality con- trol has already been performed by upstream data collection systems according to L02 recommendations. For the toll tag data, the L03 final report does not suggest that any filter- ing was performed on the data. Because toll tag travel times are highly influenced by the presence of outlier data samples, the L33 team is currently performing exploratory analysis to identify an appropriate filtering and quality-control algorithm. Point Detector Data Inputs • At least a year of 5-min detector station volumes (summed across all lanes) and speeds (volume-weighted average across all lanes) that have been quality controlled, cleaned, and imputed according to methodologies from the SHRP 2 L02 guide. Outputs • At least a year of 5-min section vehicle miles traveled (VMT), vehicle hours traveled (VHT), speed, travel time index (TTI), and travel times. steps 1. Calculate 5-min VMT and VHT at each detector station (link) using the link’s length (the distance halfway to the nearest neighboring stations in the upstream and down- stream directions): a. Link VMT = link length * 5-min volume, and b. Link VHT = link VMT/(Min(60 mph, 5-min speed)). 2. Aggregate the link-level data to section-level 5-min VMT, VHT, space mean speed (speed), TTI, and travel time: a. Section VMT = sum of link VMTs, b. Section VHT = sum of link VHTs, c. Section speed = Section VMT/Section VHT, d. Section TTI = Max(1.0, 60 mph * (1/Section speed)), and e. Section travel time (mins) = Section TTI * Section length. 3. Flag 5-min data points when less than 50% of the section’s detectors data were not working. These data points will be excluded from all downstream analysis.

60 supplemental QualIty-COntrOl experIments • Use a higher data-quality threshold in generating the section- level data by flagging 5-min data points when less than 90% of the section’s detectors were not working. Assess the differ- ence in the validation results in Task 9. • Calculate the free-flow speed along each section and use that in the TTI equations, instead of 60 mph. Assess the difference in the validation results in Task 9. Toll Tag Travel Times (San Francisco Bay Area Only) Inputs • At least a year of 5-min mean, median, minimum, 25th- percentile, 75th-percentile, and maximum travel times, and number of measured samples; and • 5-min detector station volumes. Outputs • 5-min quality-controlled median travel times. steps • To be determined following exploratory analysis currently underway by Iteris and Arizona State University. supplemental Data COlleCtIOn teChnOlOgy experIments • Collect toll tag travel times and point detector travel times on the same section. Assess the difference in the validation results in Task 9. Task 3: Calculate the Peak Hour and Peak Period (Data-Rich Only) This step is required to identify each section’s peak hour and peak period in order to group data into the data-rich study time periods. The steps outlined below adhere to those used in L03. Peak Hour Inputs • 5-min section speeds from Task 2. Outputs • Peak 60-min time period for each section, and • Average 5-min weekday speeds over a year for each section. steps 1. Subset each section’s data to include only non-holiday weekdays. 2. For each 5-min period, calculate the weekday average space-mean section speed. 3. For each section, identify the 12 consecutive 5-min periods that have the lowest average space mean speed. Peak Period Inputs • 5-min average weekday speeds from peak hour calculation. Outputs • Peak time period (of at least 75 min) for each section. steps • Identify time periods of at least 75 min where the average section speeds are less than or equal to 45 mph. If there are none, identify the 75 consecutive minutes that have the low- est average section speeds. Supplemental Time Period Definition Experiments • Test different definitions of the peak period. Task 4: Calculate Travel Time Reliability Measures The purpose of this task is to use the section-level traffic data from Task 2 and the peak hour/peak period defini- tions from Task 3 to generate yearly section travel time reliability measures for use in validation. The L03 models were developed to estimate volume-weighted travel time reliability measures. The L33 team plans to duplicate this methodology. Inputs • 5-min section data from Task 2, and • Peak hour and peak period definitions from Task 3 (data- rich only). Outputs (For Each Section and Time Period) • Mean TTI; • Percentile TTIs (10th, 50th, 80th, 95th, and 99th); • On-time statistics [percentage of trips (VMT) made within 1.1x the median travel time and within 1.25x the median travel time]; and • Failure statistics [percentage of trips (VMT) with speeds less than 50 mph, 45 mph, and 30 mph]. Steps 1. Group 5-min section data into the L03 model time periods (data-rich only): a. Peak hour: defined in Task 4; b. Peak period: defined in Task 4;

61 c. Midday: Non-holiday weekdays from 11:00 a.m. to 2:00 p.m.; and d. Weekdays: Non-holiday weekdays, 24 h. 2. Calculate the outputs using VMT-weighting (consider the VMT in the frequency of each TTI weighting). Task 5: Calculate Demand-to-Capacity Ratio Variables (Data-Rich Only) This task consists of four subtasks: (1) calculate link capacity; (2) calculate link demand; (3) calculate section critical demand-to-capacity ratio; and (4) calculate section average demand-to-capacity ratio. Task 5.1: Calculate Link Capacity The L03 project team obtained the capacity of each link from the Highway Performance Monitoring System (HPMS) wher- ever it was provided. On sections where the capacity was not listed in HPMS, it was calculated using the capacity method for planning applications from Dowling et al.’s 1997 report, which considers the number of lanes, the percentage trucks, and the peak hour factor (which was fixed in L03). The L33 team plans to duplicate this approach during validation. Inputs • Number of lanes; • Truck percentage; and • Peak Hour Factor. Outputs • Link capacity by time period. steps • Calculate capacity using NCHRP 387 methodologies. supplemental CapaCIty-estImatIOn experIment • Experiment with other capacity-estimation methodolo- gies to be determined through exploratory analysis. Task 5.2: Calculate Link Demand The L03 project team assumed that when speeds fall below 45 mph on an urban freeway the measured volume is not an accurate estimate of the demand. The L33 team plans to implement the procedure that the L03 team developed to estimate the demand during congested conditions. The L33 team will evaluate the estimation results and explore alterna- tive methods if the L03 method proves deficient. Inputs • 5-min link volumes and speeds. Outputs • 5-min link demand. steps 1. For each station, identify continuous 5-min periods when the measured speed falls below 45 mph. During these time periods, the link is assumed to be in congestion and the measured volume not representative of the demand. Single 5-min gaps during which speeds exceed 45 mph can still be included. 2. Split each congested time period into two halves. 3. The demand during the first half of congestion is assumed to be equal to the average volume measured in the two 5-min periods before the start of congestion. 4. The demand during the second half of congestion is set such that the cumulative volume measured over the con- gested period is equal to the estimated cumulative demand over the same time period. 5. Check that the two 5-min periods after the termination of congestion fit smoothly to observed cumulative volume curve. If they do not, extend the congested period to ensure a smooth transition. Task 5.3: Calculate Section Critical Demand-to-Capacity Ratio The data-rich models for the peak hour, peak period, and mid- day time periods all require the critical demand-to-capacity ratio. The L33 project team plans to compute this ratio using the L03 methodology, outlined here. Inputs • Link capacities from Task 5.1. • 5-min link demands from Task 5.2. Outputs • For each section and time period (peak hour, peak period, and midday), the critical demand-to-capacity ratio. steps 1. For each link, calculate the demand during each weekday time period (peak hour, peak period, and midday). 2. Calculate the 99th-percentile demand-to-capacity ratio for each link over all time periods in the year. 3. For a section, choose the highest 99th-percentile demand- to-capacity ratio among all the links on the section as the independent variables. Task 5.4: Calculate Section Average Demand-to-Capacity Ratio The data-rich weekday models require the average demand- to-capacity ratio of each section. The L33 project team

62 plans to compute this ratio using the L03 methodology, outlined here. Inputs • Link capacities from Task 5.1. • 5-min link demands from Task 5.2. Outputs • For each section and time period, the average demand-to- capacity ratio for a section. steps • For each link, calculate the demand-to-capacity ratio dur- ing each time period. • For a section, calculate the average demand-to-capacity ratio across all of the links on the section during the time period. Task 6: Calculate Incident-Lane-Hours-Lost Variable (Data-Rich Only) All of the peak hour and peak period models, and some of the weekday models, require the yearly incident-lane-hours-lost. L03 calculated these values wherever possible from the raw incident data. The L33 project team plans to use this approach, outlined below. Inputs • Raw, time-stamped incident data collected within the metro politan area. Outputs • For each section, incident-lane-hours-lost by time period over a year. Steps 1. Using the incident location data, subset the raw incidents to those that occurred on the section. 2. For each time period, subset the incidents to those that started in or 15 min before, ended in, or spanned the entire time period. 3. For the portion of each incident that occurred during a time period, calculate the lane-hours-lost caused by the incident using information on the lane blockage and inci- dent duration. If lane blockage and/or incident duration is not available in the incident data set, estimate them using L03 final report equations that estimate these values based on the agency incident clearance policies and the presence of shoulders. 4. For each section and time period, calculate the total incident-lane-hours-lost over the year as the indepen- dent variable. Supplemental Analysis of Incident Data • Use work zone data to calculate a work zone lane-hours- lost term. Assess how the addition of this term into the incident-lane-hours-lost variable affects the model perfor- mance in Task 9. • Use detailed incident and collision data sets to validate the L03 equations for 44 Ratio of the incident rate to the crash rate (4.545); and 44 Average number of lanes blocked per incident: (a) with usable shoulder and policy to move lane-blocking inci- dents as quickly as possible (0.476); (b) with usable shoulder and no policy to move lane-blocking incidents (0.580); and (c) no usable shoulders (1.140). Task 7: Calculate Hours of Precipitation Exceeding 0.05 in. (Data-Rich Only) Some of the peak hour and peak period L03 data-rich models require inputs of the yearly number of hours of precipitation that exceeded 0.05 in. The L03 methodology leveraged hourly precipitation data from the National Climatic Data Center. This data source will also be used in L33, and processed as follows. Inputs • Hourly precipitation values at weather stations down- loaded from the National Climatic Data Center. Outputs • For each section and time period, the number of hours of precipitation exceeding 0.05 in., for input as an indepen- dent variable into the data-rich model. Steps 1. Pick the weather station that is closest to a section. 2. Download the hourly precipitation data. 3. For each time period, calculate the number of hours when the precipitation exceeded 0.05 in. Task 8: Validate Data-Rich and Data-Poor Equations In this step, the independent variables are input into the L03 data-rich and data-poor equations, and outputs are compared with the actual reliability measures calculated in Task 4.

63 Inputs • Critical demand-to-capacity ratio for each section during the peak hour, peak period, and midday period (output of Task 5.3). • Average demand-to-capacity ratio for each section on weekdays (output of Task 5.4). • Number of hours when rainfall exceeds 0.05 in. for each section during the peak hour and peak period (output of Task 7). • Incident-lane-hours-lost for each section during the peak hour, peak period, and weekday time periods (output of Task 6). • The mean travel time, 10th-, 50th-, 80th-, 95th-, and 99th- percentile travel times, the percentage of on-time trips made within 1.1 and 1.25 times the median TTI, and the percent- age of on-time trips with 30-, 45-, and 50-mph speed thresh- olds for each section and time period (outputs of Task 4). Outputs • For each section and model, the percentage error between the predicted and measured reliability metrics. • For each model, the root mean square error across all sections. • Where possible, test the statistical significance of the com- parison between measured and predicted reliability metrics. Validation Steps To understand the performance of the data-rich and data- poor models across the regions proposed in the data collec- tion plan, the L33 project team plans to focus validation efforts on the following travel time reliability statistics predicted by the L03 models: • 80th-percentile TTI (data-poor and data-rich, all time periods); • 95th-percentile TTI (data-poor and data-rich, all time periods); and • Standard deviation (predicted by data-poor model only). The team believes that focusing on these measures will pro- vide ample insight into the performance of the L03 model forms, while allowing more resources to be spent collecting data and processing data from a wide range of sites. This will allow the L33 team to assess application guidelines for the L03 models while also gaining insight into potential model enhancements. For the data-poor prediction, the L03 project team initially used power form models to relate the mean TTI with the reliability-related TTIs. However, these equations were revised in Appendix H of the L03 final report. The final L03 data-poor equations for the 80th-percentile TTI, 95th-percentile TTI, and the standard deviation are as follows: 80th-percentile TTI 1 2.1406 ln meanTTIp ( )= + 95th-percentile TTI 1 3.6700 ln meanTTIp ( )= + standard deviation of TTI 0.71 meanTTI 1 0.56( )= − In these revisions, the L03 team changed the form of the per- centile equations to assume that the 80th- and 95th-percentile TTIs are related to the log of the mean TTI. The standard devia- tion equation keeps the power form, but modifies the coeffi- cient from the original equations. The L33 team plans to validate the three final L03 data-poor equations shown above. For the data-rich prediction, there are four equations that predict each reliability measure, one for each of the following time periods: (1) peak hour, (2) peak period, (3) midday, and (4) weekday. The data-rich equations that will be validated are as follows: peak hOur 95th-percentile TTI 0.63071 dc 0.01219 ILHL 0.04744 Rain05Hrscrite p p p= + + p p80th-percentile TTI 0.52013 dc 0.01544 ILHLcrite= + peak perIOD p p p95th-percentile TTI 0.23233 dc 0.01222 ILHL 0.01777 Rain05Hrscrite= + + p p80th-percentile TTI 0.13992 dc 0.01118 ILHL Rain05Hrscrite= + + mIDDay p95th-percentile TTI 0.07812 dccrite= p80th-percentile TTI 0.02612 dccrite= WeekDay p p p95th-percentile TTI 0.03632 dc 0.00282 ILHLaveragee= p p80th-percentile TTI 0.00842 dc 0.00117 ILHLaveragee= + To validate the data-poor and data-rich equations listed above, the L33 team will calculate the following performance measures: • Root mean square error (RMSE) Denote the predicted response values as yˆ , real response values as y, then the residual r is defined as = −ˆr y y

64 RMSE is defined as ˆ ˆ 2 2 1RMSE MSE y E y y r n i n∑( ) ( )= = −  = = RMSE can measure the magnitude of differences between the predicted and actual responses. However, there is no sim- ple benchmark or threshold for an acceptable RMSE. • Residual plots Ideally, residual r is a random variable following a normal distribution with zero mean. Plotting out the distribution of residuals provides direct impression of the goodness of fit, presence of bias, and heteroscedasticity. • Standard t-test of zero residual mean The standard t-test can be used to determine if the mean of residuals is significantly different from zero (in a statistical sense). With an unbiased model, the difference should be sta- tistically insignificant. = − µ0 t r s n where r– is the residual mean, s is the standard deviation of residuals, n is the sample size, and µ0 is the specific mean value targeted, set as zero. The end goals of the quantitative validation efforts are to determine: (1) How good are the L03 models? and (2) When and where can they be applied? To evaluate these questions, the L33 team will compare the log of residual variance as a measure of fit. When this measure is low then the model fits well; when high it fits poorly. This comparison will reveal how well the models perform during different time periods (peak versus non-peak), for different types of sections (e.g., num- ber of lanes), and across different regions with varying weather conditions and driver populations. The results of this analysis will be used to develop application guidelines for the existing L03 models and recommendations for changes to be explored in model enhancement. Task 9: Sensitivity Testing on Alternative Data Processing Approaches This analysis plan contains a number of places in the analysis chain where the L33 team would like to explore alternate methods of performing the data processing or independent variable computations. Pursuing these methods will produce multiple validation data sets on some sections. After Task 8 is performed on each data set, the validation results can be compared to gauge the sensitivity of the models to the vari- ous data processing alternatives. Conclusions This validation plan proposes to perform validation of the L03 models using data collected at up to 250 different freeway section-year combinations located in nine metropolitan areas. In comparison, the model validation performed in L03 used data from 60 section-years in a single metropolitan area. The traffic data model inputs and measured reliability statis- tics will be gathered from agency detector data feeds as was done in L03, though no L03 data will be used in the L33 vali- dation process. The incident-lane-hours-lost variable will be computed using dispatch and agency incident, crash, and work zone data sets. The L03 team used only incident data collected by the private sector in calibration and validation. Using agency incident data in L33 will allow for the assess- ment of model fit and development of application guidelines using data sets more likely to be used by agencies that are implementing the predictive models. The weather data used in L33 validation will be of the same form as that used in L03 hourly precipitation data from the National Climatic Data Center, though the L33 team is also exploring more spatially and temporally fine-grained weather data sets for enhance- ment tasks. This experimental design presented in this validation plan will allow the L33 team to confidently assess how well the data-rich and data-poor models predict travel times in different metropolitan areas. The core validation and the supplemental validation experiments will yield information that will enable the L03 team to identify and prioritize potential enhancements to the L03 models, such as the estimation of new coefficients or the development of new model forms. Reference Dowling, R., W. Kittelson, J. Zegeer. 1997. NCHRP Report 387: Planning Techniques to Estimate Speeds and Service Volumes for Planning Applications. Transportation Research Board, National Research Council, Washington, D.C.