Pilot Testing of SHRP 2 Reliability Data and Analytical Products: Washington (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

13 CHAPTER 2 Research Approach Given the complexity in each transportation project’s design, construction, evaluation, and decision making and the small sample possible to use for testing the selected products, the research team made efforts to ensure the reliability of the test results in two aspects: (1) setting up a dedicated steering committee to provide guidance and advice to the research team and (2) developing a thorough testing procedure for different types of products. 2.1 Steering Committee A steering committee for the SHRP 2 L38D research project was formed at the start of this research project. The committee members include Daniela Bremmer, Director of WSDOT’s Strategic Assessment Office and chair of the TRB Committee on Performance Measurement, Patrick Morin, Operations Manager of the WSDOT Capital Program Development and Management Office, Bill Legg, Washington State Intelligent Transportation System Operations Engineer, Shuming Yan, Deputy Director of the WSDOT Urban Planning Office, etc. They are from all relevant fields including transportation planning, traffic operations, urban corridor management, performance measurement and economic impacts, and project prioritization, and are very familiar with the past and ongoing projects suitable for this study. Principal investigator and Washington State traffic engineer John Nisbet calls regular meetings of the research team to check progress and collaborates research efforts between UW and WSDOT. He also organizes quarterly steering committee meetings to review research activities, suggest new research actions, and coordinate research efforts. 2.2 Test Procedure A systematic procedure for testing the SHRP 2 Reliability products was developed based on foreseeable needs in WSDOT’s practice. Please see Figure 2.1 for details. The test procedure covers both types of products: (1) models or procedures and (2) software tools. As shown in Figure 2.1, the test processes of the two types of products interact with each other because the computer software tools are typically the implementations of the methods or procedures. 2.2.1 Methods or Procedure Testing Models or procedures are typically developed based on assumptions. The reasonableness of these assumptions is critical to the applicability of the methods. Specific mathematical equations employed are also important, and a tradeoff between complexity and applicability must be made carefully in developing a model or procedure. Thus, the accuracy of the model or procedure needs to be evaluated. Considering that the data used in calibrating the model may not be representative to all locations and time periods, both temporal and spatial transferability must be tested.

14 Excel Based Tool Test (Application Test)Technical Model or Procedure Test Experimental Design Test Site Selection Data Compilation Test Objectives Data & Analysis Meth Data Compilation Testbed Installation Data Quality Control Testing Assumptions Spatial Transferability Temporal Transferability Test-bed Design Compare Interface User-friendly Necessary Guidance Help Documents Default Setup Layout Installation Successful Validation Algorithm Theory Calculation Scalability Various Scales Analysis and Feedback Result Analysis Conclusions Feedback/Potential Refinements Figure 2.1. General approach for pilot testing of the SHRP 2 L38 products. Following such logic, the research team developed a three-step procedure for testing the model or procedure type of products: 2.2.1.1 Experiment Design. (a) Test objectives. This step is driven by the test objectives or the key questions to answer with the experiment. Test objectives must be clearly set as the first step of the experiment design. In designing the test objectives, steps (b) though (d) are important to consider. (b) Test site selection. Random sampling from those qualified project sites is important in avoiding bias. It also allows uses of general probability theory in data analysis. Test sites should offer observations for comparative analysis. The SHRP 2 Reliability models or procedure products may include numerous control variables. To evaluate the impact of a particular variable, the conditions with and without the variable need to be observed. Also, a specific condition is better replicable to reduce the effect of uncontrolled variation and quantify uncertainty when needed. (c) Test-bed configuration design. Depending on the kinds of data needed and whether or not they are observable, further instrumentation of sensors for the desired types of data may be needed.

15 (d) Data collection and proposed analytical approach. Data collection location and time period need to be determined to support the planned tests. Given the nature of the model or procedure products to be tested in this research, simple validation of the model predicted results using field data and before-and-after analysis of specific highway treatments are sufficient in this study. 2.2.1.2 Data Compilation This step focuses on all the technical details in collecting and storing data, and making the data sets ready to use. A wide range of urban freeway and arterial data are compiled. The data collected for this study include (a) traditional static sensor data (loop, camera, etc.); (b) roadway geometric profile data; (c) incident and crash data [Washington Incident Tracking System (WITS) data]; (d) weather data; and (e) traffic operation and management data [such as active traffic management (ATM) control data]. Data quality control is an important component as low-quality data will interfere with the test procedure and may mislead the research. Data quality control procedures developed by WSDOT and UW are used to enhance data quality for the pilot testing. Data fusion and mining are performed to integrate traffic data with weather and incident data on a regional map basis to investigate travel time reliability under recurring and nonrecurring congestion conditions. More details of the data collection and quality control procedure are described in Chapter 3. 2.2.1.3 Testing In this testing step, accuracy and transferability, including temporal and spatial transferability, of the model or procedure products will be evaluated using the data collected from researchers’ study sites. 2.2.2 Computer Tool Testing All the computer tool products were Microsoft Excel–based applications. The key of the tests of such products is whether an application meets the requirements that guided its design and implementation. Specifically, the requirements may include operability, usability, performance, and scalability. An operability test includes a compatibility test of the commonly used operating systems. If the software application cannot be installed or operated in a specific operating system or Excel version, then it fails the operability test. A usability test evaluates if the software is easy to understand and use. User interface is important for user–computer interactions and thus plays an important role in usability. Evaluation of usability is based on the following factors: (1) user interface’s level of friendliness, (2) sufficient guidance and help information accessible when using the software, (3) default configurations and explanation of the input parameters needed to start the software, and (4) layout of the modules and data output. A performance test focuses on correctness and efficiency. If a software application does not implement the correct logic or method, then it fails the performance test. Even if the method

16 or procedure is correctly implemented, an application may still fail its performance test if the efficiency is in the beyond-tolerable range. The scalability test for this research project refers to whether the software tool can be applied to a much smaller or much bigger project than the ones used to develop them. Scalability is important for future applications to transportation projects with varying scales. 2.2.3 Result Analysis and Feedback A set of measures of effectiveness (MOEs) is carefully selected for each test. The computed MOEs will be compared with those used by WSDOT in practice. Over the past decades, WSDOT has completed a number of projects that are appropriate for testing and before-and-after analysis on travel time reliability. Specifically, the following projects are chosen as study projects for SHRP 2 L38:  Corridors used for the WSDOT Gray Notebook production are used to test SHRP 2 L02 products. WSDOT has been monitoring corridor travel time for the quarterly Gray Notebook performance evaluation report since 2001. The Gray Notebook provides updates on system performance and project delivery on the corridor and statewide levels. Additionally, the Gray Notebook is used for testing and evaluating products of SHRP 2 L02.  Among the Moving Washington projects, corridors along I-5 and I-405 and State Route 522 are used for testing the methods and analytical tools from SHRP 2 L08.  I-5 JBLM is chosen as a case study for testing the effectiveness and usability of the products from SHRP 2 L05 and C11. To test the five-step procedure from SHRP 2 L05, a couple of projects in this region have been prioritized within the 10-year investment strategy. By applying the SHRP 2 C11 tool on I-5 JBLM projects, both traditionally measured benefits and wider economic benefits over the past years can be analyzed, and the tool’s usability and effectiveness can be tested. At the end of each test, problems identified through the test and recommended improvements are made to help the SHRP 2 Reliability program make these tools more useful in future practice.