Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 35
35 4.2 Common Pitfalls in the desired improvement in system performance. In choos- of Before/After Studies ing between the use of travel time or delay as a performance measure, the analyst should recognize that travel time incor- The validity of any conclusions drawn from a before/after porates a component (free-flow travel time) that is often large study hinges on the validity of the assumption that all other and relatively insensitive to most facility improvements. This conditions (except for the improvement itself) are identical makes travel time a difficult measure to use for the detection when both the before and after measurements are made. It is of performance improvements, particularly over a large area generally impossible to achieve perfect validity in the real or multiple segments or facilities. world since so many conditions change over time. Delay is a much more sensitive measure for detecting A common problem for before/after studies is traffic demand performance improvements. However, delay is more volatile, grows over time. Ideally the before study is done just before the requiring more measurements in order to determine its start of construction of the improvement and the after study is average within an acceptable confidence interval. So there is performed as soon after the improvement is completed and an explicit tradeoff in terms of level of confidence one may opened to traffic. However, it takes time for people to adapt to have in the results of the before/after analysis and the cost or a new improvement, thus it is not a good idea to gather after resource requirements of that analysis. data within the first few days or weeks after a project is opened. One must find a compromise point in time when most travel- ers are thought to have adapted to the new project and the least 4.4 Determining if Conditions amount of elapsed time since the before study was completed. Are Significantly Better There also are several potential additional (often unknown) differences between the before and after conditions that can It is tempting to measure the mean travel time (or delay) affect the results, usually without the investigator's knowledge. before the improvement and the mean travel time after the Examples include changes in gasoline prices, highway im- improvements and decide that conditions are better based provements elsewhere in the region, accidents on the day of solely on a comparison of the two means. However, since you the study at other facilities in the region, etc. do not measure travel times every hour of every day of the Another common problem of before/after studies is that year, it could have been the result of plain luck, not an actual you may not be surveying all of the travelers impacted by the difference. Statistical hypothesis testing is used to determine project in your before and after studies. New travelers may if your results could have been due to luck and not the show up on the facility who were not there before it was con- improvement. structed. Odds are that the travel times of these new travelers Hypothesis testing determines if the analyst has performed on the facility were not captured in your before study of the an adequate number of measurements for the before and facility, unless other routes serving the same trip patterns also after conditions to truly tell if the improvement was effective were studied. So you may be underestimating the benefits to at the analyst's desired level of confidence. the public of the improvement if you only consider the net The test begins with the specification of a null hypothe- change in travel times on the facility itself. sis that you hopefully will be able to reject: "The measured These are weaknesses of any before/after study the analyst difference in mean travel time for the before and after must seek to minimize, but can never completely eliminate. conditions occurred by random chance. There really is no Another common pitfall, but one the analyst can avoid, is significant difference in the mean travel time between the obtaining insufficient numbers of before measurements. The before and after conditions." A statistic is computed for a number of before measurements of travel time or delay, and selected level of confidence, and if the difference between the variance among the measurements will determine the ulti- the two means is less than that statistic, then the null mate sensitivity of the before/after test. The analyst should hypothesis is accepted and it is concluded that there is consult Section 3.3 and use the methods there to determine an insufficient evidence to prove that the after condition is adequate number of measurements for the before condition. better than the before condition. The analyst can accept this outcome, or alternatively, either make more measure- ments of travel time for each condition (to improve the 4.3 Selection of Performance sensitivity of the test) or relax standards (confidence level) Measures for Before/After for rejecting the null hypothesis. Studies The specification of the problem is: Null hypothesis: The analyst needs to select the set of performance measures that will be used to determine if the improvement has resulted H0 : x - y = 0