Click for next page ( 38

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 37
37 CHAPTER 5 Identification of Deficiencies 5.1 Introduction distinguish between healthy and sick. Similarly an agency must establish what it considers to be good health for its This chapter describes how to use travel time and delay to transportation system or its vehicle fleet operations. identify real performance deficiencies in the transportation Acceptable levels for transportation system performance system and how to distinguish these deficiencies from ran- measures must usually be determined based on the dom variations in the data. A diagnosis chart is provided to agency's experience of what constitutes acceptable per- help analysts identify the likely root causes of identified travel formance for its decision makers and the constituents they time, delay, and variability deficiencies. report to. Chapter 2 provides a discussion on the selection The guidance in this chapter is designed to be applied after of appropriate measures and the setting of acceptable the analyst has identified the agency's performance standards values for each measure. and collected the data (or forecasts) on system performance. Note that when assessing deficiencies using field meas- This chapter provides limited guidance on the inclusion of urements, the agency performance standards for the facility uncertainty in the treatment of forecasted travel time and delay or the trip must be more precise than simply Level of based upon a limited set of data from California. Ideally, agen- Service D. The standard must state over how long the cies will be able to develop their own data on variability and measurement is taken and whether or not brief violations apply it in lieu of default values provided here. can be tolerated. The chapter starts by reiterating the key considerations involved in defining agency performance standards and col- lecting data for the purpose of deficiency assessments. Read- 5.3 Data Collection ers are referred to the appropriate chapter for additional The development of a data collection plan and determin- background information and guidance on selection of per- ing the required sample size for measurements are discussed formance measures, setting of performance standards, data in Chapter 3. If travel time and delay data cannot be collection, and forecasting of travel time and delay. measured directly, they must be estimated using the meth- Guidance is then provided on the statistical tests needed to ods in Chapter 6. distinguish between apparent violations of agency standards (due to sampling error) and actual violations. Additional guid- ance is provided on the incorporation of uncertainty into the 5.4 Comparing Field Data use of forecasted system performance for assessing deficien- to Performance Standards cies. Finally, the chapter provides a diagnosis chart for identi- Analysts must take great care to ensure that they have fying the likely root causes of travel time, delay, and reliability measured performance in the field using a method consistent deficiencies. to the performance standard set by the agency. For example, an agency may have a LOS standard for the peak hour of "D" for traffic signals. HCM (5) defines the 5.2 Quantifying Agency Standards threshold for LOS "D" as no more than 55 seconds of control To know if a patient is sick or not, you need to have delay averaged over the worst contiguous 15 minutes of the some established methods for measuring health (such as peak hour. Thus, there may be individual signal cycles where body temperature) and standards for each measure that the average delay for vehicles is greater than 55 seconds, but

OCR for page 37
38 if the average for the highest volume 15-minute period is less Any statistical test is subject to two types of error. The than that, then it is LOS "D" or better. Indeed almost half of probability of mistakenly accepting the null hypothesis is a the vehicles during the worst 15 minutes may experience de- Type I error, called "alpha" in the equations below. This is lays greater than 55 seconds and the intersection would still usually set quite small (e.g., at 5 percent to get a 95 percent be LOS "D." confidence level test). Analysts also must exclude delay measurements not related There also is the chance of mistakenly rejecting the null hy- to the performance standard from their computations. For pothesis. This is called Type II error and it varies with the example, HCM (5) excludes from its LOS standards delays difference between the sample means, their standard devia- caused by accidents, poor weather, etc. Only delay caused by tion, and the sample size. (Analysts should consult standard the signal control (control delay) is included in the perfor- statistical textbooks for tables on the Type II errors associated mance measurement for establishing signal LOS. Analysts with different confidence intervals and sample sizes.) The also will need to determine if holidays, weekends, and days analyst has less control over this type of error and its proba- with special events are to be excluded from the comparison bility can be quite a bit larger than the Type 1 error. to the agency performance standard. Other performance The usual approach is to adopt the null hypothesis for measures described in Chapter 2, such as the TTI, include which a Type II error (mistakenly rejecting the null hypoth- nonrecurring delay from incidents and other causes men- esis when it is really true) has the least consequences for the tioned above, but also can be calculated excluding such agency. This results in the apparently perverse approach of events. If the situation and analytical framework call for con- adopting as your null hypothesis the very condition you do sideration of nonrecurring delay in the identification of defi- not want to be true (e.g., the actual performance violates ciencies and testing of solutions, these measurements should agency standards). be left in the data computations, and the appropriate per- Analysts who wish to be very sure they do not say there is a formance measures used in the analysis. deficiency when in reality there is no deficiency will adopt the first null hypothesis above (i.e., "everything is not fine"). The test then will have a low probability (completely controlled 5.4.1 Taking Luck Into Account by the analyst) of mistakenly accepting this null hypothesis In Field Measurements (a Type I error) and in effect concluding there is a deficiency Once the standards have been set and the performance data when in reality there is no problem. have been gathered, the next task is to determine if one or more Conversely, analysts who wish to be very sure they do not say of the performance standards have been violated. With field that everything is fine, when in reality there actually is a prob- data, this is more difficult than simply comparing the results to lem will adopt the second null hypothesis (i.e., "everything is the agency standards. There is usually a great deal of day-to- fine"). Again, the test will have a low probability of mistakenly day, hour-to-hour, and even minute-by-minute fluctuation in accepting the null hypothesis and concluding there is no prob- travel times, and especially in delays, for a transportation sys- lem when in fact there is a problem. tem component. So the analyst must assess the degree "luck" An example of the first condition might be where the risk played a part in meeting or failing to meet the performance or opportunity cost for mistakenly identifying a problem standards. Statistical hypothesis testing provides the tool for when none exists is very high (e.g., condemning property ruling out luck as a contributor to meeting or failing to meet to expand a facility when the benefits of the expansion are the agency's performance standards. not statistically significant). An example of the opposite To determine whether or not you have gathered sufficient situation might involve public safety (e.g., failing to iden- evidence to establish that the agency is meeting or failing to tify a statistically significant increase in accidents at a given meet its transportation system performance standards, it is location). necessary to perform a statistical hypothesis test of the differ- For each null hypothesis the test is as follows. ence between the mean result of your field measurements and the agency's performance standard. To perform a statistical test, analysts must adopt a baseline 5.4.2 First Null Hypothesis (null) hypothesis that they then can reject if the test is suc- (Don't Cry Wolf Needlessly) cessful. The null hypothesis can either be: The analyst will reject the null hypothesis that the system fails to meet agency standards (with confidence level equal to 1. The actual performance in the field violates the agency's 1-alpha) if the following equation is true. performance standards, or 2. The actual performance in the field meets the agency's s x < q + t (1- );(n-1) (Eq. 5.1) performance standards. n