Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 37
37
CHAPTER 5
Identification of Deficiencies
5.1 Introduction distinguish between healthy and sick. Similarly an agency
must establish what it considers to be good health for its
This chapter describes how to use travel time and delay to transportation system or its vehicle fleet operations.
identify real performance deficiencies in the transportation Acceptable levels for transportation system performance
system and how to distinguish these deficiencies from ran- measures must usually be determined based on the
dom variations in the data. A diagnosis chart is provided to agency's experience of what constitutes acceptable per-
help analysts identify the likely root causes of identified travel formance for its decision makers and the constituents they
time, delay, and variability deficiencies. report to. Chapter 2 provides a discussion on the selection
The guidance in this chapter is designed to be applied after of appropriate measures and the setting of acceptable
the analyst has identified the agency's performance standards values for each measure.
and collected the data (or forecasts) on system performance. Note that when assessing deficiencies using field meas-
This chapter provides limited guidance on the inclusion of urements, the agency performance standards for the facility
uncertainty in the treatment of forecasted travel time and delay or the trip must be more precise than simply Level of
based upon a limited set of data from California. Ideally, agen- Service D. The standard must state over how long the
cies will be able to develop their own data on variability and measurement is taken and whether or not brief violations
apply it in lieu of default values provided here. can be tolerated.
The chapter starts by reiterating the key considerations
involved in defining agency performance standards and col-
lecting data for the purpose of deficiency assessments. Read- 5.3 Data Collection
ers are referred to the appropriate chapter for additional The development of a data collection plan and determin-
background information and guidance on selection of per- ing the required sample size for measurements are discussed
formance measures, setting of performance standards, data in Chapter 3. If travel time and delay data cannot be
collection, and forecasting of travel time and delay. measured directly, they must be estimated using the meth-
Guidance is then provided on the statistical tests needed to ods in Chapter 6.
distinguish between apparent violations of agency standards
(due to sampling error) and actual violations. Additional guid-
ance is provided on the incorporation of uncertainty into the 5.4 Comparing Field Data
use of forecasted system performance for assessing deficien- to Performance Standards
cies. Finally, the chapter provides a diagnosis chart for identi- Analysts must take great care to ensure that they have
fying the likely root causes of travel time, delay, and reliability measured performance in the field using a method consistent
deficiencies. to the performance standard set by the agency.
For example, an agency may have a LOS standard for the
peak hour of "D" for traffic signals. HCM (5) defines the
5.2 Quantifying Agency Standards
threshold for LOS "D" as no more than 55 seconds of control
To know if a patient is sick or not, you need to have delay averaged over the worst contiguous 15 minutes of the
some established methods for measuring health (such as peak hour. Thus, there may be individual signal cycles where
body temperature) and standards for each measure that the average delay for vehicles is greater than 55 seconds, but
OCR for page 38
38
if the average for the highest volume 15-minute period is less Any statistical test is subject to two types of error. The
than that, then it is LOS "D" or better. Indeed almost half of probability of mistakenly accepting the null hypothesis is a
the vehicles during the worst 15 minutes may experience de- Type I error, called "alpha" in the equations below. This is
lays greater than 55 seconds and the intersection would still usually set quite small (e.g., at 5 percent to get a 95 percent
be LOS "D." confidence level test).
Analysts also must exclude delay measurements not related There also is the chance of mistakenly rejecting the null hy-
to the performance standard from their computations. For pothesis. This is called Type II error and it varies with the
example, HCM (5) excludes from its LOS standards delays difference between the sample means, their standard devia-
caused by accidents, poor weather, etc. Only delay caused by tion, and the sample size. (Analysts should consult standard
the signal control (control delay) is included in the perfor- statistical textbooks for tables on the Type II errors associated
mance measurement for establishing signal LOS. Analysts with different confidence intervals and sample sizes.) The
also will need to determine if holidays, weekends, and days analyst has less control over this type of error and its proba-
with special events are to be excluded from the comparison bility can be quite a bit larger than the Type 1 error.
to the agency performance standard. Other performance The usual approach is to adopt the null hypothesis for
measures described in Chapter 2, such as the TTI, include which a Type II error (mistakenly rejecting the null hypoth-
nonrecurring delay from incidents and other causes men- esis when it is really true) has the least consequences for the
tioned above, but also can be calculated excluding such agency. This results in the apparently perverse approach of
events. If the situation and analytical framework call for con- adopting as your null hypothesis the very condition you do
sideration of nonrecurring delay in the identification of defi- not want to be true (e.g., the actual performance violates
ciencies and testing of solutions, these measurements should agency standards).
be left in the data computations, and the appropriate per- Analysts who wish to be very sure they do not say there is a
formance measures used in the analysis. deficiency when in reality there is no deficiency will adopt the
first null hypothesis above (i.e., "everything is not fine"). The
test then will have a low probability (completely controlled
5.4.1 Taking Luck Into Account
by the analyst) of mistakenly accepting this null hypothesis
In Field Measurements
(a Type I error) and in effect concluding there is a deficiency
Once the standards have been set and the performance data when in reality there is no problem.
have been gathered, the next task is to determine if one or more Conversely, analysts who wish to be very sure they do not say
of the performance standards have been violated. With field that everything is fine, when in reality there actually is a prob-
data, this is more difficult than simply comparing the results to lem will adopt the second null hypothesis (i.e., "everything is
the agency standards. There is usually a great deal of day-to- fine"). Again, the test will have a low probability of mistakenly
day, hour-to-hour, and even minute-by-minute fluctuation in accepting the null hypothesis and concluding there is no prob-
travel times, and especially in delays, for a transportation sys- lem when in fact there is a problem.
tem component. So the analyst must assess the degree "luck" An example of the first condition might be where the risk
played a part in meeting or failing to meet the performance or opportunity cost for mistakenly identifying a problem
standards. Statistical hypothesis testing provides the tool for when none exists is very high (e.g., condemning property
ruling out luck as a contributor to meeting or failing to meet to expand a facility when the benefits of the expansion are
the agency's performance standards. not statistically significant). An example of the opposite
To determine whether or not you have gathered sufficient situation might involve public safety (e.g., failing to iden-
evidence to establish that the agency is meeting or failing to tify a statistically significant increase in accidents at a given
meet its transportation system performance standards, it is location).
necessary to perform a statistical hypothesis test of the differ- For each null hypothesis the test is as follows.
ence between the mean result of your field measurements and
the agency's performance standard.
To perform a statistical test, analysts must adopt a baseline 5.4.2 First Null Hypothesis
(null) hypothesis that they then can reject if the test is suc- (Don't Cry Wolf Needlessly)
cessful. The null hypothesis can either be: The analyst will reject the null hypothesis that the system
fails to meet agency standards (with confidence level equal to
1. The actual performance in the field violates the agency's 1-alpha) if the following equation is true.
performance standards, or
2. The actual performance in the field meets the agency's s
x < q + t (1- );(n-1) (Eq. 5.1)
performance standards. n
