Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 46
Chapter 3: Measurement
Instead, in customer-driven benchmarking, a variety of
customer-driven outcome measures are treated as a group but
remain separate. Similarly, a variety of resource measures are
treated as a group but remain separate. Factors outside the
control of the agency--weather, terrain, traffic volumes--are also
treated as a group, but remain separate. Outputs have a bearing
on the analysis because they help establish the level of effort for
each benchmarking unit and the comparability of their
performances.
The idea is to simultaneously preserve each of these measures
while (1) continually bearing in mind the importance of looking
at the outcomes achieved relative to the resources used and (2)
taking into account hardship factors outside the control of each
organizational unit and their level of production.
OUTCOMES
In customer-driven benchmarking, three important kinds
outcomes can be measured:
1. Customer satisfaction,
2. Condition of assets and other attributes of roads, and
3. Value received by the customer.
Customer Satisfaction
Customer satisfaction is a topic addressed in countless books and
articles on marketing and market research, as well as in
specialized fields such as psychology. Benchmarking is
ultimately about making continuous improvements through the
identification and adoption of best practices in order to equal or
exceed the satisfaction of the customer. Measuring changes in
customer satisfaction over time provides the feedback regarding
how well you are doing.
An important measurement tool for assessing customer
satisfaction is statistically valid measures of customer satisfaction
obtained from administering a survey using random sampling.
Types of Surveys
As you plan to get started with benchmarking, important
questions you need to address are as follows:
48
OCR for page 47
What role will surveys of customer satisfaction play as a
measurement tool?
What types of survey data are currently available?
Should you develop your own survey?
Should you rely on surveys developed by others?
You will also need to address the cost and timing of developing
your surveys. If you decide to develop your own surveys, you
will also want to address the issue of what related questions and
answers you should be seeking in the survey--for example, do
you want to merely learn about customer satisfaction regarding
the department's maintenance products and services or do you
also want to learn about customer preferences and expectations,
the relative value of their preferences as they make tradeoffs, and
perhaps even what they are willing to pay?
National Quality Initiative
As mentioned previously, FHWA, AASHTO, the American
Public Works Association, and various industry associations are
supporting the National Quality Initiative (NQI) in
Transportation. The NQI develops and administers, with the
assistance of the U.S. DOT, a national survey. In May 1996, the
NQI released the results of a scientific random sample of 2,205
households that assessed customer satisfaction and preferences
regarding the nation's highway system. Summary data from the
survey's categorical questions are accurate within plus or minus
2 percent with, 95 percent confidence.1
The NQI survey included numerous questions that pertain to the
outcomes of road maintenance. It is vitally important to
recognize that the NQI survey, in attempting to determine
customer satisfaction, focuses upon important attributes of
highways. In the case of maintenance, the key issue is what the
customer satisfaction is with regard to the attributes of
maintenance products and services--for example, the NQI asks
how satisfied survey respondents are regarding the smoothness
of roads. It is not possible to solely associate the smoothness of
roads with maintenance; nonetheless, certain types of road
1
National Quality Initiative Steering Committee, National Highway User Survey, prepared
by Coopers & Lybrand and the Opinion Research Corporation, May 1996.
49
OCR for page 48
Chapter 3: Measurement
maintenance, patching potholes, and resurfacing contribute
significantly to the smoothness of roads.
Many administrators and managers in state DOTs have long
believed that the driving public placed safety above smooth
pavement in order of importance. An important result of the NQI
survey is the revelation that road users' preferences are the
reverse: they place more importance on road smoothness than
safety. Results such as this have been highly influential to
program managers in making resource allocation decisions.
During the last several years, a number of states have increased
the relative expenditures on actions that would improve
pavement smoothness. Figure 6 presents the NQI survey
questions that are the most pertinent to road maintenance.
Figures 7a through 7f show the results that were obtained from
the 1996 survey.
Thinking about the areas we just discussed, how satisfied are you with the following?
A. Traffic Flow
a. Level of congestion
b. Toll booth delays
c. Construction delays
d. Accident clean-up
B. Pavement Conditions
e. Smooth ride
f. Surface appearance
g. Durability
C. Visual Appeal
h. Appearance of sound barriers
i. Landscaping
j. Design of rest areas
k. Compatibility with the natural environment
D. Maintenance Response Time
l. Litter removal
m. Snow removal
n. Pavement repairs
o. Guardrail and barrier repairs
p. Rest area cleaning
E. Travel Amenities
q. Number of rest areas or service plazas
r. Variety of rest areas or plaza services
s. Number of emergency call boxes and radio advisory stations
t. Signs for motorist services and attractions
u. Signs for mileage and destinations
Figure 6. Sample NQI Survey Questions
50
OCR for page 49
Figure 7a. Satisfaction with Attributes of Highway System
Figure 7b. Satisfaction with Visual Appeal
i
i
Figure 7c. Satisfaction with Safety Items
51
OCR for page 50
Chapter 3: Measurement
Appearance
Figure 7d. Satisfaction with Bridge Conditions
Areas
Variety of Rest Areas
Figure 7e. Satisfaction with Travel Amenities
Figure 7f. Satisfaction with Pavement Conditions
52
OCR for page 51
Because the NQI survey provides a national baseline of data,
many states have incorporated questions from the NQI survey
into their own customer satisfaction surveys. This inclusion
allows states to compare the results obtained from their own
surveys with those obtained nationally. Kentucky, for example,
compared the results of customer satisfaction surveys conducted
in 1995 and 1996 with the national survey results.2 Potentially,
results could be compared with other states to do a simple form
of customer-driven benchmarking.
The significance of the NQI survey is that the maintenance-
related questions represent a set of widely or commonly
recognized measures of customer satisfaction. Having an agreed-
upon set of questions for assessing customer satisfaction makes it
easier to do benchmarking.
Note that the NQI survey instrument was revised in 2000 but
contains the same maintenance-related questions that were
included in the early survey. Survey comparisons between the
results of the 1995 and 2000 surveys can be found in "Moving
Ahead, The American Public Speaks on Roadways and Transportation
in Communities."
Agency Surveys
An alternative to using results from the NQI survey or to
incorporating NQI survey questions into your own questionnaire
is to develop a survey tailored to your own maintenance
products and services and to the issues in your own state, city, or
county or bridge, tunnel, and turnpike authority.
Many states are seeking more detailed insight about customer
preferences and satisfaction than the NQI survey questions can
provide, and thus have developed additional or more refined
surveys and questions.
In constructing survey questions, you will need to first define
products and services and identify their corresponding
attributes--steps in the benchmarking process discussed in Part
II. Then you will need to develop questions regarding customer
preferences and satisfaction corresponding to each attribute. You
will have to choose a suitable response scale.
2
Kentucky Transportation Cabinet, The Path, Mid-Year 1999 Report, p. 35.
53
OCR for page 52
Chapter 3: Measurement
Appendix B contains tables showing maintenance attributes and
corresponding customer outcome measures found in surveys
developed and administered by various states--for example, the
State of California has a question to assess customer satisfaction
regarding response time to emergency situations. This question
pertains to the maintenance product category of "Maintenance
Response to National Disasters." Respondents (i.e., customers)
rate their satisfaction on scale of 0 to 10, where 10 represents
"extremely satisfied" and 0 "extremely dissatisfied." This
question is intended to provide the California DOT (Caltrans)
with feedback regarding how the state does in responding to
maintenance problems associated with mudslides, floods,
earthquakes, and so on. Note that a random sample is unlikely to
include very many people who have actually been in an
emergency situation. The state was probably seeking information
regarding public perceptions of the responsiveness of Caltrans to
natural disasters, even though the respondent was unlikely to
have experienced one directly.
The Caltrans survey also included a series of related questions
intended to assess customer preferences regarding response time
for time-sensitive maintenance activities such as sign repair,
traffic delays due to maintenance, and pothole repairs.
Respondents were asked to state whether the preferred response
time should be within 15 minutes, 30 minutes, 60 minutes, 1 day,
or 1 week.
Survey Design and Administration
If you decide to develop your own survey to be used as a
benchmarking measurement tool, you should go through the
standard steps for developing sound surveys:
1. Focus groups,
2. Survey design and pretesting,
3. Coding guide and database design,
4. Sample design,
5. Administration, and
6. Summarization and analysis.
Further guidance on developing and administering surveys is
found in Appendix C.
54
OCR for page 53
Condition of Assets
The second category of outcome measures that are essential for
customer-driven benchmarking consists of condition measures.
Condition includes the condition of assets and other attributes
of roads.
Condition is important to customers from three standpoints. First,
the physical attributes of roads directly affect the experience of
road users. Examples of these attributes are pavement smoothness
and comfort, ruts and shoulder edge drop-offs, the narrowness of
bridges, the brightness of signs at night, obstructions in the
roadway, and whether ice is on the road.
Second, virtually every customer of roads pays for the roads
either directly or indirectly. The condition of roads is important to
customers of roads, if for no other reason than they do not wish to
pay higher gas and property taxes. Responsible stewardship of
the roads through proper and timely maintenance preserves the
investment in highways and streets and avoids wasting money
that could be used for more productive purposes.
Third, not only does condition relate to the physical condition of
roads, but also to the condition that results from maintenance
services such as mowing; picking up litter; trimming brush and
trees; cleaning ditches; removing drainage system blockages;
controlling erosion; cleaning rest areas; and landscaping,
including planting wildflowers.
Value Received by the Customer
The third category is value received by the customer. It is
important to remember that customers of maintenance "wear
three hats, "so to speak. One set of customers consists of those
who use the roads. This set of customers is primarily concerned
with avoiding road user costs such as travel time, vehicle-
operating costs, and accident costs.
The second set of customers consists of those who pay for the roads
and generally, but not necessarily, consists of those who use the
roads. These customers do not like it if the taxes or fees they pay
increase in order to pay for extra costs that could have been
avoided if the roads were maintained by performing the right
treatment at the right time in the right place. In other words, by not
deferring needed maintenance one avoids increased maintenance,
rehabilitation, and reconstruction costs in the future.
55
