Cover Image

Not for Sale

View/Hide Left Panel
Click for next page ( 47

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 46
Chapter 3: Measurement Instead, in customer-driven benchmarking, a variety of customer-driven outcome measures are treated as a group but remain separate. Similarly, a variety of resource measures are treated as a group but remain separate. Factors outside the control of the agency--weather, terrain, traffic volumes--are also treated as a group, but remain separate. Outputs have a bearing on the analysis because they help establish the level of effort for each benchmarking unit and the comparability of their performances. The idea is to simultaneously preserve each of these measures while (1) continually bearing in mind the importance of looking at the outcomes achieved relative to the resources used and (2) taking into account hardship factors outside the control of each organizational unit and their level of production. OUTCOMES In customer-driven benchmarking, three important kinds outcomes can be measured: 1. Customer satisfaction, 2. Condition of assets and other attributes of roads, and 3. Value received by the customer. Customer Satisfaction Customer satisfaction is a topic addressed in countless books and articles on marketing and market research, as well as in specialized fields such as psychology. Benchmarking is ultimately about making continuous improvements through the identification and adoption of best practices in order to equal or exceed the satisfaction of the customer. Measuring changes in customer satisfaction over time provides the feedback regarding how well you are doing. An important measurement tool for assessing customer satisfaction is statistically valid measures of customer satisfaction obtained from administering a survey using random sampling. Types of Surveys As you plan to get started with benchmarking, important questions you need to address are as follows: 48

OCR for page 46
What role will surveys of customer satisfaction play as a measurement tool? What types of survey data are currently available? Should you develop your own survey? Should you rely on surveys developed by others? You will also need to address the cost and timing of developing your surveys. If you decide to develop your own surveys, you will also want to address the issue of what related questions and answers you should be seeking in the survey--for example, do you want to merely learn about customer satisfaction regarding the department's maintenance products and services or do you also want to learn about customer preferences and expectations, the relative value of their preferences as they make tradeoffs, and perhaps even what they are willing to pay? National Quality Initiative As mentioned previously, FHWA, AASHTO, the American Public Works Association, and various industry associations are supporting the National Quality Initiative (NQI) in Transportation. The NQI develops and administers, with the assistance of the U.S. DOT, a national survey. In May 1996, the NQI released the results of a scientific random sample of 2,205 households that assessed customer satisfaction and preferences regarding the nation's highway system. Summary data from the survey's categorical questions are accurate within plus or minus 2 percent with, 95 percent confidence.1 The NQI survey included numerous questions that pertain to the outcomes of road maintenance. It is vitally important to recognize that the NQI survey, in attempting to determine customer satisfaction, focuses upon important attributes of highways. In the case of maintenance, the key issue is what the customer satisfaction is with regard to the attributes of maintenance products and services--for example, the NQI asks how satisfied survey respondents are regarding the smoothness of roads. It is not possible to solely associate the smoothness of roads with maintenance; nonetheless, certain types of road 1 National Quality Initiative Steering Committee, National Highway User Survey, prepared by Coopers & Lybrand and the Opinion Research Corporation, May 1996. 49

OCR for page 46
Chapter 3: Measurement maintenance, patching potholes, and resurfacing contribute significantly to the smoothness of roads. Many administrators and managers in state DOTs have long believed that the driving public placed safety above smooth pavement in order of importance. An important result of the NQI survey is the revelation that road users' preferences are the reverse: they place more importance on road smoothness than safety. Results such as this have been highly influential to program managers in making resource allocation decisions. During the last several years, a number of states have increased the relative expenditures on actions that would improve pavement smoothness. Figure 6 presents the NQI survey questions that are the most pertinent to road maintenance. Figures 7a through 7f show the results that were obtained from the 1996 survey. Thinking about the areas we just discussed, how satisfied are you with the following? A. Traffic Flow a. Level of congestion b. Toll booth delays c. Construction delays d. Accident clean-up B. Pavement Conditions e. Smooth ride f. Surface appearance g. Durability C. Visual Appeal h. Appearance of sound barriers i. Landscaping j. Design of rest areas k. Compatibility with the natural environment D. Maintenance Response Time l. Litter removal m. Snow removal n. Pavement repairs o. Guardrail and barrier repairs p. Rest area cleaning E. Travel Amenities q. Number of rest areas or service plazas r. Variety of rest areas or plaza services s. Number of emergency call boxes and radio advisory stations t. Signs for motorist services and attractions u. Signs for mileage and destinations Figure 6. Sample NQI Survey Questions 50

OCR for page 46
Figure 7a. Satisfaction with Attributes of Highway System Figure 7b. Satisfaction with Visual Appeal i i Figure 7c. Satisfaction with Safety Items 51

OCR for page 46
Chapter 3: Measurement Appearance Figure 7d. Satisfaction with Bridge Conditions Areas Variety of Rest Areas Figure 7e. Satisfaction with Travel Amenities Figure 7f. Satisfaction with Pavement Conditions 52

OCR for page 46
Because the NQI survey provides a national baseline of data, many states have incorporated questions from the NQI survey into their own customer satisfaction surveys. This inclusion allows states to compare the results obtained from their own surveys with those obtained nationally. Kentucky, for example, compared the results of customer satisfaction surveys conducted in 1995 and 1996 with the national survey results.2 Potentially, results could be compared with other states to do a simple form of customer-driven benchmarking. The significance of the NQI survey is that the maintenance- related questions represent a set of widely or commonly recognized measures of customer satisfaction. Having an agreed- upon set of questions for assessing customer satisfaction makes it easier to do benchmarking. Note that the NQI survey instrument was revised in 2000 but contains the same maintenance-related questions that were included in the early survey. Survey comparisons between the results of the 1995 and 2000 surveys can be found in "Moving Ahead, The American Public Speaks on Roadways and Transportation in Communities." Agency Surveys An alternative to using results from the NQI survey or to incorporating NQI survey questions into your own questionnaire is to develop a survey tailored to your own maintenance products and services and to the issues in your own state, city, or county or bridge, tunnel, and turnpike authority. Many states are seeking more detailed insight about customer preferences and satisfaction than the NQI survey questions can provide, and thus have developed additional or more refined surveys and questions. In constructing survey questions, you will need to first define products and services and identify their corresponding attributes--steps in the benchmarking process discussed in Part II. Then you will need to develop questions regarding customer preferences and satisfaction corresponding to each attribute. You will have to choose a suitable response scale. 2 Kentucky Transportation Cabinet, The Path, Mid-Year 1999 Report, p. 35. 53

OCR for page 46
Chapter 3: Measurement Appendix B contains tables showing maintenance attributes and corresponding customer outcome measures found in surveys developed and administered by various states--for example, the State of California has a question to assess customer satisfaction regarding response time to emergency situations. This question pertains to the maintenance product category of "Maintenance Response to National Disasters." Respondents (i.e., customers) rate their satisfaction on scale of 0 to 10, where 10 represents "extremely satisfied" and 0 "extremely dissatisfied." This question is intended to provide the California DOT (Caltrans) with feedback regarding how the state does in responding to maintenance problems associated with mudslides, floods, earthquakes, and so on. Note that a random sample is unlikely to include very many people who have actually been in an emergency situation. The state was probably seeking information regarding public perceptions of the responsiveness of Caltrans to natural disasters, even though the respondent was unlikely to have experienced one directly. The Caltrans survey also included a series of related questions intended to assess customer preferences regarding response time for time-sensitive maintenance activities such as sign repair, traffic delays due to maintenance, and pothole repairs. Respondents were asked to state whether the preferred response time should be within 15 minutes, 30 minutes, 60 minutes, 1 day, or 1 week. Survey Design and Administration If you decide to develop your own survey to be used as a benchmarking measurement tool, you should go through the standard steps for developing sound surveys: 1. Focus groups, 2. Survey design and pretesting, 3. Coding guide and database design, 4. Sample design, 5. Administration, and 6. Summarization and analysis. Further guidance on developing and administering surveys is found in Appendix C. 54

OCR for page 46
Condition of Assets The second category of outcome measures that are essential for customer-driven benchmarking consists of condition measures. Condition includes the condition of assets and other attributes of roads. Condition is important to customers from three standpoints. First, the physical attributes of roads directly affect the experience of road users. Examples of these attributes are pavement smoothness and comfort, ruts and shoulder edge drop-offs, the narrowness of bridges, the brightness of signs at night, obstructions in the roadway, and whether ice is on the road. Second, virtually every customer of roads pays for the roads either directly or indirectly. The condition of roads is important to customers of roads, if for no other reason than they do not wish to pay higher gas and property taxes. Responsible stewardship of the roads through proper and timely maintenance preserves the investment in highways and streets and avoids wasting money that could be used for more productive purposes. Third, not only does condition relate to the physical condition of roads, but also to the condition that results from maintenance services such as mowing; picking up litter; trimming brush and trees; cleaning ditches; removing drainage system blockages; controlling erosion; cleaning rest areas; and landscaping, including planting wildflowers. Value Received by the Customer The third category is value received by the customer. It is important to remember that customers of maintenance "wear three hats, "so to speak. One set of customers consists of those who use the roads. This set of customers is primarily concerned with avoiding road user costs such as travel time, vehicle- operating costs, and accident costs. The second set of customers consists of those who pay for the roads and generally, but not necessarily, consists of those who use the roads. These customers do not like it if the taxes or fees they pay increase in order to pay for extra costs that could have been avoided if the roads were maintained by performing the right treatment at the right time in the right place. In other words, by not deferring needed maintenance one avoids increased maintenance, rehabilitation, and reconstruction costs in the future. 55