National Academies Press: OpenBook

Guide for Customer-Driven Benchmarking of Maintenance Activities (2004)

Chapter: Appendix C - Guidance on Designing and Administering Surveys

« Previous: Appendix B - Catalog of Benchmarking Measures
Page 210
Suggested Citation:"Appendix C - Guidance on Designing and Administering Surveys." National Academies of Sciences, Engineering, and Medicine. 2004. Guide for Customer-Driven Benchmarking of Maintenance Activities. Washington, DC: The National Academies Press. doi: 10.17226/13720.
×
Page 210
Page 211
Suggested Citation:"Appendix C - Guidance on Designing and Administering Surveys." National Academies of Sciences, Engineering, and Medicine. 2004. Guide for Customer-Driven Benchmarking of Maintenance Activities. Washington, DC: The National Academies Press. doi: 10.17226/13720.
×
Page 211
Page 212
Suggested Citation:"Appendix C - Guidance on Designing and Administering Surveys." National Academies of Sciences, Engineering, and Medicine. 2004. Guide for Customer-Driven Benchmarking of Maintenance Activities. Washington, DC: The National Academies Press. doi: 10.17226/13720.
×
Page 212
Page 213
Suggested Citation:"Appendix C - Guidance on Designing and Administering Surveys." National Academies of Sciences, Engineering, and Medicine. 2004. Guide for Customer-Driven Benchmarking of Maintenance Activities. Washington, DC: The National Academies Press. doi: 10.17226/13720.
×
Page 213

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

217 APPENDIX C: GUIDANCE ON DESIGNING AND ADMINISTERING SURVEYS If you decide to develop your own survey to be used as a benchmarking measurement tool, you should go through the standard steps of developing sound surveys: • Focus groups, • Survey design and pretesting, • Coding guide and database design, • Sample design, • Administration, and • Analysis and summarization. FOCUS GROUPS In order to understand what issues are important to your customers, you should begin any survey process with a series of focus groups. The main reasons to use a focus group are to identify what types of questions to include in your survey and to develop insight regarding how to provide a maintenance program that is more sensitive to the needs and desires of your customers. Focus groups should be conducted in different parts of your jurisdiction to gain insight into how geography, urbanization, traffic congestion, and other factors outside the control of maintenance organizations affect customer satisfaction regarding the delivery of maintenance products and services. As much as is possible, focus group participants should be representative of all customers—including motorists, truckers, and adjacent property owners—as well as of different important categories such as gender, age, income, and ethnicity. The focus groups should be limited in size. Seven participants is a good size, but focus groups that are larger or smaller will work, too. Questions posed to focus group participants should be nondirective—do not steer respondents in any direction. Ask questions in a way that prompts focus group participants to freely discuss the issues, expectations, preferences, and satisfaction they experience as customers of road maintenance. Questions you might ask are • What products and services do you perceive the maintenance organization delivers? • What attributes of these products and services do you perceive? • Which attributes are the most important?

• What is your expectation regarding different types of maintenance—for example, snow and ice control, mowing, and pavement resurfacing? • Are you aware of other maintenance products and services that the DOT provides but that you cannot easily perceive (e.g., pavement durability, tree maintenance)? • In your opinion, what factors affect the service quality of roads? • In making travel decisions, what highway attributes have the most influence on your choices? Do you perceive that any of these attributes are related to maintenance operations? From your notes from the focus groups, you should be able to make a list of questions that are prime candidates for incorporation into a survey. SURVEY DESIGN AND PRETESTING Next you will need to design the survey. You should have someone experienced in survey design draft it. You will have to decide how it will be administered, whether by mail, phone, or some other means. The administration process will influence the survey design. If the respondent will see the questions—for example, via a mail survey—a simple, pleasing design and clear layout is essential. Regardless of the format, the questions need to be easily understood and unambiguous. Once you have drafted the survey, it should be pretested with a group of representative respondents and then revised. It may require additional pretesting and revision. CODING GUIDE AND DATABASE DESIGN When you have finalized your survey design, you will need to develop a coding guide to assist recording the results. You will also need to design the database in which you will enter the survey responses. You will have to specify a record layout for all data fields, including the order of each field, the type, and the length. This layout is usually sufficient to specify the database in a typical statistical analysis package. SAMPLE DESIGN You will need a statistician or a person with a strong statistical background to design a scientific sample for the survey. The survey should have scientific validity: you should establish target levels of statistical accuracy and confidence for the survey as a whole and for various customer groups or classes. In either case, you will want to take a random sample of customers; if you want to achieve specified levels of accuracy and confidence for subgroups, you will need to take some type of stratified random sample. For help, you should consult a statistician experienced in developing a sampling plan. Appendix C: Guidance on Designing and Administering Surveys 218

219 ADMINISTRATION When you have completed your survey design, sampling plan, coding guide, and database design, you are ready to administer the survey. While you are proceeding through these steps, you should have been making plans regarding how you will administer the survey. There are many possibilities, but the most important are the following: • Mail, • Phone, • Personal interviews, and • Internet. Your choice will depend upon the response rates you expect, the biases, and the costs. You may wish to enlist a market research firm or service bureau that specializes in conducting surveys. They will have all the tools necessary to administer the survey efficiently, including procedures for selecting a random sample and conducting computer-assisted interviews. Survey administration over the Internet requires an altogether different skill set. When your planning is complete, you will administer your survey. When the survey is completed, you will need to go through a number of additional steps before you can analyze the results: • Check the survey responses and clean the data by removing nonsensical or extreme answers or respondents. • If the survey results are not already in a database, you will need to put them in a database. ANALYSIS AND SUMMARIZATION Finally, you need to analyze and summarize the results. You should apply a standard statistical package to obtain total counts, means, standard deviations, and other standard statistics. You should use the graphical tools with the statistical package to summarize the results. USE OF DIGITAL IMAGERY Recently, Minnesota DOT (MnDOT) administered a number of innovative surveys that involve using digital imagery to help elicit information about customer preferences, satisfaction, and willingness to pay regarding different attributes of road maintenance.

MnDOT has used two survey techniques, one involving standard survey questions combined with video and the other involving the use of stated preference techniques and digital photographs. Both methods allowed survey respondents to better understand attributes of the maintenance products and services being explored. Also, in both cases, the surveys were administered to groups in a setting reminiscent of a focus group. Appendix C: Guidance on Designing and Administering Surveys 220

Next: Appendix D - Assessing Value Added to Customers »
Guide for Customer-Driven Benchmarking of Maintenance Activities Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB's National Cooperative Highway Research Program (NCHRP) Report 511: Guide for Customer-Driven Benchmarking of Maintenance Activities provides guidance on how to evaluate and improve an agency's performance through a process called "customer-driven benchmarking." The objective of benchmarking is to identify, evaluate, and implement best practices by comparing the performance of agencies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!