Cover Image

Not for Sale

View/Hide Left Panel
Click for next page ( 211

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 210
APPENDIX C: GUIDANCE ON DESIGNING AND ADMINISTERING SURVEYS If you decide to develop your own survey to be used as a benchmarking measurement tool, you should go through the standard steps of developing sound surveys: Focus groups, Survey design and pretesting, Coding guide and database design, Sample design, Administration, and Analysis and summarization. FOCUS GROUPS In order to understand what issues are important to your customers, you should begin any survey process with a series of focus groups. The main reasons to use a focus group are to identify what types of questions to include in your survey and to develop insight regarding how to provide a maintenance program that is more sensitive to the needs and desires of your customers. Focus groups should be conducted in different parts of your jurisdiction to gain insight into how geography, urbanization, traffic congestion, and other factors outside the control of maintenance organizations affect customer satisfaction regarding the delivery of maintenance products and services. As much as is possible, focus group participants should be representative of all customers--including motorists, truckers, and adjacent property owners--as well as of different important categories such as gender, age, income, and ethnicity. The focus groups should be limited in size. Seven participants is a good size, but focus groups that are larger or smaller will work, too. Questions posed to focus group participants should be nondirective--do not steer respondents in any direction. Ask questions in a way that prompts focus group participants to freely discuss the issues, expectations, preferences, and satisfaction they experience as customers of road maintenance. Questions you might ask are What products and services do you perceive the maintenance organization delivers? What attributes of these products and services do you perceive? Which attributes are the most important? 217

OCR for page 210
Appendix C: Guidance on Designing and Administering Surveys What is your expectation regarding different types of maintenance--for example, snow and ice control, mowing, and pavement resurfacing? Are you aware of other maintenance products and services that the DOT provides but that you cannot easily perceive (e.g., pavement durability, tree maintenance)? In your opinion, what factors affect the service quality of roads? In making travel decisions, what highway attributes have the most influence on your choices? Do you perceive that any of these attributes are related to maintenance operations? From your notes from the focus groups, you should be able to make a list of questions that are prime candidates for incorporation into a survey. SURVEY DESIGN AND PRETESTING Next you will need to design the survey. You should have someone experienced in survey design draft it. You will have to decide how it will be administered, whether by mail, phone, or some other means. The administration process will influence the survey design. If the respondent will see the questions--for example, via a mail survey--a simple, pleasing design and clear layout is essential. Regardless of the format, the questions need to be easily understood and unambiguous. Once you have drafted the survey, it should be pretested with a group of representative respondents and then revised. It may require additional pretesting and revision. CODING GUIDE AND DATABASE DESIGN When you have finalized your survey design, you will need to develop a coding guide to assist recording the results. You will also need to design the database in which you will enter the survey responses. You will have to specify a record layout for all data fields, including the order of each field, the type, and the length. This layout is usually sufficient to specify the database in a typical statistical analysis package. SAMPLE DESIGN You will need a statistician or a person with a strong statistical background to design a scientific sample for the survey. The survey should have scientific validity: you should establish target levels of statistical accuracy and confidence for the survey as a whole and for various customer groups or classes. In either case, you will want to take a random sample of customers; if you want to achieve specified levels of accuracy and confidence for subgroups, you will need to take some type of stratified random sample. For help, you should consult a statistician experienced in developing a sampling plan. 218

OCR for page 210
ADMINISTRATION When you have completed your survey design, sampling plan, coding guide, and database design, you are ready to administer the survey. While you are proceeding through these steps, you should have been making plans regarding how you will administer the survey. There are many possibilities, but the most important are the following: Mail, Phone, Personal interviews, and Internet. Your choice will depend upon the response rates you expect, the biases, and the costs. You may wish to enlist a market research firm or service bureau that specializes in conducting surveys. They will have all the tools necessary to administer the survey efficiently, including procedures for selecting a random sample and conducting computer-assisted interviews. Survey administration over the Internet requires an altogether different skill set. When your planning is complete, you will administer your survey. When the survey is completed, you will need to go through a number of additional steps before you can analyze the results: Check the survey responses and clean the data by removing nonsensical or extreme answers or respondents. If the survey results are not already in a database, you will need to put them in a database. ANALYSIS AND SUMMARIZATION Finally, you need to analyze and summarize the results. You should apply a standard statistical package to obtain total counts, means, standard deviations, and other standard statistics. You should use the graphical tools with the statistical package to summarize the results. USE OF DIGITAL IMAGERY Recently, Minnesota DOT (MnDOT) administered a number of innovative surveys that involve using digital imagery to help elicit information about customer preferences, satisfaction, and willingness to pay regarding different attributes of road maintenance. 219

OCR for page 210
Appendix C: Guidance on Designing and Administering Surveys MnDOT has used two survey techniques, one involving standard survey questions combined with video and the other involving the use of stated preference techniques and digital photographs. Both methods allowed survey respondents to better understand attributes of the maintenance products and services being explored. Also, in both cases, the surveys were administered to groups in a setting reminiscent of a focus group. 220