Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
217 APPENDIX C: GUIDANCE ON DESIGNING AND ADMINISTERING SURVEYS If you decide to develop your own survey to be used as a benchmarking measurement tool, you should go through the standard steps of developing sound surveys: ⢠Focus groups, ⢠Survey design and pretesting, ⢠Coding guide and database design, ⢠Sample design, ⢠Administration, and ⢠Analysis and summarization. FOCUS GROUPS In order to understand what issues are important to your customers, you should begin any survey process with a series of focus groups. The main reasons to use a focus group are to identify what types of questions to include in your survey and to develop insight regarding how to provide a maintenance program that is more sensitive to the needs and desires of your customers. Focus groups should be conducted in different parts of your jurisdiction to gain insight into how geography, urbanization, traffic congestion, and other factors outside the control of maintenance organizations affect customer satisfaction regarding the delivery of maintenance products and services. As much as is possible, focus group participants should be representative of all customersâincluding motorists, truckers, and adjacent property ownersâas well as of different important categories such as gender, age, income, and ethnicity. The focus groups should be limited in size. Seven participants is a good size, but focus groups that are larger or smaller will work, too. Questions posed to focus group participants should be nondirectiveâdo not steer respondents in any direction. Ask questions in a way that prompts focus group participants to freely discuss the issues, expectations, preferences, and satisfaction they experience as customers of road maintenance. Questions you might ask are ⢠What products and services do you perceive the maintenance organization delivers? ⢠What attributes of these products and services do you perceive? ⢠Which attributes are the most important?
⢠What is your expectation regarding different types of maintenanceâfor example, snow and ice control, mowing, and pavement resurfacing? ⢠Are you aware of other maintenance products and services that the DOT provides but that you cannot easily perceive (e.g., pavement durability, tree maintenance)? ⢠In your opinion, what factors affect the service quality of roads? ⢠In making travel decisions, what highway attributes have the most inï¬uence on your choices? Do you perceive that any of these attributes are related to maintenance operations? From your notes from the focus groups, you should be able to make a list of questions that are prime candidates for incorporation into a survey. SURVEY DESIGN AND PRETESTING Next you will need to design the survey. You should have someone experienced in survey design draft it. You will have to decide how it will be administered, whether by mail, phone, or some other means. The administration process will inï¬uence the survey design. If the respondent will see the questionsâfor example, via a mail surveyâa simple, pleasing design and clear layout is essential. Regardless of the format, the questions need to be easily understood and unambiguous. Once you have drafted the survey, it should be pretested with a group of representative respondents and then revised. It may require additional pretesting and revision. CODING GUIDE AND DATABASE DESIGN When you have ï¬nalized your survey design, you will need to develop a coding guide to assist recording the results. You will also need to design the database in which you will enter the survey responses. You will have to specify a record layout for all data ï¬elds, including the order of each ï¬eld, the type, and the length. This layout is usually sufficient to specify the database in a typical statistical analysis package. SAMPLE DESIGN You will need a statistician or a person with a strong statistical background to design a scientiï¬c sample for the survey. The survey should have scientiï¬c validity: you should establish target levels of statistical accuracy and conï¬dence for the survey as a whole and for various customer groups or classes. In either case, you will want to take a random sample of customers; if you want to achieve speciï¬ed levels of accuracy and conï¬dence for subgroups, you will need to take some type of stratiï¬ed random sample. For help, you should consult a statistician experienced in developing a sampling plan. Appendix C: Guidance on Designing and Administering Surveys 218
219 ADMINISTRATION When you have completed your survey design, sampling plan, coding guide, and database design, you are ready to administer the survey. While you are proceeding through these steps, you should have been making plans regarding how you will administer the survey. There are many possibilities, but the most important are the following: ⢠Mail, ⢠Phone, ⢠Personal interviews, and ⢠Internet. Your choice will depend upon the response rates you expect, the biases, and the costs. You may wish to enlist a market research ï¬rm or service bureau that specializes in conducting surveys. They will have all the tools necessary to administer the survey efficiently, including procedures for selecting a random sample and conducting computer-assisted interviews. Survey administration over the Internet requires an altogether different skill set. When your planning is complete, you will administer your survey. When the survey is completed, you will need to go through a number of additional steps before you can analyze the results: ⢠Check the survey responses and clean the data by removing nonsensical or extreme answers or respondents. ⢠If the survey results are not already in a database, you will need to put them in a database. ANALYSIS AND SUMMARIZATION Finally, you need to analyze and summarize the results. You should apply a standard statistical package to obtain total counts, means, standard deviations, and other standard statistics. You should use the graphical tools with the statistical package to summarize the results. USE OF DIGITAL IMAGERY Recently, Minnesota DOT (MnDOT) administered a number of innovative surveys that involve using digital imagery to help elicit information about customer preferences, satisfaction, and willingness to pay regarding different attributes of road maintenance.
MnDOT has used two survey techniques, one involving standard survey questions combined with video and the other involving the use of stated preference techniques and digital photographs. Both methods allowed survey respondents to better understand attributes of the maintenance products and services being explored. Also, in both cases, the surveys were administered to groups in a setting reminiscent of a focus group. Appendix C: Guidance on Designing and Administering Surveys 220