Cover Image

Not for Sale

View/Hide Left Panel
Click for next page ( 63

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 62
62 Standardized Procedures for Personal Travel Surveys 4.2.17 P-5: Reporting of Pretests and Pilot Surveys From a review of previous surveys, it appears that there is no consistency in reporting whether a pretest or pilot survey was performed. This would lead one to suppose that pretests or pilot sur- veys have not been conducted. There should be a standardized procedure here that the final report of a survey should document whether a pilot survey or any pretests were conducted. If none was conducted, there should also be a clear statement as to the reason why this was the case. The other major issue relates to what should be reported from a pretest or pilot survey--for example, details on how the sampling was done, sample sizes determined, elements tested and results of the tests, and any specific statistical tests of significance that were performed. There is a need for minimum reporting standards to be developed here. It is suggested that reports on recent surveys be reviewed to determine what has been documented in the past. Some of the items to be considered here should be Sample sizes and methods of drawing the samples for any pretests and pilot surveys; Nature of the design that was tested; Results of the tests, including response rate(s) and other measures of quality; and Conclusions drawn from any pretests and pilot surveys and changes implemented as a result of the pretests or pilot surveys. The documentation should include any statistical test performed to establish whether to make changes to the final survey, and anecdotal information should also be included that may have led to changes in the design of the survey and its protocols. For example, problems encountered by interviewers in using the scripts provided and questions raised by prospective respondents are all appropriate items to be included in the documentation. A report outline should be developed as the means to convey the standard for documentation of any pretests and pilot surveys conducted. 4.2.18 Q-4: Sampling Error Sampling error not only is a part of the specification of the required sample size and an input to the design of the sample, but also is an important measure of the quality of the resulting sur- vey. Sampling error of individual variable estimates is measured by the Standard Error of the Esti- mate (SEE). However, the magnitude of the measure is affected by the units of measurement of the variable under consideration, making interpretation of the value and comparison of values among data sets difficult. To eliminate this effect, the coefficient of variation (SEE divided by the estimate) provides a dimensionless measure of variation of the estimate about the mean and allows meaningful comparison among data sets. However, this does not alter the fact that sam- pling errors need to be calculated separately for each variable in question. Given the difficulties that survey planners have in communicating information about sample- size calculations to clients (Richardson et al., 1995), one would ideally like to obtain one measure of sampling error for a data set as a whole, which could be derived from an average or weighted average value calculated for a number of key variables. Unfortunately, it was not possible to devise such a measure in this project. A practical approach for assessing overall survey quality would be to use the highest sampling error obtained among a list of key variables. This idea is consistent with the idea of "total design" promoted by Dillman (1978), which suggests that the quality of a process is only as good as the weakest link in the process. It is recommended that research be conducted to determine the most appropriate variables for a combined measure of sampling error. These may include activity rates per person and household; trip rates by purpose per person and household; mode shares by trip purpose; and selected house- hold and person attributes such as vehicle ownership, household size, driver's license status, etc. Two specific approaches could be taken to determine such variables. One approach could involve