Measuring Personal Travel and Goods Movement A Review of the Bureau of Transportation Statistics' Surveys -- Special Report 277 (2003) / Chapter Skim
Currently Skimming:
B Letter Report on the Omnibus Survey Program
B Letter Report on the Omnibus Survey Program
Pages 79-100
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 79... ...
The membership of the committee is shown in En closure A The committee has been charged with reviewing the current BTS survey programs in light of transportation data needs for policy planning and research and in light of the characteristics and functions of an effective statistical agency.
|
From page 80... ...
This report presents the committee's findings and recommenda tions in four areas: the value of the Omnibus program, and three areas of action to safeguard the integrity of the program -- review of pro posed surveys, implementation of measures to improve and ensure survey quality, and steps to improve the quality of data analysis prod ucts and reports. Enclosure C provides an overview of the Omnibus Survey Program.
|
From page 81... ...
The program currently comprises two components: a monthly household sur vey and targeted surveys, up to a maximum of four per year, that address special transportation topics.2 The Omnibus program delivers timely data to inform decision mak ing, as illustrated by two security-related examples. The monthly house hold survey provides a mechanism for conducting periodic assessments of traveler reactions to airport screening processes, thereby allowing the Transportation Security Administration (TSA)
|
From page 82... ...
To take ad vantage of its potential update capability, the Omnibus monthly house hold survey would require modification, with appropriate phrasing and structuring of survey questions to ensure that data are comparable with NHTS data.3 Recommendation 1: BTS should continue its Omnibus Survey Pro gram as a relatively low-budget activity that provides timely informa tion on a range of transportation issues. SAFEGUARDING THE INTEGRITY OF THE OMNIBUS SURVEY PROGRAM The opportunity to obtain timely public opinion data on key transporta tion issues makes the Omnibus program an attractive tool for policy makers.
|
From page 83... ...
. The committee has some concerns that, in its role as a survey service organization within USDOT, BTS may be asked to conduct Omnibus targeted surveys, or add questions to the Omnibus monthly household survey, that could ultimately damage the agency's credibility as an independent provider of transportation data.
|
From page 84... ...
for the Omnibus surveys. Be cause BTS staff may not have the experience and insights needed to under stand all the policy implications of proposed surveys, these procedures should include external review of all targeted surveys.
|
From page 85... ...
Development and Testing of the Survey Questionnaire The committee questions whether the time available for formulating the survey content and testing the questionnaire is sufficient to ensure that the resulting data will provide a sound basis for analysis. Because resource limitations preclude working on several months' surveys simultaneously, the monthly schedule does not allow time to conduct a pilot survey.4 The draft survey questionnaire is reviewed by an expert panel and subjected to cognitive testing using a mall intercept.
|
From page 86... ...
" It is not clear to the committee that the very limited cog nitive testing of the survey questionnaire, conducted in a single day with a small sample, is sufficient to establish what respondents understand by the possible answers to this question, which range from "very un satisfied" to "very satisfied." · Absence of a test-modify-retest cycle: Because there is insufficient time to conduct more than one iteration of the survey questionnaire, mod ifications to the draft questionnaire in response to cognitive testing and suggestions from the expert review panel are not adequately evaluated.6 Data Collection In general, data for the monthly household survey are collected over a period of 10 consecutive days, although the data collection schedule may, 6 A total of 25 pretest telephone interviews are conducted prior to the initiation of actual calling to identify any problems associated with the data collection process, the survey instrument, specific questions, answer choices, questionnaire instructions, or question format (BTS 2002)
|
From page 87... ...
However, in the absence of additional resources, a longer data collection period for the monthly household survey would reduce the already limited time available for questionnaire development and testing, thereby heightening concerns about the limited testing of the draft questionnaire. Measures to Improve and Ensure Quality Question Design and Evaluation The formulation and evaluation of proposed survey questions should ensure that the resulting questionnaire will provide a sound basis for analysis and assessment.
|
From page 88... ...
The nonresponse bias associated with estimates from random digit dialing (RDD) telephone surveys -- such as the Omnibus monthly household survey -- is not known unless special studies are undertaken.
|
From page 89... ...
1998) .8 The average response rate was 62 percent -- significantly higher than the 43 percent currently obtained in the Omnibus monthly household survey.
|
From page 90... ...
These investigations should include examinations of the characteristics of nonresponding telephone households, longer-term follow-up of a subsample of nonrespondents to determine whether dif ferences exist between respondents and nonrespondents, and experi mental study of the use of incentives to reduce nonresponse rates. Trading Frequency for Quality The committee urges BTS to consider replacing the Omnibus monthly household survey with a similar survey done quarterly (once every 3 months)
|
From page 91... ...
Steps to Improve the Quality of Data Analysis Products and Reports Finding 4: BTS reporting of the results of the Omnibus monthly household survey does not consistently meet the quality standards expected of a federal statistical agency. The data from the Omnibus monthly survey are made available on the BTS website (www.bts.gov)
|
From page 92... ...
The expertise needed for analysis of survey data is somewhat different from that needed to design and conduct a survey, and while BTS has considerable knowl edge and experience in survey methodology, its expertise in data analysis and reporting is more limited. The committee also recognizes that BTS is anxious to disseminate its survey results to decision makers and the pub lic and thereby gain credibility for the agency.13 OmniStats is an attempt to popularize the results of the Omnibus monthly survey for a general audience -- including USDOT staff who are not technical specialists.
|
From page 93... ...
To encourage high-quality statistical reporting, it would be valuable for BTS to develop and promulgate guidelines for reporting on items such as sample design, standard errors, and response rates. These guidelines would assist BTS staff -- as well as others within USDOT -- in preparing survey reports.
|
From page 94... ...
Recommendation 4: BTS should take steps to ensure that its analyses of Omnibus survey data are technically robust and that the resulting products comply with established guidelines for the reporting of sta tistical data. CLOSING REMARKS The committee appreciates this opportunity to review and comment on the Omnibus Survey Program and hopes that the recommendations made in this report are helpful to BTS in building on its initial experience with this innovative program.
|
From page 95... ...
John Wiley and Sons, New York. ENCLOSURE C OVERVIEW OF THE OMNIBUS SURVEY PROGRAM The Omnibus Survey Program15 currently comprises two categories of survey: · A household survey, conducted monthly, that addresses a range of transportation issues; and 15The term "omnibus" refers to the capability to "load" a variety of questions on a survey instru ment scheduled to go into the field on a regular basis.
|
From page 96... ...
As such, they are subject to constraints defined in the OMB clearance package for the BTS Omnibus program.16 In particular, all sur veys under the Omnibus program are required to include questions as sessing customer satisfaction with various aspects of the transportation system -- the core function of the Omnibus program. For example, the May 2002 household survey asked, "In terms of security from crime or terrorism, did you feel more secure or less secure flying on a commercial airline in April than a year ago?
|
From page 97... ...
THE OMNIBUS MONTHLY HOUSEHOLD SURVEY The purpose of the Omnibus monthly household survey is to "monitor expectations of and satisfaction with the transportation system and to gather event, issue, and mode-specific information" (BTS 2002)
|
From page 98... ...
on contractor costs for the survey and charges $800 for each modal administration question in cluded in the survey. The Omnibus monthly household survey is distinguished from many federal surveys not only by its customer satisfaction component but also by its ability to provide quick responses to a range of questions on a con tinuing basis.19 For example, the questions for the May 2002 survey were finalized on April 26, 2002, and the final data tabulations and microdata file were made available on June 20, 2002.
|
From page 99... ...
In contrast to the monthly household survey, which relies on tele phone methods to gather data, the targeted surveys use a variety of data collection methods -- including mail out/mail back, telephone, and Web-based approaches -- depending on the purpose of the survey and the target population. For example, the 2001 Mariner Survey was con ducted primarily by mail, but telephone interviews were conducted with some nonrespondents in an effort to increase the overall response rate.
|
From page 100... ...
22The BTS role in targeted surveys includes assisting the modal administrations in designing and developing survey questionnaires, advising on sampling procedures, monitoring the survey process, and assisting in data analysis.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.