APPENDIX A
Letter Report on the National Household Travel Survey

June 18, 2002

Dr. Ashish Sen

Director

Bureau of Transportation Statistics

400 7th Street, S.W. Washington, D.C. 20590

Dear Dr. Sen:

We are pleased to transmit this first letter report of the Committee to Review the Bureau of Transportation Statistics’ (BTS) Survey Programs. This committee was convened by the Transportation Research Board (TRB) and the Committee on National Statistics1 in response to a request from BTS. The membership of the committee is shown in Attachment A. The committee has been charged with reviewing the current BTS survey programs in light of transportation data needs for policy planning and research and in light of the characteristics and functions of an effective statistical agency.2 This letter presents the committee’s consensus findings and recommendations concerning the National Household Travel Survey (NHTS).

1

The Committee on National Statistics is part of the National Academies’ Division of Behavioral and Social Sciences and Education.

2

Several National Research Council reports include discussion of the characteristics and functions of an effective statistical agency (TRB 1992; Citro and Norwood 1997; Martin et al. 2001). The present review is being conducted against the background of these earlier studies.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 APPENDIX A Letter Report on the National Household Travel Survey June 18, 2002 Dr. Ashish Sen Director Bureau of Transportation Statistics 400 7th Street, S.W. Washington, D.C. 20590 Dear Dr. Sen: We are pleased to transmit this first letter report of the Committee to Review the Bureau of Transportation Statistics’ (BTS) Survey Programs. This committee was convened by the Transportation Research Board (TRB) and the Committee on National Statistics1 in response to a request from BTS. The membership of the committee is shown in Attachment A. The committee has been charged with reviewing the current BTS survey programs in light of transportation data needs for policy planning and research and in light of the characteristics and functions of an effective statistical agency.2 This letter presents the committee’s consensus findings and recommendations concerning the National Household Travel Survey (NHTS). 1 The Committee on National Statistics is part of the National Academies’ Division of Behavioral and Social Sciences and Education. 2 Several National Research Council reports include discussion of the characteristics and functions of an effective statistical agency (TRB 1992; Citro and Norwood 1997; Martin et al. 2001). The present review is being conducted against the background of these earlier studies.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 The committee held its first meeting on February 25–26, 2002, at the National Research Council facilities in Washington, D.C. The purpose of this meeting was for the committee to review the NHTS. To this end, the committee heard presentations from representatives of BTS, the Federal Highway Administration (FHWA), and Westat;3 users of personal travel data; and researchers investigating various aspects of survey methodology, including the use of new technologies for data collection. A list of the presentations and panel discussions at the meeting is provided in Attachment B. Following the data-gathering sessions, the committee met in closed session to deliberate on its findings and recommendations and begin the preparation of this report, which was completed through correspondence among the members. In developing its findings and recommendations, the committee drew on information gathered at its first meeting, articles in the technical literature,4 and the experience and expertise of individual members. The committee would like to thank all those who contributed to this review through their participation in the February meeting and their responses to follow-up questions. The assistance of Joy Sharp of BTS and Susan Liss of FHWA is particularly appreciated. In summary, the committee found that data from the Nationwide Personal Transportation Survey (NPTS) and the American Travel Survey (ATS) have proved useful to a variety of entities for the purposes of analyzing policy issues, setting funding priorities, and monitoring trends in travel behavior. The committee believes that data from the NHTS, which has superseded the NPTS and ATS, will prove similarly useful. Therefore, the committee recommends that BTS continue to collect, analyze, and disseminate data on personal travel within the United States. Nevertheless, the committee identified opportunities for the agency to improve its personal travel surveys in terms of both their value to a wide range of users and the quality of the data provided. In particular, the committee recommends that BTS consider developing a family of personal travel surveys aimed at meeting the needs of a variety of users. These surveys are likely to differ in content, coverage, methodology, and frequency. 3 Westat conducted the pretest of the NHTS and has a major role in the conduct of the 2001 survey. 4 A list of all nonproprietary materials considered by the committee is available from the Public Records Office of the National Academies (e-mail: publicac@nas.edu).

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 The remainder of this report is organized as follows. First, some background information is provided on the NHTS and its predecessor surveys, the NPTS and the ATS. Survey nonresponse and approaches to its reduction are then discussed. The users of the NPTS and ATS data are identified and their uses of these data reviewed. Finally, the committee presents its recommendations to BTS for improving the agency’s personal travel surveys.5 THE NHTS AND ITS PREDECESSOR SURVEYS The purpose of the NHTS is to provide a timely inventory of personal travel within the United States. The survey provides information on local and long-distance trips, including miles traveled by mode, the purpose of the trip, and the demographic characteristics of traveling households. The uses of the NHTS, as indicated by the uses of its predecessor surveys, are likely to include policy analysis at the national and local levels, monitoring of trends, benchmarking, and calibration of models for forecasting. In the absence of the NHTS, nationwide personal travel data available from the federal government would be limited to journey-to-work trips reported in the decennial census and the new American Community Survey. There would be no source of nationwide data on increasingly important non-work-related travel, which would be much harder to investigate. The 2001 NHTS is surveying 25,000 households nationwide. In addition, nine add-on surveys are being collected at the request, and expense, of several state and local agencies to increase the sample size in places of interest to those agencies.6 By purchasing an add-on sample, a state or metropolitan planning organization (MPO) receives both the national random samples for its area and the additional local area or state samples it has purchased. The add-on surveys are gathering data from an additional 40,000 households using the same methodology and instruments 5 At the time of writing, the design of the 2001 NHTS has been finalized, and data collection is ongoing. Therefore the committee’s recommendations address future versions of the NHTS (and other personal travel surveys) for which designs have not yet been determined. 6 Add-on areas for the 2001 NHTS comprise five states (Hawaii, Kentucky, New York, Texas, and Wisconsin) and four local planning organizations (Baltimore, Md.; Des Moines, Iowa; Lancaster County, Pa.; and Oahu, Hawaii).

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 as the basic NHTS, thereby avoiding the compatibility issues that typically arise when separate surveys are undertaken.7 The NHTS supersedes two earlier personal travel surveys: the NPTS and the ATS. The NPTS investigated daily travel within the United States. This survey was conducted five times (1969, 1977, 1983, 1990, and 1995); the irregular frequency was determined, in large part, by the availability of funds. The questions included in the survey were constrained to support survey-to-survey trend analysis. The ATS investigated long-distance travel within the United States and state-to-state person-trip flows. This survey was conducted in 1977 as a component of the Census of Transportation and again in 1995, when it was conducted for BTS by the U.S. Census Bureau (FHWA 1998). The cost of the 2001 NHTS is estimated to be approximately $10 million. The combined cost of the 1995 NPTS and the 1995 ATS was approximately $22 million.8 The cost breakdowns per household for the 1995 NPTS, the 1995 ATS, and the 2001 NHTS are as follows:9 Survey Total Cost ($) No. of Households Cost per Household ($) 1995 NPTS 4,096,000 21,000 195 1995 ATS 18,000,000 67,000 269 2001 NHTS 10,275,000 25,000 411 These costs include survey design, pretesting, data collection and editing, and data set preparation. For the 1995 NPTS and the 2001 NHTS, slightly more than half the cost per household is spent on data collection, and slightly less than half on survey design, pretesting, and data set preparation. Integration of the ATS and NPTS was recommended as a means of providing “useful data for federal, state, and MPO analysis and planning purposes, including consistent estimates of daily and long-distance household travel patterns, in a more cost-effective manner than two separate surveys, 7 According to a recent article on standardizing household travel surveys (NCHRP 2002, 1), the wide diversity in design, application, and analysis of these surveys makes it difficult, if not impossible, to compare results between one survey and another. 8 The costs for the 2001 NHTS are given in 2001 dollars; the costs for the 1995 surveys are given in 1995 dollars. 9 Budget data provided to the committee by BTS staff, March 8, 2002.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 neither of which provides a complete picture of household transportation” (Citro and Norwood 1997, 139). Before deciding to proceed with an integrated survey, BTS investigated issues associated with the combination of the NPTS and ATS into a single survey. For example, the 1999 conference “Personal Travel: The Long and Short of It” (TRB 2001), sponsored in part by BTS, addressed both methodological and content issues relating to the merging of the NPTS and ATS. BTS conducted a pretest of approximately 1,750 households to investigate the feasibility of a combined NPTS/ATS instrument. The pretest used a number of different survey designs. These designs were selected to assess both the feasibility of using a combined survey instrument for daily and long-distance travel and methods for improving response rates. Although a combined data set for long-distance and daily travel should facilitate in-depth analysis of overall travel patterns, there were concerns that the burden placed on households participating in the survey could result in a loss of data quality. The pretest included a qualitative analysis of respondent burden using interviewer feedback and interview monitoring to provide insights into problem areas.10 On the basis of the results of the NHTS pretest, BTS concluded that combining the NPTS and ATS into one survey is feasible. The agency also identified the most cost-effective survey design from among eight options considered for collecting nationwide data on U.S. travel.11 The pretest results allayed concerns about overburdening respondents by asking them to report travel in trips of all lengths.12 The NHTS offers both advantages and disadvantages vis-à-vis the earlier surveys. A major advantage is the inclusion of more complete data for trips in the 30- to 100-mile range, which were poorly represented in the NPTS and ATS (TRB 2001, 12). Reporting of all travel by a single sample of households will also facilitate comparisons of local and long-distance travel. Previously the NPTS and ATS provided two different 10 NHTS 2001: The Pretest and Key Tests (www.bts.gov/nhts/keytests.html). 11 NHTS 2001, Pretest Analysis Highlights, Update October 30, 2000 (www.bts.gov/nhts/update103000.html). 12 The extended survey interview to collect information on all trips made on the travel day plus all long-distance travel during the travel period takes approximately 15 minutes per person, with an additional 3 to 4 minutes for the state add-ons.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 samples with different criteria for sampling persons within a household.13 Possible disadvantages of the combined survey include additional complexity in survey-to-survey trend analysis incorporating data from earlier surveys and a much-reduced sample size for long-distance travel. The 1995 ATS surveyed 67,000 households, whereas the 2001 NHTS will survey only 25,000 households. The continuing interest in improving intermediate and long-distance travel services defines a public policy need for high-quality data on longer trips. SURVEY NONRESPONSE Some level of nonresponse occurs in every voluntary survey.14 A number of reports and papers discuss the reasons for nonresponse in household travel surveys (see, for example, Stopher and Metcalf 1996; TRB 1996; Zimowski et al. 1997). Some of these reasons, such as interviewer quality and the changing characteristics of telephone usage (see below), are not specific to travel surveys. However, the complexity and content of household travel surveys impose a significant burden on respondents and thus can result in a substantial fraction of nonrespondents. For example, to understand travel patterns and to provide data for estimating travel behavior models, travel surveys collect household and individual demographic characteristics as well as detailed descriptions of all trips taken during a specified time period. Trip data, which are needed for most or all persons in the household, include origin and destination, purpose, time of day, mode(s) of travel, and other characteristics. A household may make as many as 20 or 30 trips in a day. Recalling and reporting on these trips involve a major effort on the part of respondents, and there is some evidence that overall response rates decrease as a result of greater respondent burden. Consequently, some survey experts believe that household travel surveys have reached or surpassed their most cost-effective length (TRB 1996). 13 The ATS included information about trips made by all persons in the household, regardless of age, whereas the NPTS included trips made by household members aged 5 or over (FHWA 1998). 14 Nonresponse includes both unit nonresponse (i.e., the failure of a household to participate in a survey at all) and item nonresponse (i.e., the failure to complete a component of the survey, or the failure of the required number of household members to participate).

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 In addition, household travel surveys often involve a two-stage process: a recruitment interview to obtain information about the household is followed by an extended data-gathering interview to collect information on household travel. The 2001 NHTS is an example of such a two-stage survey. Since each interview has an accompanying level of nonresponse, the overall response rate—given by the product of the response rates for the two interviews—tends to be lower than that for a survey involving only one interview. Nonresponse is a major concern because of the potential for bias and the resulting implications for data quality.15 If the travel behavior of non-respondents is not significantly different from that of respondents, there may be no significant bias. However, there is reason to believe that the travel patterns of survey nonrespondents are significantly different from those of respondents. For example, in a survey that relies exclusively on telephone interviewing methods, those who travel extensively and are not home to answer the phone (high-income, high-mobility groups) are likely to be underrepresented, whereas those who are usually home to answer the phone and do not travel much (e.g., the elderly) may be overrepresented. Thus, the 2001 NHTS, which relies on telephone contacts for data collection, may give too much weight to the travel patterns of those who do not travel much and too little weight to the travel of more mobile groups in society—a serious deficiency for a personal travel survey. The underrepresentation of certain socioeconomic groups (e.g., low-income groups such as the urban poor and persons without phones) and certain age groups (e.g., teenagers, particularly boys) may also introduce bias into the survey results. The changing characteristics of telephone usage are reducing the effectiveness of current telephone survey methods and may be introducing bias. Coverage and response rates are declining for the following reasons: Consumer resistance: Unsolicited phone calls, especially telemarketing calls, compete for respondents’ attention and have a negative effect on response rates. These calls encourage households to adopt defensive measures, such as screening calls by using caller identification devices 15 Nonresponse error is only one of the factors affecting data quality. Other factors include sampling error, coverage error, and measurement error (see, for example, Sammer 2000).

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 and answering machines and requesting that their names and phone numbers be added to “do not call” lists.16 Cell-phone-only households: Cell phones are excluded from random digit dialing (RDD) lists because of the pricing structure, which may require users to pay for incoming calls. There are concerns that these exclusions may lead to sampling bias because of the increasing number of cell-phone-only households. Language difficulties: The number of households in the United States for which the first language is not English is growing. Telephone communication may be far more difficult than some other types of interaction (e.g., face-to-face interviews) for survey respondents who are not native English speakers. Various sources report typical response rates for household travel surveys. According to Zimowski et al. (1997, i), household travel surveys conducted in recent years by a combination of telephone and mail methods have typically obtained response rates in the range of 25 to 40 percent. A review of 55 U.S. household travel surveys conducted between 1989 and 1995 revealed similar rates (Stopher and Metcalf 1996). The mean response rates were 49.9 percent for the recruitment interview and 69.5 percent for the extended data-gathering interview, with considerable variation about the mean in both cases. Overall response rates varied from 10 to 75 percent, with a mean of 36.4 percent. In response to a question from the committee, one survey expert noted that response rates for personal travel surveys are generally in the high 50 percent range for recruitment interviews and on the order of 70 to 75 percent for extended data-gathering interviews, giving overall response rates in the high 30 percent to low 40 percent range.17 The overall response rate for the 1995 NPTS was 37.2 percent (51.6 percent for the recruitment interview and 72.1 percent for the extended data-gathering interview). The final household response rate for the 16 “Do not call” list legislation aims to prevent telephone sales solicitation calls rather than calls for the purposes of bona fide research. However, there is concern among some survey practitioners that consumers may mistakenly believe the lists prevent all unsolicited phone calls from strangers. Such consumers are likely to be antagonistic toward telephone interviewers seeking to recruit survey participants. 17 As reported by Johanna Zmud of NuStats in a presentation to the committee on February 26, 2002.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 1995 ATS was 85 percent—a relatively high value for a household travel survey. BTS indicated to the committee that the markedly different response rates for the 1995 NPTS and the 1995 ATS are largely attributable to three factors: the sampling frame, the data collection mode, and the data collection methods.18 Sampling frame: The ATS used an area-frame sample of households that had very recently been used by the Census Bureau in its Current Population Survey. In contrast, the NPTS used a list-assisted RDD sample. With RDD samples, there is always a significant percentage of households that cannot be contacted and contribute to the nonresponse rate. In the case of the 1995 NPTS, “no contacts” made up almost 7 percent of the sampled phone numbers. The use of an established sample of addresses rather than an RDD sample contributed to the higher response rate for the ATS vis-à-vis the NPTS. Data collection mode: The 1995 NPTS relied exclusively on computer-assisted telephone interviewing (CATI) methods. The 1995 ATS used primarily telephone interviewing, but in-person interviews were conducted with some respondents who could not be reached by telephone. Although such in-person interviews increase survey costs, they can be beneficial in increasing the response rate. Data collection methods: The 1995 NPTS used restrictive data collection methods to improve data quality over that obtained in the 1990 version of the survey. The definition of a “usable household” was rigorously constrained, a 6-day data collection window was applied, and the use of proxy reporting was limited. In the case of the 1995 ATS, the data collection methods were less restrictive, thereby contributing to the higher response rate vis-à-vis the 1995 NPTS. For example, the 1995 ATS considered a household to be complete if only one adult member provided information for everyone in the household. In contrast, the 1995 NPTS required 50 percent or more of the adults in the household to be interviewed for the data to be included in the survey. The committee notes that, in addition to the above differences, the 1995 ATS was conducted by the Census Bureau whereas the 1995 NPTS 18 Information provided to the committee by BTS staff, April 30, 2002.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 was conducted by Research Triangle Institute. There is evidence that the Census Bureau achieves higher response rates in voluntary surveys than other survey organizations (see, for example, NRC 1979). BTS anticipates that the response rates for the 2001 NHTS will be about 60 percent for the recruitment interview and about 75 percent in the extended interview, giving a total response rate of 45 to 50 percent. Although this anticipated overall response rate for the 2001 NHTS is relatively high compared with response rates for household travel surveys conducted by MPOs across the United States (Stopher and Metcalf 1996), it is low compared with the response rates for the 1995 ATS and for other federal policy-related surveys, as illustrated by two examples. The National Household Education Survey (NHES) is a telephone survey of the noninstitutionalized civilian population of the United States. Households are selected for the survey using RDD methods, and data are collected using CATI procedures. Response rates for the screener interview were 73.3 percent for NHES:95 and 69.9 percent for NHES:96. Response rates for the extended interviews varied according to subject, being highest for Early Childhood Program Participation (90.4 percent) and lowest for Youth Civic Involvement (76.4 percent). The resulting overall response rates were in the range of 53.4 to 66.3 percent. The response rates for the NHES have decreased since the early 1990s, falling from an average of 72 percent in 1991 and 1993 to an average of 58 percent in 1996. An investigation of the response rates for the 1991, 1993, 1995, and 1996 surveys indicated that no single factor, such as length of the interview, can be used to predict response rates. A number of factors, including survey objectives, approaches to screening households, and interactions between interviewers and respondents, must be considered in assessing the impact of survey design and procedures on response rates in RDD surveys (National Center for Education Statistics 1997). The National Health Interview Survey (NHIS), initiated in 1957, is the principal source of information on the health of the civilian, noninstitutionalized, household population of the United States. In response to requirements for enhanced topic coverage, survey questionnaires in the period following the 1982 revision became increasingly unwieldy, running almost 300 pages and requiring interviews that averaged 2 hours. The resulting burden on respondents, interviewers, and the data collec-

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 tion budget, together with declines in both response rates and data quality, led the National Center for Health Statistics to redesign the questionnaire. The revised NHIS, fielded since 1997, is conducted using computer-assisted personal interviewing (CAPI). The total household response rate for the 1999 NHIS was approximately 87.6 percent (Centers for Disease Control and Prevention 2002). The Office of Management and Budget (OMB), which must give approval for all federally funded surveys, has expressed reservations about proceeding with the 2001 NHTS because of the low response rates for the predecessor surveys—notably the NPTS—and resulting concerns about nonresponse bias. According to OMB, “levels of response below the levels expected of such surveys will mitigate against valid, generalizable results.”19 OMB has given BTS conditional clearance to proceed with the 2001 NHTS on the understanding that the agency will investigate the high nonresponse rate and find ways to reduce it in the future. BTS is aware of many of the factors contributing to nonresponse for the NHTS and has supported related research investigations. For example, BTS was one of the sponsors of the TRB Personal Travel Survey Round-table.20 Participants in this meeting discussed survey methodology issues and identified problem areas and research needs relating to RDD methodology and low response rates. BTS and FHWA have drafted a nonresponse research plan for the 2001 NHTS. Research using both in-house and contract resources will investigate omissions resulting from noncoverage or nonresponse, the differing demographic and travel characteristics of respondents and nonrespondents, and possible adjustments to correct for any bias. In addition, investigations of potential changes in methodology for the next version of the NHTS will continue. These include the use of alternative definitions of a “usable household” in terms of the percentage of adults who respond and the feasibility of collecting long-distance travel information by asking respondents about their most recent trip.21 Several strategies that may help reduce nonresponse rates in the NHTS have been explored through field experimentation. For example, the 19 As reported in the Notice of Office of Management and Budget Action, Terms of Clearance—2001 NPTS/ATS. 20 This group met on September 21–22, 2000, and again on March 19–20, 2001. 21 Information provided to the committee by BTS staff, April 18, 2002.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 dated. For example, the 1995 ATS data are now too out of date to be useful, and the 2001 NHTS data will be of limited use by 2004. BTS does not have a formal process for identifying data users or modifying its surveys to meet user needs. Mailing lists, publications citing the surveys, and information on website usage provide BTS and FHWA staffs with some indications of by whom and how the data are used. However, there are no major, structured outreach activities to identify and query users of the NPTS/ATS. Questions have been added to the NHTS to meet the needs of specific users (e.g., questions on walking and biking trips in response to a request from the Centers for Disease Control and Prevention), and some modifications were incorporated in the 1995 NPTS to meet MPO needs. Nevertheless, such changes appear to be implemented on a largely ad hoc basis. During the NHTS pretest, an effort was made to identify the types of users likely to use the combined NPTS/ATS results to support public planning and policy activities, and to define what information these users are likely to need (KPMG 1999). Information was gathered through telephone interviews with users of 1995 NPTS and ATS data and from other professionals with extensive experience and insights into state and local planning and modeling needs and issues. Budget limitations precluded the use of other outreach methods such as user panels, user conferences, and ongoing user involvement programs. The six-member BTS Advisory Committee, appointed by and reporting to the Director, meets two to three times a year and provides some guidance on information product needs. The members of this committee are high-level managers and researchers, some of whom are not primary data users but may receive reports and analyses ultimately traceable to BTS data. RECOMMENDATIONS Recommendation 1 BTS should consider developing a family of personal travel surveys aimed at meeting the needs of a variety of users. These surveys are likely to differ in content, coverage, methodology, and frequency. Data from the NPTS and the ATS have proved useful to a wide range of organizations and individuals for investigations of policy issues and for

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 benchmarking. However, declining survey response rates (see, for example, Stopher and Metcalf 1996, 14) have resulted in growing concerns about possible bias in the data collected. The conditional clearance from OMB allowing BTS to proceed with the 2001 NHTS reflects a general concern about the validity of survey results when response rates are low. Low response rates in household travel surveys are attributable to a variety of factors, including the complexity of such surveys, the growing resistance to surveys in general and telephone surveys in particular, and the changing patterns of communications access to American households. Furthermore, the diversity of analysis and decision needs to be met by BTS’s national travel surveys suggests that it may become increasingly difficult to meet user requirements for both quality and subject coverage with a single, periodic national household travel survey.28 Therefore, BTS should consider a variety of survey options for measuring personal travel and should not necessarily limit its efforts to a cross-sectional household travel survey conducted once every 5 years using CATI methods. The agency should investigate a range of survey designs and supporting technologies that offer the potential to keep ahead of the growing challenge of collecting household travel data and to meet the current and emerging data needs of a variety of users. Additional, structured efforts are needed to identify these users and their data requirements. The following recommendations elaborate on the need for outreach to users and potential users (Recommendation 2), efforts to improve survey response rates and data quality (Recommendation 3), and opportunities for research into methodologies for transportation surveys (Recommendation 4). Recommendation 2 BTS should develop a formal program for identifying and interacting with current and potential users of its personal travel surveys to better understand their data needs and their perspectives on issues such as data quality. The following approaches are suggested. 28 Information on the identity of users and their data needs is provided in the earlier discussion of data use and data users.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 BTS should consider establishing two formal advisory panels: A policy committee to advise on user needs and interface with users and potential users, and A technical advisory committee to advise on issues such as survey design and research leading to improved methodologies. BTS should consult with a range of constituencies—including those outside of the Washington, D.C., area—as part of the development of its personal travel surveys. Survey topics and possible new surveys should be discussed with potential users to assess their interest and inform subsequent survey development. Timeliness in providing survey results should also be considered. A survey that does not meet the needs of users in terms of both content and timeliness is not a worthwhile investment of resources. The community of users of BTS products is not limited to the current users. Different, better products are likely to attract, and satisfy, more users and thus may increase the cost-effectiveness of the surveys themselves. BTS should view the identification of user needs as an ongoing process and should endeavor to anticipate data needs relating to emerging and future policy issues. Data needs evolve over time as travel, infrastructure conditions, and the national agenda change. For example, renewed interest in high-speed, intercity rail services suggests the need for timely and reliable data on long-distance travel. Recommendation 3 BTS should continue its efforts to improve survey response rates and data quality, taking advantage of a range of design concepts and new technologies. Survey Design The selection of survey designs should be governed primarily by the purposes to which users will put the data, even though resource availability will inevitably influence design decisions. An understanding of user requirements for data quantity and accuracy is essential to determining the most cost-effective method of obtaining high-quality data that fulfill users’ needs. In particular, the sample size should be determined on a

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 rational statistical basis, while taking account of resource constraints and requirements for geographic coverage. It is not clear to the committee whether the current NHTS sample size is driven by data quality needs or simply constrained by the available budget. Given a fixed budget, there are two possible options for addressing the trade-off between data quantity and quality (Richardson et al. 1996). Either the decision to obtain data of a specified quality controls the quantity of data that can be collected, or specification of the quantity of data to be collected dictates the data quality. It is the committee’s understanding that, under current circumstances, many users are more concerned about the quality and content of the data than about sample size. Nevertheless, trading quantity for quality can have detrimental consequences if the data set becomes too small to be representative of behaviors of interest. For example, reductions in the sample size can result in inadequate data to assess the travel behavior of specific groups within the population, such as minorities or low-income households. In some instances, random stratified sampling techniques may be helpful in capturing sufficient data on “rare” behaviors—for example, trips involving walking, biking, or transit use—to permit calibration of models. For example, a geographically stratified sample of households might be used, where subgeographies containing larger fractions of transit users are oversampled. Demographic data from census or local administrative records would then be used to determine the oversampling rate and develop appropriate weighting factors. This method is an efficient way to gather data that, after weighting, can be used to characterize the behavior of the population as a whole. To the extent possible within resource constraints, the frequency with which data are collected should be based on rates of change of travel patterns and the factors affecting them. In general, data for major surveys should be collected every 5 years, and preferably more frequently. One strategy that helps spread survey costs over time is the use of continuous data collection. For example, if a survey is to be conducted every 5 years, it may be advantageous for budgeting purposes to spread the sample out over the 5-year period to obtain more timely data that can be cumulated over time to yield the desired sample sizes for subgroups or national estimates. Another concept that BTS may wish to consider is the use of a

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 continuous sampling and updating strategy to capture dynamic behaviors and maintain the freshness of data sets. Many of the behaviors of interest to policy makers and researchers are dynamic, involving the responses of households and individuals to changing circumstances and factors. Therefore, BTS—in consultation with its policy and technical advisory committees—should consider using longitudinal panel surveys as a means of capturing information on behavioral dynamics. Such surveys could either supplement or replace traditional cross-sectional household travel surveys.29 Panel surveys can be essential for understanding location choices and moving behaviors, which have important influences on travel needs. Coupled with appropriate contextual data on transportation supply, they can also reveal the effects of changes in the transportation system that can guide future investment and service planning decisions. Regardless of the survey design(s) selected, two features are important for obtaining high-quality data: A full-feature pilot survey conducted at the beginning of the project, and A follow-up investigation of nonresponse at the end to establish whether bias is present. New Technologies To overcome nonresponse and other data quality challenges, there is a need for multiple data collection methods (multi-instrument designs) that use different ways to reach people and measure their behaviors. Some encouraging results have been obtained using Internet-based travel diary surveys and handheld electronic travel diaries.30 29 According to Lawton and Pas (1996), longitudinal household travel surveys can take a number of forms: repeated cross-sectional surveys, before-and-after surveys, and panel surveys. The panel survey, which is the most commonly used longitudinal survey method in transport planning, is a repeated survey of the same sample of respondents. The period between surveys depends on the behavior being analyzed. 30 Adler, T. 2002. Applications of Technology to Travel Survey Data Collection. Presentation to the Committee to Review the Bureau of Transportation Statistics’ Survey Programs, Washington, D.C., February 25. Guensler R. 2002. Applications of Technology in Future Travel Survey Methods. Presentation to the Committee to Review the Bureau of Transportation Statistics’ Survey Programs, Washington, D.C., February 25.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 Internet-based travel diaries permit the implementation of user-friendly features such as context-sensitive instructions, a help feature, automatic addition of intrahousehold shared trips, and the ability to complete the diary in a series of work sessions at times convenient to the respondent. In addition, respondent-interactive geocoding provides a variety of ways for those completing the diary to describe the location of trip origins and destinations, including addresses, place names, and map pointing. Internet-based travel diaries also permit cost-efficient, high-capacity survey data retrieval and can increase unit and item response rates and resulting data quality. However, the up-front design cost is relatively high, and such diaries are susceptible to disruption by computer viruses and hackers. Any use of Internet-based travel diaries will necessitate a multi-instrument survey design. Such diaries are limited to households with Internet access (currently more than 50 percent of all households31) and have an associated socioeconomic bias, with Web respondents having higher incomes and being younger than the population average. In addition, respondents’ level of literacy is likely to influence the ease with which they can use an Internet-based diary. In-vehicle and personal GPS data loggers are being investigated in the United States and overseas as means of gathering travel data. Certain measurement errors (e.g., in distance traveled) can have a significant effect on personal travel data needed by some users. The use of GPS tracking has the potential to provide insights into measurement errors associated with both the distance traveled and the number of trips reported. For example, a pilot study in Atlanta indicated that more accurate information on the number of trips is obtained by using an automated data logging device than by using a travel diary.32 Despite the early promise of data loggers, the associated issues of privacy and confidentiality require further study before such technologies can be widely used to gather survey data. 31 As of September 2001, 50.5 percent of U.S. households had Internet access in their home, and 56.7 percent of the total U.S. population lived in households with Internet access (U.S. Department of Commerce 2002). 32 As reported in a presentation to the committee by Randall Guensler on February 25, 2002.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 Recommendation 4 BTS should assume a leadership role in research into methodologies for transportation surveys to help ensure that issues relating to survey quality are investigated and the results incorporated into the agency’s future surveys. The agency should Work with the user community and researchers to identify priority areas for study. Issue peer-reviewed grants for research to encourage and leverage investigations of methodological issues by organizations outside of the federal government, including universities and small businesses. Funding topics should be developed with assistance from the technical advisory board. Act as a clearinghouse for research activities relating to personal travel surveys. These activities fit well with the general model of a federal statistical agency (Martin et al. 2001). Some specific research opportunities are outlined below. An appropriate level of transparency in the grant award process is needed to encourage improvements in overall research quality. For example, those submitting research proposals could be provided with copies of proposal reviews and further guidance to help them make their proposals more responsive to BTS’s requirements. The committee notes that external research is often a useful way to benefit from specialized expertise not available in-house and may offer the opportunity to experiment with a variety of methods in parallel. Such research can also be a valuable source of independent validation of methods and strategies. New Survey Methods and Techniques Given the shrinking response rates in personal travel surveys conducted using traditional telephone methods, BTS needs to invest immediately in research into new survey methods and techniques for implementation in 5 to 10 years’ time. BTS and its user community have a vested interest in understanding the implications for survey research of trends in household structure, travel and activity patterns, and technologies, so that survey methods can be continuously adapted to ensure the availability of essential travel data. Such adaptations are likely to involve pursuing new technologies for data collection and investigating advanced statistical

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 techniques. The latter may include methods for updating and blending data collected at different time periods or under a continuous sampling scheme, techniques for detecting and adjusting for nonresponse bias, and tools for random stratified sampling. Panel Survey Methodology The use of longitudinal panel surveys requires effective methods for panel selection, retention, replacement, tracking, and data updating. There is a need to understand the cost–quality–usefulness trade-offs of panel data compared with cross-sectional approaches. Some lessons have been learned from transportation surveys conducted using longitudinal panel designs, including the Dutch National Mobility Panel, the Puget Sound Transportation Panel, and the German KONTIV33 survey (Lawton and Pas 1996), but more research is needed to develop such approaches for a national study in the United States. Empirical Investigations of Nonresponse To date, relatively few empirical studies have investigated nonresponse in travel surveys. The question of how large a response is required to support the various applications of NHTS data should be addressed and answered. Investigations are needed to understand the implications of nonresponse in terms of bias and generalizability. Research is needed into approaches that may be helpful for (a) reducing nonresponse rates in general and (b) obtaining travel survey data from underrepresented groups, such as high-income households that travel extensively, non-English speakers, teenagers, and large households. Addition of Contextual Data For many users, nationally collected travel data could become significantly more useful for both policy analysis and model estimation if contextually defined supply (level of service) data were linked to travel behavior and demographics. Although the development of such contextual data requires the use of detailed geographic locations, the resulting 33 The Continuous Survey on Travel Behavior (KONTIV) was conducted in the former West Germany in 1976, 1982, and 1989. A new travel survey, “Mobility in Germany,” is being conducted in 2001/2002 (www.kontiv2002.de/engl/background.htm).

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 information is unlikely to compromise the confidentiality of individual households. For example, information on the number of retail jobs within a half-mile or 10-minute walk of a household would fit not only the household in question but also thousands around it. Nonetheless, efforts to link contextual data to travel behavior and demographics must address two potentially conflicting requirements—the retention of sufficient geographic information for data to be useful and the suppression of any information that could undermine the confidentiality of individual households. Advances in network modeling and the expanded availability of powerful computational resources should facilitate the linkage of contextual data to travel behavior and demographics, but remaining obstacles include limits on the availability of supply data and the absence of analysis tools for measuring service attributes as a function of respondent location. There is a need for significant methodological research associated with the addition of contextual data, but the payoff in terms of expanded usefulness of the resulting data sets may be very large. CLOSING REMARKS The committee appreciates this opportunity to review and comment on the NHTS and looks forward to continuing to work with BTS staff, contractors, and the professional community as a whole in its forthcoming reviews of the Omnibus and Commodity Flow Surveys. Sincerely yours, Joseph L. Schofer Chair Committee to Review the Bureau of Transportation Statistics’ Survey Programs Attachment A: Committee membership34 Attachment B: Data-gathering activities at the first committee meeting 34 The information provided in Attachments A and B is made available elsewhere in this report. Therefore, these attachments are not reproduced here.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 REFERENCES Abbreviations FHWA Federal Highway Administration NCHRP National Cooperative Highway Research Program NRC National Research Council TRB Transportation Research Board Centers for Disease Control and Prevention. 2002. 1999 National Health Interview Survey (NHIS) Public Use Data Release: NHIS Survey Description. Division of Health Interview Statistics, National Center for Health Statistics, Hyattsville, Md. Citro, C. F., and J. L. Norwood (eds.). 1997. The Bureau of Transportation Statistics: Priorities for the Future. Panel on Statistical Programs and Practices of the Bureau of Transportation Statistics, National Research Council, Washington, D.C. FHWA. 1998. Personal Travel: The Long and Short of It, Issues Involved in Analysis Using the NPTS and the ATS. Office of Highway Policy Information, Washington, D.C. KPMG. 1999. American Travel Survey Users and Uses. Lawton, T. K., and E. I. Pas. 1996. Resource Paper for Survey Methodologies Workshop. In Conference Proceedings 10: Conference on Household Travel Surveys: New Concepts and Research Needs, TRB, National Research Council, Washington, D.C., pp. 134–153. Martin, M. E., M. L. Straf, and C. F. Citro (eds.). 2001. Principles and Practices for a Federal Statistical Agency, 2nd ed. Committee on National Statistics, National Research Council, Washington, D.C. National Center for Education Statistics. 1997. An Overview of Response Rates in the National Household Education Survey: 1991, 1993, 1995, and 1996. Report 97-948. nces.ed.gov/pubs97/97948.html. NCHRP. 2002. The Case for Standardizing Household Travel Surveys. Research Results Digest, Number 261. TRB, National Research Council, Washington, D.C., Jan. NRC. 1979. Privacy and Confidentiality as Factors in Survey Response. National Academy Press, Washington, D.C. Oak Ridge National Laboratory. 2000. Citations and Uses of the NPTS Data. July 30, 1999; updated July 20, 2000. Richardson, A. J., E. S. Ampt, and A. H. Meyburg. 1996. Nonresponse Issues in Household Travel Surveys. In Conference Proceedings 10: Conference on Household Travel Surveys: New Concepts and Research Needs, TRB, National Research Council, Washington, D.C., pp. 79–114. Sammer, G. 2000. Summary of Workshop on Quality Indicators. In Transportation Research Circular E-C008: Transport Surveys: Raising the Standard, TRB, National Research Council, Washington, D.C., pp. II-E/15–II-E/22.

OCR for page 54
Measuring Personal Travel and Goods Movement: A Review of the Bureau of Transportation Statistics' Surveys- Special Report 277 Stopher, P. R., and H. M. A. Metcalf. 1996. NCHRP Synthesis of Highway Practice 236: Methods for Household Travel Surveys. TRB, National Research Council, Washington, D.C. TRB. 1992. Special Report 234: Data for Decisions: Requirements for National Transportation Policy Making. National Research Council, Washington, D.C. TRB. 1996. Workshop Summary and Research Problem Statements: Nonresponse Issues. In Conference Proceedings 10: Conference on Household Travel Surveys: New Concepts and Research Needs, National Research Council, Washington, D.C., pp. 27–40. TRB. 2001. Transportation Research Circular E-C026: Personal Travel: The Long and Short of It. National Research Council, Washington, D.C. gulliver.trb.org/publications/circulars/ec026/ec026.pdf. U.S. Department of Commerce. 2002. A Nation Online: How Americans Are Expanding Their Use of the Internet. Economics and Statistics Administration, National Telecommunications and Information Administration. www.ntia.doc.gov/ntiahome/dn/Nation_Online.pdf. Zimowski, M., R. Tourangeau, R. Ghadialy, and S. Pedlow. 1997. Nonresponse in Household Travel Surveys. Report DOT-T-98-4. Prepared for the Federal Highway Administration by the National Opinion Research Center, Chicago, Ill.