. "3 Sample Surveys: Overview, Examples, and Usefulness in Studying Aviation Safety." An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press, 2009.
The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
An Assessment of NASA’s National Aviation Operations Monitoring Service
dispersed (in space or time) units, the cost of locating and collecting data from the units can be high. Censuses also require more resources to train the data-collection staff.
Data from sample surveys can be collected and analyzed in a more timely manner than can data from censuses.
It is difficult to control nonsampling errors in large or difficult-to-reach populations. For example, collecting data from all the units in these populations would require a large and dispersed administrative staff, which would be harder to train and closely supervise. Nonresponse problems are also harder to manage in a larger operation.
If a census is repeated over time, it would require all the units in the population to be repeatedly monitored. For human populations, this would place considerable burden on the respondents and could lead to decreased cooperation and higher nonresponse rates.
3.2 THE USE OF SAMPLE SURVEYS IN THE GOVERNMENT SECTOR
Sample surveys are used routinely by countries around the world to collect and analyze data in order to inform policy decisions, allocate resources, and assess national needs. Various federal agencies in the United States (for example, the Census Bureau, the Bureau of Labor Statistics, the Bureau of Justice Statistics) have been conducting or sponsoring sample surveys to obtain high-quality data about the state of the economy, health, education, crime, and other issues. The U.S. decennial census is mandated by the U.S. Constitution for the purpose of allocating congressional seats, but virtually all other demographic information used for policy making is collected on a sample basis. Several other countries have replaced their censuses with sample surveys (for example, Germany in 1987 and France in 2004). Even in the United States, the long form for the census was replaced in 1996 by the American Community Survey, a monthly sample survey of about 250,000 households. 5
The rest of this section describes one particular survey, the National Crime Victimization Survey (NCVS), in some detail as it shares some key similarities with the NAOMS survey: it uses multiple data sources, many of which are self-reported; it is a national survey designed to collect sensitive data (crime versus aviation safety); and it must protect respondent confidentiality. It is also informative to see how this survey started and how it evolved over time. See Appendix E for additional examples of federal surveys.
Several sources of crime data are used to inform U.S. policy decisions and to allocate funding for criminal justice to the states. The Uniform Crime Report (UCR) began in the late 1920s when the International Association of Chiefs of Police recognized a need for reliable data on crime in the United States in order to measure the effectiveness of local law enforcement and to provide data to help fight crime. In 1930, the job of collecting, summarizing, and publishing the UCR was turned over to the Federal Bureau of Investigation (FBI), which received data on monthly counts of eight types of crimes as well as the number of arrests for an additional 21 crimes from police jurisdictions. Although participation in the UCR is voluntary, over 98 percent of all police agencies in the nation reported to the UCR for at least some months in 2005.6 However, the system had some weaknesses in that some police agencies reported on a wide range of characteristics of all crimes, whereas others submitted information on a more limited set of crimes.
In the 1970s, the criminal justice community determined a need for more in-depth information about reported crime incidents. The Bureau of Justice Statistics commissioned a study to determine how the UCR could be improved to meet these needs. Based on that study, the UCR was further refined into what is now known as the National Incident Based Reporting System (NIBRS). By 2007, only about 25 percent of the U.S. population was covered by a police jurisdiction reporting to the NIBRS,7 which has not yet replaced the UCR for national crime statistics. However, even if the NIBRS included data from all jurisdictions, there would still be gaps in the available information about crime because not all crime is reported to the police.
U.S. Census Bureau, American Community Survey (ACS), Washington, D.C., available at http://www.census.gov/acs/, accessed July 15, 2009.
Nathan James and Logan Richard Council, How Crime in the United States Is Measured, CRS-RL34309, Congressional Research Service, Washington, D.C., 2008.