the new measures will not be consistent with what was done before. Although changes in methodology will make some data users unhappy, a new methodology may be equally or perhaps more fit for use and more practical to implement. This may mean that agencies and decision makers will have to think hard about who the key data users are, as well as what information and policy needs have to be satisfied.

An example of a transition to a new methodology in the U.S. federal statistical system is the transportation research community’s transition from using the census long-form sample to using the American Community Survey as a source of transportation data. At the start of this process, they were reportedly quite unsure about the idea of using data that were based on a rolling sample and that would usually be 2 or 3 years old, as opposed to the data from the census long-form sample, which could be up to 10 years old. This is a good example of breaking away from the way things have been done with the goal of improving the fitness for use, and now they may have something better than what they had before.

Another way of thinking about the issue of acceptability is to question what are considered official statistics. Some people argue that an actual enumeration is the only legitimate way to count the population, but the statistical community knows that this is not the best approach to obtain most of the data. The question is how far is the statistical and survey community really willing to go to innovate. When will model-based estimates be widely accepted as official statistics? There have been and continue to be challenges to almost all forms of statistical methodology applied to the census. But the statistical system is in a position that it could be releasing a lot more official numbers that are model-based, and indeed there are some areas in which model-based estimates are well accepted, such as unemployment statistics that are adjusted through a sophisticated time-series model.

There has been considerable talk of Google’s consumer price index (CPI) recently. If Google develops a method that tracks the online sales of groceries, it will probably reflect the price of groceries in stores fairly well. The index will, of course, be based on a biased sample, with not nearly the right coverage of grocery stores, but if there is demand to get a leading indicator of the CPI without having to wait for data to arrive from an agency whose field representatives are visiting stores or calling people and asking what they paid for a gallon of milk, the Google CPI, or a more disaggregated version of it, can be useful for statistical modeling.

However, this does not mean that the statistical community should be accepting all new methodologies that come along. There is still an important role for statistical agencies, perhaps as gatekeepers, because raw administrative data and unvetted Internet surveys are not going to necessarily yield very good statistics.

Zaslavsky also reflected on the discussions about the use of different modes

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement