Systematic research and development (R&D) to improve census methodology, pretest changes in procedures, and evaluate census results is an integral part of the modern census. Such R&D was not possible in the early decades of census-taking because there was no permanent census office with appropriate staffing, and statistical methods for testing and evaluation were in their infancy. An exception that illustrates the importance of research and testing occurred for the 1850 census: a temporary Census Board, created in 1849, solicited outside expert advice that resulted in marked improvements in the design of the census forms that were used by U.S. marshals to collect the data. The improvements were based on an 1845 census of Boston directed by Lemuel Shattuck, a founder of the American Statistical Association (Hacker, 2000a).
A permanent Census Bureau was established in 1902, and the 1910 census included perhaps the first major experiment conducted in conjunction with a U.S. census. In order to boost awareness of the 1910 census and save interviewing time, census enumerators in large cities (100,000 population or greater) distributed “advance schedules” a few days prior to the beginning of the actual count. Although respondents were encouraged to fill out the schedule in advance of the enumerator visit, enumerators were not permitted to simply accept a filled-out form as a response to the census; simply “prepar[ing] the way for the enumerator by announcing his approaching visit and informing the people precisely of the questions to be answered” was the primary aim of the experiment, while potential time savings and gains in accuracy were described as the “secondary object” (U.S. Department of Commerce and Labor, 1911:115–116). Three decades later, the Census Bureau’s plan for the 1940 census included the first formal, structured set of evaluations of census content and quality in a decennial census; the procedural history of the 1940 census commissioned by the Bureau describes a series of analyses of the data quality for individual question items as well as limited work to estimate the level of underenumeration in the census (Jenkins, 1983:96–104). The 1950 census included the first systematic set of pretests and experiments to plan the enumeration, along with the first set of experiments conducted as part of the census and a large number of post-census evaluations (Goldfield and Pemberton, 2000a).
The Census Bureau uses several terms to delineate its census R&D activities, including pretest or test, experiment, and evaluation. Although usage is not always consistent, in this terminology, “tests” involve collection of data from respondents between censuses. Some tests shake down operations and uncover potential problems in the field; other tests are more properly termed experiments, in that they involve designed comparisons of alternative questionnaires, field procedures, and other aspects of the census. The Bureau, however, typically reserves the term “experiment” for instances when data collection to compare alternative methods is conducted as part and parcel of