Alternative Questionnaire Experiments
The past three decennial censuses have all included some form of Alternative Questionnaire Experiment (AQE), designed to test responses to different versions of either the census short-form or long-form sample questionnaire. The 1980 AQE tested two “matrix”-type forms more conducive to the optical sensing technology then used to extract electronic data from microfilmed questionnaire images against a more respondent-friendly design; the 1990 AQE tested five different variations of the census long-form questionnaire (DeMaio and Bates, 1992; Martin et al., 2003).
The 2000 incarnation of the AQE was broader in scope than its predecessors, incorporating three separate subexperiments. One of these focused on the census long form and the branching and “skip” instructions that are meant to guide respondents through the questionnaire (Redline et al., 2002). Another tested different versions of the race and Hispanic origin questions and changes in reporting induced by those questionnaire features (Martin, 2002). It is the third component of the 2000 AQE—“An Experiment to Improve Coverage Through Revised Roster Instructions” (Gerber et al., 2002)—that is germane to this panel’s study; hence, in this report, we will often use “2000 AQE” to refer specifically to the portion of the experiment that studied roster and residence concepts.
The full 2000 AQE consisted of eight panels of 5,000 households (10,000 were included in one of the panels); five of these focused on changes to the cues used to help respondents navigate through the census long form, and one presented the 2000 census short-form questions but used the design conventions of the 1990 census. The two panels of key interest from the perspective of census residence rules were a modified short form with revised residence and roster questions and a control group consisting of the standard 2000 census short form. Both of these panels were mailed to approximately 5,200 selected households, and both experienced return rates on the order or 73 percent. The sample of households was stratified by “high coverage areas” (low proportions of minorities and renters in the 1990 census) and “low coverage areas” (high proportions), and were drawn from the Master Address File for mailout/mailback areas of the country (Martin et al., 2003).
Respondent households that returned the revised-roster questionnaire and the 2000-census control questionnaire by mail and included phone numbers were sampled so that the respondents could be reinterviewed (4 months later) and the results used to study the effects of design changes. In total, 4,218 households were selected for reinterview; 2,958 interviews were completed.
The specific changes to the residence question and its related instructions that were tested by the 2000 AQE are described in Box 6-2.