|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. APPENDIX A
|
||||||||||||||||
 |
 |
Page 56 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 56
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
APPENDIX A
THE CENSUS BUREAU’S SUGGESTED TOPICS FOR RESEARCH
The following chart was provided to the panel by the Census Bureau as a partial summarization (augmented by several other reports and presentations) of their deliberations as to the research topics that should be considered for either experimentation during the 2010 census or evaluation shortly after. The leftmost column provides an identification key for each topic along with a short series of either questions or a brief discussion that defines the topic. The next block of columns provides criteria that should be used to help rank these topics, initiated by a high-medium-low ranking of the resulting importance of the topic. The criteria are anticipated impacts on cost, quality of data, whether the topic would require a new census component process, and whether it was accomplishable. Finally, the last block of columns provides information on whether the topic was better suited to 2010 or 2020 and whether a census environment was needed to assess alternatives to current census processes.
OCR for page 57
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
This page intentionally left blank.
OCR for page 58
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
TABLE A-1 2010 Census Program for Evaluations and Experiments: Appendix to Summaries of Suggested Research
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
A. Coverage Measurement
A.1 Census Coverage Measurement (CCM) is the program that will answer the question: How accurate was the coverage of the population?
1
Yes
Yes
No
Yes
Yes
Yes
Yes
A.2 How effective is the CCM interview and subsequent processing in determining the members of the household at each housing unit on CCM interview day and the usual residence of each household member on Census Day? When there are errors in determining household membership and usual residence, what are the causes and what are the possible remedies? What are the effects of recall errors and reporting errors on the CCM interview?
1
No
Yes
Yes
Yes
Yes
Yes
Yes.
(Note: Some things can be done outside the census.)
A.3 Can we start to learn if comparing the history of census operations with the CCM results in the sample blocks can helpus explain how and when errors occur, and also suggest potential remedies?
1
No
Yes
No
Yes, as a feasibility study in small number of blocks in the CCM sample.
No
Yes
Maybe
OCR for page 59
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
A.4 Our knowledge about Group Quarters (GQ) coverage is very limited. Efforts to estimate GQ coverage in 1980 and 1990 were limited (in both scope and success), and the GQ population was out-of-scope for the 2000 Accuracy and Coverage Evaluation (A.C.E). Since the GQ population will also be out-of-scope for CCM in 2010, we need to consider that the problems are likely to be different for different types of GQs (e.g., college dorms vs. nursing homes vs. migrant farm worker camps).
1
No
Yes
No
Yes
Always a problem
No
Yes
Yes
A.5 How can we develop a standard of comparison for household membership on CCM interview day and usual residence on Census Day? What are the effects of recall errors and reporting errors on the CCM interview? Candidate methods for developing the standard include ethnographic studies matched to the census and CCM interviews, respondent debriefings following a CCM interview, and an in-depth Living Situation Survey.
2
No
Yes
Yes
Yes, with limitations (scope must be very limited, and any “standard” cannot be expected to get exact truth).
Yes
Yes
Yes, Ideally
A.6 Can administrative records augment CCM fieldwork from telephone follow-up to reduce cost and improve CCM data quality? For example, administrative records information may aid in confirming which enumerations linked in the computerized search for duplicates are the same person when the determination cannot be made in the field. An evaluation of A.C.E. Revision II estimates of duplication in Census 2000 using administrative records information demonstrated potential for improving CCM data quality in this manner.
2
Yes
Yes, unclear
N/A
Yes
No
Yes
Yes
OCR for page 60
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
B. Race and Hispanic Origin
B.1 Evaluate alternative race and Hispanic origin questions to include (1) double-banking of response categories and shared write-in spaces, (2) modified examples to follow the Advisory Panel recommendations, (3) separate evaluation of the features of 2005 National Census Test (NCT) panel 6, to better understand how each influences Hispanic and race reporting, and to inform future decisions, (4) modified Hispanic question that allows multiple Hispanic reporting (Y/N, yes multiple types). The latter must be tested in both Nonresponse Followup (NRFU) and the mailout. Samples must adequately represent small groups. Re-interview is needed to assess data quality.
1
No
Yes
No
Yes
No
Yes
Desirable especially for small groups
B.2 Develop a combined race and Hispanic origin question.
3
No
Yes
No
Yes
Yes
Yes
Yes
B.3 Conduct Research to support rules for editing problematic race and Hispanic origin responses (e.g., Y/N responses to Hispanic origin). A goal is to better understand respondent intent of write-in entries in the presence of, and in the absence of, marking checkboxes.
1
No
Yes
No
Yes
No
Yes
Yes
C. Coverage Improvement
Address List Development
C.1 How accurate was the final address list?
1
No
Yes
No
Yes
Yes
Yes
Yes
C.2 How should we deal with updating the address frame coming out of the 2010 Census, so that we can avoid a large and expensive address canvassing operation in the future, or so that the operation could be conducted at a much reduced cost?
1
Yes
Yes
Yes
See Below (C.2.a)
Yes
Yes
Not sure
C.2.a How can the quality of the address frame be improved with a more scientific extract process?
1
Yes
Yes
Yes
Yes, we need evaluations to demonstrate that this can be done.
Yes
Yes
Not sure
OCR for page 61
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
C.2.b How can we use additional information (like the Delivery Sequence File in rural areas, American Community Survey (ACS) Time of Interview data, Carrier Route data, and the National Change Of Address file) to improve address list maintenance?
1
Yes
Yes
Yes
Yes, we need evaluations to demonstrate that this can be done.
Yes
Yes
Not sure
C.3 Can we target Address Canvassing activities better?
1
Yes
No
No
Yes, there are concerns about the ability to reliably match persons with common names and across long distances
Politically acceptable – Can we convince stakeholders that we don’t need to do an address canvassing in their jurisdiction, while we do need to do it in the neighboring jurisdiction.
Yes
Yes
No; need 2010 data
C.4 How accurate were the data collected in Address Canvassing? How can we improve Address Canvassing quality?
1
No
Yes
No
Yes
Yes
Yes
Yes
C.4.a How well does automated Global Positioning System (GPS) collection work in terms of completeness and accuracy of GPS coordinate data?
1
No
Yes
No
Yes
Yes
Yes
Yes
C.4.b How can we improve GPS collection—increase human intervention, improve automated collection, both?
1
No
Yes
No
Yes
Yes
Yes
Yes
C.5 How can we improve address list maintenance, operational procedures, and enumeration of small multi-unit structures (2-10 units)?
2
No
Yes
No
Yes
Yes
Yes
Yes
OCR for page 62
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
Administrative Records
C.6 How can we avoid the need for followup and use administrative records to:
Identify coverage problems?
Identify and classify duplicates?
Resolve potential coverage problems identified by the coverage probes.
1
Yes
Yes
Yes
Yes
Privacy concerns and issues with file access. Stakeholders have expressed reservations about the use of administrative records
Not sure
Yes
Yes
OCR for page 63
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
Coverage Followup (CFU)
C.7 Does Coverage Followup actually work?
How effective is it?
Is CFU effectively identifying omissions?
Is it introducing bias?
How do recall and reporting errors affect its determination of residency, and hence erroneous enumerations (EEs)?
How can we afford to follow up on more coverage improvement cases?
Is the expense of CFU worth the coverage gain?
Can certain categories of response to coverage questions be automatically coded, or field coded by interviewers to reduce follow up workload?…
What recall and reporting problems affect ability to identify: missed people the respondent had in mind when filling out the undercount question? Are enumerators screening out people who are eligible to be listed?
How to optimize which cases are coded for CFU?
1
Yes
Yes
Yes
Yes
Yes
Yes
Yes
C.8 Develop and experimentally evaluate alternative designs for coverage followup instruments. Alternative methodologies might involve dependent questions; self-response by all relevant household members; immediate follow-up; and other methodological improvements to facilitate recall and reporting in CFU (i.e., conduct an integrated experiment).
1
Yes
Yes
Yes
Yes
Yes
Yes
Yes
OCR for page 64
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
C.9 To what extent did nationwide person matching improve the identification and removal of duplicates of housing units and persons in the census? In particular, what improvements can be made in the identification and removal of census duplicates of persons across some distance given the challenges created by chance agreements of names and birth dates?
1
Yes
Yes
Yes
Yes there are concerns about the ability to reliably match persons with common names and across long distances.
Yes
Yes
Yes
C.10 Develop and experimentally evaluate alternative designs of the undercount (and overcount?) questions in the mail form to effectively identify census coverage errors for follow-up. Variations might include format (open vs. closed) and wording of questions and response categories, and placement in the form.
1
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Residency Rules/Questionnaire Design
C.11 Implement and experimentally evaluate alternative residence rules and presentation of roster instructions in paper and other modes, including the National Academy of Sciences (NAS) recommendations to ask a sufficient number of residence questions to determine residence, and to obtain alternative addresses. Panels would be included in an alternative questionnaire experiment (AQE) and would require a coverage re-interview. Cognitive testing is needed for development, along with research on reading behavior and use of flashcards or other ways of presenting instructions. Alternative approaches might include:
de facto approach
Worksheet approach
Alternate address elsewhere
2
No
Yes
No
Yes
No
Yes
Yes, for some aspects
OCR for page 65
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
Be Counted
C.12 What effects does the Be Counted Program have in filling gaps in coverage?
Is it worth it?
Is including it better than trying to ensure people are counted in other ways?
Does it introduce coverage errors?
3
No
Yes
No
Yes
Public perception—of missing people or not giving them an opportunity to be enumerated in the census if not for the Be Counted program.
Yes
Yes
Yes
General
C.13 How accurate was vacancy/occupancy status of housing units in Census? Are there ways to improve accuracy?
1
No
Yes
No
Yes
Yes
Yes
Yes
C.14 Through ethnographic research, can we learn more about American Indian and Alaska Native households, Hispanic households, and immigrant communities that might result in different methods for enumeration? This research could provide insight into CFU and Census Coverage Measurement (CCM) to understand deficiencies.
2
No
Yes
No
Yes
No
Yes
Yes
OCR for page 66
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
D. Field Activities
Automation
D.1 What was the impact of adding expanded automation to field data collection for Address Canvassing, Nonresponse Followup (NRFU) and Census Coverage Measurement Personal Interview (CCM-PI)? Did we gain in efficiency? Did we see cost savings? Did automation contribute to operational improvements? Should we use the hand-held computers (HHCs) in operations other than Address Canvassing, NRFU and CCM-PI in 2020(e.g., U/E)?
1
Yes
Yes
Yes
Yes
Yes
Yes
Yes
D.2 What was the impact on field staff of using HHCs to conduct field data collection operations? Did using HHCs help us to improve the effectiveness and efficiency of field staff? Did using the HHC help us improve the productivity of field workers? What impact did the HHC have on field staff training?
2
Yes
Yes
Yes
Yes
Yes
Yes
No
Training
D.3 How can enumerator training be improved?
Can we make enumerator training more efficient/effective through the redesign of enumerator training materials and job aids?
Can we make enumerator training more effective by expanding the use of technology-based training?
1
Yes
Yes
Yes
Yes
Yes
Yes
OCR for page 67
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
D.4 How can we better prepare enumerators to ensure they are effective and efficient in their job? What is the optimal contact strategy for NRFU? How many contacts should we make for NRFU? (Requires experimental design).
1
Yes
TBD
No
This research requires an experiment design with different contact strategy in different locations. Implementing such a design in the census environment may not be practical or worthwhile
Yes
Yes
Yes
D.5 How can enumerator training be improved to reduce/minimize errors that may be introduced by interviewers? The focus is on interviewers’ contributions on coverage errors (in NRFU, CFU, and CCM-PI) in particular, but also errors in other short form items. Possible research approaches might include: an interviewer variance study, in which interviewer assignments are randomized, or assigning a sample of mail returns to NRFU enumerators for reinterview.
2
No
Yes
Yes
There are studies that could be contributions to done to determine interviewer contribution to error. However, we are not sure if training could be improved or changed to address those contributions.
Yes
Yes
Yes
OCR for page 68
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
Quality Control (QC)
D.6 How can Global Positioning System (GPS) Technology be used as a QC tool for field work, e.g., to identify curbstoning or inefficient field work?
1
Yes
Yes
Yes
Yes
Privacy concerns—GPS Tracking of employees
No
Yes
Yes
D.7 How can the QC design for field operations be improved to be more effective/efficient?
How much does the QC improve the quality of the census operations? Does the QC have a high probability of identifying data falsification and/or violation of procedures?
Is there an efficient way to verify the QC work? Is it worth it to verify the QC work?
2
No
Yes
No
Yes
Yes
Yes
Yes
D.8 Can a batch level approach to re-interview sampling improve efficiency and/or effectiveness of field re-interview operations?
3
Yes
Yes
No
Yes
No
Yes
Yes
E. Language
E.1 Can an alternative design for the bilingual English/Spanish questionnaire result in improved data?
1
Yes
Yes
Yes
Yes
Backlash; referring to lowered mail response from non-Spanish-speaking populations
Yes
Yes
Yes
E.2 Is there a better or more efficient way to stratify the mailing of the bilingual forms?
2
Low
Low
Yes
Yes
No
Yes
No
OCR for page 69
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
E.3 What information can systematic observations yield about how census enumerators are obtaining information from households with little or no understanding of English? Are there changes we can or should make to our methodologies and practices to improve these interviews?
2
No
Yes
No
Yes, but may present complex stratification issues
Yes
Yes
Yes
E.4 Can we obtain better mail response and/or higher quality data by mailing a Language Assistance Guide Booklet that depicts the questionnaire in Five (5) languages?
1
Yes
Yes
Yes
Yes
No
Yes
Yes
F. Mode Effects
F.1 What is the magnitude of the effects of mode on responses to 2010 census questions? This research would compare the 2010 mail mode content to the adaptation of the specific content items for other modes used in the 2010 census. An example would be comparing the 2010 mail form relationship question, which shows all 14 categories, to the proposed 2010 telephone-adapted version, which asks an open-ended question. In general, this study would examine response distributions (or reliability and other data quality measures) for the 2010 mail items compared to the adapted versions used in 2010 for other modes; comparable random samples would be ideal to avoid self-selection confounds.
1
No
Yes
No
Designing an experiment that reduces/ eliminates the self-selection bias is complex and may not be feasible from a field/budget/ schedule standpoint.
Yes
Yes
Yes
OCR for page 70
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
F.2 What are the effects of mode for alternative adaptations of the 2010 census questions in non-mail modes? This research would compare the 2010 mail mode content to alternative adaptations of the specific content items for other modes. These alternative versions for the non-mail modes are adaptations, which show promise in terms of providing comparable data to the mail form, but were not used in the 2010 census. We would examine response distributions (or reliability and other data quality measures) for the 2010 mail items compared to the alternative adapted versions for other modes; comparable random samples would be ideal to avoid self-selection confounds.
2
No
Yes
No
Designing an experiment that reduces/ eliminates the self-selection bias is complex and may not be feasible from a field/budget/ schedule standpoint.
Yes
Yes
Yes
G. Content
G.1 What are the combined effects on the data of all questionnaire changes made in the 2010 mail questionnaire?
1
No
Yes
No
Yes
Yes
Yes
Yes
G.2 What are the consistency and reliability of reporting in the 2010 census?
1
No
Yes
No
Yes
Yes
Yes
Yes
G.3 How well do questions perform in interviews?
2
No
Yes
No
Yes
Yes
Yes
Yes
G.4 How comparable are Census 2010 data and American Community Survey (ACS) data?
1
No
Yes
Yes
Yes
Yes
Yes
Census data required but not environment
G.5 Do current methods for identifying the householder/Person 1 perform well in all modes? If not, can improved method(s) be developed?
2
No
Yes, possibly
No
Yes
No
Yes
No
OCR for page 71
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
H. Self-Response Options
H.1 How can we improve alternatives for increasing mail response? An experiment for 2006 showed that a deadline plus delayed mailing of questionnaires improved the mail response rate by two percentage points? This should be replicated in the 2010 Census as an experiment to see if results hold up in a census environment, and to get good data on timing of returns under a deadline. If design permits, effect of deadline messaging and compressed schedule could be teased out.
1
Yes
Yes
No
Yes
No
Yes
Yes
H.2 Experiment testing an additional contact reminder after replacement questionnaire. This contact would contain stronger language, relative to the reminder postcard and replacement questionnaire, indicating that failure to comply would mean inclusion in the Nonresponse Followup (NRFU) workload (more expense, etc). Different types of contacts could be tested such as postcard, full size letter, phone message, etc.
1
Yes
Yes
No
Yes
No
Yes
Census environment is optimal but at least site test is required to utilize NRFU message.
I. Special Places/Group Quarters
I.1 Did the revised Group Quarters (GQ) definitions improve the identification and classification of GQs (GQs versus housing units, and by type)?
1
No
Yes
No
Yes
Yes
Yes
Yes
I.2 Evaluate methods for improving GQ data collections by: 1) assessing the yield from the various sources used to update the MAF/TIGER database (MTDB), as well as assessing how the various census operations update the MTDB; 2) studying effects of allowing a Usual Home Elsewhere in more types of GQs; and 3) collecting additional information to assist with unduplication of college students.
2
No
Yes
Yes
Yes
Yes
Yes
Yes
OCR for page 72
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
I.3 The National Academy of Sciences recommends, “The U.S. Census Bureau should participate in a comprehensive review of the consistency of content and availability of prison records. The accuracy of prisoner-reported prior addresses is uncertain, and should be assessed as a census experiment. A research and testing program, including experimentation as part of the 2010 census, should be initiated by the Census Bureau to evaluate the feasibility and cost of assigning incarcerated and institutionalized individuals, who have another address, to the other location." (National Academy of Sciences (NAS) report entitled “Once, Only Once, and in the Right Place – Residence Rules in the Decennial Census (September 2006), pp 9-10).”
3
No
No
Yes
Maybe—the scope of this comprehensive review of prisoner reported address information would be massive if all levels of correctional facilities (e.g., federal, state, local, and private) were included in the review. Additionally, the Census Bureau has concerns about the feasibility of actually collecting the prisoner address information.
There could be possible implications for other types of GQs. This topic has generated much discussion and varying views among census stakeholders.
Yes
Yes
Yes
J. Marketing/Publicity/Paid Advertising/Partnerships
OCR for page 73
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Topics and Questions
Criteria
Considerations
Rank
1 = H
2 = M
3 = L
Cost (Big Payoff)
Quality
New to Census
Accomplishable
Other Criteria
For 2010?
For 2020?
Census Environment Required
J.1 How effective was the communication strategy for improving response and accuracy of the census?
How do the separate components of the communications strategy contribute to the improvements (e.g., advertising, partnerships, etc.)?
How effective were the targeted messages at reaching specific audiences?
Did the communications strategy change attitudes or behavior toward and/or increase awareness of participation in the census?
1
Yes
Yes
No
The efforts in 2000 to evaluate the Advertising/ Marketing approach was inconclusive. Whether or not we can evaluate the 2010 approach depends on the solution put forth by the vendor and our ability to develop reliable technology.
Yes
Yes
Yes
K. Privacy
K.1 Test alternative presentation and placement of privacy messages in cover letter, etc.
1
Yes
Yes
No
Yes
No
Yes
Yes
K.2 Monitor public concerns about privacy and confidentiality in a series of quick-turnaround surveys conducted during the census to provide U.S. Census Bureau executives with timely information about emerging concerns and issues. Data from monitoring surveys can also augment (or replace) traditional outreach evaluation surveys, which are slow and do not provide useful information on a timely basis.
1
No
No
Yes
Yes
Can help decision makers respond to emerging issues, crises
Yes
Yes
Yes
OCR for page 74
Experimentation and Evaluation Plans for the 2010 Census: Interim Report
Criteria and Considerations for Assessing Proposed Research Topics and Questions
Criteria:
Cost (Big Payoff) – [Yes/No] Will results potentially lead to substantial cost savings in the 2020 Census?
Quality – [Yes/No] Could results conclusively measure effects on data quality?
New to Census – [Yes/No] Does the question address operations that are new since Census 2000, experienced significant procedural change, or experienced significant issues during Census 2000?
Accomplishable – [Yes/No] Will data be available to conclusively answer the question? Will there be a high demand of resources to address and answer the question? Are complex or untested methods foreseen to address and answer the question?
Considerations:
For 2010 – [Yes/No] Is this research question intended to assess an operation in the 2010 Census?
For 2020 – [Yes/No] Is this research question intended to assess a 2010 Census operation to inform the 2020 Census?
Census Environment Required? [Yes/No]
SOURCE: 2010 Census Program for Evaluations and Experiments—Appendix to Summaries of Suggested Research (planning document shared to the panel by the U.S. Census Bureau, April 13, 2007).