Summary
The survival of a democracy necessarily requires that successive generations of the population embrace its institutions, procedures, and practices. Since 1790, the U.S. census has been such an institution. It is a reoccurring civic ceremony in which everyone counts, and it reaffirms a commitment to equality among all. The census is a cornerstone of American democracy because political representation is explicitly tied to population counts. The constitutional purpose of the census is apportioning the U.S. House of Representatives among the states. Congress enacted Public Law 94-171 [1975] to ensure that the U.S. Census Bureau provides relevant census data to the states for redistricting. The census also serves as a vehicle for fairness in other contexts, such as the distribution of funds for federal programs. Census data underlie government and business planning, as well as countless private and public uses such as forming the denominators for public health rates.
It is within this context of the importance and multifaceted purposes of the decennial census that the charge for assessing the quality of the 2020 Census was set forth in the Statement of Task for this consensus study. Specifically, the Census Bureau sponsored this study with a very general but expansive statement of task delineating three key requirements:
- Review information from the Census Bureau on the data collected as well as various process measures and indicators of data quality obtained as part of 2020 Census operations;
- Review other available information, such as results from Demographic Analysis (DA), process measures and preliminary results from the Post-Enumeration Survey, and analyses of administrative records; and
- Consider the results from evaluations of similar indicators from the 2010 and 2000 Censuses.
In this final report of the Panel to Evaluate the 2020 Census, we build on the frameworks for understanding census quality developed in our interim report,
bringing to bear the results of original data analysis performed by a dedicated subgroup of the panel. This panel was granted unprecedented access to data and resources, with the subgroup performing its work behind the Census Bureau’s information technology firewall; we are grateful for this access and acknowledge that very few if any of the data and information requests we made were denied. That said, we also note that all presented results and analyses from our work have been subject to the Census Bureau’s standards for information disclosure review. Hence, we have endeavored to be as thoroughgoing as possible in our analyses, but are necessarily limited in what we can report, constrained not only by the lack of available time and resources to address all areas of interest, but also by the mechanics and constraints on reported data of the Census Bureau’s disclosure avoidance policies and infrastructure.
It is also important to note at the outset that it is inherently difficult, if not impossible, to fully evaluate a census while it is still in progress, yet that is the case with the 2020 Census. The 2020 Census data products have been greatly delayed, principally (as we discuss in detail) due to complications in implementing new techniques for confidentiality protection, and the Census Bureau’s own evaluations and assessments have been delayed as well. Accordingly, it is inevitable that there will be discrepancies between what we say in this report and what is learned in the final 2020 Census data products and assessment reports.
The overriding, signature achievement of the 2020 Census is that there was a 2020 Census at all. This statement is meant as high praise for the work of Census Bureau staff in conducting the census amidst an unceasing array of challenges, notably a months-long delay in field operations due to the onset of the COVID-19 pandemic. The Census Bureau successfully carried out a census under exceptionally difficult circumstances, not only because of the hard work of its staff, but also because it had honed its development efforts for 2020 on a small number of innovation areas. These key innovations were enabling self-response via the internet, shifting most precensus Address Canvassing work from field visits to in-office review of imagery and other data, reengineering field management and case handling systems, and permitting the use of administrative records data to provide enumerations for some nonresponding households. In this report, we provide detailed chapters on these innovation areas and other major aspects of the 2020 Census in building a comprehensive assessment—starting with examination of a longstanding indicator of census and survey quality. This Summary describes each in turn, stating the full text of some (but not all, for brevity) of our conclusions and recommendations.1
___________________
1 A full list of conclusions and recommendations is presented in Appendix A, and a Glossary and List of Abbreviations to aid the reader follows in Appendix B.
AGE HEAPING AS AN INDICATOR OF DATA QUALITY
Age heaping refers to a phenomenon whereby the ages of some survey subjects are misreported as favored ages—typically, but not necessarily, those ending in 0 or 5. Age heaping is important for two reasons. First, some uses of census data call for detailed age distributions at the state and local level that are not available from DA. Second, age heaping may be an indicator of other additional data-quality issues in the census, affecting both coverage and characteristics.
Conclusion 3.1: Age heaping (excess responses of ages ending in 0 or 5) is a well-recognized indicator of data quality in censuses. Age heaping was much more prevalent in the 2020 Census than in 2010, by an estimated factor of at least 2.5 for ages between 23 and 62 years, for which heaping is most common. This increase was almost completely due to increases in the rate of heaping during Nonresponse Followup (NRFU)—especially from proxy respondents but also from NRFU household respondents. Age heaping was rare in both censuses for self-responses, whether by paper or internet (in 2020 only), and there was no evidence of age heaping associated with administrative records.
Conclusion 3.2: Findings about age heaping raise concerns about the overall quality of data collected during 2020 Nonresponse Followup (NRFU), especially from proxy respondents but even from some household members. Other quality indicators, including item nonresponse rates, numbers of people requiring imputation of all characteristics, and coverage errors, reinforce the difficulties of obtaining accurate data in NRFU and indicate that the difficulties increased in 2020 compared with 2010. The greater extent of quality problems in NRFU in 2020 compared with 2010 may be attributable to the COVID-19 pandemic (including the resulting delay in the NRFU schedule), but other factors might also be involved, which would need to be addressed in 2030 Census planning.
COVERAGE MEASUREMENT IN THE 2020 CENSUS
Every census since 1790 has missed some people who should have been counted, counted some people twice, and counted other people who should not have been counted (e.g., someone born after Census Day). The balance of undercounted and overcounted people determines whether the census experienced a net undercount or overcount for the nation as a whole. In addition, people can be missed in one place and erroneously counted in another place according to census rules regarding usual residence. Also, people may misreport information such as age, or fail to report race or ethnicity, for
example, and have race or ethnicity erroneously imputed. These kinds of errors will not affect coverage for the nation but will add to net under- or overcount in a geographic area or population group. “Differential” net undercount or overcount—that is, the difference in coverage rates between population groups or geographic areas—is more concerning than the national net undercount or overcount. Principal methods for estimating census coverage (Box 4.1) include DA and the Post-Enumeration Survey (PES).
Conclusion 4.2: Comparing the 2010 and 2020 Census population coverage estimates from the Post-Enumeration Survey (PES):
- Differences among race and ethnic groups widened substantially in 2020, with adverse implications for use of the data to allocate fixed resources, such as representation, funding, and services. Some groups saw increases in net overcounts, particularly non-Hispanic White Alone people and Asian people, and other groups saw increases in net undercounts, particularly Black people, Hispanic people, and American Indians and Alaska Natives on reservations.
- Increases in net undercounts in 2020 for groups such as Black and Hispanic people were driven by increases in omissions from the census (as measured by a lower match rate for people in the P-sample to the E-sample in the PES). Omissions, in turn, were higher in census tracts with lower rates of self-response.
- Net overcounts and undercounts were more pronounced for people of various groups in rental compared with owner housing, particularly in 2020 compared with 2010.
Recommendation 4.1: For the Demographic Analysis (DA) coverage evaluation method, the U.S. Census Bureau should conduct research on:
- The reasonableness of the assumption for incompleteness of Medicare enrollment in the high series, which produced coverage estimates that diverged greatly from the low and middle series for people ages 75 and older, through an appropriate match study;
- The suitability of Medicare data for coverage evaluation of the entire population ages 65 and older as in the 2010 Census and for improving the census count of this age group in 2030;
- The reasonableness of the methods for assigning Hispanic and non-Hispanic births for coverage estimates for children and young adults, which produced wide differences between the low and high DA series in 2020;
- Methods for narrowing estimates of net international migration, which affect estimates for Hispanic people and the total population; and
- Methods for developing experimental subnational DA estimates, starting with young children.
Recommendation 4.3: For the Post-Enumeration Survey (PES) method, the U.S. Census Bureau should:
- Perform sensitivity analyses, based on plausible assumptions, to put error bounds around such key operations as matching and imputation of match and enumeration status, to evaluate the quality of the 2020 PES and plan such analyses from the outset for the 2030 PES;
- Plan analyses for the 2030 PES of components of error for census operations in exhaustive and mutually exclusive categories that are as comparable as possible between 2020 and 2030;
- Seek adequate funding to increase the PES sample size in 2030 to at least 2010 levels; and
- Experiment with modeling to estimate net undercoverage and overcoverage for more detailed geographic and demographic groups for which direct estimates could not be provided in 2020.
MASTER ADDRESS FILE
Over its entire existence, the fundamental concept of the U.S. census has been to count each person in the right place—at their location of usual residence. Accordingly, it follows that the quality of the decennial census is inextricably linked to the quality of the Census Bureau’s geographic resources and, in particular, the address list that is used to mount census data-collection efforts. Prior to the 2000 Census, the address list used as the operational frame to conduct and monitor progress in the decennial census was constructed anew each decade, prior to and during the census. The 2000 Census began the practice of maintaining an ongoing Master Address File (MAF) to support the Census Bureau’s household survey programs as well as the decennial count. The MAF is intended to be a repository for all living quarters in the United States, including not only housing units but also group quarters (such as college and university student housing, health care/nursing facilities, and correctional facilities) and transitory locations (such as marinas, recreational vehicles, and hotels/motels). The most striking impression from analysis of metadata for the 2020 MAF is also the least surprising: the MAF is largely composed of addresses that
have been on the rolls for many years, and continuity in the MAF over time necessarily dominates change and churn.
Conclusion 5.1: The U.S. Census Bureau’s approach of retaining a fluid Master Address File database and filtering operational extracts from that database is a sound one, provided that the filters are continually studied and updated to improve their efficacy.
Recommendation 5.1: The U.S. Census Bureau should continue to research and refine the filters it applies to the Master Address File to derive functional operational extracts, with the intent to reduce the number of addresses cancelled during collection (i.e., flagged as deletes in Nonresponse Followup and other field operations). Along those lines, the Census Bureau should continue to study ways to partner with and more fully utilize U.S. Postal Service data (including the Delivery Sequence File) and potential sources for address addition and revision information between censuses, such as address data recoverable from administrative records extracts, with particular eye toward more regular updating in areas without mail delivery.
Recommendation 5.2: The rules and scope of participant-input address-building operations of the census require clarification, including the Local Update of Census Addresses, New Construction, and Count Review. The U.S. Census Bureau should continue to improve ways to work with user-supplied input or data resources, and make it easier for state, local, and tribal authorities to supply input in usable form, while more clearly laying out the expectations for those address-building operations.
SELF-RESPONSE TO THE CENSUS
Two major census quality indicators are coverage (all eligible households are counted once and in the right place) and the quality of the data collected from each household in terms of accuracy and completeness. Evaluations have uniformly shown, including in 2020, that self-responses to the census are higher quality than those obtained in Nonresponse Followup (NRFU) and other in-person enumeration operations. The Census Bureau had a major success in 2020 with the use of the internet as a self-response mode in addition to paper and telephone and in achieving a high return rate (self-responses as a percentage of occupied housing units). The Census Bureau used a “wave” mailing strategy (multiple mailings) to encourage self-response—pushing
internet response (Internet First) in areas more equipped to use that response mode (e.g., areas with more households with broadband) and giving other areas the option up front to respond by paper or internet (Internet Choice). People could also respond via the internet without using the census ID number on printed materials (Non-ID enumerations).
Item nonresponse is an important data-quality element. Self-Response generally yields lower item nonresponse than Nonresponse Followup (NRFU) (see Figure 6.2). In comparing methods of Self-Response, the internet appears to have less item nonresponse for most data items. Telephone Self-Response appears to have slightly more item nonresponse compared with internet, though very few households used the telephone to respond. Paper has the most item nonresponse among the 2020 Self-Response modes, except for the age item.
Conclusion 6.1: Going from a paper-only 2010 Census to a majority Internet Self-Response 2020 Census is a commendable achievement, and it is particularly commendable that the U.S. Census Bureau’s internally developed Primus system was able to be swapped in immediately prior to the census with excellent performance throughout the count. (Primus was the backup system pressed into primary service after concerns surfaced regarding the outsource-developed system.) The distinction between Internet First and Internet Choice contact strategies was a reasonable starting position.
Recommendation 6.1: The U.S. Census Bureau should continue to investigate the performance of the contact strategies used in the 2020 Census, to increase overall self-response (particularly via the internet) in 2030, more in concordance with international experience with online census response. This investigation should include review of the effectiveness of the 2020 Census’s segmentation into Internet First and Internet Choice strategies, as well as the timing and the operative contact/enumeration strategy (e.g., Internet First or Internet Choice mailings or Update Leave delivery) underlying successful Non-ID enumerations in 2020.
NONRESPONSE FOLLOWUP
In the 2020 Census, there was substantial geographic variation in the proportion of households that self-responded and accordingly in the proportion enumerated by NRFU. There was also geographic variation in the proportion of households enumerated by each NRFU response mode. Overall, NRFU enumerations were of poorer quality than self-responses, with some NRFU response modes worse than others. Geographic variation in the quality of data
collected is likely associated with variations in populations’ social, economic, demographic, and housing characteristics.
Conclusion 7.4: Nonresponse Followup (NRFU) enumerations result in more item nonresponse and poorer coverage than self-responses. NRFU item nonresponse for race and Hispanic origin exceeded 15% in the 2020 Census. Proxy enumerations had greater than 35% item nonresponse for race and Hispanic origin. Results suggest that census tracts with higher proportions of people with lower socioeconomic status (SES) characteristics were more likely to be enumerated using modes that resulted in higher item nonresponse. The net result is that 2020 Census data on race, Hispanic origin, and age are less accurate in those census tracts than in those with higher proportions of higher SES characteristics. This result is most relevant for redistricting. State and local officials are defining political districts with data that vary in how accurately they reflect the age and racial and ethnic composition of the population. Census data are particularly prone to error in areas with high percentages of people who are Black, Hispanic, or American Indian and Alaska Native living on reservations.
Recommendation 7.1: As a goal for the 2030 Census, the U.S. Census Bureau should consider a major reduction in the use of proxy interviewing for enumeration, if not the elimination of proxy reporting in all but very limited circumstances. Work toward this goal should be predicated on the results of research that only the Census Bureau can conduct, reexamining and comparing proxy reports in the 2020 Census with the information that would have been available from administrative records or third-party data resources.
USE OF ADMINISTRATIVE RECORDS FOR ENUMERATION IN THE 2020 CENSUS
Administrative records data have been used in previous censuses for many purposes, including as input to updating the Master Address File and in the Demographic Analysis method of coverage evaluation, but not to replace self-response or NRFU. In the 2020 Census, use of administrative records for enumeration in cases where a self-response was not received and one enumerator NRFU visit also failed to produce a return was a successful innovation. Administrative records generated almost 4% of occupied housing unit enumerations and over 14% of vacant housing unit enumerations. Further research and testing to determine how to expand this application of administrative records to reduce the NRFU workload is warranted. However, movement toward an administrative records-based census—enumerating a substantial proportion of
the population using records data without a contact attempt—is a long-term proposition that does not appear feasible for the 2030 Census.
Conclusion 8.1: Based on the information currently available, the U.S. Census Bureau deserves commendation for the meaningful steps made in the limited use of administrative records as an alternative to the enumeration of some nonresponding households in the 2020 Census. The Census Bureau acted with appropriate levels of both boldness and caution in settling on conditions under which a nonresponding household at a designated residence would be enumerated using records information after only one enumerator visit rather than six, yet still ensuring that single in-person visit. Moreover, this change in approach was preceded by an extensive history of high-quality experimentation and testing.
Conclusion 8.2: Moving to the use of administrative records to enumerate a substantial proportion of the population without a contact attempt does not seem feasible for the 2030 Census. Rather, it appears to be a long-term proposition, requiring extensive testing, research, and engagement with Congress, other stakeholders, and the public. Not only are there operational and technical issues to resolve and data-quality standards to determine and assess for geographic areas and population subgroups, but there is also the need to consider the ramifications of switching from a household/address-based focus to a person-based focus and to address issues of legal, constitutional, and public acceptability. It will be essential for the U.S. Census Bureau to be as transparent as possible about its research and testing and to have open and constructive dialogue with all parties.
Recommendation 8.1: The U.S. Census Bureau should continue its research and development program concerning the best ways to use administrative records as a supplement to decennial census operations. Potential uses of administrative records include expanding enumeration of limited subsets of the 2030 Census population in the Nonresponse Followup workload, reducing proxy responses and whole-person imputations, and possibly redressing the long-standing net undercount of children ages 0–4.
MEASUREMENT OF THE GROUP QUARTERS POPULATION
Group quarters (GQs) are places where people live or stay in a group living arrangement, and that are owned or managed by entities or organizations providing housing and/or services for the residents. These services may
include custodial or medical care as well as other types of assistance. GQs differ from typical household living arrangements because the people living in them are usually not related to one another. Moreover, GQs also include those who have no fixed address and are enumerated at places where they receive services, such as in shelters for those experiencing homelessness. Group Quarters Enumeration is always difficult, and the COVID-19 pandemic-induced operational adjustments were particularly hard to grapple with in this respect. As with other parts of the 2020 Census, Census Bureau staff rendered laudable and extraordinary efforts to adapt to the conditions and make the GQ count as complete as possible. Yet it is only fair to reiterate that not all of the 2020 Census challenges in Group Quarters Enumeration were unique to the circumstances of 2020.
Conclusion 9.1: The enumeration of nonhousehold group quarters (GQ) locations in the 2020 Census experienced major shortcomings, some of which can be directly attributed to COVID-19 pandemic adjustments to the data-collection scheduling (e.g., the scuttling of prearranged plans for in-person enumeration worked out in the Group Quarters Advance Visit operation). Yet several important operational and conceptual shortcomings cannot be attributed solely to the pandemic, such as the mismatch between census content and the U.S. Department of Education’s interpretation of permissible data provision under federal privacy law, the shortfalls in the enumeration of people in military GQs, as well as the fluctuation in status (and resulting problems in the address list) in more fluid GQ types like group homes.
Recommendation 9.2: Group quarters (GQs) require more than updated address information; they require continually updated contact strategies as well. The U.S. Census Bureau should develop “selected advance contact” protocols with GQ facilities by type (and their parent companies, as appropriate), akin to follow-ups performed with businesses, and state, local, and tribal partners should be engaged in this update activity.
Recommendation 9.3: In light of the 2020 experience, the U.S. Census Bureau should convene group quarters (GQ) stakeholders in a discussion of more effective and useful options for electronic provision of data for GQ residents, and on the issues involved in obtaining access to administrative records for GQ facilities. The completeness of such records should also be assessed by GQ type.
MEASUREMENT OF RACE AND ETHNICITY
The quality of the race and ethnicity data collected in the 2020 Census is of central concern—these data are among the most important a U.S. census collects, and the census historically has not counted all racial and ethnic groups equally well. It is important to note that “race” and “ethnicity” are social-political constructs and often associated with national origins. They are also fluid, depending in part on the format of the survey or census questions and who is the respondent (self, other household member, proxy). The measurement of race and ethnicity in the 2020 Census was complicated by several factors. While an expected change by the U.S. Office of Management and Budget of the measurement standard from the two-question format for race and ethnicity to a combined question did not occur, the Census Bureau made important changes to the existing questions. These changes, which included the provision of write-ins for the White and Black categories and expanded data capture and coding, undoubtedly played a role in the greater diversity measured in the 2020 Census.
Conclusion 10.1: The 2020 Census depicted a more diverse nation than did the 2010 Census. The White Alone population declined significantly, particularly among Hispanic people; the Hispanic population increased; and the Two or More Races population increased substantially, particularly among Hispanic people. It is unknown the extent to which these changes were due to demographic changes (e.g., more multiracial children); changes in self-identification; the provision of write-in origins (perhaps not intended to indicate racial identification) made possible by the format and processing changes to the 2020 Census race question; and other effects of the 2020 Census format and processing changes.
Conclusion 10.2: The 2020 Census had poorer quality data on race and ethnicity compared with the 2010 Census in terms of coverage error (net overcounts and undercounts), rates of missing and imputed responses, and the noise infused by the 2020 Census Disclosure Avoidance System.
- The 2020 Census exhibited larger coverage errors compared with 2010 for race and ethnic groups. There were increases in net overcounts for some groups, particularly non-Hispanic White Alone people and Asian people, and increases in net undercounts for other groups, particularly Black people, Hispanic people, and American Indians and Alaska Natives. Consequently, differences among groups widened significantly in 2020, with adverse implications for uses of census data to allocate fixed resources, such as representation, funding, and services.
- The 2020 Census had higher imputation rates (to account for missing and inconsistent responses) for race and ethnicity compared with 2010, particularly for responses not provided by a household member via the internet, a mailed-back paper questionnaire, or telephone. Moreover, the 2020 Census had much higher imputation rates for race and ethnicity for people in group quarters compared with the already high rates in 2010.
- The 2020 Disclosure Avoidance System injected noise of considerable magnitude into the Redistricting File for small race and ethnicity groups in small governmental jurisdictions, such as incorporated places and American Indian tribal lands and Alaska Native villages.
Recommendation 10.1: The U.S. Census Bureau should conduct research to determine how the changes in format and processes that were made in the 2020 Census and in the American Community Survey beginning in 2020 affected the distributions of race and ethnicity. Such research should use qualitative, quantitative, and simulation methods to ascertain: how respondents viewed and used the 2010 and 2020 formats; trends in multirace reporting by age, sex, race, and ethnicity; how samples of 2020 respondents would have been categorized using the 2010 format, data capture, and coding; and the implications of differences in write-ins by response mode (e.g., more write-ins for internet responses) on race distributions among geographic areas. The Census Bureau should communicate the results of this research to data users to assist them to understand the implications of the changes for time series, and should also use the findings to inform 2030 Census planning.
Recommendation 10.2: To improve the quality of reporting on race and ethnicity in the census, American Community Survey (ACS), and other federal data collections, the U.S. Office of Management and Budget (OMB) should revise Statistical Policy Directive No. 15 to adopt a combined, check-more-than-one race/ethnicity classification with both Hispanic and Middle Eastern or North African as main categories, in addition to White, Black, American Indian and Alaska Native, Asian, and Native Hawaiian and Other Pacific Islander. OMB should formalize any changes to the race and ethnicity reporting standards as soon as practicable, to permit the U.S. Census Bureau to implement the new race/ethnicity question in the ACS and in the 2030 Census. OMB should allow agencies to collect data only for the major categories in the combined
question when added detail (as in the 2020 Census) would impose undue administrative burdens. A decision by the Census Bureau on including added detail and write-in spaces, along with expanded data capture and coding procedures, for a combined race/ethnicity question for the ACS and the 2030 Census should await completion of research on such matters as how respondents viewed the 2020 question format and the effects of response-mode differences in write-in responses on diversity among geographic areas.
Recommendation 10.4: The U.S. Census Bureau should consult as early as possible with the redistricting community, which includes state legislators, redistricting commissions, political parties, and political consultants, among others, to determine the optimum set of tabulations to include on the 2030 Redistricting File, whether the current race and ethnicity categories are retained or the proposed revisions take effect. The consultation should include consideration of streamlining the number of race/ethnicity categories in the file and combining blocks with small populations (using input from localities) to maximize the ability to protect confidentiality while maintaining accuracy.
IMPACT OF DISCLOSURE AVOIDANCE CHANGES ON 2020 CENSUS DATA PRODUCTS
Regardless of unique challenges forced upon the 2020 Census, other challenges were the result of the Census Bureau’s own decisions. Chiefly, the Census Bureau decided in the very late stages of 2020 Census planning to completely replace its Disclosure Avoidance System (DAS) for protecting confidentiality in census data products with an entirely new approach—one that had not been tested, prototyped, or deployed in the population census context. While confidentiality protection is a critically important responsibility of a statistical agency, this decision was made without appropriate consideration and balance regarding the utility of resulting census data products to fulfill the many important functions of census data. In short, the new DAS was not ready for use in 2020 Census production, degrading the value of 2020 Census data products in terms of both quality and timeliness. Moreover, lingering questions remain about both the simulated database reconstruction attack that motivated the new DAS and the degree of confidentiality protection that was ultimately realized through final parameter settings. This has arguably harmed the 2020 Census data products, some of which are not set to be released until late 2024, and the reputation of the Census Bureau.
Conclusion 11.1: In an era of Big Data, linkage technology, and other aspects of today’s computational and data environment, it is difficult for the decennial census to balance the need for accurate data for small areas and small population groups with an adequate level of confidentiality protection. When evaluating methods and levels of confidentiality protection, stewardship of census data for the public good requires balancing societal needs for the information—particularly for equity in representation, fund allocation, and many other uses—with plausible harms of breaches of confidentiality.
Conclusion 11.2: The decision of the U.S. Census Bureau to respond to the threats to confidentiality protection at a very late date in 2020 Census planning with a new, more complex Disclosure Avoidance System (DAS) using differential privacy-based algorithms went counter to long-standing principles of decennial census planning. The approach had not been tested in a census environment nor had the ability of the algorithms to handle critical user data needs been assessed; the Census Bureau had no backup plan should implementation of the new DAS prove challenging. The decision to continue to deploy the new DAS in the face of serious implementation problems has resulted in marked delays in delivery of data products, with some variables and types of geographic units of questionable utility, and other variables and geographies not provided at all. In addition, it is not clear that the chosen privacy budgets for the various 2020 Census data products, with high values of the ∊ parameter that trade off accuracy with confidentiality protection, provide much actual protection.
Conclusion 11.3: Sustained, interactive communication with data users in advance of major changes to data products from a census or long-running survey is critical for a statistical agency to maintain credibility with and serve its user community. The late decision to adopt new confidentiality-protection techniques for the 2020 Census led to untimely and inadequate user communication on the part of the U.S. Census Bureau. The comment periods after the release of each demonstration file for the Redistricting and Demographic and Housing Characteristics Files were valuable, but many users lacked the staff and resources to respond adequately.
Recommendation 11.1: For 2030 Census data products, the U.S. Census Bureau should adopt the risk-utility framework recommended by the Advisory Commission on Data for Evidence Building, which accepts that disclosure risk is a continuum, that not all data items are equally sensitive, and that federal
statistical data need to be accessible and useful for a wide range of users and uses.
Recommendation 11.2: At a minimum, the 2030 Census should provide as-collected (i.e., unaltered) total population counts for all governmental units (states, counties and equivalents, minor civil divisions, incorporated places, recognized tribal areas, school districts) and quasi-governmental units (census county divisions, census-designated places, and tribal statistical areas), no matter how small. In addition, census block population totals should add up to block groups, census tracts, counties, and states.
Recommendation 11.3: For the 2030 Census data product plan, the U.S. Census Bureau should begin immediately on a multipronged research program with ample testing and opportunities for feedback and dialogue with the data user and stakeholder community, broadly defined. The goal should be an end-to-end plan by 2027–2028 for producing a suite of 2030 data products that serve user needs, are appropriately protected, and meet the time schedule of the 1990–2010 Censuses.
Recommendation 11.5: The U.S. Census Bureau should welcome initiatives to add language to appropriate legislative vehicles, such as the Foundations for Evidence-Based Policymaking Act of 2018, that prescribes responsibilities and penalties for data users, in addition to agency staff, for willful, harmful disclosure of confidential information.
LEARNING FROM 2020, PREPARING FOR 2030
Envisioning the 2030 Census and beyond, we echo the general guidance of previous National Academies of Sciences, Engineering, and Medicine panels. To effect major change in census conduct, the Census Bureau should focus its primary attention on a small and manageable number of major innovation areas and pursue a rigorous program of testing and systems development to address those areas. At the same time, there is danger in excessively casting the 2020 Census as the result of a one-off set of conditions. There is still much to be learned from the 2020 experience for shaping the 2030 Census and its successors, such as a program of earlier and sustained In-Office Address Canvassing including portions that were descoped for budgetary reasons. We encourage documenting and improving upon the contingency procedures that had to be developed and invoked extemporaneously in 2020 (e.g., enhancing
work with community and state/local/tribal government partners), many of which may hold promise for 2030 and beyond.
Recommendation 12.1: To prioritize its testing and development of both procedures and technical systems for the 2030 Census, the U.S. Census Bureau should take the following as primary objectives:
- Maximize self-response to the census, including better matching of contact and communication strategies to the desired response mode, with particular attention to hard-to-reach, at-risk populations;
- Improve the quality of data in Nonresponse Followup, including reducing if not eliminating the use of low-quality proxy reporting when there is an alternative available;
- Reduce gaps in coverage and data quality associated with race, ethnicity, and socioeconomic status;
- Improve the quality of address listings and contact strategies for all living quarters, including group quarters; and
- Realign the balance between utility, timeliness, and confidentiality protection in 2030 Census data products.
It is important to note a deliberate omission from this list of suggested priority goals. Though we encourage the Census Bureau to continue its solid work on using administrative records data as a supplement to census operations, including judicious use of records data to enumerate some nonresponding households, we do not support a movement toward an administrative records-based census in which records would be used as original enumeration for a large percentage of the population. We do not believe that the vast array of conceptual, technical, and legal issues associated with a move toward a records-based census are resolvable in time for production of the 2030 Census. It is vitally important that the goals and designs of the 2030 Census be developed in true partnership with census data users; myriad community stakeholders; and state, local, tribal, and government partners—all partnerships that make the decennial census the essential, grand civic ceremony upon which the nation relies.
