Abstract
The census is a cornerstone of American democracy because political representation is explicitly tied to population counts. Census data also underlie government and business planning, allocation of billions of federal dollars to states and localities, and countless public and private uses, such as forming the denominators for public health rates. Thus, the quality of a decennial census, in terms of fulfilling these fundamental purposes, is necessarily difficult to assess before all major data products from that census are developed and released. Yet such is the task of this Panel to Evaluate the Quality of the 2020 Census, in light of the truly extraordinary circumstances of the 2020 Census of the United States. In this report, we provide a detailed explication of a few fundamental messages, which we succinctly outline here.
The overriding, signature achievement of the 2020 Census is that there was a 2020 Census at all. This statement is meant as high praise for the work of U.S. Census Bureau staff in conducting the census amidst an unceasing array of challenges, notably a months-long delay in field operations due to the onset of the COVID-19 pandemic. The Census Bureau successfully carried out a census under exceptionally difficult circumstances not only because of the hard work of its staff, but also because it had honed its development efforts for 2020 on a small number of innovation areas. These key innovations were enabling self-response via the internet, shifting most precensus Address Canvassing work from field visits to in-office review of imagery and other data, reengineering field management and case handling systems, and permitting the use of administrative records data to provide enumerations for some nonresponding households. As the census approached, the Census Bureau made many scope adjustments to its planned operations, and the pandemic and other adverse conditions forced a great many other revisions. Collectively, the focus on a small number of innovations helped the Census Bureau react as well as could be expected to the difficult conditions that faced 2020 Census operations.
Error—in the statistical sense of deviation from the unknown true value—is an inevitable part of the census, and the complete absence of error is an unrealistic and unattainable standard. Accordingly, it is not surprising (nor is it meant as criticism of Census Bureau work) that there were distinct quality problems with the 2020 Census, especially when compared with the 2010 Census. The most vivid illustration of these problems is the phenomenon of age heaping in 2020 Census results (the disproportionate reporting of age in approximate, round-sounding values ending in 0 or 5), which our analyses demonstrate to be principally a function of proxy reporting in the Nonresponse Followup (NRFU) operation. The quality of the count of college and university students was particularly hard-hit by the timing of the COVID-19 pandemic and the fieldwork stoppage. Lastly, compared with 2010, increased differential coverage with respect to race and ethnicity (Hispanic origin) was present in the 2020 Census, based on analyses of 2020 Census self-return rates, NRFU operational data, and the 2020 Post-Enumeration Survey.
Regardless of unique challenges forced upon the 2020 Census, other challenges resulted from the Census Bureau’s own decisions. Chiefly, in the very late stages of 2020 Census planning, the Census Bureau decided to completely replace its methods for protecting the confidentiality of census data (termed Disclosure Avoidance System [DAS]) with an entirely new approach—one that had not been tested, prototyped, or deployed in the population census context. While confidentiality protection is a critically important responsibility of a statistical agency, this decision was made without appropriate consideration and balance regarding the utility of resulting census data products to fulfill the many important functions of census data. In short, the new DAS was not ready for use in 2020 Census production and substantially degraded the value of 2020 Census data products in terms of both quality and timeliness. Moreover, lingering questions about both the simulated database reconstruction attack that motivated the new DAS and the degree of confidentiality protection that was ultimately realized through final parameter settings have arguably harmed 2020 Census data products, some of which are not set to be released until late 2024, and the reputation of the Census Bureau.
Envisioning the 2030 Census and beyond, we echo the general guidance of previous National Academies of Sciences, Engineering, and Medicine panels: to effect major change in census conduct, the Census Bureau should focus its primary attention on a small and manageable number of major innovation areas and pursue a rigorous program of testing and systems development to address them. As priority goals for 2030 research and development, we suggest: (1) maximize self-response to the census; (2) improve the quality of data in Nonresponse Followup, including reduction if not elimination of low-confidence proxy reporting when a good alternative is available; (3) reduce gaps in coverage and data quality associated with race, ethnicity, and socioeconomic status; (4) improve the quality of address listings and contact strategies for
conventional housing units and group quarters locations alike; and (5) realign the balance between utility, timeliness, and confidentiality protection in 2030 Census data products. In addition to the five priority goals, we encourage research and development of ways to extend the use of administrative records to support the 2030 Census along the lines of the 2020 Census. We do not, however, support major movement toward an administrative records-based census as part of the 2030 Census design. That movement, if desired, is a much longer-term proposition for which the research, evidentiary, and legal base must be carefully developed. In the panel’s judgment, goals and designs for the 2030 Census should be developed in true partnership with census data users and the myriad community of stakeholders and state, local, tribal, and federal government partners that make the census the vital, grand civic ceremony that it is.
