– 6 –

Self-Response to the Census

A goal of every census is to count the entire population once, only once, and in the correct location. A related goal is to obtain complete and accurate data on the census form questions, such as race and ethnicity, age, and housing tenure (owner/renter), for people and households. The 1960 Census was the first to use self-response as the primary means of enumeration, based on research that showed wide variation in the quality of responses obtained by census enumerators (National Research Council, 2010:22–24). Evaluations of census quality in terms of coverage of the population and completeness of responses to individual questions have uniformly shown, including in 2020, that self-responses to the census are of higher quality than those obtained by interviewers in Nonresponse Followup (NRFU) and other in-person enumeration operations (see Box 6.1, which calls Figures 6.1 and 6.2, and Chapter 3).

The 2020 Census achieved a self-response rate—the percentage of all addresses that the U.S. Census Bureau endeavored to enumerate—that was roughly comparable to the 2010 Census mark, about 66–67%. The 2020 Census self-response return rate—the percentage of occupied housing units that self-responded—was about 74% according to information provided by the Census Bureau for the analysis in this chapter, examining the performance of the 2020 Census Self-Response options by characteristics of groups of census tracts.¹ The

___________________

¹ Comparisons of self-response and self-response return rates between 2020 and 2010 are not straightforward because the 2010 Census had a higher proportion of households that had no opportunity for self-response because they were enumerated by in-person procedures only (see Appendix C). We do not provide a self-response return rate for the 2010 Census because of this denominator problem and because rates that can be calculated for 2010 from the Census Bureau’s

solid self-response in 2020 occurred despite the COVID-19 pandemic, which curtailed local outreach and communication efforts, and despite low levels of public trust in government.² This achievement was vital to containing the workload for NRFU, which the COVID-19 pandemic made especially challenging to conduct.

In this chapter, we describe and assess the 2020 Self-Response modalities and outcomes. We first detail the Self-Response operations for 2020 (with a summary for 2010 as a frame of reference), highlighting the very successful

___________________

Operational Quality Metric releases (U.S. Census Bureau, 2021b:Address Resolution by Household Size) differ from official rates for that census (see Letourneau, 2012). We note that the 74% self-return rate for 2020 cited in the text also differs from rates that can be calculated from the same operational metrics. A forthcoming 2020 Census operational assessment report should provide definitive rates.

² See Pew Research Center (2022). The percentage of people agreeing that the federal government can be trusted to do the right thing “just about always” or “most of the time” has remained at or below 24% since 2007.

introduction of the internet as a primary Self-Response mode in 2020 (see Appendix C for more details).

We then examine variation in self-response among groups of census tracts categorized by social, economic, and housing characteristics from the American Community Survey (ACS). Variation in self-response inevitably means variation in data quality among areas and groups. When measured by completeness of population coverage, variation in quality has adverse implications for the equity of allocations of U.S. representatives, funds, and services based on census results. Variation in completeness and accuracy of characteristics can also be detrimental to areas and groups to the extent that imputation procedures for item missingness and edits to correct inconsistent responses may also be inaccurate. Both kinds of variation can erroneously reduce or amplify true differences among geographic areas and population groups and thereby bias comparisons of many kinds of rates that use census data for denominators—for example, mortality rates by age, sex, race, and ethnicity in counties or smaller areas.

Another issue with variations in completeness of coverage and accuracy of characteristics is that, for the balance of the decade, decennial census data become the statistical foundation for the Census Bureau’s annual population estimates for states, counties, cities, and towns. In turn, the population estimates have many uses, including as denominators and as weighting controls for the ACS and other surveys. Variations in data quality among geographic areas and population groups can undermine the relative accuracy of population estimates and denominators from the census.³

6.1 SELF-RESPONSE STRATEGIES AND PROCESSES

Except for a small-scale experiment with internet response in the 2000 Census, the primary Self-Response mode in the 1970–2010 Censuses continued to be paper questionnaires as in 1960, with a small number of self-responses obtained by telephone. Peer countries began offering the internet as a self-response mode as early as the 2001 census in Spain and the 2006 census in Canada and several other countries and expanded its use substantially in censuses in 2010–2011 (National Research Council, 2010:290–301). The Census Bureau benefited from their experience when it made the decision to support Internet Self-Response for the 2020 Census, as recommended by the National Research Council (2011:1,18–20). The Census Bureau also benefited from its own experience in successfully using the internet as a primary self-response mode in the ACS, beginning in 2013. Moving to operationalize and promote Internet Self-Response in 2020 proved to be one of the major innovations and success stories of this census. Internet Self-Response was crucial to enabling the Census Bureau to surmount the hurdles for the census enumeration posed by the COVID-19 pandemic. Responses with a paper questionnaire were also important in achieving higher-than-2010 self-response and return rates.

6.1.1 2010 Census Self-Response Procedures

In 2010, bar-coded paper questionnaires with a return envelope for self-response were mailed to addresses in areas of the country with predominantly “city-style” addresses (street names and house numbers). These Mailout/Mailback areas comprised type of enumeration area (TEA) 1, which accounted for 91% of 115.9 million total addresses. Another 8% of addresses in

___________________

³ The Census Bureau’s population estimates currently use only population totals from the 2020 Census. For demographic characteristics, they use 2010-based estimates trended forwarded. The reason has to do with the new Disclosure Avoidance System (DAS), using differential privacy-based methods. The 2020 DAS has precluded the availability of the necessary demographic data for the population estimates until an (as of fall 2023) unspecified date. See Chapter 11 for a detailed discussion of the new confidentiality-protection methods and their implications for data quality for small geographic areas and population groups.

TEA 2 (Update/Leave areas) and TEA 7 (Urban Update/Leave areas) had the questionnaire dropped off by a census employee with a return envelope for self-response. The remaining 1% of addresses were enumerated in person (Johanson et al., 2011:15).

The mailing strategy in TEA 1 consisted of three mailings to every address: (1) an advance letter informing households that the census questionnaire would be arriving soon; (2) a mailing package containing the questionnaire; and (3) a reminder postcard. A fourth mailing, which contained a replacement questionnaire, was sent to 15 million addresses in Mailout/Mailback areas expected to have low response and another 25 million nonresponding addresses in Mailout/Mailback areas expected to have a moderate level of self-response.

The Census Bureau also included a “Be Counted” operation, which encouraged people to pick up and complete an unaddressed (“Non-ID”) questionnaire at a library or other public location. This procedure added the people in 676,000 housing units to the 2010 Census (addresses provided by respondents had to match a housing unit address on the Master Address File (MAF) to be counted—see Jackson et al. (2012:Table 27)).

6.1.2 2020 Census Self-Response Procedures

During the 2010–2020 intercensal period, the Census Bureau made a commitment to heavily promote Internet Self-Response in 2020, to contain costs and improve data quality. The Census Bureau tested a variety of strategies to improve its mailout procedures, to develop a user-friendly online questionnaire and log-in page, and to best ensure that the cloud-based internet operation would not crash under heavy loads or be attacked; Box 6.2 provides additional comment on this internet response platform. The Census Bureau also developed procedures for “Non-ID” online response whereby people who had mislaid their 12-digit internet ID codes or had not received them could provide their addresses and complete their enumerations on their cell phones or laptops.

Testing resulted in generation of a more elaborate “wave” mailing strategy (pioneered in Canada) to encourage self-response for addresses in TEA 1. In 2020, TEA 1 comprised 95% of the total of 151.8 million addresses, while TEA 6 (Update Leave), in which census enumerators delivered paper questionnaires accompanied by letters with an internet ID code to respond online, comprised another almost 5%. The remaining TEAs involving in-person enumeration (Update Enumerate and Remote Alaska) comprised less than 1% of addresses.⁴ Advertising and outreach activities supported all of these enumeration strategies.

___________________

⁴ See https://www.census.gov/newsroom/press-releases/2019/tea-viewer.html.

Internet First and Internet Choice

For TEA 1, the first part of the mail strategy was to divide areas into “Internet First” and “Internet Choice.” Internet First areas were those for which the majority of residences had relatively easy access to the internet—specifically, census tracts not characterized as Internet Choice (see below). For Internet First areas, the plan was to start with a letter from the Census Bureau, followed by a second letter, then a postcard, followed by a letter with a paper questionnaire included, and lastly an “It’s not too late” postcard. Each letter informed residents about how to be enumerated online and provided them with a 12-digit ID to use for internet response.

Internet Choice areas were census tracts with low overall self-response rates in the ACS and one of the following: low ACS internet response rates, high rates of people ages 65 and older, or low rates of internet subscribership, estimated from 5-year ACS data for 2013 through 2017. The sequence of mailings for Internet Choice areas was to be a letter with a questionnaire enclosed along with a link for responding online, followed by a letter, a postcard, a letter with a questionnaire, and then an “It’s not too late” postcard.

The first three mailings for both Internet First and Internet Choice areas went out on schedule beginning in mid-March 2020. The COVID-19 pandemic led to delays of 1–3 weeks for the remaining two mailings because of the need to protect Census Bureau workers who assembled mailing packages (see Table C.2 in Appendix C). In response to the pandemic, the Census Bureau added a sixth mailing of a postcard with an internet link sent in late July to nonresponding households in Internet First and Internet Choice areas. The Census Bureau sent a seventh mailing of a letter with a paper questionnaire in late August–early September to nonresponding Internet First households in low-response census tracts. In addition, the Census Bureau sent a postcard to 4.5 million U.S. Postal Service (USPS) boxes in areas where there was no USPS carrier delivery service. Finally, the Census Bureau extended the deadline for self-response from July 31, 2020, to October 15, 2020.

Non-ID Response

In 2020, the Census Bureau widely advertised the Non-ID response option via the internet (people could also respond without an ID by telephone), with considerable success. Before accepting a Non-ID response, the Census Bureau had to verify the address. Of Non-ID responses submitted on behalf of fully 17.6 million households, only 3.6% had to be discarded because the address did not match the MAF and could not be verified in the field (see Table C.3). In all, Non-ID responses constituted 9.4% of household enumerations, compared

with 55.6% for internet and telephone ID responses and 11.6% for paper-based responses (see Table 3.4).⁵

Evaluation of Self-Response Strategies and Processes

With the available data, we were unable to analyze the 2020 Census mailing and contact strategies beyond gross effects, such as somewhat earlier elevated levels of paper-questionnaire take-up in Internet Choice areas. We also could not disentangle the effects of messaging and mail strategies (e.g., Internet First and Internet Choice) in total or by geographic areas or population groups, nor how response mode choices were affected by the COVID-19 pandemic.⁶ These are areas in which the Census Bureau could usefully mine its 2020 data to plan informative tests and experiments to determine optimal self-response modes and enumeration operations for the 2030 Census.

6.2 ANALYSIS OF VARIATION IN SELF-RESPONSE

Chapter 3 provided evidence of the overall superior quality of Self-Response—whether measured by population coverage, item and whole-person nonresponse, or age heaping—compared with responses obtained by enumerators. This chapter focuses on variations in self-response among small areas (census tracts, which average about 4,000 people) grouped by characteristics such as income and availability of broadband. Variations have implications for equity for many uses of census data because areas with low self-response will, other things equal, have lesser-quality data than other areas.

6.2.1 Previous Work on Self-Response for Census Tracts

Previous research on self-response for census tracts includes a report of the 2020 Census Quality Indicators Task Force (2020:App. 2), O’Hare (2021), and O’Hare and Lee (2021). These reports all used publicly available self-response rates for which the denominator included not only occupied housing units but also vacant units and addresses that were found not to be residential or not to exist. This denominator can confound comparisons between areas that differ in vacancy rates, for example, but the results of the three aforementioned studies are nonetheless informative.

___________________

⁵ Little information is available on the quality of Non-ID self-responses compared with ID responses. One analysis from the 2020 Post-Enumeration Survey found that Non-ID self-responses were less likely to be correct enumerations and more likely to include whole-person imputations than ID self-responses, whether by internet, telephone, or paper, and than NRFU enumerations with the household head. Non-ID responses were better in these regards than NRFU enumerations with proxies and other household members (see Table 3.4).

⁶ Chapter 10 has limited data on response mode choices by race and ethnicity.

The 2020 Census Quality Indicators Task Force (2020) compared 2010 and 2020 self-response rates for census tracts, counties, and states. Focusing on census tracts, the 2010 and 2020 Census rates were highly correlated, but the 2020 distribution of rates had more tracts at the high and low ends of self-response compared with 2010. The Task Force also provided evidence that census tracts with low internet response ended up with lower overall self-response—in other words, paper and telephone did not compensate in these tracts.

O’Hare (2021) found a high correlation between self-response rates for the nation’s approximately 84,000 census tracts in the 2010 and 2020 Censuses, but also found that 4,000 tracts (about 5%) had an increase in self-response of 10 percentage points or more, and 8,000 tracts (about 10%) had a decrease in self-response of this magnitude between 2010 and 2020. Using ACS 5-year data to identify majority populations in each census tract, O’Hare determined that low-income tracts had the highest percentage with a decrease in response of 10 percentage points or more (23%), followed by Hispanic tracts (20%), low-education tracts (19%), Black Alone or in Combination tracts (15%), and immigrant or renter tracts (14% each).

O’Hare and Lee (2021:10–14) summarized their method of analyzing variations in self-response rates for census tracts. They identified historically undercounted population groups from research on prior censuses and identified census tracts with high concentrations of each group (defined as 50% for some groups and 33% for others). They excluded census tracts in Remote Alaska and Update Leave areas. O’Hare and Lee (2021:5) also found high correlations of 2010 and 2020 self-response rates and relatively low self-response rates in 2020 in census tracts with concentrations of Black people, Hispanic people, American Indian or Alaska Native people, foreign-born people, people with low incomes, people with a high school diploma or less education, people who rent, people who recently moved, and people who live in crowded housing. Our analysis of self-return rates (self-response as a percentage of occupied housing units) has similar results but shows more detail by mode of self-response and by examining the range of concentration of each group across all census tracts divided into ventiles (groups accounting for five percent of the total).

6.2.2 Panel’s Analysis of Self-Return Rates

The Census Bureau provided the panel with self-response data from the 2020 Census operation—specifically with self-return rates for census tracts where the denominator is occupied housing units, not all addresses in the census. The operational data used (e.g., self-response completions by time) are not publicly available. Only operational metrics and ACS public data were used. Data for census tract percent of occupied housing units responding by Self-Response (internet, paper, telephone), and census tract 2016–2020 5-year ACS data from

Demographic Profiles were merged. These data were used to calculate zero-order bivariate correlation coefficients between each variable in the merged dataset. The correlation coefficients between each modality and each ACS variable were graphed from high to low for each modality.

For each ACS variable, the values were used to rank census tracts, and they were then grouped from highest to lowest into up to 20 categories (ventiles). Census tracts with missing data for an ACS variable were dropped. The means of each ACS variable were calculated for each ventile category, as was the mean of the percentage of households enumerated using each self-response mode. The resulting data are then employed to generate line graphs illustrating the variation in modality from one ventile to the next and across all ventiles.

Self-Response Correlations with Census Tract Social and Economic Characteristics

Zero-order bivariate correlations provide some indication of the association between two variables. They do not control for confounding factors. Figures 6.3–6.6 illustrate correlation coefficients between social, economic, and housing characteristics for total Self-Response, internet, paper, and telephone, respectively. Each plot is sorted from highest to lowest coefficient. Total Self-Response is positively associated with higher socioeconomic status (SES) indicators (median rooms, income, owner-occupied units, and non-Hispanic White), and negatively associated with lower SES indicators (percent of renter units, percent of vacant housing units, percent Hispanic, unemployment rate). While percent having broadband was the strongest association with Internet Self-Response, internet response looks similar to the total, implying that Internet Self-Response was driving total Self-Response.

Thus, those census tracts with a high percentage of households with broadband are likely to be represented more accurately in 2020 Census data than those tracts with small percentages of households with broadband. This sort of pattern, in which Internet Self-Response is positively associated with an ACS variable, suggests better-quality data for the populations living in those census tracts with greater prevalence of the characteristic represented by the variable.

Most households completed the census by internet, and paper was the second-most-common modality. Correlations by SES appear to be the opposite for paper compared with those we saw with Internet Self-Response, with lower SES characteristics positively associated with paper response. Given that paper has the highest item nonresponse compared with other self-response modalities, those census tracts with higher percentages of lower SES indicators are more likely to be represented in the 2020 Census with poorer-quality data.

Telephone Self-Response accounted for a small percentage of the total enumeration, yet it was positively associated with indicators of lower SES.

Given lower levels of item nonresponse for telephone compared with paper, telephone appears to slightly offset the quality issue for lower SES indicators and lower quality associated with paper.

Self-Response Ventile Plots

The percentage of enumerations completed by Self-Response mode is illustrated in Figures 6.7–6.12. Plots were generated for a select set of ACS variables illustrating the percent enumerated by each mode of Self-Response for each ventile of the ACS variable. Figure 6.7(a) illustrates household size. Larger and smaller households appear less likely to self-respond compared with those in the 2.5–3 persons-per-household range. Figure 6.7(b) illustrates the percentage of adults with bachelor’s degrees or higher. Those census tracts with a small percentage of people with bachelor’s degrees or higher appear to be much more likely to respond by paper compared with internet. Thus, census tracts with lower educational attainment are more likely to have poorer-quality 2020 Census data than census tracts with higher educational attainment. Figure 6.8(a) illustrates median household income, and the patterns by modality look very similar to those for percent with a bachelor’s degree or higher.

Figures 6.9–6.10 focus on basic race and ethnicity groupings as well as linguistic isolation. We see lower Self-Response and Internet Self-Response for areas where smaller percentages of population consisted of non-Hispanic White people. Patterns for percent African American people, Hispanic people, and people who speak a language other than English or speak English less than very well appear to be similar to each other, with each having lower Self-Response and Internet Self-Response, and higher Paper Self-Response. Areas with high percentages of non-Hispanic White people and areas with low percentages of African-American people, Hispanic people, and people who speak a language other than English appear to be less likely to respond by internet and more likely to respond by paper. We mapped these ventiles and the census tracts appeared mostly in rural areas. Thus, the Census Bureau’s strategies for Update Leave and Internet Choice, and perhaps limited internet access may result in poorer-quality data in these areas.

Figures 6.11–6.12 focus on housing stock characteristics. The plots demonstrate that areas with higher percentages of rental housing, vacant housing, and large housing-complex housing all have lower Self-Response and Internet Self-Response; however, they also have lower Paper Self-Response, resulting in a higher percentage of units in those census tracts being enumerated with NRFU. Areas with housing-related characteristics associated with lower SES are likely to be presented in the 2020 Census with poorer-quality data.

6.3 CONCLUSIONS AND RECOMMENDATIONS

The majority of self-response questionnaires in the 2020 Census were submitted via the internet; paper remained an important response channel, and a small share of returns by telephone complemented the results. Stated as such, this result is so obvious and unsurprising that it is easy to forget that adoption of the internet as the primary response channel was a seismic change for U.S. census taking, that internet response was eschewed entirely in the 2010 Census, and that it took time and concerted effort for the Census Bureau to put a firm stake in the ground for internet data collection in the decennial census. The U.S. 2020 Census experience built from international experience with internet collection in censuses, in particular Canada’s refinement of “wave methodology” for planning its sequence of mailings and other communications to maximize response yield in the quinquennial census. Using its own survey information and modeling, the Census Bureau divided its Self-Response TEAs into Internet First and Internet Choice segments, with still-nonresponding households in the latter being slated to receive a paper questionnaire earlier than those in the former. The 2020 Census mailing strategy was affected by the COVID-19 pandemic adjustments, including last-minute addition of blanket reminder mailings, and the internet response channel weathered the risky, last-second reversion to the home-grown Primus system for data collection. However, it is difficult to imagine that the 2020 Census could have been completed without internet data collection as a foundation, including the provision for Non-ID response without reference to a physical-mail access code.

Proportional distribution of enumeration modes varies geographically and with social, economic, and housing characteristics. Given that some modes

of enumeration result in poorer-quality data, if a larger proportion of a population is enumerated by modalities associated with poorer data quality, it is implied that the data quality for those populations will be poorer. While the Census Bureau systematically imputes missing values, the error associated with imputation implies that data quality is likely to be better for those items collected directly from householders.

Another issue with less-than-accurate data on age, race, and ethnicity is that, for the balance of the decade, census data become the statistical foundation for many programs. In addition to this decennial data foundation varying in accuracy across age, race, ethnicity, and probably sex, introducing differential privacy compounds questions about the accuracy and utility of 2020 Census data for some programs and small-area analyses. Programs that use census data and the demographic-detail data as their data foundation will be less than accurate, and the amount of error present in census data will likely be compounded for programs such as postcensus population estimates and projections. Census demographic-detail data are used by numerous statistical programs as denominators for calculating age, sex, race, and ethnicity-specific rates. Less-than-accurate denominators can potentially minimize or amplify true differences between geographies and subpopulations—an issue that is even greater for small geographic areas or analyses of small subpopulations.

This page intentionally left blank.