– 7 –

Use as Denominators for Rates and Baseline for Estimates

Eddie Hunsinger (California Department of Finance) moderated a session on using census data as denominators for rates and as the baseline for population estimates. The session included four speakers: (1) Nancy Krieger (Harvard T.H. Chan School of Public Health) on public health and health equity concerns; (2) Mandi Yu (National Cancer Institute) on cancer incidence and mortality rates; (3) Alexis Santos-Lozada (Pennsylvania State University) on the effects of noise injection into census data on mortality rates for small geographic areas; and (4) Jeff Hardcastle (State of Nevada) on the effects of noise injection on state population estimates and projections.

7.1 PUBLIC HEALTH AND HEALTH EQUITY QUESTIONS

Nancy Krieger (Harvard T.H. Chan School of Public Health) said she spoke as a population health scientist and a longstanding census data user with a focus on issues of health equity. For a practice she adopted for all presentations, she started by acknowledging that she was on indigenous land and offered respects to the indigenous holders of these lands and also to the knowledge formed by critical indigenous perspectives.

Her presentation addressed four issues, each illustrated by empirical examples using census data for monitoring, analyzing, and improving population health and health inequities: (1) counts, which were critical for denominators for rates; (2) place, referring to census-derived, area-based metrics to characterize regions or areas; (3) time, referring to data needed to analyze temporal trends in

health and health inequities, as well as temporal discontinuities in data over time; and (4) counts for resources and representation, which together were critical societal determinants of health and health inequities. In each case, her question was how census data might be affected by differential privacy.

Krieger noted that raising such issues as a population scientist was nothing new. After all, public health has long had an important input into the conduct of the U.S. census (see Krieger, 2019, which focused on census and public health issues, prompted by concerns about the possible impact of inclusion of a citizenship question). The article discussed public health’s role in a shift in 1850 from census schedules that had one line per household to one line per individual nested within households. Krieger said that this change was impelled by the “notorious” 1840 Census, whose flawed schedule format led to spurious data suggesting that slavery protected black people from insanity, whereas freedom drove them mad.

The article also discussed the long history of major problems affecting census data for indigenous populations and new data linkage efforts that used public health data to help with this problem. Lastly, the article discussed the development of census tracts starting in the 1910 census, which given their use for public health monitoring and interventions were initially called “sanitary areas” until 1930.

The issue of differential privacy offered yet another opportunity for public health to be not only a user but also a shaper of the content of census data. Equally longstanding has been the inerconnectedness of census data and health equity, as illustrated by the pathbreaking study published in 1826 by Louis-Ren Villerm, a prominent French physician and economist. For his work, he used the new and unprecedented Parisian census data of 1826 to calculate for the first time mortality rates in relation to socioeconomic measures, specifically a neighborhood measure of wealth. He used this approach because extant mortality records lacked socioeconomic data, a problem that has continued to affect much of U.S. health data to this day. It should be clear that census data and public health have been vital partners.

7.1.1 Denominators

The calculation of rates and comparisons of the effects of systematic differential error with nondifferential error must have numerators and denominators:

If the denominator was spuriously decreased by differential privacy, the rate would be artificially deflated. If the denominator was spuriously inflated, the rate would be erroneously depressed. These obvious points needed to be stated, especially given concerns raised in prior presentations: differential privacy could

result in systematic bias regarding whose denominators were inflated, leading to underestimates of rates of health outcomes, whether disease, disability, or death, and, conversely whose denominators were deflated, leading to overestimation of these rates.

Krieger noted that prior presentations suggested that denominators would be spuriously increased in smaller and more rural areas on a “spine,” and the opposite for bigger cities or in AIAN areas “o the spine.” As a thought experiment, she encouraged participants to imagine what such systematic error would do to local as well as national understanding of the recent and unprecedented rising death rates among middle-aged U.S. adults due to opioid use and the premature onset of cardiovascular disease. Would we miss the excess elevation in smaller, more rural areas, and underestimate the effect on cities? Or consider the new federal initiative to address the persistent ongoing ravages of HIV/AIDS, infection rates of which are especially elevated in nonurban areas in southern states, particularly young black men. Would their rates be artificially deflated, suggesting the epidemic was less bad than it actually was? She said these questions warrant serious investigation.

7.1.2 Census-Derived Area-Based Metrics

Krieger cited the approach of the Public Health Disparities Geocoding Project,¹ which she initiated in the 1990s to overcome the problem of missing socioeconomic data in most kinds of health records. In one study, Krieger and her colleagues investigated multiple census-derived, area-based socioeconomic measures at both the census tract and block group levels. They came to the conclusion that public health monitoring by state health departments, cancer registries, and hospitals could be validly done using census tract data on the percentage of persons below the poverty line.

As an example of the application of such data, Krieger displayed a census tract map of Washington, DC, showing rates of asthma among adults overlaid on a map showing the percentage of children below poverty, revealing almost identical distributions. How would differential privacy affect these data, which are used to understand who is burdened by disease and where services need to be directed?

Krieger cited another example of the use of census tract data as part of examining health outcomes by their societal determinants: the national databases supported by the Robert Wood Johnson Foundation and the U.S. Centers for Disease Control and Prevention, which included the County Health Rankings, 500 Cities, and City Health Dashboard databases.²

___________________

¹ See https://www.hsph.harvard.edu/thegeocodingproject/.

² These are described at https://www.rwjf.org/en/library/collections/better-data-for-better-health.html. Similar applications include the County Health Rankings, 500 Cities, City Health

Krieger asked a related question of how differential privacy would affect more complex, place-based measures, both within and across geographic levels, such an Index of Concentration at the Extremes (ICE).³ Similarly, how would differential privacy affect an analysis of the effects of “redlining,” which institutionalized racial segregation nationwide, for census tracts in one city compared with another? Because redlining maps cut across census tracts, it would be necessary to apportion the percent of population in a given census tract in relation to these areas using census block data. How would differential privacy affect such analyses?

7.1.3 Time Trends

Krieger provided an example of time-trend analysis of infant death rates stratified by county income quintile from 1960 to 2010, comparing the black population and other populations of color to the white population overall by county and quintile. A companion analysis used 1999–2010 data that allowed greater refinement in categorization of racial and ethnic groups and permitted examination of rate differences for diverse racial and ethnic groups in relation to nonwhite Hispanics. Together, such analyses were important because they showed that current inequities needed to be interpreted in light of the magnitude of health inequities in different historical periods. How would such time-trend analyses be affected by differential privacy?

Krieger cited two illustrations of the analytical problems posed by discontinuities in data and the resources required to address them that involved public health data. One was the 1997 revision of Statistical Policy Directive 15, issued by the U.S. Office of Management and Budget, which changed how racial and ethnic data were to be collected, and among other things, required construction of bridge estimates for population counts to permit temporal comparisons over time. A second example did not alter census data per se but instead how they were used. This concerned the shift to the 2000 standard population (standard million) for age standardization of vital statistics and cancer data, which had until the late 1990s been respectively standardized to the 1970 and 1940 standard millions. This shift required reissuance of older health agency data updated to the new age standard so that trends could be meaningfully assessed over time. It also required warning data users not to compare age-standardized rates over time unless they were sure they were using the same age standards. Would resources

___________________

Dashboard databases, National Equity Atlas, U.S. Small-Area Life Expectancy Estimates Project, and The Opportunity Atlas.

³ In brief, the ICE requires first demarcating the extremes of a chosen distribution, such as privileged versus not in a given area (defined using such variables as poverty and race and ethnicity); then subtracting the number of people in the worst-off extreme from the number in the best-off and then dividing the result by the relevant population total. The index ranges from −1 to 1, with −1 indicating that everyone is deprived and 1 that everyone is privileged.

be available to address how the introduction of differential privacy might affect temporal comparisons?

7.1.4 Population Health and Health Equity

Krieger asked how differential privacy would affect two other critical uses of census data that matter vitally for population health and health equity: allocation of federal funds for health and political representation. According to an analysis by the Census Bureau published in 2017, there were 132 programs that used census data for fund allocation, which in fiscal year 2015 were funded at approximately $675 billion. Among the top 18 programs, each with over $4 billion in funds, fully 13 directly affect health and healthcare. Similar concerns could be raised with regard to within-state allocations of state funds. With regard to political representation, which is in turn linked to political districts, one recent study made novel use of data on gerrymandering to show its relationship with the location of Superfund sites. In the words of the study authors, “minority populations are effectively ‘gerrymandered out’ of the white and lower environmental hazards districts” (Kramar et al., 2018). How would differential privacy affect the resources and representation that together constitute key societal determinates of health?

7.2 RATES OF CANCER INCIDENCE AND MORTALITY

Mandi Yu (National Cancer Institute, NCI) acknowledged her collaborators from NCI, Zaria Tatalovich and Kathleen Cronin. She began by talking about the Surveillance, Epidemiology, and End Result (SEER) program, which NCI established and has supported since 1973.

7.2.1 Surveillance, Epidemiology, and End Result (SEER) Program

The SEER program collects and publishes cancer incidence and survival data from population-based central cancer registries across the nation. The data collected include patient demographics, primary tumor sites, tumor morphology, stage at diagnosis, first course of treatment, and follow-up for vital analysis. SEER is widely used for research and can provide information for policy making and public health surveillance. The registries in SEER cover about 35 percent of the U.S. population. The Centers for Disease Control and Prevention (CDC) has a sister program, the National Program of Cancer Registries, started in 1992, and the National Center for Health Statistics within CDC runs a mortality statistics program that began in the 1930s. Census data have been a very important part of the federal cancer surveillance system since 1980.

The cancer statistics produced and released by SEER include statistics that require population data as denominators—rates of cancer incidence (new cases) and mortality rates. Trends in these rates over time are an important product from SEER. Another important product is cancer survival and prevalence data, as well as data on the risk of developing or dying of cancer. SEER releases new data every year on April 15 down to the county level by age, sex, race, and ethnicity. The denominator data used in the SEER reports comes from the Census Bureau’s Population Estimates program’s July 1 county level estimates.

7.2.2 Features of Population Estimates

SEER needs population data at the county level for several reasons. First, SEER users expect county data. Moreover, some registries constitute data collections from only one county (e.g., Los Angeles County with 10 million people). Consequently, having good data for states but not counties can be a problem if differential privacy methods are applied to the population estimates. SEER users also rely on having a high degree of accuracy for the 19 age groups in the population estimates, particularly over time. Race categories are further complicated by differential compliance with U.S. Office of Management and Budget standards for multiple race reporting. SEER is still working on improving reporting of cancer data from hospitals using multiple race reporting, so trend analyses of cancer rates will continue to be carried out using single race data.

7.2.3 Implications of Differential Privacy

SEER users will be concerned about the effects of differential privacy methods on the time series of population estimates from the Census Bureau, just as they would be with any new methodology that could affect trends. As an example, the 2010 Census implemented improved editing procedure for race and ethnicity, which resulted in a larger number of Hispanics who checked “some other race” being assigned to AIAN using internal data. SEER looked at the effects of these changes on the time series of population estimates from 2000 to 2010 and found discrepancies. The Census Bureau was willing to reedit the 2000 data to be comparable with 2010, for which SEER was very appreciative.

Yu concluded by saying that it is important to publish uncertainty measures for the 2020 Census data. Uncertainty measures, even in a gross form similar to measures of sampling error, could be very useful to inform the impact of differential privacy on the specific statistics of interest. The NCI has previous experiences of developing statistical methods to incorporate sampling errors associated with population denominators in reporting uncertainties of cancer rates. Publication of uncertainty measures by the Census Bureau would valuable opportunities for the user community to contribute to ways of alleviate

the impact. She also suggested that the Census Bureau should think about helping users understand the implications of differential privacy methods for comparability between 2010 and 2020 Census data.

7.3 IMPACT ON CRITICAL RATE CALCULATIONS, PARTICULARLY FOR SMALL AREAS AND DEMOGRAPHIC COMMUNITIES

Alexis Santos-Lozada (Pennsylvania State University) introduced himself as a population scientist, a faculty member at Penn State, and a member of the Pennsylvania Population Network. Penn State launched a new initiative to connect faculty members with local governments, and most of the requests have been to produce analyses of census data.

7.3.1 Effects of Differential Privacy on Mortality Rates

Santos-Lozada compared the original 2010 dataset with the 2010 Demonstration Data Products (DDP), using each data set as the denominators for death rates at the county level. He obtained the county-level death counts from the CDC WONDER online data system for disseminating public health data.

Differences in county total population ranged from a minimum of 853 people to a maximum of 4,259 people. More importantly, the ranges differed among racial and ethnic groups. The biggest differences were for Hispanics, with a range of negative 950 people to 2,966 people at the county level. In percentage terms, there were large changes for Hispanics in small counties, such as Kalawao County, Hawaii; Rock County, Nebraska; and Garfield County, New Hampshire.

Looking at mortality rates, the two data sets were close for the total population and for non-Hispanic whites, but there was more spread for non-Hispanic blacks and Hispanics. Santos-Lozada noted that race detail was not available for all counties from CDC WONDER.

Translating the data into ratios of the mortality rates in the two data sets, a mortality rate ratio above 100 meant that the original 2010 data had a higher rate than the DDP data, while a rate below 100 meant that the original data had a lower rate than the DDP data. For the total population, there was little variation by county in the ratios and similarly for non-Hispanic whites (they were all close to 100). For racial and ethnic minorities, though, there was much greater variation. Moreover, for the smaller counties for which mortality data were not available for racial and ethnic groups, the variation was likely to be larger still. These kinds of differences could affect understanding of health conditions and health disparities for minority populations in the United States.

Santos-Lozada said that the governments in the Pennsylvania Population Network that request data analyses were small areas that did not have the

capability to do such analyses themselves. The application of differential privacy methods to the 2020 Census data could make it much harder for Santos-Lozada to help these communities.

Santos-Lozada wondered what the implications of differential privacy would be for identifying pockets of growth for minority populations across the United States. In the case of Centre County, Pennsylvania, it did not have many Hispanics 20 year ago, but now it has a fair number. There is a Colombian restaurant and a Venezuelan restaurant, and there are physicians moving in to take care of the Hispanic population. These people probably would not have been detected if the application of differential privacy methods had reduced small concentrations of Hispanics or other racial and ethnic groups in small areas.

7.3.2 Socioeconomic Analysis

Santos-Lozada used five-year American Community Survey (ACS) data on socioeconomic characteristics for counties to see if there were any significant associations with mortality rate ratios under or over 100. The results indicated that counties with higher proportions of non-Hispanic blacks, Hispanics, and people living below the poverty threshold were more likely to have mortality rate ratios over 100. Conversely, areas with higher proportions of people under age 18 or aged 65 and older, higher unemployment, lower outmigration, and being located in the South were more likely to have mortality rate ratios less than 100. All these groups are critical for understanding and reducing public health problems and health disparities.

7.3.3 Concluding Comments

Santos-Lozada said his bottom line was that the application of differential privacy methods in the 2010 DDP injected patterns of change in the denominators for mortality rates. These changes would likely make the work of population health scientists more difficult down the road. The largest variations, when compared with the original 2010 data, were for small areas and racial and ethnic minorities. Inaccurate denominators for mortality rates would reduce our understanding of the demographic profile of certain areas of the nation and of health disparities in disadvantaged areas. Santos-Lozada argued for the need to control the population counts for smaller geographies for racial and ethnic groups.

7.4 HOUSING AND POPULATION COUNTS: IMPLICATIONS FOR LOCAL ESTIMATES AND PROJECTIONS

Jeff Hardcastle (State of Nevada) said he would discuss state-produced population estimates, using Nevada as an example. His presentation covered the impact of the application of differential privacy methods to census data on the housing-unit method for developing population estimates, the numbers that states and localities use for policymaking and public safety, as well as questions and observations about differential privacy.

7.4.1 State-Produced Population Estimates

Nineteen states produce their own population estimates, and of those, about 10 produce estimates by single years of age, race, sex, and Hispanic origin (based on the most recent survey by the Federal-State Cooperative for Population Estimates). Any changes in data availability and the cell counts would likely impact the work that states do across the country.

Nevada’s program of population estimates and projections began in 1987 and formed the basis for distributing $79 million in revenue in fiscal year 2019 among 82 governmental units. As two other examples, Washington State distributed $200 million and Florida $449 million on the basis of population estimates. State-produced population estimates are tied to several key variables from the census, as well as some local data. The census supplied housing unit counts to compare to local assessor counts, plus occupancy rates and persons per household. Hardcastle then used the Census Bureau’s population estimates in a regression model to average with the housing unit-based estimates to hopefully produce a more accurate estimate.

Both data sets have error. The census data have geocoding errors, imputation errors, and swapping errors. Local governments have issues with staffing levels, skill sets, and definitional issues. Hardcastle said some assessors in Nevada provide inconsistent housing counts from year to year.

Looking at the impacts of differential privacy across the country, there seemed to be a cut point at about 50,000 population where the occupancy rate went up for counties below that population threshold and down for those over 50,000. Hardcastle found substantial differences for three Nevada counties after comparing occupancy rates in the original 2010 data and the DDP. It has always been hard to reconcile data from local county assessors with census data, and the tension would just be increased by the introduction of differential privacy.

In terms of state code and statutes, Nevada would now have to review state law to account for margins of error in the census data and thus the population estimates. The Nevada legislature meets only biennially and would be starting from scratch to determine how to handle variations in the revenue allocation formula. The process was likely to be quite contentious. As it is, Hardcastle had

been involved in an administrative hearing (or court case) over a 200-housing-unit discrepancy between local data and census data.

Moreover, it would be hard to make changes in state law and code without a very good story or vocabulary that was readily explainable to elected officials. Hardcastle could not see himself trying to sit down with the city manager from Elko to tell him that his population estimates and revenue were impacted by a privacy-loss budget ϵ. Local policy makers would not want to deal with inference but rather with their perceptions and the impact on their constituents. Data that did not accord with local knowledge would generate questions and debates.

7.4.2 Households with Different Service Needs

Hardcastle showed differences in distributions across census tracts between the 2010 Summary File 1 (SF1) data and the DDP for households containing married families with children, for which there was quite a variation. A similar analysis for seniors living alone showed much more dramatic differences. These kinds of differences would make it a much harder lift for targeting public services.

7.4.3 Concluding Comments and Questions

Hardcastle wondered how differential privacy interacted with census residence rules. With regard to trade-offs, such as between more detailed geography and more detailed characteristics, Hardcastle agreed with Ken Hodges’ earlier suggestion to reduce the number of race categories. Also, there were some superfluous census-designated places in Nevada because they had no real ties to local unincorporated towns in the county.

Hardcastle offered an admittedly off-the-wall suggestion from his understanding that the threat to privacy stemmed from digital scraping, matching, and the like. Perhaps some rarely used tables could be available as paper products. There could still be a disclosure avoidance issue, but it would take quite a bit of work to take the data out of the paper, enter them into a spreadsheet, and work with them. He offered this suggestion in the spirit of looking for ideas to reduce the burden on the privacy budget and increase the availability of data that so many people need at reasonable geographies that were vital for public health, safety, welfare, and policy decisions. Hardcastle said that there was a need to think outside the box.

7.5 FLOOR DISCUSSION

Helen Nissenbaum (Cornell Tech) said that she understood that social scientists, statisticians, demographers, and public health officials who used

statistical methods were trained in dealing with bad data. All population data had factors that affected their so-called accuracy: nonresponse, lies, misunderstandings, people moving, and so on. If one really wanted an out-of-the-box solution that would ensure greater accuracy, government agents could come to the door with weapons and watch people fill out the forms. Instead, though, we use statistical methods to help us get useful data without compromising our fundamental values. Drawing an analogy to differential privacy, could not statisticians take noisy data and extract accuracy?

Krieger replied that “fundamental values” included not only privacy but also equity. If differentially private methods systematically distorted the data, then that would be a concern at the level of values. She further pointed out that there are different kinds of public health data such as vital statistics, which are fairly complete. While there are classification issues with how mortality data are coded, the data themselves do not have huge amounts of error. The same is true with cancer registries, which are not based on surveys. To say that data are always messy is not quite accurate. A key concern for vital statistics is ensuring the consistency of the denominators so that temporal trends will be valid, which would be a concern with differentially private methods as applied to census data. Systematic changes in denominators across the country would matter for monitoring trends, which in turn would matter for population health programs. Moreover, there is a question of resources available to understand the effect, which is not something that each and every place should have to figure out on its own. Yu added that, yes, public health researchers do deal with a lot of noise but it appears to her that the noise introduced into denominators by differentially private methods has obscured the true signal. Also, Yu said, there needs to be a better way to communicate differential privacy and its effects. In reply, Santos-Lozada asked who was concerned about privacy and who was benefiting from the availability of useful data for making good decisions. Hardcastle agreed that all data have errors, but he tends to view census data through the lens of his experience working in retail. The census to him is like an inventory, a once in 10 years hard count of the population. Retailers, in comparing inventory with purchase orders, receipts, and the like, might find out that the numbers do not match. This finding would help them figure out what might be going on, such as shoplifting or overpaying for goods and services. More generally, one could make a bad decision with good data, but it would be much harder to make a good decision with bad data. Hardcastle said that the key in putting out noisy data is to be open and honest about where the holes are, which to date had been a problem with differential privacy for census data.

Abraham Flaxman (University of Washington) wondered what would be an acceptable level of variation for epidemiological rates. Santos replied that the ranges he included in his presentation were not driven by any rule but were put in because they captured the bulk of the data. His concern was with outliers and with not having a sizable change in mortality rates and other indicators that

used census denominators for racial and ethnic minorities. Krieger added that one naturally has to deal with issues of uncertainty, but if one added imprecision in systematic ways, then it would be hard to understand the actual population health situation. People would, regardless of how they were counted, go on getting sick and dying, and not knowing accurately where those burdens were distributed would be really problematic.