Tasked, as the workshop was, to focus on nonfederal uses of American Community Survey (ACS) data, many of the speakers in several of the workshop sessions touched on issues involved in using ACS data to inform state and local government. Users at the state, local, and tribal levels—whether working directly for governmental units and affiliated agencies or participating in the Census Bureau’s State Data Center network to disseminate data broadly—are a sufficiently large and important sector of the ACS user base to warrant a dedicated workshop session. Included in this mix are users who focus on the general issues of rural policy, an area of study for which the ACS presents special benefits and burdens: While those interested in urban policy and large populated areas face the “problem” of handling a wealth of ACS estimates of different vintages (1-, 3-, and 5-year estimates all covering large population groups), workers in rural policy must rely exclusively on the 5-year estimates.
Section 5–A summarizes work using the ACS to examine ancestry, migration, education, and—particularly—racial and ethnic diversity for the Minnesota state government. The contrast in rural and urban uses of the data is spotlighted in Sections 5–B and 5–C, the first of which describes work by the Rural Policy Research Institute and the second (complementing the applications by The New York Times discussed in Section 4–C) illustrating the capability to map ACS findings for small-area neighborhoods in New York City and State. Section 5–D discusses the usability of ACS data to describe the unique geographic and demographic characteristics of individual chapters within the Navajo Nation. The
workshop steering committee asked Jacqueline Byers of the National Association of Counties to serve as leader for the discussion of these presentations; her own comments from the perspective of county government users, and the general discussion, are summarized in Section 5–E.
On a continuous, daily basis, the Minnesota State Demographic Center (SDC) fields questions and works on projects that depend critically on the ACS. These include questions from other state agencies,1 nonprofit organizations, media outlets, or the general public—whether through the center’s dedicated email “helpline” or other means. Not wanting to minimize those smaller, frequent requests for information, Susan Brower (Minnesota state demographer) began her comments by saying that she would focus on a few larger, longer-term projects to suggest the substantive range of the center’s work with the ACS to inform state policy.
She started by describing recent work making use of the ACS’s ability to zero in on trends in very specific population groups. Over the past four decades, Minnesota has become home to increasing numbers of Asian and Pacific Islanders, in large part reflecting arrivals of refugees fleeing repressive governments in southeast Asia. As Brower described, the mix of particular Asian Pacific ethnic groups in Minnesota is considerably different from that of the entire U.S. population, based on her analysis of 2008–2010 ACS data (via the Integrated Public Use Microdata Series [IPUMS]). In the general U.S. population, the proportions of southeast Asian groups (e.g., Cambodian, Hmong, Vietnamese) and east Asian groups (e.g., Chinese, Japanese, Korean) are roughly on par with each other, with east Asian groups holding a slight plurality and a somewhat smaller percentage of south Asian groups (e.g., Asian Indian, Tibetan, Bangladeshi). By comparison, southeast Asian groups make up a clear majority of Minnesota’s Asian population, with a markedly smaller share of east Asian groups and slightly fewer from south Asian groups. In specific, the largest single Asian Pacific ethnic group in Minnesota is the Hmong population; the state’s approximately 3,000 Tibetans make Minnesota rank second among U.S. jurisdictions in Tibetan-Americans; and the state’s concentration of Karen (a Burmese–Thai minority group) is believed to be the largest outside of southeast Asia (Council on Asian Pacific Minnesotans, 2012:i, 4).
In spring 2012, the Minnesota SDC collaborated with the state’s Council on Asian Pacific Minnesotans—a state agency tasked to report to the governor and legislature on issues important to Minnesota’s Asian community—on a broad
1The Minnesota State Demographic Center is administratively housed in the state Department of Administration.
report on the demographic and economic condition of Minnesota’s Asian Pacific ethnic groups. The work resulted in an April 2012 report, State of the Asian Pacific Minnesotans (Council on Asian Pacific Minnesotans, 2012), that made extensive use of ACS data as well as counts from the 2010 census. Brower said that her center’s role in this report was to perform the analysis and (examining the standard errors on the estimates) demonstrate to the Council that the results were reliable and stable.
For the report, Brower chose to perform special tabulations from the IPUMS 3-year microdata (2008–2010) from the ACS, profiling the nativity, ancestry, and socioeconomic and housing conditions of the state’s major Asian subpopulations. She said that she found it necessary to build the special tabulations, rather than use the detailed tables (which she made certain to describe as “easily accessible” from the Census Bureau’s American FactFinder interface), to provide reassurance that the analysis referenced all of the Asian population. In particular, she said that she and the Council wanted to respect the very different cultural and historical backgrounds of the different Asian Pacific subpopulations and, therefore, carefully address each subpopulation separately. She said that the American FactFinder tables tend to describe results for the Asian population overall, and she and the Council needed finer resolution than that for this project. To illustrate the point, Brower noted that the two largest segments of Minnesota’s Asian population—Hmong (26 percent) and Indian (15 percent)—are “at very different ends of the socioeconomic spectrum” and in background in coming to Minnesota. The major influx of the Asian Indian population occurred recently (over the past 10 years or so) and are more likely to be fluent in English.
Analysis of the ACS data suggests great diversity by specific ethnicity group in the broader Asian-ancestry population in Minnesota. Looking at “all Asian” ancestry people in Minnesota in the 2008–2010 ACS data, 21 percent hold a bachelor’s degree or higher. But that figure masks very different educational attainment by specific groups; the majority Hmong population includes only 14 percent with a bachelor’s degree or higher; Asian Indians are second largest in size in Minnesota’s population but tend to be better educated, leading other Asian subpopulations at 82 percent with a bachelor’s degree or higher. The state’s Chinese and Korean subpopulations count over 50 percent with at least an undergraduate degree; only 9 percent of the state’s Cambodians and 6 percent of Laotians have a degree. Additional interesting results were generated by looking at nativity, comparing the ACS data with the 1990 and 2000 census long-form samples. Brower found that the percent of native-born Asian Minnesotans with at least a bachelor’s degree has steadily decreased over the decades—to the extent that, as of the 2008–2010 ACS data, foreign-born Asians in Minnesota were more likely to have an undergraduate degree than native-born Asians.
Brower briefly described two other recent projects in which the Minnesota SDC used ACS data to inform specific policy uses. In April 2012, the Min-
nesota SDC performed analysis of 5-year (2006–2010) ACS PUMS data at the request of the Minnesota Department of Human Rights. Specifically, the department wanted to focus on equal employment opportunity (EEO) compliance for a very precise economic sector—the construction industry—to see if state-mandated hiring goals were being met on construction projects funded by the state. Recalling the earlier question from Census Bureau staff on the practical import of a few weeks or months in the timing of ACS data releases (Section 2–F), Brower noted that the timing does matter—and that this EEO work was one example where the ACS’s current release schedule gives it a unique advantage over other products. She said that the Census Bureau plans to release special tabulations for EEO compliance at the end of calendar year 2012, and that these special tabulations will facilitate much more detailed analysis than SDC’s PUMS work. However, waiting until the end of 2012 was not a viable option—the state human rights commissioner faced a deadline to get the data on hiring goals during the state legislative session because 2012 is a revenue bonding year. Major capital projects were in line for the session, and the state wanted to make sure that hiring goals were in line with expectations. SDC plans to repeat the analysis when the special tabulations become available later this year, to permit the human rights department to update its goals.
Just as ACS data are used at the federal level to allocate federal funds, so too are they used by state governments to distribute funds to localities. Brower recounted that, in May 2012, SDC helped the Minnesota Department of Employment and Economic Development score applications under the Small Cities Development Grants Program. These grants—made from money allocated by the U.S. Department of Housing and Urban Development—are intended to help small, rural communities fund infrastructure and housing improvements. The grant monies are distributed according to a formula based on the percentage of persons in poverty at the city place level; SDC compiled data from the 5-year 2006–2010 ACS data to use in the formula.
Brower said that SDC is currently working on a longer-term planning project, expected to culminate in a themed version of the center’s annual report to the governor and legislature, tentatively titled The Time for Talent. The basic goal of the report is to make use of the ACS’s detail on educational attainment and occupation to document the levels of talented workers in the state, and to use its data on place of birth to get a read on the movement of talented workers in and out of the state. For this work, Brower said that the SDC took the slightly unusual step of looking at four individual-year ACS PUMS files; cognizant of the economic recession, they were concerned that some interesting trends might be masked by looking at 3-year or 5-year data as a whole. She said that the numbers that they obtained from the 1-year files were seemingly too small to be stable, but they determined that the average of those single years seemed to perform better. She emphasized that the work was still under way at the time of the workshop, and that additional analysis would attempt to use
a Person Question 15a on the 2000 census long-form questionnaire asked “Did this person live in this house or apartment 5 years ago (on April 1, 1995)?” and included “No, outside the United States” as a response. Question 15b asked “Where did this person live 5 years ago?” and permitted specification of a city, county, state, and ZIP Code.
b Person Question 15a on the 2012 ACS questionnaire (with similar versions on earlier questionnaires) asks “Did this person live in this house or apartment 1 year ago?” and includes “No, outside the United States and Puerto Rico” as a response. Question 15b asks “Where did this person live 1 year ago?” and permits specification of an address, city, county, state, and ZIP Code.
SOURCES: Tabulation from 2000 Census data and Integrated Public Use Microdata Series, 2007–2010; adapted from workshop presentation by Susan Brower.
Some illustrative high-level results from the work are shown in Tables 5-1 and 5-2. Table 5-1 suggests that Minnesota continues to draw large numbers of workers with a bachelor’s degree or higher from the neighboring state of Wisconsin—and nearly as many from outside the United States. Taking a longer-term, life-course look by examining educational attainment and state of birth, Table 5-2 suggests that Minnesota ranks fairly high in retaining degreed workers who were born in the state; only the country-sized economies of Texas and California, as well as North Carolina, have higher “staying power.”
SOURCES: Tabulation from single-year Integrated Public Use Microdata Series, 2007–2010; adapted from workshop presentation by Susan Brower.
She concluded by reiterating that Minnesota’s state government makes heavy use of ACS data for three principal purposes: developing population profiles of Minnesota-specific groups, conducting research to support development of state policy and distribution of funds, and crafting special reports to help with long-term planning. She also commented that, shortly before the workshop, the SDC hosted a new forum for boosting public awareness of ACS data: its first “Minnesota Data Opener” (a takeoff on the opening of fishing season, which began at roughly the same time), in partnership with Minnesota Public Radio. The data opener took the form of an online chat. SDC published a “workbook” of data on education (derived principally from the ACS) in advance, and invited the public to look at the data themselves and comment on what they found most interesting. In particular, the “data opener” included a contest, asking people to produce some kind of visualization from the data. They were very encouraged by the results—it was an “engagement project using data,” and the results were numerous, interesting ways of presenting data for the SDC to consider moving forward.
Kathleen Miller—program director of the Rural Policy Research Institute (RUPRI) at the University of Missouri—described RUPRI’s recently completed Geography of Need project, an effort to compare the needs for various human
services in rural America compared to urban areas. RUPRI undertook the Geography of Need project in collaboration with Colleen Heflin (also from the University of Missouri) and with funding from the Kresge Foundation. The work grew out of Kresge’s interest in recent years in redesigning their human services portfolio. Particularly in light of the economic recession and funding pressures created by federal and state budget deficits, Kresge tasked RUPRI to develop a national overview of how needs for services vary by geography and urbanization, and what types of needs for services occur in different combinations in different places. That regions of the country—many of them poor—are characterized by persistent poverty is well known, but the way in which needs for services vary within those areas is much less clear.
As RUPRI set about conceptualizing and planning this study, Miller said they did so by following three basic rationales:
- Human service needs depend on the characteristics of the population in need: For instance, military veterans have unique needs to services, and also have existing access to an array of services not open to other populations (e.g., U.S. Department of Veterans Affairs health centers). Needs for human services can also vary across the life course—the elderly have very different needs than do young children—and in combination with geography or other factors (e.g., a town whose composition changes due to either a large outmigration of young people or a rapid inmigration of retirees relative to one with very young populations).
- Specific local conditions affect the demand for and provision of human services: These local conditions could include cultural differences or language barriers (linguistic isolation). In rural settings, transportation barriers are of particular concern. As Miller put it, “all the services in the world are not going to help if you can’t get to them”; if someone lives in a rural area an hour or more away from any service—“everything from a gallon of milk to your pediatrician”—and their car breaks down, they are stuck.
- Human service needs are also a function of the economic structure of an area: Again, as mentioned above, some rural areas are characterized by persistent poverty and limited employment opportunities, and that will affect the range of needed services.
Starting work, Miller and RUPRI faced the task of deciding what level of geographic resolution to pursue in mapping the landscape of need. State-level analysis was simple to rule out as being too coarse, but going down to finer levels of geography like Public Use Microdata Areas (PUMAs) or tracts raised issues of “poor data and smaller sample size.” The basic task to construct a national-level portrait put additional constraints on the problem: The researchers had to think in terms of a set of indicators that could be calculated for all units at the chosen resolution (and not just cities in metropolitan areas), and the idea of
Ultimately, RUPRI settled on counties as the appropriate unit—a choice with certain undeniable benefits. The number of county-level equivalents in the United States (3,141) is large but not overwhelming, and their boundaries are generally well understood and stable over time. More pertinent, counties are also the natural operational geography for many human services providers (e.g., county health and welfare agencies), so counties seemed to mesh well with the project’s goals. To be sure, though, counties also raised some disadvantages because they range widely in both population and land area, their size along either dimension can obscure important urban/rural distinctions within county borders, and whole counties might be less likely to stand out with respect to some needs/risk criteria than a purely rural community.
Cinching the choice of county as the level of resolution was what Miller called a “fortunate” bit of timing—active work on the Geography of Need study began around December 2010, coinciding with the release of the first 5-year ACS estimates. The release of the 2005–2009 5-year ACS estimates was the first exposure to ACS data and first update of socioeconomic data since the 2000 census for about 40 percent of counties—and, for purposes of rural data analysis, “this was early Christmas.”
The approach taken by Miller and RUPRI was to construct a data set of 12 demographic and economic indicators for all counties. While most of these indicators could be calculated directly from the 5-year ACS data, RUPRI favored data resources from the U.S. Bureau of Economic Analysis and the U.S. Department of Veterans Affairs over the self-report data in the ACS, on income from government transfer payments and veterans status, respectively. RUPRI also turned to the Census Bureau’s annual population estimates program to derive the population denominators used in its calculations and to its annual model-based Small Area Income and Poverty Estimates for the county poverty rate. RUPRI’s final set of county-level indicators included:
- Percent of the population age 65 and over;
- Working-age dependency ratio;2
- Fertility rate;
- Foreign-born population;
- Percent of population in racial or ethnic minority groups;
- Percent of population living in subfamily arrangements;3
2Segmenting the population into “working-age” (e.g., from ages 15–64) and “dependents” (the complement, e.g., children under age 15 and seniors over age 64), the working age dependency ratio is simply the ratio of dependents to working-age.
3Subfamilies are families that live in the household of some other person—for instance, an unmarried mother living with her children in the house of her parents or a married couple (with or
- Percent of population with less than a high school education;
- Veterans as a percent of total population;
- Percent of households receiving Supplemental Nutrition Assistance Program (SNAP/Food Stamp) benefits;4
- (County) poverty rate;
- Percent of households without a vehicle available; and
- Share of personal income from government transfer payments.
Having constructed a set of indicators, RUPRI derived a very simple index measure: Each indicator was separately sorted and ranked, and a county was assigned a 1-point risk factor if it fell in the bottom 10 percent (and 0 otherwise). Summing the scores across counties produced a Human Services Need Index score between 0 and 12; these scores are mapped in Figure 5-1. Miller said that the red-shaded areas (those with more than 3 or even up to 9) on the map highlight some areas where RUPRI expected to see hardship high levels of human services need: Appalachia, the “Black Belt” of the South, the Mississippi Delta, the Texas–Mexico border region, the California Central Valley, and American Indian lands in the Great Plains and the Four Corners. What also struck RUPRI staff was the vast swath of the country with 0 risk factors and the remote nature of many of the highest-scoring counties, suggesting the possibility of significant transportation barriers and difficulty in service delivery.
RUPRI’s full analysis of these data tries to probe what is behind the red-shaded areas on the map—what combinations of the 12 indicators/risk factors seem to show up with greatest frequency, and where. Miller said that it was not surprising to see one frequent combination—low educational attainment and high poverty. But, for some parts of the country, a less-expected combination (high elderly population and veterans’ status) proved most prevalent.
Table 5-3 breaks down the index scores by whether the county is or is not part of a metropolitan area. Strikingly, over half of metropolitan counties exhibit 0 risk factors, compared to only about one-third among nonmetropolitan counties. On the high end of the scale—between 3 and 9 risk factors—the difference is even more stark, covering only about 9 percent of metropolitan counties but just over one-quarter of nonmetropolitan counties. Subsequent analysis suggested that noncore counties (those containing no urban cluster with at least 10,000 residents) are “very overrepresented” among the nonmetropolitan counties with high risk factors.
without children) living with some other relative. The concept is described in greater detail on the IPUMS project website at http://usa.ipums.org/usa/volii/subfamilies.shtml.
4Housing Question 12 on the 2012 version of the ACS questionnaire asks “IN THE PAST 12 MONTHS, did anyone in this household receive Food Stamps or a Food Stamp benefit card? Include government benefits from the Supplemental Nutrition Assistance (SNAP). Do NOT include WIC or the National School Lunch Program.” Only a yes/no checkbox response is permitted.
SOURCES: Calculated using 2009 Population Estimates, 2005–2009 American Community Survey, and 2009 SAIPE estimates (U.S. Census Bureau) and data from the U.S. Department of Veterans Affairs and the U.S. Bureau of Economic Analysis Regional Economic Information System. Analysis and mapping by Rural Policy Research Institute. Adapted from workshop presentation by Kathleen Miller
|Number of Risk Factors||Metropolitan Area Counties||Non-Metropolitan Area Counties|
|0||585 (53.2%)||690 (33.8%)|
|1||267 (24.3%)||464 (22.7%)|
|2||149 (13.5%)||346 (16.9%)|
|3–5||91 (8.3%)||459 (22.5%)|
|6–9||8 (0.7%)||84 (4.1%)|
SOURCE: Calculations by Rural Policy Research Institute, University of Missouri; adapted from workshop presentation by Kathleen Miller.
Wrapping up, Miller commented that the ACS is hardly perfect for studying and understanding rural America. Revisiting the point that a large fraction of U.S. counties fall below the 20,000 population cutoff—and so have to wait for 5-year estimates from the ACS—she conceded that there are grounds to be concerned about the sample size for rural counties, about large margins of error, and less timely estimates (only two nonoverlapping 5-year periods per decade for the purest indication of change over time). But—in short—the ACS “is the best data and often the only data we have” for rural analysis. RUPRI’s particular analysis of human services risk factors would not have been possible without the 5-year estimates; the work has helped RUPRI develop hypotheses for further research and helped the sponsoring Kresge Foundation retool its portfolio of grants dealing with human services.
Shifting from ACS analysis of intensely rural areas to the survey’s use in explaining trends in the nation’s largest city, Steven Romalewski described recent work in the mapping of ACS estimates and—in particular—first steps in reflecting margins of error in the plots. Romalewski directs the Mapping Service in the City University of New York’s (CUNY) Center for Urban Research, which uses geographic information systems (GIS) software to design a variety of online applications using Census Bureau data. Of particular note, and in support of the 2010 census, Romalewski helped develop an easy-to-use mapping application for the Census Bureau’s tract-level planning database—a tabulation of 2000 census long-form samples used to calculate an index score (not unlike Miller’s additive Human Services Risk Index in the previous section) that indicated the “hard-to-count” nature of each tract. Other applications developed by the CUNY Mapping Service made use of the long-form-sample data from the
Like Miller’s RUPRI, Romalewski’s Center for Urban Research and Mapping Service is housed in an academic setting, but works with and for a variety of constituencies with very practical data needs: local organizations, state and city agencies, and other general researchers. Part of the problem the Mapping Service faces in moving more fully into ACS-based products is meeting the expectations that have developed in their partners’ (and clients’) minds. The Mapping Service has become known for online interactive mapping that allows users to zoom in very closely; their clients really need, and expect to be able to, drill down to fine-grained tract or block group data.
Accordingly, by way of providing initial impressions of the ACS data, Romalewski offered two notes of empathy with Miller’s concerns over ACS use for data analysis. First, Romalewski said that he and his colleagues try to adhere to the counsel of the Census Bureau and the National Research Council (2007b) to refrain from comparing estimates for overlapping time intervals; given that the fine levels of geography of greatest pertinence to CUNY’s users are only available in the 5-year products, Romalewski also feels the frustration of twice-a-decade comparisons. (Like Miller, he was quick to add that twice-a-decade is far preferable to the alternative.) And, second, Romalewski shared the concern that small counts in individual city tracts and block groups can produce “unacceptably high margins of error.”
With those comments as prelude, Romalewski said that—with margins of error now much more prominent in the ACS estimates—he and his colleagues are grappling with two fundamental questions as they update their applications and try to most effectively (yet accurately) present ACS small-area data. Those questions are:
- Is geographic aggregation necessary? That is, can we have confidence in visualizing spatial patterns by plotting tract (or even block group) data, or is selective aggregation of some small areas necessary to get acceptable precision?
- Should (or must) “high margin-of-error” areas be denoted or visually flagged in some way, as through cross-hatching or other means? Put another way, is it possible to create robust choropleth maps even with uncertain data?
He hastened to add that “I don’t have any conclusions” as yet; working through these issues is an ongoing exercise. In that spirit, he spent the remainder of his presentation briefly displaying a mix of applications and first-cut approaches: applications where thorny presentation issues remain to be made in converting from long-form sources to the ACS and initial attempts to flag potential problem areas in static maps.
As examples of his center’s ongoing mapping projects, he began by showing screenshots from the Long Island Index project, an interactive map server
located at http://longislandindexmaps.com. At that site, interested users can generate overlays from a variety of sources, including data from county planning authorities and the New York and U.S. Departments of Education. The application currently draws population and housing data from the 1990, 2000, and 2010 census summary files, and generates maps down to the block group level. The specific screenshot he chose to display zoomed in on central Long Island (Suffolk County) and displayed the percentage of occupied housing units with no vehicles available. He noted that a number of organizations on Long Island are keenly interested in this specific data item in order to understand access to food sources and for planning transportation routes. Romalewski would return to this specific view—households without cars in central Long Island—to illustrate the Mapping Center’s concerns about presenting ACS data. He then briefly displayed a screenshot of a block-level map of the New York metropolitan area map, with each block shaded according to the basic race category making up the plurality of the block’s total population. This map server, accessible at www.urbanresearchmaps.org/comparinator/pluralitymap.htm, permits users to look at data for 14 other large metropolitan areas in addition to New York. The specific view Romalewski chose included a vertical slide bar that the user can manipulate; 2000 census data appear to the left of the line and 2010 data to the right.5 Similar views could be generated from ACS data, albeit not as fine as the block-level counts.
Maps are an especially powerful way of making data interpretable to the broader public, and Romalewski said that this power (and responsibility) motivates the Mapping Center’s concern in trying to accurately convey levels of uncertainty in mapped data. He mentioned that he teaches an introductory GIS course for graduate students in the Pratt Institute’s city and regional planning program and is struck that even in that setting of fairly sophisticated users, it is common to hear comments along the lines of “The map said this.” When data are presented in map form, people more readily assume that it is the truth—even though the margins of error underlying the tints of areas on the map might undercut that level of certainty. Romalewski said that he and his colleagues are working with and learning from numerous sources as they explore different cartographic techniques for conveying uncertainty, including David Wong and colleagues at George Mason University (work on software extensions for the ArcGIS package)6 and the Cornell University Program on Applied Demographics. More recently, an informal working group of interested parties formed in New York City—including the CUNY Mapping Center, the city’s Department
5Similar formatting is used in an online application portraying legislative districts in New York state, with block-level counts and district boundaries overlaid on each other—pre-2012-redistricting on the left and post-redistricting on the right. This project is located at www.urbanresearchmaps.org/nyredistricting/map.html.
6Wong contributed a short methodological overview of his work to overlay coefficients of variation on ACS-based maps to the workshop’s case study/agenda book.
of City Planning, and Ford Fessenden and colleagues at The New York Times (see Section 4–C).
Again, Romalewski emphasized, the problem is very much open and he has not yet reached any conclusions about preferred approaches; he moved swiftly through a series of approaches, with their attendant pros and cons. One approach is to visually mask areas with high levels of uncertainty:
- For a map of Asians and Pacific Islanders as a percentage of total population at the census tract level within New York City—2000 census data in one pane and 5-year 2005–2009 ACS estimates in the other—Romalewski sought to flag “tracts with unreliable estimates.” Different percentages of the Asian population were portrayed in shades of green (or a light yellow tint for 0–10 percent Asian population) but—for the 5-year ACS pane—tracts with a coefficient of variation of 25 percent or more were shaded grey. Major trends register strongly in the maps—the extreme growth of the Asian population in parts of Queens (see Sections 4–C and 7–A) and historic concentrations in lower Manhattan and parts of Brooklyn. But the “problem” is that there are many such “unreliable” tracts—the ACS-based map is overwhelmingly grey; the map avoids the false precision of the 2000 census pane where many of the ACS-grey tracts show up in the yellow-tinted 0–10 percent Asian category, but the presentation of so many “unreliable estimate” tracts might lead viewers to question the reliability of the data as a whole.
- For a map of tract-level percentages of population living in poverty (income under 150 percent of the poverty line) in the Chicago metropolitan region,7 Romalewski demonstrated a more overt “mask.” The poverty levels were mapped in red-orange shades, and he displayed a version with and without pale blue dots marking tracts with coefficients of variation of 25 percent or more. Again, the attempt at an honest representation of the variability inherent in the estimates winds up calling the reliability of the whole map into question because there are so many blue dots. Perhaps worse, the blue dots very literally mask the picture of poverty where census tracts are smaller and most dense: The dots make it physically difficult to see the shades of poverty in the city of Chicago itself.
Another approach is to confront the margins of error directly—to plot the lower and upper bounds of a confidence interval estimate for a location rather than a single point estimate. To illustrate the point, Romalewski revisited the view of block groups in Suffolk County, shaded based on the percentage of housing units with no access to a vehicle, using 5-year 2006–2010 ACS estimates.
7The maps were clipped to make a vertical, portrait-orientation presentation and so focus on the core of the Chicago area in Illinois; the map coverages spill over into northern Wisconsin and northwest Indiana but the view is truncated, as it is for outer-ring Illinois counties like DeKalb and LaSalle.
SOURCES: Calculated from 2006–2010 American Community Survey, Table B25044; adapted from workshop presentation by Steven Romalewski.
Figure 5-2 shows the map based on the lower bound of an interval estimate for the no-car percentage, subtracting the margin of error from the estimated number of no-car households and dividing by the total number of occupied housing units. He said that this might be considered a “worst case” map for no vehicle access. Most people on Long Island have cars, so low percentages dominate the picture, but Romalewski said that the pattern of block groups with high percentages is fairly similar to the pattern based on 2000 census long-form data. But when the view is switched to the upper bound on the estimates (Figure 5-3), the volatility in the data becomes clear; the map is saturated in the red and orange tones connoting the highest levels of no vehicle access because large standard errors drive up the upper bound. It is a “world of difference” between the two maps and—for purposes of envisioning specific transportation routes and policies—it suggests that the ACS data exhibit extreme uncertainty at the fine, block-group level.
Romalewski’s next series of maps looked at the same geographic area—Suffolk County, this time looking at census tracts rather than block groups—but a slightly different data source. As discussed in greater detail in Chapter 7,
SOURCES: Calculated from 2006–2010 American Community Survey, Table B25044; adapted from workshop presentation by Steven Romalewski.
the Census Bureau issued a special tabulation from 5-year 2006–2010 ACS data for use in legislative redistricting and more easily assessing the equity of newly drawn districts. Known as the Voting Age Population by Citizenship and Race tabulation—or CVAP for short—the file contains counts of voting-age U.S. citizens (and their associated margins of error) for geographic units from the nation as a whole down to the census block group level. The maps Romalewski displayed using CVAP data grew out of work that the Mapping Center has been doing with a group involved with boosting civic engagement among the Long Island population; in particular, this set of maps examined the percentage of the total CVAP population in each tract that is of Hispanic origin. Given this application area, one instinctive “answer” to the problem of volatility in the ACS estimates—switching to a more aggregate level of resolution like county subdivisions or defined villages—is not really viable; higher-level geographic aggregates are far from optimal for purposes of planning specific, targeted voter registration drives or activities affiliated with schools.
Using the CVAP data, Romalewski began by repeating the approach of generating separate maps based on lower-bound and upper-bound estimates for per-
cent Hispanic. The results were similar to, but more muted than, those for the no-car variable by block group; a cluster of high-Hispanic voting-age population in the northwest section of the town of Islip stands out, but numerous tracts flare up to higher levels (and deeper shading on the map) in the upper bound map due to large margins of error. He displayed the results from a few other approaches:
- “Triangulation”/corroboration with related, auxiliary data: In this specific setting, he displayed a map that looked at the Hispanic voting-age (18 and older) population from the 2010 census summary data file. This is not the same population because it is not restricted to citizens, but it can serve as an intuitive check—or an effective, true upper bound—on the CVAP data. The high-Hispanic-concentration areas in Suffolk County that registered strongly in the CVAP maps show up on the 2010 summary file map—northwest Islip, Brentwood, Huntington. And one could construct a “corrected” map through this triangulation—for instance, an area in East Shoreham shows up in the 11–25 percent Hispanic shading level on the upper-bound CVAP map but that percentage is logically capped at the 6– 10 percent category because that is the 2010 census percentage Hispanic-and-voting-age for the tract (without regard to citizen status).
- Omission of unreliable-estimate areas: A variant of the masking approach, another take would be to visually omit tracts with highly unreliable estimates (coefficient of variation 25 percent or greater) rather than call attention to them. In practice, this took the form of leaving the unreliable-estimate tracts unshaded (white). Though he repeated that he has made no conclusions, he did suggest that this approach is undesirable because—in general—“you want to try to avoid losing information from the map,” and this feels like going out of the way to not present the whole picture.
- Overlay some visual marker on areas where margins of error could shift estimate to a different color category: In this sample map, Romalewski shaded the Suffolk County tracts in some tint of green based on Hispanics as a percentage of total CVAP population, but overlaid a dot-mesh layer on tracts where the interval estimate could place the estimate in one of the other green-tint ranges. He said that his basic reaction to this picture is that it is “okay to an extent,” but it seems to put a very high burden on the map’s readers to sort out what the map is really telling them.
He closed by noting that the Cornell Applied Demographics program is doing particular work on techniques to flag potentially unreliable areas in an interactive, online mapping application, focusing the viewer’s attention on the possible pitfalls and permitting a more detailed look at the reason for the flags by clicking on them.
Using ACS data to craft a demographic and economic portrait of the Navajo Nation is a very different proposition from using it to explore patterns in New York City. The Navajo Nation lands cover a land area in the Four Corners region of the Southwest that is roughly comparable to the state of West Virginia—across which is spread a resident population of roughly the same size as the single neighborhood of Flushing, Queens. However, the ACS is proving just as useful a resource, as Lester Tsosie (from the Navajo Nation’s Division of Economic Development) described in his workshop presentation. His workshop slides were actually a direct copy of a briefing that Tsosie’s division had prepared for tribal leaders in April 2012 as a briefing on the “economic landscape” of the Navajo Nation, so the core of his remarks at the workshop was actually a pure demonstration of the way in which ACS data are used to address the questions of policy makers.
For the benefit of the workshop presentation, though, he began with some basic overview of structure. The Navajo are one of roughly 550 American Indian tribes in the United States and—in both its membership population and the area of its tribal land holdings—is the largest of those tribes. The tribe maintains its own membership rolls, but respondents to the decennial census and the ACS can self-identify as Navajo in answering the question on race. Per a graph in his slides, the 2010 census reported just over 308,000 self-identified Navajo in the total U.S. population; of these, the majority (about 56 percent) lived “on reservation,” within the Navajo Nation’s tribal lands. That figure represents a decreased share of Navajo living on tribal land compared to the 2000 census (about 61 percent); Tsosie said that much of this migration from living on tribal land to living elsewhere is principally about Navajo people going to large urban centers for employment (and educational) opportunities.
Functionally, the Navajo Nation is a semiautonomous territory with its own “national” (tribe-level) executive, legislative, and judicial branches. It is divided into 110 chapters that perform some local government functions; these chapters vary greatly in size and population—and 110 chapters partition the on-reservation population of about 173,000 quite finely—but the chapters are a natural and meaningful level of resolution for looking at demographic data.
Tsosie’s economic and demographic briefing for tribal leaders centered around four choropleth maps, working with four variables calculated from 5-year 2006–2010 ACS data:
- Families in poverty: Per Tsosie’s calculations, on the order of 13.8 percent of the total U.S. population has income levels below the poverty line; within the Navajo Nation, that proportion is nearly tripled (37.7 percent). Based on the map, the Division of Economic Development superimposed black circles to draw tribal leaders’ attention to 10 particularly prominent
clusters of poverty within the Navajo Nation—some single-chapter and geographically compact (e.g., the Alamo and Tóhajiilee chapters in New Mexico) and others more sprawling (e.g., a cluster of roughly 15 chapters centered around the Manuelito and Tsayatoh chapters and spreading across the Arizona–New Mexico border).
- Percent of civilian labor force unemployed: Again, viewed as a whole, the Navajo Nation’s unemployment rate (estimated for the presentation at 15.6 percent) exceeds—and is nearly double—the national level. For this map, deriving the percent unemployed by chapter from the ACS data, the Division of Economic Development did not superimpose any graphic device to call attention to particular clusters because the areas with the highest unemployment rates (higher than 36 percent) stand out very prominently on the map: a north-south line of New Mexico chapters running from Whiterock to Smith Lake and four chapters in Arizona near the eastern border of the Hopi Indian Reservation, which is geographically an enclave in the main Navajo Nation landmass.
- Median household income: Tsosie’s department’s calculations suggest that median household income in the Navajo Nation ($26,232 per year) is just under half of the median income for the United States as a whole. Mapped by chapter, the higher median-income chapters tend to be on the edges of the Navajo Nation’s land area—most strikingly, in a line along the western border of the nation from Lechee and Coppermine in the north to Leupp in the south (or, roughly, from Page, Arizona, to just east of Flagstaff, Arizona). In particular, Tsosie said that his division is using this map (in combination with the unemployment map) to reevaluate the division’s strategic development plans, seeing how the maps coincide (or not) with developing commerce centers within the Navajo Nation.
- Percentage of people whose spoken language is something other than English: As a crude measure of speakers of the Navajo language, the Division of Economic Development calculated the percentage of each chapter’s population that reported speaking a language other than English; presumably, this mainly identifies speakers of the native Navajo language and its dialects. About 70.7 percent of the entire on-reservation Navajo Nation population speak a non-English language, per the presentation’s calculations. Though there are pockets of non-English concentrations on the northern and eastern edges of the Navajo Nation’s lands, the general impression from the map is a radial pattern—strongest concentration in the chapters in the northeast corner of Arizona and spreading (to lower concentrations) away from that center. This map, and the underlying data, are now being examined by education authorities within the Nation, because it does have implications for the service populations of educational institutions. As an aside, Tsosie also noted that this map also serves as a
After constructing these maps, Tsosie paid credit to the consistency of the ACS as a data source. The Navajo Nation lands spread across three U.S. states and cover (or otherwise interact with) a host of county and local governments. Touching on so many other jurisdictions, Tsosie said that he knows from experience that it is very difficult for an organization like his Division of Economic Development to coordinate, work with, and obtain information from all those other sources. Information is also not necessarily coded or recorded in the same way in the various jurisdictions; data available in (and obtainable from) the state of New Mexico may be markedly different from that in Arizona. By comparison, he said that the ACS at least provides a stable, consistently-applied-and-coded source for the kinds of data products and tools that he wants to develop for tribal leaders.
Tsosie then walked through a quick demonstration of his divison’s website, http://www.navajobusiness.com, from which a variety of additional data analyses and reports are available.8 The Division of Economic Development fields some data collection activities on its own—for instance, the Navajo Shopping Preference Survey described in one issue of their Navajo Economic Data Bulletin in 2012 and a visitor survey—but Tsosie said that they are increasingly turning to the ACS population data and related sources. On one dedicated subpage of the site,9 Tsosie and colleagues performed a service for the individual Navajo chapters by extracting chapter-specific data from the most recent 5-year 2006–2010 ACS data as well as the 2010 census Summary File 1 tabulation. Admittedly, Tsosie said, some of these extracts are simply PDF files of tables constructed using the American FactFinder interface while others reformat the data into easier-to-work-with Excel files. However, the division’s work in putting the documents in one place means that the chapters have a single source and do not need to learn the nuances of American FactFinder on their own. Tsosie said that this repository of reports—and the ACS’s coverage in Navajo Nation—has been very well received by the chapters, and has been particularly appreciated by officials from the tribal government and individual chapters involved in preparing grant proposals.
Tsosie also lauded the topic coverage of the ACS as being useful in tribal decision making and resource allocation. One of the tabulations presented on the site—and one that Tsosie asked to be included in the case study/agenda book for the workshop—are chapter-level estimates to the question
of whether the housing unit has complete plumbing facilities or not. He said that the questions of whether the housing unit contains a flush toilet or plumbing facilities are often mentioned in discussions of whether the ACS is too intrusive, and they are sometimes lampooned. But he said that the questions are important to ask because—in many parts of the Navajo Nation, in other American Indian reservations, and in other rural communities—the answer to them is “no.” Indeed, paging through the chapter-by-chapter listings, quite a few of the 110 chapters have a majority of housing units lacking complete plumbing facilities. If one takes plumbing for granted, the questions might seem frivolous, but in poor, rural areas the information is essential for documenting current conditions and working on changed policies.
Similar to Miller’s observation that ACS data are certainly not perfect for rural analysis—but that they are the best and sometimes the only data available (Section 5–B)—Tsosie noted that he has seen some of the same quirks as others when working with small-area, small-population data. Looking at detailed data on household income, he said that he has seen one or two pockets of unusually high values. Like Plyer’s anomalous high-wealth section in New Orleans’ Lower Ninth Ward (see Section 3–C), the ACS estimates seem to show that “there are supposedly some very rich Navajos out there—we haven’t found themyet.” But, in his assessment, the benefits of the ACS data outweigh these quirks.
That said, Tsosie said that he needed to mention one special kind of “burden” associated with the ACS and, with it, one direction in which additional information from the Census Bureau could be a credibility boost. That “burden” is the obligation to not only detail the statistical and survey methodology of the ACS, but also explain the precise mechanics of how the data are collected. Though tribal and chapter leaders appreciate the value of the numbers they are beginning to receive from the ACS, there is a great deal of confusion over exactly how the data are collected—on and off reservation lands—on an ongoing basis. The data collection process is much more visible in the decennial census, with its high-profile publicity push and its structured interactions with tribal authorities on how to conduct the enumeration.10 But exactly how the Census Bureau operates on tribal lands in off-census years, and how it regularly collects ACS information in on-reservation lands—while respecting the reservation’s sovereignty—remains a great mystery to outside observers. Based on the comments that Tsosie and colleagues have received, speculation on how the ACS data are actually collected in the Navajo Nation ranges from “put[ting] out a table outside a Bashas’ [Grocery Store] to get the information” by interview to “just get[ting] the data off of the different [web] sites out there on the Navajo Nation.” Tsosie conceded that he and his colleagues are not really able
10In the 2010 census, tribes were given options on the degree to which in-person field visits would be used relative to mail (incoming or outgoing) methods. Most American Indian lands—including the Navajo Nation—were counted via Update/Enumerate methods—field staff knocking on every door, updating their address lists as necessary, and completing the questionnaire through personal interview (rather than simply leaving them to be returned by mail).
to dispel or correct these speculations because there does not seem to be good documentation of the true procedures in areas like the Navajo Nation without a great deal of conventional, city-style, mailable addresses. He also remarked that his efforts to “get some clarification” through the Census Bureau’s Denver regional office has been met with “very limited success.”
Revisiting that comment during the session’s discussion period, Tsosie added that part of the concern over how the ACS data are collected on Navajo lands stems from longstanding issues of trust. As he put it, “our historical legacy is not always on good grounds with the federal government,” and he conceded that it can be “very difficult to extract data from our people sometimes because of what the federal government has done to our people” in the past. But he closed by commending a great benefit of the ACS: its continual presence and its coverage of demographic information within the Navajo Nation serves to “validate our existence”; it “validates our sovereignty” by chronicling that the Navajo are and “will continue to be there on our home lands,” and so the ACS is extremely valuable to the tribe.
Asked to moderate the discussion session, National Association of Counties (NACo) Research Director Jacqueline Byers began by noting that the 2010 census was the fourth with which she has worked with the decennial census in a professional capacity, whether in her current role at NACo or when working in state government. Now, working with NACo members and helping them use ACS data, she said that—“finally”—she doesn’t need to “apologize that the data is ten years old anymore … and it is a wonderful thing.” Byers also said that she wanted to follow up on Tsosie’s comments on the complete plumbing and flush toilet questions—and remark specifically on the discussion of privacy issues in the workshop session dedicated to ACS burden (summarized in Chapter 8, but a session that immediately preceded this state/local/tribal session during the workshop). She said that she finds it an “interesting contradiction in how people approach privacy” that “people tell everything about themselves on Facebook”—that you can barely slip and fall on the street (or anything else embarrassing) without it “going up on YouTube within half an hour”—yet will “have a problem with telling [the Census Bureau] how many toilets they have or what time they leave for work in the morning.”
Turning to counties’ perspectives on the ACS, Byers said that counties are an often-underestimated force in public policy; within the states, counties (and county-level equivalents) are typically the administrative units that “are charged with the responsibilities of making things work.” Counties’ work involves a great deal of planning—developing services and then ensuring that they are delivered to those who need them. Accordingly, “counties do everything with
data,” and so are making extensive use of the ACS. Particularly in small, rural counties, the 5-year ACS data are playing a major role in counties’ economic development plans—documenting the education and skill sets in the areas in order to attract new business (and jobs). As mentioned in the previous sessions, counties also make use of the ACS for targeting services for children and for foreign language assistance; for making transportation decisions like road repair programs and development of mass transit routes; and for developing public health and safety plans (and prepare for disasters). She noted seeing a recent news story talking about the millions of dollars being spent by counties to deal with unexpected consequences of tough economic conditions—additional funds to help the homeless and stock food banks to keep up with demand and funds spent to maintain foreclosed properties so that values of surrounding properties do not completely collapse. Increasingly, she said, counties are turning to the ACS to inform these kinds of interventions because they can get the information easily and frequently, especially in large urban counties.
She then asked each of the presenters for their responses to two questions—questions she admitted were “not particularly provocative” but ones important to the current debate on the nature and future of the ACS. First, she asked simply: How would the state governments, tribal authorities, and other client agencies described in the presentations do what they need to do without the ACS?
- Miller said that for her applications—trying to look at measures covering the whole country and trends in rural America in particular—“we would be very limited in what we could do and what we could say,” to the extent that it is hard to imagine other alternatives. There are some other county-level data resources that might be brought to bear, but none has the same level of detail as the ACS.
- Brower answered that her center would most likely have to turn to state administrative records sources, but that there would be “no comparison” with the range of analysis that can be done with the ACS.
- Romalewski followed up on that comment, saying that he imagined that administrative records from state and local governments would be used more frequently, but he said that these would likely be less consistent and reliable than systematic collection of ACS data. He said that some of this spottiness in records data would arise from inconsistency in collection and coding at the most local levels; even in a large metropolitan county like Suffolk County on Long Island, the county “sometimes doesn’t have much say over how the individual towns collect the information,” and that’s “not a good situation.”
- Tsosie said that tribal governments like the Navajo Nation would likely have to turn to the federal Bureau of Indian Affairs (BIA) and try to access the records that BIA maintains. However, he said that he suspected that much of that data “would be outdated and probably not very useful.”
For more detailed survey-type data, he said that the recourse could be to hire consultants or survey research firms, but that those costs could be prohibitive.
Byers’ second question for the panel of speakers—and the rest of the workshop attendees—was what they specifically think would happen to their data projects and their ability to use ACS data if response to the survey were made purely voluntary rather than mandatory.
- Miller said that there are already difficult-to-reach and low-response-rate populations in the mandatory ACS—the most rural areas in the United States counting as one of those. She said that she suspected that a voluntary ACS would impair the effective sample size of the survey, and that the groups that are already most difficult to reach would be more likely to be missed in a voluntary ACS.
- Brower agreed, saying that “we wouldn’t have the confidence that we have [now] in the representativeness” of the population. Given her center’s analyses, she said that she would particularly worry about a distorted view of low-income or linguistically isolated populations.
- Romalewski said that, not being a statistician, he could not really judge the specific effects of what happens—but he expected that his mapping applications would be worse as a result.
- Tsosie recalled his concern about exactly how ACS data are collected on American Indian lands at present under the mandatory collection; if it is made voluntary, “forget it, [tribal areas] won’t get anything.”
Byers closed with some personal comments about the legislative debate on the future of the ACS. She said that this workshop—and its assemblage of ACS stakeholders—is an important effort. However, she said, a critical perspective not represented in the room is one that is critical to consider going forward: that perspective is “the average citizen who is filing the complaints” with the Census Bureau and their congressman about the ACS. Byers says that she sees this as a major educational challenge and opportunity—boosting awareness in the mind of the average citizen about how data from the census and ACS affect their lives. For example, the average person putting children on the school bus in the morning might not fully realize that—very often—it is ACS and census data that play major roles in planning the school bus service (e.g., ensuring that the children have a seat in the bus), in making sure that they have a classroom to go to at the end of the ride (school siting to meet demand), that there is a teacher standing in front of the room (school staffing decisions), and so forth.