Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 79
4
American Community
Survey
PUT FORWARD AS A POTENTIAL REPLACEMENT for the decen-
nial census long form in 2010, the American Community Sur-
vey (AC S) is a maj or household survey anticipated to include
250,000 housing units each month. The hope underlying the ACS is
that, when fully operational, the survey will provide continuous infor-
mation on demographic characteristics, social welfare participation, ect-
ucation en c! health status, commuting patterns, distribution anc! fre-
quency of crime, and other important attributes of the population of
the Unitecl States.
Until now, the equivalent of the ACS has only been concluctect on
a limited scale. Pilot data collection of the ACS began in selectee! test
sites in 1996 a geographic base that reached 31 sites by 1999 and the
resulting data have fee! into reports of the feasibility of quality data col-
lection.~ In conjunction with the 2000 census, a larger-scale prototype
Come of the 31 test sites are blocks of adjacent counties, but most are single
county sites. Hence, in the dialogue that has emerged regarding the ACS, "31 test
sites" and "31 counties" are used fairly interchangeably even though the test sites span
36 counties (U.S. Census Bureau, 2003a).
79
OCR for page 80
80
PLANNING THE 2010 CENSUS
ACS began operations, involving 700,000 households per year. Data for
this survey, known as the Census 2000 Supplementary Survey (C2SS),
were first collected in 2000, anct data collection continucct at this level
in 2001 anc! 2002.2
Original plans called for the ACS to begin full fielc! implementation
in 2003, a schedule that would support publication of small-area esti-
mates in 2008. However, congressional stalemate on the budget for fis-
cal year 2003 clelayoc! full implementation by at least 1 year; moreover,
the fiscal year 2003 budget totals approved by Senate appropriators fell
well short of the funcis neeclec! for full ACS deployment. Funding in
support of full fielct implementation in 2004 anct 2005 is unclear at this
timed
In this chapter, the panel offers its interim assessment on the ACS.
As we will describe in detail, some benefits of the ACS are nearly in-
ctisputable, key among them the increased timeliness of the data rel-
ative to traditional long-form census estimates. But the new data re-
source brings with it new challenges in evaluation anct estimation, anct
the Census Bureau neecis to bolster the case for the ACS by provicI-
ing stakeholclers with information on the ways in which ACS infor-
mation shoulc! be usec! in a variety of contexts. In orcler for the ACS
to replace the long form, it must be clemonstratect that the ACS can
acloquately meet all of the unique functions of long-form social anc! cle-
mographic clata. The panel is supportive of full implementation of the
ACS. However, we recognize that much remains to be clone in articu-
lating the strengths and the weaknesses the challenges as well as the
new opportunities of the ACS as a replacement for the long form.
CONDUCTING THE ACS
When the ACS is fully fielclecl, it will use as its sampling frame the
same Master Aciciress File (MAF) usec! by the decennial census. The
2The Census Bureau refers to the latter two data collections as SSO1 and SS02 the
2001 and 2002 Supplementary Surveys respectively.
3 Under the funding levels provided by the Bush administration s proposed budget
for fiscal year 2004, questionnaire mailing for a full-scale ACS could begin during the
fourth quarter July-September) of 2004. Field work for follow-up would be deferred
until after September 2004, pushing the considerable expense of field interviewing into
the fiscal 2005 budget process. Prior to the fourth quarter mailing, data would continue
to be collected in the 31 test sites and at the C2SS levels (Lowenthal, 2003a>.
OCR for page 81
AMERICAN COMMUNITYSURVEY
81
annual sample of housing units chosen for participation in the survey
will be cliviclec! into monthly mailout panels, anc! each month's panel
will be a systematic sample across the complete acictress list. Thus, it
is intenclec! that each month's sample will be a representative sample
(approximately 480) of the population of each area of the United States.
However, this simplified version of the sample selection process will
be complicated by alterations similar to practices currently usect in the
decennial census long form, including oversampling of small geographic
areas.
The ACS is intenclec! to be aciministerec! primarily via mailout/
mailback. However, the proposed ACS techniques to follow up with
households that clo not return the mail form cliffer from decennial
census practice. All mail nonresponclents will be initially followoc!
up by computer-assistecl telephone interviewing (CATI) during the
month following questionnaire mailout, if there is an available phone
number. After CATI follow-up, a random one-thirct of the remaining
nonresponclents will be clesignatec! for follow-up by fielc! enumerators.
The precise nature of this sequential follow-up process remains to be
cleterminecl; there are tentative plans to sample areas with low mail
anct telephone response rates at a higher fraction rather than a strict
one-thircl random sample; this oversampling may help to make sample
variances more comparable across areas.
The stagewise nature of ACS follow-up leacis to another important
design feature, which is that all of the information collected in a given
month will be usect as inputs for that month's estimates. That is, a
particular month's estimates may include mailback responses from
the present month's systematic sample of housing units but will also
include completed telephone anc! personal interviews from 1 anc! 2
months prior, respectively. This design choice is advantageous in that
it simplifies data processing anc! production loac! there is no neec!
to wait until month t + 2 for final resolution of all the housing units
chosen in month t before processing responses already submittecl. But
it cloes raise complex methodological challenges, including the choice
of weighting methods to aciciress unit nonresponse.
While the size of this survey will make some direct small-area es-
timates possible, the estimates for areas under 65,000 population typ-
ically will be produced by aggregating information over either 3 or 5
years, depending on the size of the area. At this time, moving averages
OCR for page 82
82
PLANNING THE 2010 CENSUS
are planned to be used for these aggregate-year estimates, though other
possibilities coulc! be consiclerec! in the future.
It is the neect for a 5-year winclow to produce cletailect small-area es-
timates that puts a firm constraint on the ciate of full ACS deployment.
The initial plans for full deployment in 2003 would produce small-area
estimates in 2008, allowing some time for the new ACS figures to gain
acceptance as a long-form replacement. Hence, to match the long-form
data production schedule of the 2000 census, the absolute cleaciline for
full implementation of the ACS is 2007, which would permit the pub-
lishing of national estimates analogous to those from the long form in
2012.
STRENGTHS OF THE ACS
A great strength of the ACS relative to other national household
surveys is its large sample size, which allows it to provide small-area
information on the American population population characteristic
profiles for counties, cities, anct other local areas. Over a 5-year period,
the survey's sample size will approximate that of the census long form,
supporting the production of estimates for small anct nonstanclarct
geographical areas, such as school districts anc! traffic analysis zones.
In acictition anct again given the large sample size information will
be available for population groups defined by factors other than ge-
ography, including racial and ethnic groups, age classes, occupational
groups, anct educational anct health categories. (Tabulations can also
be prepared for subpopulations with some combination of these
characteristics.)
While the census long form can only provide these small area pro-
files in once-per-clecacle snapshots, the ACS collects information con-
tinuously throughout the clecacle. Therefore, the ACS has the impor-
tant advantage of providing estimates of the intercensal dynamics of
small-area changes in the many variables listed above. Such estimates
have been almost nonexistent up to now anct can provide important in-
formation for policy initiatives en c! public anc! private planning.
The increased timeliness of the ACS estimates relative to census
long form estimates is a very substantial benefit. ACS data products are
at most 3.5 years out of ciate when released; census long form data procI-
ucts are never less than 2 to 2.5 years out of clate anct can be as much
as 12.5 years out of ciate. Presently, using census data to develop lower
OCR for page 83
AMERICAN COMMUNITYSURVEY
83
bouncis on the amount of year-to-year change that occurs for various
estimates for example, poverty rates involves examining census-to-
census differences and dividing by 10; this annual change may be mea-
surec! directly uncler the ACS.
The ACS may eventually permit researchers to develop an inte-
gratec! framework for more accurate small-area estimation, perhaps
combining one or more waves of ACS data with results from aciminis-
trative records, other household surveys, anc! the short-form decennial
census. This broacler perspective views the ACS as a supplement to
the social anc! demographic information currently collected by existing
surveys anct administrative records systems. There are a variety of
synergies that can be imagined between ACS anc! household surveys
such as the Current Population Survey, jointly using each to improve
the information collected by the other.
Relative to the decennial census, the prime advantage of a full-
flecigect ACS to the Census Bureau is the resulting prospect of a
short-form-only census. Though the census long form was only
aciministerect to a 1-in-6 sample in the 2000 census, the operational
burclen is tremendous; completed long forms constitute a mountain
of paper, and each form must be unstapled (running the risk of pages
being mishancilecI) before processing.
There are also reasonable arguments that the ACS may provide
more accurate information than the census long form. ACS data woulc!
be collected uncler more controlled circumstances by more experienced
interviewers. Moreover, by spreading the clemanct on respondents to
provide cletailec! personal anc! household information over the clecacle,
the ACS may also be less susceptible to flaws anct inaccuracies that
may arise from nonresponse in a once-a-clecacle measurement. During
the 2000 census, concern over the perceived intrusiveness of the long
form questions was well publicized, leacling to the conjecture albeit
one that has not been empirically documented that this concern may
have negatively impacted response rates on long form questions anti,
accorclingly, hurt the accuracy of long-form clata.
COSTS OF THE ACS
The great advantages of the ACS timeliness anc! accuracy must
be offset against the costs of implementing the program. Given that
it cannot "piggyback" on some of the infrastructure proviclec! by the
OCR for page 84
84
PLANNING THE 2010 CENSUS
decennial census, one might assume that the ACS coulc! cost more than
the marginal cost of the long form it is replacing. However, the Cen-
sus Bureau has argucct that operational efficiencies will make a short-
form-only census complemented by ACS a less expensive option than
a mixect long-anct-short-form traditional census. In congressional tes-
timony on May 13, 2003, Census Bureau director C. Louis Kincan-
non commented that "our current estimates indicate that three com-
ponents of the 2010 Census LACS, MAF/TIGER Enhancements, anc!
early planning/testing] will cost approximately $11.2 billion. However,
if we change course right now en c! revert to a traditional census, the
cost will increase to more than $12 billion anct perhaps much more."4
In its original presentation of its 2010 census strategy, the Census
Bureau argucct that most of the acictitional costs of ACS can be paid
for through the associated greater efficiencies in the 2010 census. Ac-
corcling to the bureau, these savings would result by eliminating the
collection anct processing of long-form information cluring the clecen-
nial census, through improvement of MAF/TIGER, anc! through use
of hanct-helct data collection crevices to facilitate fielct follow-up of mail
nonresponclents. As the panel notec! in its letter report (National Re-
search Council, 2001c), we have not seen validation of this claim based
on empirical evidence anc! suggest that a fuller cost-benefit analysis of
the ACS would help bolster the case for the survey.
ACS INFORMATION AS A REPLACEMENT FOR
LONG-FORM INFORMATION
Our basic theme in this report is the importance of integration
within the census process, and in that spirit the panel urges the Census
Bureau to make a stronger case for the ACS and its role in the broader
census context.
At the most basic level, the case for the ACS as a replacement for
the census long form is an easy one the ACS's content is patterned on
the long form, so the ACS will succeed in collecting the same set of data
items as the long form. That information will, moreover, be collected
4The remarks are quoted from the director's prepared testimony before the U.S.
House Subcommittee on Technology, Information Policy, Intergovernmental Rela-
tions, and the Census at a hearing on the ACS's potential to replace the census long
form in 2010.
OCR for page 85
AMERICAN COMMUNITYSURVEY
85
anc! released on a much more timely basis than the census long form a
significant benefit, anc! an improvement over the long form. But two
basic questions remain to be fully answered in bolstering the case for
the ACS.
First, is the ACS able to satisfy all of the neects currently acictressect
by long-form ciata? This larger question can in turn be cliviclec! into
at least two aspects. The first stems from the fact that ACS estimates
will for all but the largest population or geographic groups be based
on averages across multiple years of data. Hence, the question arises:
are there applications using the census long form for which substitu-
tion of a moving average-type estimate from the ACS would be inap-
propriate? The seconct subquestion is how well ACS estimates match
other estimates of the same phenomena not only how ACS finclings
compare with census long-form results but also how ACS estimates
compare with other survey measures. (We will briefly discuss another
aspect of the question whether the ACS can provide specific break-
clowns en c! analyses, to the same extent that the long form cloes in a
later section on the group quarters population.)
The seconct fundamental question of interest is: What is the quality
of ACS estimates anc! data relative to the census long form? Specifically,
what can be said about error both bias anct variance in data collected
through the ACS, and how does that compare with the census long
form? It is also important to consider the level anct possible patterning
of ACS unclercoverage, just as it is important to analyze the same with
respect to the census long form.5
In the following sections, we offer some initial comments on these
two basic questions. The panel recognizes that there are no absolutes
in weighing the prospective ACS against the census long form that all
options involve tracle-offs of both costs anct benefits, anct that the ACS
cannot reasonably be expected to be better than the census long form
in every respect (ancl vice versa). We are optimistic that increased Cen-
sus Bureau attention to informing data users anc! stakeholclers (whether
long-term users of the long-form data or newcomers) about the unique
5For this discussion, we make the simplifying assumption that survey undercover-
age, relative to the census, may be considered a component of nonresponse. (This is a
simplifying assumption because some natural causes of survey undercoverage, like any
incompleteness of the operational ACS address list, are of course not exclusively due
to missing data.)
OCR for page 86
86
PLANNING THE 2010 CENSUS
features anc! challenges of working with ACS data will build a stronger
case for the survey.
ESTIMATION USING THE ACS
Adequacy of Moving Averages as Point Estimates
A basic concern regarding the American Community Survey as a re-
placement for the census long form is whether ACS estimates -which,
particularly for small areas or groups, would be moving averages of mul-
tiple years' data points can take the place of fixed point-in-time esti-
mates. Obviously, ACS estimates have one clear advantage in that those
fixect point-in-time estimates could, for the census long form, refer to
a point as much as 12 years ago. More to the point, though, the con-
cern is whether funct allocation formulas or other public anct private
planning neecis for demographic data can be aciciressec! using a combi-
nation of data from multiple years. The Census Bureau has issucc! a
ctraft report that attempts to acictress users' concerns about this shift
(Alexancler, 2002), en c! Zaslavsky anc! Schirm (1998, 2002) outline the
advantages and disadvantages a locality may experience through use of
either a moving average or a clirect (census) estimate.
The crux of the debate on this point is that a moving average is a
smoother! estimate; by averaging a particular time periocl's data obser-
vation with those within a particular time winclow, the resulting esti-
mate is meant to follow the general trend of the series but not be as
extreme as any of the inctiviclual points. The ramifications of this basic
concept emerge when moving average estimates are entered into sensi-
tive allocation formulas or compared against strict eligibility cutoffs. A
smoothed estimate may mask or smooth over an individual year drop in
level of need, thus keeping the locality eligible for benefits; conversely,
it may also mask incliviclual-year spikes in activity and thus disqualify
an area from benefits. It is clear that the use of smoothed estimates is
neither uniformly advantageous nor ctisacivantageous to a locality; what
is not clear is how often major discrepancies may occur in practice.
One basic conceptual answer to this conundrum is to not use mov-
ing averages and instead use sample-basecl estimates from individual
years. These estimates would be unbiased in terms of probability but
OCR for page 87
AMERICAN COMMUNITYSURVEY
87
coulc! be highly variable; this woulc! affect aspects of formula grants
such as "holcI-harmless" provisions.6
A relater! worry that has been expressed about moving averages is
that, by incorporating estimates from other time periocls, the estimates
for a given time perioc! coulc! be substantially biasec! anc! will not truly
reflect the conditions for that given time period. The outstanding em-
pirical question is assessing the bias that may result from averaging over
3 years of data comparer! to 5, en c! trying to weigh the magnitude of that
bias against the bias associated with using a long-form estimate that is
up to 12 years olcI. Intuitively, it is sensible that, when examining data
series in which change is substantial between census years, moving av-
erage estimates woulc! be preferable to seriously ciatec! estimates. When
there is little change through the clecacle, there should be little ctiffer-
ence between the two estimates. However, since this is an empirical
question, the Census Bureau should carry out research that helps to
evaluate this tracle-off.
The continuous measurement properties of the ACS give it unique
advantages over the decennial snapshots available from the census long
form, but they also raise a final, related point of concern regarding mov-
~ ~ . . . .
ng averages. ~~ eat Issue IS assessing year-to-year cad range In a c ata series.
It is incorrect to use annual estimates baser! on moving averages over
several years when assessing change since some of the clata are from
overlapping time periods en c! hence identical. At the least, the results
will yielct incorrect estimates of the variance of the estimates of change.
Therefore, users shoulc! be cautioned about this aspect of the use of
moving averages. Along the same lines, moving averages present the
same types of problems when they are usec! as clepenclent variables in
various statistical models, in particular time series models, anct in some
regression models. Therefore, the Census Bureau coulc! bolster the case
for the ACS anct potentially help relieve users' concerns if it would pro-
cluce a user's guicle that details the statistical uses for which moving
averages are and are not intenclecl, the problems they pose to users, and
. . .
means to overcome them.
6A "hold-harmless" provision in a funding formula is one that limits the amount by
which an allocation can change from one year to another; for instance, under a 70 per-
cent hold-harmless level, a unit's allocation may only decrease by up to 30 percent. In
a hold-harmless situation, an unusually volatile observation one year due to increased
variability could mean that the unit's allocation may remain out of true alignment for
several cycles due to the amount of allocation automatically carried over.
OCR for page 88
88
PLANNING THE 2010 CENSUS
Though the prospect of using moving averages rather than long-
form census estimates cloes raise legitimate concerns anc! will have im-
pact on users, the panel judges that the benefit of more timely infor-
mation collected by the ACS outweighs these concerns. That said, it is
important that the Census Bureau strive to minimize the impact of the
change anc! work to eclucate users anc! stakeholclers about the nature of
the change.
Comparing ACS/C2SS to the Census Long Form
Thus far, we have outliner! from conceptual anc! theoretical perspec-
tives the issues surrounding the acloquacy of ACS estimates to replace
the long form. It is also natural to aciciress the question from a more
pragmatic point of view: the ACS anct the census long form purport
to measure the same basic phenomena, but clo the resulting data from
both series actually tell the same story?
Comparisons of how the ACS or C2SS estimates match census
long-form estimates implicitly treat the census long-form data as an
effective "golc! stanciarcI" a questionable assumption at best, given
that it discounts the various (anct sometimes substantial) sources of
error to which the long form is subject. First, the long-form data for
small areas are subject to substantial sampling error. In aciclition, as
mentioned above, the long form is also subject to nonresponse, and for
some sample items, the amount of item nonresponse for the long form
in the 2000 census was extremely high (National Research Council,
2001a).
Love (2002) has iclentifiect a number of sources of differences be-
tween the ACS (or C2SS) anc! long-form census estimates that com-
plicate any direct comparison. These include: different reference clates;
different mocles of follow-up of nonresponse; different criteria usec! to
clecicle if a response is acceptable; different edit anct imputation tech-
niques; different methods for data capture anc! processing; differences
as to whether or not proxy interviews are accepted (they are not ac-
ceptec! by ACS but they are accepted for the decennial census); cliffer-
ences as to who is an eligible respondent; anct different weighting pro-
ceclures usect to acictress nonresponse anct sampling (e.g., the weighting
of the long-form estimates to the 100 percent data). The reference pe-
rioct associated with a question item is of particular interest for ACS
OCR for page 89
AMERICAN COMMUNITYSURVEY
89
estimates, since annual averages will be the average of responses cor-
responcling to 12 different reference periods, clepencling on when the
questionnaire was applied There are also differences in the target popu-
lation. For example, the ACS cloes not currently include group quarters
in its survey, but the census cloes.
Work on comparing the ACS (test sites) anct C2SS estimates to cen-
sus long-form estimates has been initiated by the Census Bureau. To
ciate, what is known is that there are some large differences; generally,
these differences can be explained by the amount of sampling error in
the two surveys (U.S. General Accounting Office, 2002a). However,
examination of complete-sample C2SS data suggests large differences
for the number of housing units lacking complete plumbing facilities
anct for the number of unpaid family workers. Also, at the state level, a
large number of C2SS estimates clifferec! from the long-form estimates
by at least 10 percent, including the number of workers commuting us-
ing public transportation, the number of households with income above
$200,000, the number of housing units lacking complete plumbing fa-
cilities, anct the number of renter-occupiect units with gross monthly
rent of $1,000 to $1,499.
The Census Bureau neecis to complete this analysis, including the
contribution of sampling variance, for all years of data collection, anct
attempt to identify the sources of differences other than sampling error.
A priority of this analysis should be responses related to resiclency, but
all responses should be examined.
QUALITY OF ACS ESTIMATES
The error associated with ACS data may be clecomposect into sam-
pling error (sample variance) anct nonsampling error, the latter of which
can be further separated into error clue to nonresponse anc! measure-
ment error clue to various causes.
At the most basic level, sampling error in the ACS will be slightly
larger than that for the long form because the total ACS sample size
over a 5-year period will be slightly smaller than that for the census
long form. On its own, this difference is unlikely to have a substantial
impact on users. However, sampling error clue to initial mail and CATI
nonresponse is widely variable and could be appreciable in some small
OCR for page 92
92
PLANNING THE 2010 CENSUS
responses on both the short anc! long forms. For example, for age,
the census imputation rate was 3.6 percent, whereas for the C2SS it
was 2.4 percent. Salvo anct Lobo (2002) report that the allocation rate
~ · 11 1 1 · · · ~ ~ 1 _ ~ ~ ~ ~ ~ ~ -
tessent1ally the same as the Item Imputation rate' tor the ~t)t)t) Ale In
Bronx County was typically much higher in the 1990 long form than in
the 2000 ACS, ancI, further, that this difference was strongly relater! to
the lower quality of fielct data collection for census long-form informa-
tion in comparison to the ACS. The U.S. General Accounting Office
(2002a) reports on preliminary work carried out by the Census Bureau
for long-form items in which the imputation rates were slightly higher
than for the 2000 C2SS. The Census Bureau intends to extenct the
analysis of imputation rates to all long-form items in the near future.
Since there was no content follow-up for the 2000 census long form,
it is very reasonable to expect that the gap for long-form items will be
even more pronounced than the observer! difference for short-form
items. We point out that there are other differences in administration
between the ACS (C2SS) anc! the census long anc! short forms (e.g.,
the ACS uses CATI en cl CAPI) that complicate this comparison, some
of which are cliscussec! below. However, it seems correct to anticipate
that the ACS clata will be founct to be subject to less item nonresponse
for long-form information than the census.
Quality of Imputed Responses
Rates of unit anct item nonresponse are only partially informative
as measures of the ultimate error clue to nonresponse. This is because
the imputation en cl weighting routines that the Census Bureau uses to
treat item en c! unit nonresponse (anc! survey unclercoverage) can off-
set some of the information loss, clepencling on the extent to which
the various assumptions used to support the imputation methods hold
(e.g., responses missing at ranclom). Therefore, measures of the qual-
ity of imputations are an important additional measure of the impact of
Item anc unit nonresponse.
This impact could be measured using either a reinterview survey or
through matching to a more reliable source of data (possibly aclmin-
istrative records or highly reliable household surveys). Both of these
approaches are problematic. Reinterview surveys of appreciable sample
size are expensive and require high-quality interviewing to elicit higher
OCR for page 93
AMERICAN COMMUNITYSURVEY
93
quality responses than proviclec! earlier. Matching studies are limited
by the availability of higher quality, comparable information a cliffi-
cult stanclarct to meet. The Census Bureau is in the process of carrying
out a matching study comparing C2SS responses to those for the 2000
census short form, on the unclerstancting that both sets of responses are
subject to error.10
Some interesting work on responses to race anc! ethnicity questions
has been carried out (Bennett anct Griffin, 20021. A less satisfying vari-
ant of this analysis coulc! still be carried out for small geographic aggre-
gates, for example, comparing census anc! ACS frequencies anc! means
for responses at the tract level, which would overcome the inability to
match incliviclual long-form responses. Some of this work is being con-
cluctect by the Census Bureau anct is ctiscussect below. Historically, there
were matching studies of census responses to Internal Revenue Ser-
vice (IRS) anct Current Population Survey (CPS) clata for earlier cen-
suses (Bureau of the Census, 1964, 1975b),l1 en c! excellent reinterview
studies were clone in the 1970s anct 1980s (Bureau of the Census, 1970,
1975a). Also, limited research on the quality of the imputations for
1990 were carried out by Thibaucleau (2002), but comparable work has
not been carried out for 2000.
Measurement Error
Measurement error consists of differences between the response
that was intenclec! by the survey designers given a householcl's char-
acteristics anct the response that was actually captured. Possible
contributors to measurement error include: misunclerstancling of a
question by the responclent, collecting data for the wrong time periocl,
responding in the wrong units, transposing digits, making errors in
capturing the response, intentional lying by either the respondent or
the fielc! enumerator, anc! so on.
fondue to the design of the C2SS specifically, the provision that the same respon-
dent would not receive both the census long form and the C2SS this matching is only
feasible for characteristics on the census short form.
~~Confidentiality concerns in the 1980s and 1990s led the IRS to restrict access to
data, even for statistical purposes, thus precluding further census matching studies in
recent decades. More recently, the IRS has facilitated limited administrative records
research by the Census Bureau using IRS data with appropriate safeguards.
OCR for page 94
94
PLANNING THE 2010 CENSUS
it is reasonable to assume that, generally, the measurement error
in ACS will be either comparable to, or very possibly somewhat less
than, that for the census long form. The reason behind this argument
follows from ACS design specifications: the ACS interviewing staff will
be more experienced than short-term census enumerators, anct ACS
interviewers are forbiciclen to use proxy respondents.
One challenge in comparing measurement error between the ACS
anct the census long form is reconciling the different definitions of res-
iclence in the two systems.l2 These definitions are both valic! anc! cle-
fensible, anct each may have particular advantages in different contexts,
but their basic differences complicate comparison. Moreover, the ACS
stages data collection over 3 months, anct this may incluce error clue
to temporary vacancies and frequent moving. For analytic purposes,
the moving time window of the ACS may present difficulties in inter-
preting quantities like income. Each interview's snapshot is intended
to capture a responclent's income for the 12 months preceding the in-
terview, as opposed to a fixed April-to-April reference frame; this may
. . . .
camp. 1cate time series comparisons.
TOPICS FOR FURTHER RESEARCH AND DESIGN
CONSIDERATION
A substantial agenda of outstanding operational anc! methoclologi-
cal issues would have to be acictressect in a fully operational ACS. Some
of these issues shoulc! be tackled in the near future in orcler to gener-
ate the maximum benefits from use of the ACS as part of an integrated
framework of estimates.
Voluntary versus Mandatory Response
The law governing conduct of the census imposes penalties on
"whoever, being over eighteen years of age, refuses or willfully neglects
Tithe census attempts to capture "usual residence" the location where respon-
dents usually live or spend most of their time. By comparison, the ACS captures "cur-
rent residence," the place where the respondent is at the time of the interview. More
precisely, the ACS uses a "Two Month Rule"; any respondent at a sampled household
unit who has been living at the location for more than 2 months is considered a current
resident (U.S. Census Bureau, 2003a>. This can create differences for migrant workers
or "snowbird" retirees who live for lengthy periods in different areas of the country.
OCR for page 95
AMERICAN COMMUNITYSURVEY
95
. . . to answer, to the best of his knowledge, any of the questions on any
schedule submitted to him in connection with any census or survey"
enablectin other parts of the census cocle (13 USC ~ 241(a)~.l3 In
aciclition, it is a crime to willingly give false answers to such censuses or
surveys (13 USC ~ 241 (b) ~ . Accordingly, census mailings in 2000 as
in previous years prominently featured notices that "your response is
required by law."
The Census Bureau has argucc! that the ACS is intenclec! to replace
the mandatory census long form anct, hence, the ACS should be con-
cluctec! on the same mandatory basis as the census. The General Ac-
counting Office has concurred with the bureau that it has statutory
authority to conduct the ACS and that it has the authority to require
responses (U.S. General Accounting Office, 2002b). The distinction
between voluntary anct mandatory conduct is a significant one because
it is believed that the "requirecl by law" verbiage on census forms plays
. . .
a rot e In raising response rates.
However, early congressional discussion of the nature en c! content
of the ACS led inctiviclual members of Congress to suggest that the
ACS be concluctec! on a voluntary basis. Accordingly, the Census Bu-
reau is conducting part of the 2003 Supplementary Survey (the proto-
type ACS) on a voluntary basis; this test includes replacing "requirecl by
law" verbiage with a more generic appeal (U.S. Census Bureau, 2003a).
The response rates, including item nonresponse rates, on the voluntary
surveys will be compared with a control group receiving manciatory-
response questionniares, as well as to the 2001 anct 2002 Supplementary
Surveys. The Census Bureau anticipates being able to report initial re-
sults of this test to Congress in August 2003, anct the basic question of
mandatory response is an important one to have settled early.
Interaction with Intercensal Population Estimates and
Demographic Analysis Programs
One high-priority research area shoulc! be the development of mocI-
els that combine information from other sources household surveys,
administrative records, census data, anc! so forth with ACS informa-
tion. One prominent example of this is the interplay of estimates from
However, the census code does provide that respondents cannot be compelled to
disclose their religious beliefs or affiliation (13 USC § 241 Achy.
OCR for page 96
96
PLANNING THE 2010 CENSUS
ACS and the population estimates program from the Census Bureau.
At this point, it is planned that estimates from the ACS are to be con-
trolled to postcensal population estimates. However, this should not
be considered a one-way street. It is also possible for ACS to be used
to provide the population estimates program with improved estimates
of internal and external migration, fertility, household size, and vacancy
status. The resulting improved population estimates could then be used
as improved marginal totals to which to control ACS estimates. Fur-
ther, the ACS also provides direct information on population size, and
a joint estimate from population estimates and from the ACS is con-
ceivable. The Census Bureau needs to carry out research on how the
ACS can be used to improve intercensal population estimates. Further-
more, the Census Bureau needs to examine how existing household
surveys could change their posts/ratification practices (controlling to-
tals by age, race, and sex) given the collection of ACS data.
The potential for ACS to provide improved estimates of internal
and external migration also suggests the importance of exploring the
potential interactions between the ACS and population estimates
derived by demographic analysis. Demographic analysis uses aggre-
gate data on birth, death, immigration, and emigration to produce
population estimates by ace. sex. and race.
1 1 1 1
____~_ ---- -- r---~-~
Demographic analysis
was a key benchmark used to evaluate coverage in the 2000 census,
but it has significant limitations. First, estimates of immigration and
emigration particularly those of illegal immigration are inherently
difficult to produce with precision. Second, existing administrative
records used to generate demographic analysis counts facilitate only
the most basic racial comparisons white and black but do not permit
direct estimation of Hispanics and other groups. The Census Bureau
should consider ways in which the ACS might inform demographic
analysis estimates, including more refined estimators of birth among
the foreign-born population and of internal migration.
Other possibilities for instance, using ACS and household sur-
vey information jointly in regression models to provide improved esti-
mates of the frequency of crime or unemployment could also be ad-
dressed as a research topic.14 Another high-priority research area would
· · · . . . ~ ·.. .
14The use of models that combine information from other sources has implications
for the sample designs of the major household surveys and is a future research topic
OCR for page 97
AMERICAN COMMUNITYSURVEY
97
be identification of better procedures for weighting anc! imputation, to
aciciress nonresponse anc! unclercoverage in the AC S; the hope woulc!
be to develop procedures that are, in a sense, optimized for ACS survey
data, anc! not simply borrowoc! from procedures usec! on the decennial
census long form.
.
.
Group Quarters
The intent of the census long form is to provide information on
characteristics of the entire population. This means not only the popu-
lation resicting in housing units but also those living in group quarters-
such places as college dormitories, military barracks, prisons, anc! mecli-
cal anct nursing facilities. Nonresponse to the census long form anct the
neec! to impute for nonresponse may detract from the overall reliability
of census long-form data, but those data clo at least allow users to make
some inference about the group quarters population. Accorclingly, the
complete elimination of the census long form anc! the possible loss of
data on the group quarters population is an obvious concern of some
census stakeholclers.
In its ciraft operational plan, the Census Bureau has inclicatec! that
the ACS will be aciministerect to a 2.5 percent sample from the bureau's
group quarters roster (U.S. Census Bureau, 2003a). It remains to be
cleterminect how acloquate this may be for monitoring this important
population group, especially for small levels of geography.
SUMMARY AND ASSESSMENT
In 1995, a previous Committee on National Statistics (CNSTAT)
panel related to the decennial census offerec! its comments on an idea
"which the Census Bureau has recently been investigating:"
to drop the long form from the census and substitute a con-
tinuous measurement survey that is, a large monthly sur-
vey of perhaps 200,000 to 500,000 households. By averag-
ing the results of the monthly surveys over a period of 3 to
5 years, more timely long-form-type clata, accurate enough
of great potential interest. Use of these models and connections to external programs
such as the ACS may permit other household surveys to reallocate sample to areas in
which estimates are less reliable.
OCR for page 98
98
PLANNING THE 2010 CENSUS
for use in relatively small geographic areas, could be pro-
clucecI....
In its preliminary work, the Census Bureau has spec-
ulatec! that the costs of the new continuous measurement
survey over a clecacle could be roughly offset by the cost
savings from ciropping the long form from the census anc!
by other cost reductions that might be achieved in inter-
censal operations....
Although we believe that the proposer! continuous
measurement system deserves serious evaluation, we
conclude that much work remains to develop credible
estimates of its net costs anc! to answer many other funcia-
mental questions about data quality, the use of small-area
estimates based on cumulated data, how continuous mea-
surement could be integrated with existing household
surveys, anc! its advantages compared with other means
of providing more frequent small-area estimates. In our
judgment, it will not be possible to complete this work
in time to consider the use of continuous measurement
in place of the long form for the 2000 census (National
Research Council, 1995:91.
Eight years later, faced with the task of offering advice on making
the vision of continuous measurement a reality in the 2010 census, the
similarity between the arguments then and now is uncanny. Similar, too,
are the points of concern; the current panel is harct-pressect to improve
upon the basic summary of concerns outliner! by our predecessors. We
are, however, much more sanguine that a compelling case can be macle
for the ACS and that it is a viable long-form replacement in the 2010
census.
In summary, the panel appreciates the enormous potential benefit of
the ACS of having a program for continuous measurement of social
anct demographic variables of key national interest. The ACS presents
a unique source of timely information that coulc! be extremely useful
to public and private planning and that could be used to support more
effective and targeted fund allocation. The potential benefits of the
ACS are self-evident and require little salesmanship. However, what
cloes require fuller justification is how these benefits offset the costs of
OCR for page 99
AMERICAN COMMUNITYSURVEY
99
the program and, more fundamentally, how the program works as a true
long-form replacement. The panel is optimistic that such a compelling
case can be macle, though it will take continucct evaluation work and
research.
Recommendation ACS-1: The Census Bureau should
carry out more research to understand the differences
between and relative quality of ACS estimates and long-
form estimates, with particular attention to measure-
ment error and error from nonresponse and imputation.
The Census Bureau must work on ways to effectively
communicate and articulate those findings to interested
stakeholders, particularly potential end users of the data.
The fact that the Census Bureau has not clone more in comparing
the data collected from the 31 ACS test sites, the C2SS, and the 2001
and 2002 Supplementary Surveys with the data collected by the 2000
census long form is disappointing. Such analyses would help assess the
quality of ACS data and would be helpful in making the argument for
transition from the long form to the ACS. This deficiency is probably
clue to limited analytic resources at the Census Bureau and creates an
argument for "farming out" this analysis to outside researchers. Fur-
thermore, since access to local information is very useful in interpret-
ing the results, the Census Bureau should explore whether local experts
might be interested in assisting in this effort.
Recommendation ACS-2: The Census Bureau should
make ACS data available (protecting confidentiality)
to analysts in the 31 ACS test sites to facilitate the
comparison of ACS and census long-form estimates
as a means of assessing the quality of ACS data as a
replacement for census long-form data. Again, with
appropriate safeguards, the Census Bureau should re-
lease ACS data to the broader research community for
evaluation purposes.
Recommendation ACS-3: The Census Bureau should is-
sue a user's guide that details the statistical implications
OCR for page 100
100
PLANNING THE 2010 CENSUS
of the difference between point-in-time and moving av-
erage estimates for various uses.
Part of a fuller justification of the ACS necessarily involves a cost-
benefit assessment: enumeration of all benefits and costs, measurement
or postulation of the benefits and costs, and comparison with costs and
benefits Including data collection and processing) of the status quo
approach (the census long form). The panel acknowledges that it is
difficult to put a price tag on the value of more timely data, but coming
to terms with cost-benefit tracle-offs is an important part of assessing
the program. Estimates of the possible error in ACS and long-form
estimates as a function of the clateciness of the data need to be factored
into any comparison. This can be clone by adding them to estimates of
mean squared error. Such comparisons will be somewhat approximate
in several respects, but the resulting assessments will be more reflective
of the relative utility of these two sets of estimates.
ACS Funding
The panel looks forwarc! to further discussion on the methoclolog-
ical challenges associated with the ACS but, at this particular time, our
most fundamental recommendations regarding the ACS must be very
pragmatic in nature. In our letter report (National Research Coun-
cil, 2001c), we strongly urged the Census Bureau to make contingency
planning a focus of its planning efforts, with particular attention to the
funding levels for the ACS. The difficulty of securing fiscal year 2003
funcling for the anticipated full launch of the ACS underscores the im-
portance of that recommendation.
Implementation of the ACS woulc! allow the 2010 census to consist
only of the short-form questionnaire, a design feature that is too crit-
ical and too wicle-reaching to leave unresolved until late in the clecacle.
The short-form-only census would facilitate broacler Internet clata
collection and the use of MCDs to collect respondent ciata; it woulc!
reduce the data collection effort and simplify use of multilanguage
forms. A late reemergence of the need for long-form clata collection
woulc! remove any efficiencies the Census Bureau hac! clevelopec! from
its s treamline ct cle sign .
OCR for page 101
AMERICAN COMMUNITYSURVEY
101
Funding for the ACS is, of course, not a decision of the Census
Bureau but of Congress. Accorclingly, in building a compelling case for
the ACS, the Census Bureau neects to work in concert with congres-
sional liaison. The importance of making a decision on general 2010
census structure within the next 2 years early in the clecacle must be
emphasized; the role of the ACS in that structure must be articulated.
Furthermore, it must be stressed that support for the ACS cannot be
erratic; major changes in sample size over the course of the program
could severely compromise use of the ACS as a vital component of a
coorclinatec! set of estimates. The panel is encouraged by statements in
a recent hearing on the ACS that indicate that congressional authorizers
are aware of the importance of making a clear decision regarding ACS
funcling. Specifically, at a May 13,2003, hearing on the ACS, Represen-
tative Adam Putnam (R-FL), the chairman of the House Subcommittee
on Technology, Information Policy, Intergovernmental Relations, anc!
the Census, commented in his opening statement:
I am also very aware that we are rapidly approaching the
point where the Census Bureau neects to know one way or
the other if there will be a long form in the 2010 census
or will the ACS be the new survey tool. It's funclamen-
tal to a successful 2010 Census that we let the Census Bu-
reau know as soon as possible how the Congress expects
the Census to be concluctecI. I'm hopeful that we can con-
tinue to work together to resolve these final remaining is-
sues, anc! that Congress can make a final determination on
full functing for the ACS in the near future.
Given our panel's charge, the most basic question we face is
whether the ACS is a satisfactory replacement for the census long form
(anc! therefore something that shoulc! be the foundation of 2010 census
planning as it has become). We recognize that significant estimation
and weighting challenges must be aclclressecl; the survey's costs, bene-
fits, and uses must also be clearly articulated in order to convince users
anc! stakeholclers of the surveys' effectiveness. However, we clo not see
any looming flaw so large in magnitude that full ACS implementation
should be set asicle.
We therefore encourage full congressional funding of the ACS. It is
important, though, that Congress recognize that funding of the ACS
OCR for page 102
102
PLANNING THE 2010 CENSUS
should be viewoc! as a long-term commitment. As Representative Put-
nam noted in his comments, it is important that Congress send a clear
signal (whatever it clecicles) regarding the ACS. The benefits of the ACS
will be jeoparclizec! if the survey program is faced with oscillating bucI-
get commitments; cuts in functing (anct with them reductions in sample
size) will impair the overall quality of the survey, with first and most
pronounced impact on the ability to produce estimates for small geo-
graphic areas and population groups.
Contingency Planning
In the meantime, the Census Bureau must begin contingency plan-
ning to be prepared should support for the ACS not be forthcoming.
Some possibilities include: reinstitution of the long form in 2010;
implementation of a 1-year ACS (e.g., like the C2SS) to run simulta-
neously but not buncilect with the census; greatly increasing the sample
size en c! revising the content of the Current Population Survey; or
greater use of administrative records supplemented with other survey
ciata. The costs and benefits of these various approaches need to be
clevelopect and presented for review so that decisions on the ACS can
be fully informed. Also, planning neecis to be started on the most likely
of these or other contingencies so that the bureau is well preparecl.l5
Recommendation ACS-4: The Census Bureau should
identify the costs and benefits of various approaches to
collecting characteristics information should support
for the full ACS not be forthcoming. These costs and
benefits should be presented for review so that decisions
on the ACS and its alternatives can be fully informed.
Tithe Office of Inspector General of the Census Bureau's parent agency, the U.S.
Department of Commerce, has expressed similar concerns. "If the Bureau does not
receive sustained ACS funding throughout the decade, it may be unable to eliminate
the long form for 2010"; consequently, the Census Bureau's planning for 2010 should
"include a contingency plan for use of the long form" (U.S. Department of Commerce,
Office of Inspector General, 2002:iv).
Representative terms from entire chapter:
acs data
