Proceedings, Third Workshop Panel to Review the 2000 Census (2001) / Chapter Skim
Currently Skimming:
Proceedings
Proceedings
Pages 1-87
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
I think it is an important meeting. As you all know, the role of our panel is to evaluate the census, but one extremely important aspect of that task is to evaluate census coverage, including any possible disproportionate undercount and to evaluate any steps that the Census Bureau might take to adjust those numbers.
|
From page 2... ...
some of you who have good memories will know that after the 1990 census the term "failed census" was fairly frequently used. It was used loosely, that is, without defining whet constituted a successful and/or a failed census.
|
From page 3... ...
It is extremely clifficult to predict the partisan consequences of adjustment or nonadjustment, and I do not need to talk about all that in this room. The point, however, and this is why this meeting is so very, very important, is we need to bury that phrase just as we needed to bury the phrase that the 1990 census was a "failed census." It is very bad for the federal statistical system and the Census Bureau to live under the clouct that somehow it has the will and the capacity to design a census knowing beforehand what the likely partisan outcome of that census will be.
|
From page 4... ...
will meet our expectations and, indeed, we will evaluate both the census and the Accuracy and Coverage Evaluation early next year to decide which set of data to denominate as the P.L. (Public Law]
|
From page 5... ...
Obviously, that decision will be based on only technical considerations and scientific analysis to the best of our capacity. Now I want to return to the larger theme about Secretary Mosbacher's observation following the 1990 census about the possibility of designing a census to have a known partisan outcome.
|
From page 6... ...
make all data that we use in that decision process available to the public.
|
From page 7... ...
I will give you just one example that John Thompson used in a session the other day. Let us say that we find that demographic analysis suggests that there is a large differential undercount, so we release that.
|
From page 8... ...
The data may have partisan consequences but the Census Bureau is not partisan and I really urge those in the room, especially those on the panel, who may be critics of dual-systems estimation and that is fine, as you all know, we have no trouble with that we really hope that the criticism of clual-systems estimation is articulated on scientific and technical grounds and not because somehow the Census Bureau can manipulate data, which it does not know how to clot As I have said many times publicly, we do not have experts in voting behavior, we do not have experts in redistricting, we wouic! not know how to go about trying to design a census that would have a known partisan outcome, and especially we do not even know who will be running the Census Bureau or which particular party will have appointed him, when this decision is made.
|
From page 9... ...
The census edited file is a big deliverable, because that goes into the computer matching and then the clerical matching for the Accuracy and Coverage Evaluation Howard will probably talk a little bit about that. TaY Waite tells me that we have started the computer matching for the A.C.E.
|
From page 10... ...
We have our clerks trained and they are now doing some practice clusters and everything else, so that we will be able to begin clerical matching as soon as things get through the computer matching pipeline, so we are on a roll here and very happy with where we are.
|
From page 11... ...
There is a document on the quality of the census processes [Baumgardner et al., 20014. There is a document on demographic analysis "Robinson, 20014.
|
From page 12... ...
quality, that is, the various kinds of errors you get in dual-systems estimation matching bias, correlation bias, et cetera; and then part of that review of A.C.E. quality is a discussion of how one might synthesize those errors into an overall measure.
|
From page 13... ...
Also, we have to Took at not just the net national coverage, what can be n~easured by the demographic analysis, for example, but also the gross errors, the omissions, the fictitious and other erroneous enumerations, the level of census imputations, and we have to review census heterogeneity. It is one thing to say in 1990 that the undercount was 1.8 percent net national; how did that distribute by geographical area, by group, by city, by whatever else, how did that heterogeneity result from overcounts or unclercounts, whatever?
|
From page 14... ...
One of the most important things from demographic analysis is, of course, the sex ratios. How do the sex ratios of the census compare to what you wouIc3 expect from demographic analysis?
|
From page 15... ...
Remember the actual demographic analysis, the traditional data series, is black'nonblack, so someone who chooses white and Asian does not affect that, it is only people who choose black and something else. If that is, inclee cl, a large group, then that can really fuzzy the historic series.
|
From page 16... ...
DR. HOGAN: All right, at this point of the analysis we will have at an aggregate level how the census compared to the demographic analysis, the demographic estimates, and the full-count review.
|
From page 17... ...
Had we not done an A.C.E. at all, we would still have monitored the census mailout, the census enumeration process, the census address list compilation processes; we would have had various quality assurance management measures that said how well that census was done before we even dreamt of an A.C.E.
|
From page 18... ...
I see really very much, based on the dress rehearsal experience, that at the end of this is not, yes, it is a great census or it is a super great census, but here is what we know about how the census was conducted, here is a story. From demographic analysis we have a story about how good the census was.
|
From page 19... ...
result in the kinds of patterns that the PES measured. That method of building address lists, by the way, we changed, and I want to point that out.
|
From page 20... ...
South Carolina was the one unadjusted site. Nevertheless, we treate(1 it as if it had to be adjusted within the same time frame, the nine months from Census Day.
|
From page 21... ...
One of the things that I would like you to talk about a little bit more, and maybe that should be for more detailed discussion later, is the whole unduplication process. We could defer that until we get into more detail, because you are giving the overview and I do not want to interrupt that, but I do not want to have that omitted.
|
From page 22... ...
Why don't you tell us a little bit about that, since we have time?
|
From page 23... ...
First, let's assume, using whatever processes they have done in the censuses on cluplicated people, housing units, and whatever else, and we have a file, if after that point there are still duplicates in it, or fictitious things or anything else, but let's focus on duplicates, then we have to, as part of the PES or the dual-systems estimates, as part of the E-sample, specifically, measure the number of duplicates on a statistical basis, based on the sample, and subtract that out from the census count. What we are trying to Lo, as explained in my last talk here, is to, first, try to measure the number of people correctly counted in the census and then we are going to try to measure the proportion of all people who were correctly countecl.
|
From page 24... ...
There is an evaluation of the primary selection algorithm, I believe, on the books. The results of that will not be available until 2003, late 2002, at the earliest, so the kind of careful after-the-fact evaluation to really see if it did what it was intenclec!
|
From page 25... ...
file, for example, into line with true geographic locations, true GPS [Global Positioning System] coordinates, which will then allow us to do a lot of very innovative things and build in the address list and send enumerators out, but this is the king!
|
From page 26... ...
However, there may be a clozen or more things that are going to go into making that decision. We have decided not to treat each of those inclividual components, the variance, the match rate, or the correlation bias as dichotomous decisions; if above this threshold, then, yes, if below that threshold, then, no.
|
From page 27... ...
I agree with what you said about if there is a place that is local, where the census was very bad, A.C.E. will probably be relatively neutral or perhaps improve it a little bit.
|
From page 28... ...
as they existed in January, we notched the housing units from the census to the A.C.E. The reason we clid this was to help us draw our targeted extended search sample to facilitate the matching clerks and also to allow us to identify which housing ~ , ~7 units had already returned a census questionnaire to allow us to start the telephoning, because we did not want to telephone until they returned the census .
|
From page 29... ...
DR. YLVISAKER: I am a little bit confounded with expectations, consistency, and so on, that is, we fire!
|
From page 30... ...
to make sure. There had to have been a housing unit there that was not already on our address file front which we did not already have a response.
|
From page 31... ...
While I think the initial match of the January housing unit file is very hard to interpret, we do have a final housing unit match that will be done as part of the evaluation program. We will start as soon as we are done with people and that will evaluate the quality of the final census address list and, I think, will allow linking back to which process initially put this actress on the list, so as part of that evaluation some of the questions that you are asking might be answered.
|
From page 32... ...
Census Day noninterviews [NIi, then characteristic imputations, how many cases were acTUed to this cell because we imputes] the characteristics and only the characteristics?
|
From page 33... ...
AND UNADJUSTED CENSUS COUNTS COMPARE DR. HOGAN: Let me go on to the next document twe can take these two C3OCuments together)
|
From page 34... ...
Second, it gives you, actually, the base results, the coverage correction factors that will be used, the undercount rates, that one can begin, then, by summarizing them to various levels, to compare to demographic analysis, ant! it gives some measures of uncertainty, the coefficients of variation that one can start analyzing relative to the size of the populations.
|
From page 35... ...
Remember, we have essentially three kinds of missing data in the A.C.E.: we have whole household non-interviews where we got a refusal and could not get an interview; we have cases where we got an interview but we clid not get sex or we did not get age or race, or Hispanic origin; and, finally, we have cases in the P-sample where we could not cletern~ine whether the person lived there on Census Day or, having determined that he/she] lived there on Census Day, we couIct not determine definitively whether he/she]
|
From page 36... ...
The quality assurance program, again, was ciesignecT prin~arily to make sure the people were visiting the right housing units anti getting an interview. They could fait to visit the right housing unit, either
|
From page 37... ...
DR. ZASLAVS KY: There is a concern that Larry has brought to our attention about the potential for the telephone interviewing to introduce correlation bias because of the fact that the only way to be interviewed by telephone is if you returnect the census form, so that to the extent that there is some sort of mode effect associates]
|
From page 38... ...
Obviously, there is a certain type of correlation bias that is associated with the type of person who is in the household but, at least in operational terms, typically in 1990 an interviewer would have gone out to those two housing units not knowing whether anybody had responded. In this situation, now, the unit that responded, there was a phone call, there were attempts made by a phone call and, if that was not successful, then there was an enumerator knocking on the door.
|
From page 39... ...
It could cause correlation bias, but correlation bias occurs only if that response is correlated with response to the A.C.E. If we gave up after telephoning, it certainly would be.
|
From page 40... ...
To the extent that this feeds into correlation bias we pick it up when we start looking at correlation bias specifically in some of the later things, but how to isolate this aspect of what caused the correlation bias from other aspects, the heterogeneity, the telephone- we also had a number of letters we turned over to David Whitford of people saying "I already participated in the census, I'll never participate in your A.C.E.," which is another form of correlation bias in the other direction. We will have evidence of correlation bias at the end of the day.
|
From page 41... ...
DR. PREVVITT: That is a deputy director question.
|
From page 42... ...
file, reclistricting, and Summary File 1; everything else wouIc3 be adjusted and available only adjusted. We would basically be saying the data we support or we denominate as redistricting data are the adjusted data, so that is what we are endorsing.
|
From page 43... ...
It is the first John Long could speak to this much better than I can) , we put out two detailed summary files for short-form data.
|
From page 44... ...
In Summary File 1 we will have the full set of age data for them, but it is basically still the same 1 00-percent characteristics and that level of detail.
|
From page 45... ...
If all you lool~ecT at were variances, you clearly would just choose one. However, I think, at least at one level, you can step back and say, yes, the precise definition of the African American domain for Census 2000 does not con~pletely agree with 1990, but it is close enough that you can compare the variance on the black populations and see if there was a reduction or about the same, or increased, just as I think almost everybody is going to Took at the level estimates for the black population in 1990 and 2000 and ask if they roughly measure the same thing.
|
From page 46... ...
We may come back to this a little bit, but a number of things that I talked about earlier say that the interviewing the matching is a good one, where we have how the interviewing went by census regional offices.
|
From page 47... ...
What we have done so far is we have gone through the census compared to DA [demographic analysis) , we have compared the census operations, and we looked at the A.C.E.
|
From page 48... ...
If, on the other hand, because of the use of proxies or different survey instruments or the difference of respondents, if we ask people on the P-sample, "Where did you live," and they said "Here," but they were counted somewhere else, and we went to the somewhere else ant! asked, on the E-sample, the neighbor, "Did the people live here," and they say, "Yes, he lived here," we wouIcl have an inconsistent reporting of Census Day address.
|
From page 49... ...
An issue was raised at the last meeting, I think it was a very good issue, and we went back and looked at it, which is that we have an imbalance with college students that, because of the timing of the census and the A.C.E. interviewing, we will have far more people who moved into the housing units from dormitories and
|
From page 50... ...
As with all of these things, you balance what you can balance, but to get back to the conversation about the telephone interviews in the A.C.E., when we realized that we were going to be able to get many more cases by phone than we thought we would and we raised the kinds of questions you raised earlier, what are the down sides of that, the huge upside is you are getting data much closer to Census Day, so we thought some of these kinds of errors would be reduced by maintaining the flow of telephone interviews. An example of the larger point, that is, where we are going to have to balance all of this off and we have sort of got it all in front of us.
|
From page 51... ...
One of the big advantages in the DSE literature on procedure C is that the matching is just a lot easier to control, because you have to clear only with people who five cT in that block on Census Day. I think we will have good evidence here on the level of matching error.
|
From page 52... ...
The other 30-or-so percent it spits out to the clerks, the techs, and the analysts, and they are using judgment in terms of whether it is the same housing unit. They actually have and can bring up on the computer screen the census maps with their map spots, the A.C.E.
|
From page 53... ...
DR. YLVISAKER: How about cases where the primary selection algorithm [PSA]
|
From page 54... ...
Those will be done and available in a couple of years but certainly not by March. Missing data now I think we have touched a little bit on this, so I think we can go through this a little bit more quickly.
|
From page 55... ...
1.6, "Balancing Error" hogan, 2001i, let me say a little bit about this, because it is one of those that is not intuitive. Essentially, we want the same rules applied in the P-sample when we say a person was correctly enumerated tin the E-sampleJ.
|
From page 56... ...
We have reintroduced a concern with balancing error that we have to Took at, and we are going to Took at this basically by looking at the results of the targeted extended search, what we found in the surrounding blocks in terms of misses, geocoding errors, what we had to impute in terms of the missing (lata for a targeted surrounding block search to see that our design, our new design, our TES design, is working. That is, as I said, a new design; it was not clone in ciress rehearsal, even, it was not done in 1990, and we have to see if it is working.
|
From page 57... ...
Here, as in the inconsistent reporting the census data addressed, we will have some information in terms of how the follow-up went, the kinds of census errors the follow-up hall, (foes that Took reasonable, is that pattern of census fabrication, census duplication, consistent with what we wouIcl expect?
|
From page 58... ...
I mean, we do get census questionnaires that clearly say Donald Duck, that clearly say Fido. There are smart Decks out there that we do find and identify in the mailback questionnaires.
|
From page 59... ...
, there is a table in which we tried to show one of the ways we hoped to deal with the issue. For demographic analysis, we said before that the race detail that we actually have to work with is blac:k/nonblack.
|
From page 60... ...
Since it fish still too early to know and we have only a little bit of fragmentary information from a few previous studies such as the dress rehearsal and now a little bit from the American Community Survey, it is hard to know what the total amounts are going to be, but there is speculation that particularly for the black and African American population the differences may not be particularly large for those two groups. We will have to wait to see what happens.
|
From page 61... ...
with estimates of components of change, so we have an estimate of the Hispanic population on April 1, 2000, which is a benchmark. One thing we can clo is we can adjust for unclercount; at least 1990 for that population using the PES results, so that will give us a rough benchmark that is not truly independent like national demographic analysis for blacks and nonblacks, but we can use that to roughly assess how close are the census results and the A.C.E.
|
From page 62... ...
may give us some information to the overall assessment of results, and we can do it, but it is certainly not part of the main tables in this prototype report. We really do not show anything specific for Hispanics, because that does not have the same independent basis of the national demographic analysis, which goes back to 1 93 5 for births and deaths and then for Medicare we use for the population over 65, essentially independent of the census we are evaluating but, true, for the Hispanic benchmarks it is based on 1990 with the adjustment carried forward, so it has problems in measurement of the population of 1990.
|
From page 63... ...
or the census, it is not to say which one of them is wrong but it is a flag to say that there is something wrong here and it may be demographic analysis but it may be the A.C.E. In that sense it is one more arrow in the quiver and even as problematic as the other race pieces are carried forward from 1990, they still provide what may be potentially valuable indicators of problems in one data system or the other.
|
From page 64... ...
and their differences, that demographic analysis can be useful to point out differences but that one needs to be rather carefuT in interpreting them and, certainly, they would not be a method for adjustment by themselves. Is that right?
|
From page 65... ...
That is where we would, I think, go to the most differences. I think demographic analysis, as has been pointed out, has much less to say about the Hispanic population than it does about the total population and the black population, but it can say a little bit.
|
From page 66... ...
It would be surprising if somehow we ran a PES where it disappeared. We measured this in 1990 in comparison with the demographic analysis, specifically the demographic analysis sex ratios, although we just had the discussion on the DA that there are some limitations and uncertainty there If you accept the DA sex-ratio estimates, you can measure the correlation bias, at least among mates, by taking the A.C.E.
|
From page 67... ...
What we do not know is how to measure it, except by comparison to an outside standarcl, such as demographic analysis. I think we have a good understanding of correlation bias, but that is different from being able to measure it precisely.
|
From page 68... ...
The other thing that I think is really worth pointing out with synthetic error is it is there before we clo the A.C.E., before we do the adjustment. Matching error was not there before we clid the A.C.E., there is no matching error in the census; if there is error in the A.C.E.
|
From page 69... ...
Then they do post-stratum level estimates and they carry that back down locally, and then you can actually compare the undercount that you got for Ventura County or for Georgia together with the synthetic estimates YOU Ret from the carrying-down process. , O DR.BELL: But they are not using the undercount coverage correction factors, they are using, in essence, a substitution rate correction factor, right?
|
From page 70... ...
On the other hand, you coup have a correlation between the census and the A.C.E. capture probabilities, that caused correlation bias, but if that heterogeneity in the census was not geographically localized but had a little bit of this group ant!
|
From page 71... ...
It would not tell you anything specific to poststrata, because I do not think you could disaggregate clown to post-strata at that level of geography, but perhaps by saying, okay, the direct estimates of population are systematically higher than what the synthetic estimate is saying for this cluster of 20 or 30 blocks that would make up a congressional district and, for this cluster of 20 or 30 blocks it is consistently Tower.
|
From page 72... ...
Contamination error: this is if, somehow, having conducted the A.C.E. listing and whatever else in the block, it means that within the sample blocks the census gets conducted differently, that has been a concern.
|
From page 73... ...
it a Tot more for Hispanics and somewhat less for whites and a little bit more for Asians, how did that improvement impact the various groups that we are going to have to assess accuracy over.
|
From page 74... ...
Related to the total error moclel, or what I call synthesizing the quality of the A.C.E., is how do you compare that to the census? In 1990 we had the Toss function analysis, where we had models at all sorts of levels assuming with and without correlation bias for states, for large cities, for cities of 100,000 I forget how else we split it we had model after model, squared error versus absolute error, went through all of them and had a number of results there.
|
From page 75... ...
Loss functions after that might be of some use.
|
From page 76... ...
~ ~ — — ~ ~ , ,, ~ The real decision, in my mind, is you take what this says and the uncertainty around this kind of analysis and do you believe the targets or do you not believe the targets? You compare that with demographic analysis an(l, again, do you believe demographic analysis, (lo you not?
|
From page 77... ...
DR. STOTO: One way to avoid the difficulty of the loss function is not to use a loss function but to think about patterns.
|
From page 78... ...
We also have the multiplicity estimator for the service-based enumeration, an estimator where we show up at, say, a homeless shelter it is a little bit more complicated than this. We show up at a homeless shelter and we ask who is here now and how often do you use homeless shelters.
|
From page 79... ...
DR. VVELLS: But it seems that the A.C.E., in some of the evaluations it is just not clear, the correlation bias, synthetic bias, various things.
|
From page 80... ...
We do not even try to measure the response error in the census and the synthetic error in the census or the correlation bias in the census before we apply the results for apportionment. We are putting the A.C.E.
|
From page 81... ...
I think Howard mentioned the propylene, it has come up in a couple of ways. When we run the primary selection algorithm we decide that this is the census.
|
From page 82... ...
DR. BAILAR: I think in previous census years, where we thought about the correction or adjustment of the census, we have used only one measure of the accuracy of the census and that came from demographic analysis, so we were looking at primarily was there an undercount, was there a differential, so I was glad to see some emphasis on looking at the errors in the census itself.
|
From page 83... ...
I think you have some ideas about that and maybe that could not be formally prespecified, but the rationale for looking for certain kinds of things could be articulated a little bit more explicitly before the process gets under way. The other comment I would make is that I do not want I will be the advocate for not downplaying the quantitative comparisons and loss functions and those types of analyses.
|
From page 84... ...
The second thing is, I think that one of the questions that we really have not addressed today, or maybe when I was not here was how we are thinking about the decision process. Sometimes we were talking about being able to prove that the A.C.E.
|
From page 85... ...
It is extremely valuable to have had the panel as a part of this process over the last year and a half, extremely important to have the Academy have the 2010 committee to begin to move from where we already are in terms of how we will approach 2010. It is appreciated and, obviously, we will all read your report carefully.
|
From page 86... ...
Obviously, we go into it on the assumption that it will make an improvement, but if it turns out that we think it clic! not, then we will simply either it dicI not because it cTid not pan out the way we hope it will pan out or because it is not necessary, because, To and beyond, we diet count 100 percent of the American population (cloubtful, really doubtful, because I got too many letters from people saying, "No matter what you do, you are never going to count me," and some of whom actually sent in one hundred!
|
From page 87... ...
Thomas Jefferson were among the first to say that it was not enough people. It is interesting that instead of the argument being among the various states or even within the various states, the argument was between the United States and the other countries of the world in trying to show that we really were a growing country with growing power and, in order to have growing power, we had to have more people.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.