2
Sampling Frame Development and Maintenance

The sampling frame for the American Community Survey (ACS) is based on the Master Address File (MAF), which is a database maintained by the Census Bureau and used to conduct the decennial census and supports some of its major survey programs. The MAF is an inventory of known housing units, group quarters, and selected nonresidential units in the United States. MAF records contain mailing address, location information, and additional attributes of each residence.

The Census Bureau developed the MAF in preparation for the 2000 census with the intent of keeping it updated as a continuous resource; previous censuses built their address lists anew each decade. During the past decade, the primary sources of MAF updates have been regular “refreshes” from the U.S. Postal Service Delivery Sequence File (DSF): which, as the name implies, is the Postal Service’s inventory of mail delivery points. Some updates have also been generated by clerical operations, such as the Master Address File Geocoding Office Resolutions, and by field observations, whether through the Community Address Updating System in primarily rural areas or through feedback from ACS nonresponse follow-up attempts.

Most of the procedures developed to maintain the MAF are focused on keeping the inventory of housing units current. The strategy for updating the inventory of group quarters (GQ) facilities is less comprehensive, and the sources used for updates related to them are less adequate.

To develop a GQ sampling frame for the ACS, the ACS office begins with MAF extracts, including an inventory of group quarters from the most recent census and a list of group quarters closed on Census Day to be checked again during the ACS fieldwork. Information from these files is then supplemented with a list of federal prisons and detention centers obtained from the Federal Bureau of Prisons. The Census Bureau also uses its own Internet queries to update a variety of group quarters, such as state prisons and migrant worker camps. As feasible, the Census Bureau also relies on military liaisons to update the list of military GQ facilities.

Information about the group quarters selected into the sample is further updated with data obtained from field representatives, who complete a Group Quarters Facility Questionnaire during their initial visit to a facility. Problems encountered during field visits are researched by headquarters staff, and this research often provides new information about the status or location of a facility. The updates resulting from the fieldwork are used to update the ACS GQ universe for the following year’s sample.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 9
2 Sampling Frame Development and Maintenance The sampling frame for the American Community Survey (ACS) is based on the Master Address File (MAF), which is a database maintained by the Census Bureau and used to conduct the decennial census and supports some of its major survey programs. The MAF is an inventory of known housing units, group quarters, and selected nonresidential units in the United States. MAF records contain mailing address, location information, and additional attributes of each residence. The Census Bureau developed the MAF in preparation for the 2000 census with the intent of keeping it updated as a continuous resource; previous censuses built their address lists anew each decade. During the past decade, the primary sources of MAF updates have been regular “refreshes” from the U.S. Postal Service Delivery Sequence File (DSF): which, as the name implies, is the Postal Service’s inventory of mail delivery points. Some updates have also been generated by clerical operations, such as the Master Address File Geocoding Office Resolutions, and by field observations, whether through the Community Address Updating System in primarily rural areas or through feedback from ACS nonresponse follow-up attempts. Most of the procedures developed to maintain the MAF are focused on keeping the inventory of housing units current. The strategy for updating the inventory of group quarters (GQ) facilities is less comprehensive, and the sources used for updates related to them are less adequate. To develop a GQ sampling frame for the ACS, the ACS office begins with MAF extracts, including an inventory of group quarters from the most recent census and a list of group quarters closed on Census Day to be checked again during the ACS fieldwork. Information from these files is then supplemented with a list of federal prisons and detention centers obtained from the Federal Bureau of Prisons. The Census Bureau also uses its own Internet queries to update a variety of group quarters, such as state prisons and migrant worker camps. As feasible, the Census Bureau also relies on military liaisons to update the list of military GQ facilities. Information about the group quarters selected into the sample is further updated with data obtained from field representatives, who complete a Group Quarters Facility Questionnaire during their initial visit to a facility. Problems encountered during field visits are researched by headquarters staff, and this research often provides new information about the status or location of a facility. The updates resulting from the fieldwork are used to update the ACS GQ universe for the following year’s sample. 9

OCR for page 9
OPPORTUNITIES FOR INCREASED COLLABORATION TO IMPROVE THE GQ INVENTORY Despite the efforts described, the various updating operations are not as efficient at maintaining the GQ sampling frame as are the procedures for maintaining the household frame. A major inefficiency in the Census Bureau’s address update operations is that there is only limited collaboration among the divisions to integrate address updates and corrections resulting from work related to individual programs. The Census Bureau has been working on securing a budget and establishing procedures that will allow for more thorough updating of the MAF and the closely associated Topologically Integrated Geographic Encoding and Referencing (TIGER) digital mapping system. This is also the right time to consider a more integrated, agency- wide approach toward the MAF as a complete inventory of living quarters, both “traditional” housing units and group quarters. Some units in the Census Bureau have long-standing partnerships with states and localities and rely on these for local information. However, these partnerships are often established on the basis of the needs of a specific program, without maximizing coordination with other Census Bureau units that may have similar needs. For example, the Federal-State Cooperative for Population Estimates program involves states in assisting the Census Bureau’s Population Estimates Program (PEP) to produce subnational population estimates. The State Data Center program is another partnership between the Census Bureau and the states, which facilitates the dissemination of data and other assistance to meet local needs. Many state partners supply information related to group quarters in a variety of nonuniform formats, and they could possibly be doing more to assist the ACS program needs if efforts were better coordinated across the different Census Bureau units. Recommendation 2-1: The Census Bureau should establish a mechanism for ensuring that information useful for the updating of living quarters, especially GQ living quarters, from existing state and local partnerships is more efficiently shared among different Census Bureau divisions and programs. This effort should begin with a review of possibilities for information sharing to improve the quality of the Master Address File, followed by the development of an operational plan for improved information flow and cross-unit sharing. Working more closely with a large number of states and localities will present challenges. Establishing formal agreements with the approximately 39,000 functioning local governments, or even a subset of them, would be a major undertaking. Data availability varies greatly across local sources, and processing and standardizing these data may involve substantial resources. Perhaps a more practical alternative to consider would be heavier reliance on state demographic offices that maintain their own inventory of group quarters, some using them to generate their own estimates for state and local geographic areas. These offices could supply lists of facilities or estimates of the GQ populations as part of a formal program. These types of agreements may be particularly 10

OCR for page 9
useful in large urban areas, where there are more GQ facilities with complex living arrangements (Goldenkoff, 2010). Recommendation 2-2: The various units in the Census Bureau should collaborate to undertake an investigation of opportunities that may exist to obtain more detailed records or state-generated estimates of group quarters from state demographic offices and other state and local partners. Given that many group quarters operate as licensed establishments, the inventory of them could be improved by collaboration with the Census Bureau’s economic statistics directorate and using the North American Industry Classification System, which classifies business establishments. The panel is aware that an effort to explore these opportunities is currently under way. Recommendation 2-3: The ACS office should continue to collaborate with the Census Bureau’s economic programs to explore the possibility of using business registries for additional information regarding GQ facilities. ACS efforts to collect detailed information from GQ populations are not unique in the federal statistical system. For example, the Current Population Survey includes residents of noninstitutional group quarters in the sample (even though uses different residence rules).Targeted surveys of specific GQ types exist as well. For example, the National Center for Health Statistics conducts a national survey of nursing home residents and is launching a new survey of residential care facilities. The Bureau of Justice Statistics regularly conducts administrative censuses of correctional facilities of various types and administers surveys to their occupants, and the Office of Juvenile Justice and Delinquency Prevention does the same for juvenile facilities. The National Center for Education Statistics collects data from students, including residents of college dorms. The data collected by these agencies, and possibly others, may be useful to complement the Census Bureau’s efforts to maintain and update the sampling frames for certain types of group quarters, especially given that the Census Bureau often serves as the data collection contractor for other agencies sponsoring studies of these populations. Closer collaboration with the Census Bureau would also be beneficial to other agencies that have to invest significant resources in maintaining the sampling frames for their surveys. Recommendation 2-4: The Census Bureau should continue to explore opportunities for partnering with other federal statistical agencies that collect data from GQ populations and identify opportunities that could lead to a more efficient updating of the GQ inventory. 11

OCR for page 9
INTEGRATING UPDATES FROM FIELD REPRESENTATIVES The processes currently in place for updating the GQ inventory are more likely to identify and remove out-of-business or out-of-scope records from the sampling frame than to locate and add new records, creating the impression that the GQ population is shrinking. Some of the additions are found to be ineligible after they are added to the sampling frame, and sometimes the reason is that the address is in fact a housing unit. Anecdotal evidence suggests that it is not unusual for housing units to be converted to certain types of group quarters, such as group homes for adults, and then back to housing units over relatively short periods of time. Complex housing arrangements in some urban areas can include apartment buildings of conventional housing mixed with small group quarters—for example, for populations with special needs. By the end of 2010, the ACS computer-assisted personal interview (CAPI) instrument used for interviewing households will have new functionality to allow field representatives who are assigned to interview housing units to also collect basic information (such as address, contact information, GQ type, and maximum occupancy) about group quarters that have been identified in the housing unit sample. This will greatly increase the efficiency of updating operations and should also reduce costs. Enabling field representatives to collect information about a housing unit that was included in the GQ sample has also been considered. Recommendation 2-5: The Census Bureau should identify ways to improve and expand procedures for integrating updates received from field representatives into the GQ frame. An operations plan needs to be constructed that permits the transfer of information from field workers in the regional offices to the geography division at headquarters, so that new GQ facilities can be added to the address list and changes in status of existing addresses can be reported—for example, by removing facilities that no longer exist and adding newly converted facilities. The Census Bureau should also continue to pursue the development of procedures that will allow for more efficient updating of the household sample with cases that have been converted from group quarters to housing units. SCOPE OF COVERAGE Due to difficulties described above associated with maintaining the sampling frame, the GQ sample contains a relatively high percentage of ineligible cases as identified during the facility-level data collection phase. This includes cases that are determined to be housing units instead of group quarters and group quarters that no longer exist—for example, because the facility has been closed. The ACS also classifies some GQ types as permanently out of scope and excludes them from the sampling universe because of operational limitations, privacy concerns, and legal requirements, which are different for a continuous survey than for the census. The excluded GQ types are domestic violence shelters, soup kitchens, regularly scheduled mobile food vans, targeted nonsheltered outdoor locations, crews of commercial maritime vessels, and natural dangerous encampments. 12

OCR for page 9
An additional challenge is introduced by a combination of the data collection methodology and the seasonal nature of some group quarters. Examples are college dormitories or military facilities, which may have no residents during some of the data collection months when they are included in the sample. As discussed later, sampled GQ facilities are randomly assigned an interview month throughout the course of the year, and the same facility may be in the sample during more than one month. Table 2-1 summarizes the distribution of the main GQ-level outcome codes for those included in the 2008 ACS sample based on an internal evaluation of the sampling frame conducted by the Census Bureau (Williams, 2010). A facility case is considered completed at the GQ level if a field representative collects basic information and a resident roster from the facility. Once the names of the residents are collected, the actual respondents can be sampled and approached for an interview. TABLE 2-1 Distribution of GQ-Level Outcomes for Facilities Sampled in 2008 Number in Percentage Sample (unweighted) GQ-Level Outcomes Completed 13,610 76.4 Refusals, unable to locate, and other nonresponse 33 0.1 Eligible but unoccupied at the time of survey 1,694 9.5 Ineligible: 2,482 13.9 No longer exists 928 5.2 Converted to housing unit 1,049 5.9 Domestic violence shelter 17 0.1 Out of scope 488 2.7 SOURCE: Williams (2010). While inability to locate a sampled facility or refusal to participate do not seem to represent serious problems for the ACS GQ sample, eligible facilities that are unoccupied at the time of the survey and ineligible cases make up close to one-quarter of the sample. Tables 2-2 and 2-3, also based on the Census Bureau’s internal research, show that the rates of cases that fall into one of these two categories differ considerably by GQ type and size. The Census Bureau stratifies facilities by size: the “small” stratum includes group quarters with 15 or fewer residents, as shown on the frame, and the “large” stratum includes those with more than 15 residents. Table 2-2 shows that the GQ types with the highest rates of noninterviews because the facility is unoccupied at the time of the survey are college housing, military facilities, “other health care facilities,” homeless shelters, and “other noninstitutional facilities.” In the case of college dorms, one out of four of the large dorms sampled is not interviewed because the facility is unoccupied at the time of the survey. Table 2-3 illustrates that the sample includes many ineligible cases as well, especially among small group quarters. For example, approximately half of the small military facilities, homeless shelters, and nursing homes are found to be ineligible. The relatively high rates of ineligible and eligible but unoccupied cases raise concerns about overcoverage, in addition to the previously discussed undercoverage issues related to deficiencies in updating the GQ inventory. The ACS sample estimates 13

OCR for page 9
are controlled to population estimates obtained from the PEP program. However, as discussed in Chapter 4, these population estimates have their own limitations and are unlikely to be able to compensate for all of the problems discussed above. The significant effort spent in the field pursuing GQ facilities that cannot lead to interviews warrants a closer look from a cost-benefit perspective. There may exist procedures that could be put into place to improve the frame before sampling, or at least before cases are assigned to field representatives. This should be evaluated in the context of the relative costs of the additional research that will have to be conducted in-house compared with the costs associated with closing out the cases in the field. It is also important to consider how the two methods compare in terms of the quality of the information available to make a determination about a facility’s status—in other words, whether one method or the other is less prone to error. In some cases there is a significant lag between the time when updates to the MAF are received and the time the sample is generated. To reduce the percentages of group quarters that no longer exist or have been converted to housing units in the sample, this lag time should be examined to identify possible opportunities for increased efficiency. Another GQ facility type with relatively high rates of no residents is military facilities. When address updates are received by the Census Bureau, the challenge often becomes matching information from the different sources and identifying potential duplicates. More information about the quality of the updates received from such sources as the Department of Defense and the Defense Manpower Data Center (DMDC) is needed to assess whether replacing outdated lists of military facilities with updates from these sources, without spending additional resources on matching and reconciliation, may be justified. 14

OCR for page 9
TABLE 2-2 Distribution of Eligible But Unoccupied Rates by GQ Size for Facilities Sampled in 2008 Number of Sample Percentage of Cases Cases Unoccupied But Eligible Number in Percentage of Large Small Large Small Sample Sample (>15) (≤15) (>15) (≤15) GQ Type Correctional facilities for adults 3,482 19.4 3,373 109 1.1 5.5 Juvenile facilities 330 1.8 241 89 7.5 6.7 Nursing facilities/skilled nursing facilities 4,256 23.8 4,075 181 0.5 1.7 Other institutional facilities 474 2.6 348 126 9.5 22.2 College/university student housing 4,872 27.2 4,672 200 25.2 17.0 Military group quarters 790 4.4 697 93 16.9 15.1 Emergency and transitional shelters 557 3.1 390 167 4.1 13.8 Group homes intended for adults 2,279 12.7 955 1,324 1.9 2.6 Other noninstitutional facilities 872 4.9 356 516 14.3 11.8 Total 17,912 100 15,107 2,805 9.8 7.5 SOURCE: Based on tabulations provided by the Census Bureau, August 11, 2010. 15

OCR for page 9
TABLE 2-3 Distribution of Ineligible Rates by GQ Size for Facilities Sampled in 2008 Number of Sample Percentage of Cases Cases Ineligible Number in Percentage of Large Small Large Small Sample Sample (>15) (≤15) (>15) (≤15) GQ Type Correctional facilities for adults 3,482 19.4 3,373 109 2.8 13.8 Juvenile facilities 330 1.8 241 89 14.5 22.5 Nursing facilities/skilled nursing facilities 4,256 23.8 4,075 181 10.3 48.6 Other institutional facilities 474 2.6 348 126 20.1 28.6 College/university student housing 4,872 27.2 4,672 200 5.1 30.5 Military group quarters 790 4.4 697 93 16.5 55.9 Emergency and transitional shelters 557 3.1 390 167 28.2 52.1 Group homes intended for adults 2,279 12.7 955 1,324 34.3 35.3 Other noninstitutional facilities 872 4.9 356 516 20.5 39.1 Total 17,912 100 15,107 2,805 8.3 36.6 SOURCE: Based on tabulations provided by the Census Bureau, August 11, 2010. 16

OCR for page 9
SAMPLE REDESIGN OPTIONS An overall question to consider is whether the sampling design—which relies on two separate samples, one for housing units and one for group quarters—is efficient in the case of every GQ type. In addition to the issue of duplication between the two frames, some of the living quarters move back and forth between them. For example, as mentioned earlier, some group quarters are converted to housing units, then back to group quarters, in the course of relatively short periods of time. Currently, if a group quarter is converted to a housing unit or if a housing unit turns out to be group quarter, the case is closed out and an interview is not completed. Greater field efficiencies might be derived with alternative procedures. Small group quarters are disproportionately more likely to be converted to housing units by the time a field representative visits the premises than larger facilities, and some GQ types, such as homeless shelters and juvenile detention centers, are also more susceptible to this type of change (Williams, 2010). The Census Bureau should evaluate whether redesigning the sample based on either GQ size or type would improve the efficiency of the data collection. As discussed in Chapter 3, small and large group quarters are not sampled the same way, which raises the question of whether it would also be worthwhile to continue research on whether a cutpoint other than 15 or fewer for the expected number of residents would be more efficient for defining “small” facilities. A model relating an indicator of being in scope to the measure of size may be helpful in this regard. Possibilities for redesigning the sample would include dropping small group quarters (or certain types of them) from the GQ sample or adjusting the data collection procedures to make it possible to interview residents of these facilities as part of the household data collection. In the latter case, for these living quarters, a determination about whether they are a household or a GQ facility would be made only during the interview rather than in advance. Naturally, this approach is more appropriate for the noninstitutional group quarters, and field representatives would not encounter some types at all while working a primarily household sample. For example, they would be unlikely to come across a large correctional facility. Integrating these two data collection efforts would be a substantial undertaking, and the subsampling procedures would have to be applied on the fly while the facility interviews are in progress. Nevertheless, there could be cost savings if the process involving these cases could be simplified. The Census Bureau could retain the goal of covering virtually the entire U.S. population without having to maintain a sampling frame for some of the difficult-to-cover sizes of group quarters. It is also possible that this approach would improve the household coverage rate. An alternative option would be to drop certain types of group quarters (e.g., homeless shelters) from direct sampling instead of dropping group quarters of a certain size. Given that changes in GQ size are likely to be more frequent than type changes, this design may be more practical. GQ types that are especially likely to be found ineligible after a field visit or that may be characterized by seasonality—such as group homes intended for adults and other noninstitutional group quarters—may also turn out to be most likely to be picked up in the household sample if the procedures could accommodate these types of cross-sample interviews (U.S. Census Bureau, 2009). In summary, the two redesign options discussed above are: 17

OCR for page 9
1. Pick up some segments of the GQ universe only through the household sample. The segments could be defined by size or type. These segments would be represented in estimates of the total population, but the samples would probably be too small to produce reliable results for the group quarters population separately. 2. Completely exclude some GQ segments from the ACS universe. Both of these options have the advantage of eliminating the need to maintain a sampling frame for certain segments of the universe. The first option would require that field representatives be able to conduct a household or GQ interview, depending on what is found in the field, or that a field representative refer a facility to the central office for interviewing at a later time. These changes would have to be carefully considered in the context of data user needs, especially if the GQ estimates are important for calculating total population estimates. However, as discussed, a small number of GQ types are already excluded from the ACS data collection for a variety of reasons, and ACS estimates are controlled to be consistent with the PEP estimates for all group quarters (U.S. Census Bureau, 2009). Excluding more facilities would surely raise new concerns, but it is important to evaluate whether some are contributing disproportionately more to the cost of the data collection than the use of the data justifies. As discussed above, other sources that could provide population counts of the residents of certain types of group quarters should also be considered when evaluating this issue. Recommendation 2-6: The Census Bureau should conduct a general evaluation of the reasons for the relatively high rates of ineligible and eligible but unoccupied group quarter facilities in the ACS sample and determine whether there are practical ways to reduce these rates for all or some group quarters. Recommendation 2-7: The evaluation should take into consideration the costs associated with determining that a facility is ineligible or unoccupied and how these costs would change if some of the work were performed before a case is sent to the field. 18