Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 83
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary 9 Health Insurance Coverage in the American Community Survey: A Comparison to Two Other Federal Surveys Joanna Turner and Michel Boudreaux University of Minnesota, State Health Access Data Assistance Center With the passage of national health care reform legislation, there is a growing need for in-depth information on health insurance coverage across key population subgroups and across state and local areas. The American Community Survey (ACS), which added a question about health insurance coverage in 2008, has the potential to serve as a major information resource to support the implementation and evaluation of health care reform at the federal, state, and local levels. This paper describes the ACS health insurance data, discusses some of the methodological issues that arise in collecting them, and shows key estimates from the ACS compared with estimates from other federal surveys, paying particular attention to estimates for children. In comparing the ACS estimates with other data sources, we also pay particular attention to the impacts of differences in question design and data processing of the estimates. The paper begins with an overview of the ACS, including a discussion of the health insurance question. The next section compares key uninsurance estimates from the ACS with estimates from the Annual Social and Economic Supplement to the Current Population Survey (CPS ASEC) and the National Health Interview Survey (NHIS), briefly discussing challenges in collecting health insurance coverage in the surveys and the likely implications of differences in data processing. The final section summarizes the implications of our findings for the role of the ACS in tracking insurance coverage over time. This paper recaps the preliminary analysis of Turner, Boudreaux, and Lynch (2009), building on that work to
OCR for page 84
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary further explore differences among the surveys. The focus here is on providing an overview for analysts who may not be familiar with the ACS, the CPS ASEC, or the NHIS estimates of health insurance coverage. AMERICAN COMMUNITY SURVEY The ACS is a nationwide survey designed to collect and produce economic, social, demographic, and housing estimates on an annual basis. It is conducted in all U.S. counties and Puerto Rico municipios and participation is required by law. About 3 million housing unit addresses are sampled annually throughout the United States and Puerto Rico. There are separate housing unit (HU) and group quarters (GQ) samples. GQ include nursing homes, correctional facilities, military barracks, and college/university housing, among others. The sample coverage of the ACS is different from other surveys that gather information about health insurance coverage. For example, neither the CPS ASEC nor the NHIS samples institutional GQ, residents of Puerto Rico, or active-duty military members. Furthermore, neither the CPS ASEC nor the NHIS uses a sample that draws from every county equivalent across the United States. ACS data are collected continuously using independent monthly samples. The ACS uses three modes of data collection for HUs. All sampled households are mailed a paper survey. All mail nonrespondents are followed up with an attempt for a telephone interview, and a sample of roughly one in three telephone nonrespondents is followed up with personal visits. The nonresponse interviews are conducted using computer-assisted instruments—computer-assisted telephone interviewing (CATI) or computer-assisted personal interviewing (CAPI).1 Respondents living in GQ facilities complete their forms using a different procedure based on the size of the facility. Some respondents fill out the paper form, and some forms are completed by field representatives. The application of sequential modes in tandem with mandated responses leads to a high unit response rate. The official response rate is derived by estimating the number of completed interviews over the estimated number of units that should have been interviewed. This procedure omits units that are ineligible for the survey (such as businesses) and those that did not respond to the mail or phone interview but were not sampled for personal visit follow-up. In 2008, the response rate was 98 percent.2 This approach to calculating the response rate has appealing 1 Telephone nonresponse follow-up is conducted through CATI, and personal visit nonresponse follow-up is conducted through CAPI. 2 Available: http://www.census.gov/acs/www/acs-php/quality_measures_response_2008.php [May 2010].
OCR for page 85
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary qualities when using the response rate as a proxy for nonresponse bias. However, alternative measures of the response rate may also be helpful. The Census Bureau publishes the number of originally sampled units and the number of final interviews for each state. For the United States as a whole, 2.8 million HU addresses were selected, and 1.9 million addresses completed an interview.3 The Census Bureau publishes ACS single-year estimates for areas with populations of 65,000 or more, 3-year estimates for areas with populations of 20,000 or more, and 5-year estimates for all statistical, legal, and administrative entities. The health insurance coverage data and all new content added to the 2008 questionnaire will have the first 3-year estimates released in 2011, based on 2008-2010. The first release of 5-year estimates will be in 2013, based on 2008-2012. Multiyear estimates from the ACS differ from average annual estimates from the CPS ASEC. The multiyear data from ACS represent a period estimate derived from data collected for 3- and 5-year periods.4 The CPS ASEC multiyear estimates are 2- and 3-year averages, but they are not based on pooled data. Thus, while the ACS and the CPS employ a conceptually similar moving-average concept, they are distinct.5 Health Insurance Coverage Question The ACS questionnaire has two sections. In the housing characteristics section, the respondent answers questions for the household. In the personal characteristics section, the respondent answers a set of person-level questions for each member of the household. The health insurance coverage question is asked for each person in the household. The respondent is instructed to report each person’s current coverage by marking “yes” or “no” for each of the eight coverage types listed (labeled as subparts a to h). The question text is reproduced below: Is this person CURRENTLY covered by any of the following types of health insurance or health coverage plans? Mark “Yes” or “No” for EACH type of coverage in items a–h. 3 Retrieved on May 13, 2010, from: http://www.census.gov/acs/www/acs-php/quality_measures_sample_2008.php [July 2010]. 4 More detail on the interpretation of ACS estimates is available in “Statistical Issues of Interpretation of the American Community Survey’s One-, Three-, and Five-Year Period Estimates” at: http://acsweb2.acs.census.gov/acs/www/Downloads/MYE_Guidelines.pdf [July 2010]. 5 Available at: http://www.census.gov/acs/www/Downloads/JSM2007_Beaghen_Weidman.pdf [July 2010].
OCR for page 86
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Insurance through a current or former employer or union (of this person or another family member) Insurance purchased directly from an insurance company (by this person or another family member) Medicare, for people 65 and older, or people with certain disabilities Medicaid, Medical Assistance, or any kind of government-assistance plan for those with low incomes or a disability TRICARE or other military health care VA (including those who have ever used or enrolled for VA health care) Indian Health Service Any other type of health insurance or health coverage plan. The respondent is asked to write in the type of coverage for household members reported to have another type of health insurance or health coverage plan in item h. The health insurance question is intended to capture comprehensive plans.6 Plans that cover only specific health services, such as dental plans, or are limited to coverage due to an accident or disability are not considered health insurance coverage. Furthermore, it is important to note that subpart d intends to capture all public health insurance programs and is not just an estimate of Medicaid coverage. Missing responses to the question subparts a to g were assigned a “yes” or “no” response through editing and hot-deck imputation. During the editing process, write-in answers describing or naming the type of “other” health insurance or health coverage plan in subpart h were classified into one of the first seven categories. Hence, only the first seven types of health coverage are part of the microdata file; subpart h and the write-in are not included. Using the complete edited data, people were considered insured if they had a “yes” in at least one of the coverage types: employer- or union-based plan; a private plan purchased directly; military health care; Medicare, Medicaid, or other public programs; or veterans (VA) health care. People who had no reported health coverage or whose only health coverage was Indian Health Service are considered uninsured. Indian Health Service alone is not considered comprehensive coverage (State Health Access Data Assistance Center, 2005). The types of health insurance are not mutually exclusive; people may be covered by more than one type at the same time. 6 A guide to help the respondent complete the survey form is provided along with the paper questionnaire.
OCR for page 87
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Item Completeness and Imputation Rates This section examines patterns of nonresponse and imputation rates for the 2008 ACS. Respondents with complete item response had a “yes” or “no” to each of the first seven types of coverage (subparts a through g on the mail questionnaire). Respondents with no complete items had neither a “yes” nor a “no” to all seven items. The remainder had partial response, meaning that the respondent had at least one item with “yes” or “no,” but not all. This analysis of nonresponse does not include write-in responses and excludes respondents that were sampled in 2007 and returned their paper survey in 2008.7 The percentage of people with responses (either “yes” or “no”) to all, some, and none of the seven item subparts is presented in Table 9-1. Across all modes, 73.0 percent of people had a “yes” or “no” response to each item; 23.2 percent responded to at least one but not all items, and 3.8 percent left all the items blank. This varied by mode, with mail respondents the least likely to provide complete item response, at 51.8 percent. In the GQ population, 81.0 percent had complete health insurance data. People in HUs interviewed by telephone (CATI) or in person (CAPI) were the most likely to give complete item response, at 96.1 percent. This pattern reflects both differences in the instruments and differences in the composition of people in each mode. In addition to classifying the write-in responses, the editing process applied logical edit rules. If a respondent marked “yes” to one and only one of the types and all other subparts were left blank, the types associated with the blanks were assigned values of “no.” For example, a respondent marked “yes” for employer-provided coverage (subpart a) and left the rest blank. The edited final response for that person would be a “yes” for employer- or union-based coverage and a “no” to all of the others: direct purchase, Medicare, Medicaid, military health care, VA, and Indian Health Service. The assumption was made that if a respondent checked one of the types of coverage as “yes” and left the rest blank, that these blanks were implied “no’s.” This process turned some partial responses into complete responses, and they were not considered imputed. This editing choice was the result of analysis of the pattern of responses to the paper form. Table 9-1 also presents the weighted allocation rate—the percentage of people who had an answer to at least one of the health insurance types obtained through hot-deck imputation. In the population considered, 9.7 percent had at least one health insurance variable imputed. 7 Just over 27,000 forms, although included in the 2008 data, actually used the 2007 instrument. Thus, they could not answer the health insurance question and their values were fully imputed.
OCR for page 88
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary TABLE 9-1 Item Nonresponse and Imputation Rates for ACS Health Insurance Coverage, 2008 Universe: U.S. Population, All People Who Responded to a 2008 Questionnaire All Modes HU – Mail HU – CATI/CAPI GQ Estimate SE Estimate SE Estimate SE Estimate SE All People (number in thousands) 300,349 31 153,826 1,073 138,276 1,091 8,247 (x) Before Editing Percentage with complete item response 73.00 0.15 51.80 0.05 96.10 0.03 81.00 0.28 Percentage with at least one but not all response 23.20 0.14 43.80 0.05 1.20 0.02 7.10 0.16 Percentage with no complete items (all nonresponse) 3.80 0.02 4.40 0.02 2.70 0.03 11.80 0.23 After Editing Percentage with at least one health insurance type allocated 9.70 0.04 14.70 0.04 3.80 0.03 15.60 0.25 NOTES: SE = standard error. (x) Rounds to zero. Subpart h of the question (any other type of health insurance or health coverage plan) is excluded from the nonresponse calculation. SOURCE: Data from U.S. Census Bureau, 2008 American Community Survey, Turner et al. (2009).
OCR for page 89
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Comparisons to Other National Surveys Data users and policy makers who have relied on other surveys for estimates of health insurance coverage will be interested in how ACS estimates compare with other sources. The CPS ASEC and the NHIS are both widely used sources of estimates of health insurance coverage, although neither is a gold standard. Both surveys produce estimates that are particular to their own contexts, question wording, and processing regimens. The following sections briefly describe the surveys, discuss the structure of the health insurance questions, and present coverage estimates from the ACS side-by-side with estimates from the CPS ASEC and the NHIS. ANNUAL SOCIAL AND ECONOMIC SUPPLEMENT TO THE CURRENT POPULATION SURVEY Design The CPS is a monthly survey that the Census Bureau conducts for the Bureau of Labor Statistics to provide data on labor force participation and unemployment. Data on health insurance coverage are collected through the ASEC, which is administered February through April. About 76,200 households are sampled per year. The CPS ASEC sample is the civilian noninstitutionalized population of the United States.8 CPS ASEC data are collected through a combination of telephone and personal visit modes using computer-assisted instruments. The Census Bureau publishes CPS ASEC estimates of health insurance coverage for the nation and all states. Health Insurance Coverage Question The CPS ASEC income and health insurance coverage questions are asked at the household level, that is, “Does anyone in the household…?” If the answer is “yes,” the CPS ASEC goes on to ask “Who…?” This is distinct from the ACS questionnaire, which asks all the questions about each person individually, that is, “Does this person…?” From a cognitive and operational perspective, each approach has benefits and challenges. The CPS ASEC asks respondents to recall their insurance status for the prior calendar year (January through December). Hence, respondents 8 Members of the armed forces living off post or with their families on post are included if at least one civilian adult lives in the household.
OCR for page 90
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary need to recall insurance coverage for a period that began 14 to 16 months prior to the interview. The question series covers a comprehensive list of insurance types that include public program names specific to the state in which the interview is conducted. Finally, if the person does not indicate coverage, a verification question asks specifically about his or her coverage status.9 The CPS ASEC health insurance question set and editing result in an estimate intended to be of those uninsured for all of the previous calendar year. Previous research has indicated that the long reference period is a limitation of the CPS ASEC methods, with the estimate of the uninsured too high for a “full year” measure and more closely approximating a point-in-time estimate.10 NATIONAL HEALTH INTERVIEW SURVEY Design The NHIS is an ongoing survey conducted throughout the year by the National Center for Health Statistics to monitor the health of the nation. It has been conducted since 1957. The NHIS consists of a Basic Module, including the Family Core, the Sample Adult Core, and the Sample Child Core, as well as several supplements that vary from year to year. In recent years, slightly less than 35,000 households were interviewed. The NHIS sample is the civilian noninstitutionalized population of the United States. NHIS data are collected through an in-person survey using computer-assisted interviewing. The sample for the NHIS includes data from the 50 states and the District of Columbia. However, it is not designed to provide state-level estimates; the lowest level of geography publically available is census region. Health Insurance Coverage Question Like the ACS, the NHIS asks the respondent about insurance status and coverage type at the time of the survey. The NHIS also asks if the respondent has been uninsured for at least part of the year prior to the interview and if the respondent has been uninsured for more than a year at the time of the interview. The question series includes a comprehensive 9 For more information on health insurance coverage in the CPS ASEC, see: http://www.census.gov/hhes/www/hlthins/hlthins.html [October 2010]. 10 See appendix C of “Income, Poverty, and Health Insurance Coverage in the United States: 2008” P60-236(RV), for more information about the quality of the CPS ASEC health insurance estimates, available: http://www.census.gov/prod/2009pubs/p60-236.pdf [October 2010].
OCR for page 91
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary list of insurance options that include public program names specific to the state in which the interview is conducted, as well as open-ended response options. A verification question is included to confirm that respondents who did not respond that they were enrolled in any insurance program are, in fact, uninsured. The NHIS also edits variables based on supplemental information that the interviewer may collect, such as statements or insurance cards that respondents display.11 HEALTH INSURANCE QUESTION DESIGN ISSUES The ACS is the first major federal mailout-mailback survey to include health insurance questions. The ACS uses a set of health insurance categories that are similar in conceptual scope to other surveys, like the CPS ASEC and the NHIS, but there are methodological differences that highlight the limitations of soliciting health coverage information in a mailout-mailback environment. The ACS, since it utilizes a paper survey instrument, does not allow the customization of questions to reflect the specific state health programs (or Medicaid/Children’s Health Insurance Program funded programs) for which residents of a particular state or locality can apply. The CPS ASEC and the NHIS, in contrast, are conducted entirely through computer-assisted instruments and are able to use state-specific public program names in questions. This mechanism has been shown to help respondents identify their enrollment in public health programs (Eberly et al., 2009). However, at this time the extent to which the lack of state-specific program names biases ACS estimates is uncertain. Another limitation of a mailed paper survey instrument is the inability to use customized questions or wording for subgroups of the population, such as children. The CPS ASEC and the NHIS, in contrast, utilize specific questions aimed at children to ensure that coverage under the Children’s Health Insurance Program is reported. Although the ACS collects information on a number of important health insurance covariates, such as housing characteristics, public program participation, socioeconomic status, and functional limitations, it lacks a number of covariates that are found in the CPS ASEC and the NHIS. These include general reported health and disability status and detailed income and employment measures and, in the NHIS, health conditions and health care utilization. The ACS health insurance coverage question uses a clearly defined current coverage measure, referred to as “point in time,” that is easily interpreted. In contrast, the CPS ASEC asks respondents to report any 11 For more information on the NHIS, see: http://www.cdc.gov/nchs/nhis.htm.
OCR for page 92
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary coverage they had in the preceding calendar year. Numerous studies have found that this question format downwardly biases estimates of coverage. Indeed, the CPS ASEC bias is so severe that it is closer to other surveys’ point-in-time measures than it is to other sources of all-year coverage (Congressional Budget Office, 2003; Davern et al., 2007a). Despite these challenges and deficiencies, the ACS has a number of attributes that benefit data users and researchers. Most notably, its large sample size (about 30 times that of the CPS) and certain selection of all U.S. counties allow it to produce estimates for state and substate geographic areas and for key population subgroups. COMPARISON OF HEALTH INSURANCE COVERAGE ESTIMATES In order to compare 2008 ACS data with data from the 2009 CPS ASEC (2008 calendar year estimates) and the 2008 NHIS public-use files, we defined health insurance characteristics in the CPS ASEC and the NHIS as similar to ACS rules.12 In this way, variables for each of the seven ACS health insurance types were created for the CPS ASEC and the NHIS. These comparisons illustrate how the ACS estimates of the uninsured fit with these other national surveys. The 2008 ACS point-in-time estimates are compared with the CPS ASEC 2008 all-year uninsured estimates and the 2008 NHIS point-in-time estimates. In addition to the uninsured measure, differences in survey design may influence the results. All comparative statements have undergone statistical testing and are significant at the 95 percent confidence level unless otherwise noted. It is important to note that, unlike the CPS ASEC and the NHIS, the ACS edits for nonresponse did not use a rules-based assignment of health insurance coverage (called consistency or coverage edits). In the ACS, these types of edits are being implemented in the 2009 estimates and are discussed in further detail in the Data Processing section of this paper. Table 9-2 shows the baseline rates of health insurance coverage from the three surveys. The ACS health insurance coverage rate was 84.9 percent, not statistically different from the NHIS rate of 85.2 percent. This high level of consistency is a good sign for the ACS, which is conceptually similar to the NHIS, as they both measure current coverage. The CPS ASEC health insurance coverage rate was 84.6 percent. Although the statistical test of the difference between the ACS and the CPS ASEC showed evidence of difference, these two estimates do not appear meaningfully different—both round to 85 percent of the population. Table 9-2 shows that the ACS direct purchase rate is 14.2 percent, 12 For example, the CPS ASEC estimate of military health care was separated into TRICARE/other military health care and VA coverage.
OCR for page 93
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary which is 5.3 percentage points higher than the CPS ASEC and 7.6 points higher than the NHIS. Given that overall levels of health insurance coverage are similar, the direct purchase results suggest that the ACS is classifying direct purchase differently from the alternative surveys, but that at aggregate levels respondents are consistently indentifying that they are covered by some form of coverage. The ACS direct purchase estimate is particularly worrisome, as previous research has shown that the CPS ASEC overestimates administrative totals of the direct purchase population (Cantor et al., 2007). As an all-year measure, the CPS ASEC should theoretically exceed the point-in-time measure of the ACS. As such, the ACS rate is likely to be an overestimate of the direct purchase population. Possible reasons for the higher estimate of direct purchase coverage are discussed in the Direct Purchase section of this paper. The health insurance coverage rates for children under age 18 are also shown in Table 9-2. Both the ACS and the CPS ASEC estimate that 90.1 percent of children have health insurance coverage, and the NHIS estimates that 91.0 percent of children have health insurance coverage. The difference between the ACS and the NHIS is statistically significant but not meaningful. The ACS had a higher percentage of children under age 18 with employer-sponsored insurance, 56.2 percent, than the NHIS rate of 54.0 percent. The ACS had a lower percentage of children under age 18 with employer-sponsored insurance than the CPS ASEC rate of 58.9 percent. The ACS estimated a higher proportion of children with direct purchase coverage, 9.2 percent, than the CPS ASEC rate of 5.1 percent or the NHIS rate of 3.4 percent. The ACS found fewer children under age 18 with coverage from a public means-tested health insurance program, 27.8 percent, than the CPS ASEC rate of 30.3 percent or the NHIS rate of 31.4 percent. This difference may reflect methodological differences in the data collection processes—including the fact that the 2008 ACS, unlike the CPS ASEC and the NHIS, does not include a consistency edit. Table 9-3 shows uninsured rates by selected demographic and economic characteristics from the three surveys. The uninsured rate for non-Hispanic whites was not statistically different among the surveys: 10.6 percent in the ACS, 10.8 percent in the CPS ASEC, and 10.6 percent in the NHIS.13 The ACS had a lower percentage of uninsured non-Hispanic 13 The surveys allow respondents to choose more than one race. Except for the Multiple Race category, race groups discussed in this paper refer to people who indicated only one racial identity among the six major categories: white, black or African American, American Indian and Alaska Native, Native Hawaiian and Other Pacific Islander, and Some Other race. The use of single-race population in this paper does not imply that it is the preferred method of presenting or analyzing data. A variety of approaches are used.
OCR for page 98
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary blacks, 18.0 percent, than the CPS ASEC rate of 19.0 percent, but higher than the NHIS rate of 16.4 percent. Similar to the results for non-Hispanic whites, the uninsured rate for Hispanics was not statistically different among the surveys: 31.5 percent in the ACS, 30.7 percent in the CPS ASEC, and 31.6 percent in the NHIS. In the CPS ASEC and the NHIS, the sample sizes are small for the remaining race categories and are better interpreted when viewed over a longer term. All race and ethnicity categories are presented here for completeness. The ACS had a slightly lower uninsurance rate for people living in poverty, 29.1 percent, than the CPS ASEC rate of 30.4 percent, but higher than the NHIS rate of 26.5 percent. There are substantial differences in the measurement of income among the three surveys, which may impact the estimates of health insurance coverage by income level. For an indepth analysis of income data in major federal surveys, see Czajka and Denmead (2008). Although there are some statistically significant differences across other measures examined, those differences tend to be quite small in magnitude. Table 9-4 shows uninsured rates for children under 19 years by selected demographic and economic characteristics. The uninsured rate by age category is not statistically different among the three surveys. For children under age 6, 8.6 percent were uninsured in the ACS, 8.7 percent in the CPS ASEC, and 7.6 percent in the NHIS. For children aged 6 to 11, 9.7 percent were uninsured in the ACS, 9.2 percent in the CPS ASEC, and 9.0 percent in the NHIS. A similar pattern was seen for children aged 12 to 18. As was true for the overall population, the uninsured estimates from the three surveys are quite consistent for key subgroups of children. Data Processing As a whole, the ACS produces estimates of health insurance coverage that are remarkably similar to the CPS ASEC and the NHIS. One potential source of variation could be the different data processing regimens used by the three surveys. Data processing differences in the ACS (consistency edits) and CPS ASEC (imputation bias) are particularly important because there is a high probability that they will change in coming years. Thus, the differences produced in this analysis may not be reflective of differences in future years. In the sections that follow we describe these data processing differences and compare the three surveys after accounting for them. Consistency Edits The CPS ASEC and the NHIS employ consistency edits (also called logical coverage edits) that deterministically assign public coverage to
OCR for page 99
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary people who do not report it. In the CPS ASEC, coverage is assigned to likely enrollees based on a set of rules developed by the Census Bureau in consultation with an independent technical advisory group. The rules apply to Medicare, Medicaid, and military coverage. For example, people over age 65 who report Social Security or Railroad Retirement income are assigned Medicare. This data processing technique is motivated by two assumptions: (1) coverage is generally under reported in surveys, and (2) health insurance is a particularly difficult concept for respondents and is prone to more response error than other socioeconomic concepts. To our knowledge, no study has carefully examined the quality of such edits. However, after consultation with policy and data experts, the Census Bureau determined that these edits reduce the level of error at aggregate population levels. As mentioned, the 2008 ACS data file has not been edited in this manner. However, in summer 2009, the Census Bureau, in consultation with outside experts, developed a set of edit rules for use in the ACS and will implement them starting with the 2009 ACS (Lynch et al., 2010). The Census Bureau will not retroactively release 2008 data with these edits; however, the State Health Access Data Assistance Center (SHADAC) is in the process of releasing edited microdata through the Minnesota Population Center’s Integrated Public Use Microdata Series (IPUMS).14 Imputation Bias in the CPS ASEC The CPS ASEC imputation routine is known to produce less private dependent coverage than is expected from the explicitly reported distribution (Davern et al., 2007b). This problem stems from a rule in the imputation routine that restricts dependent coverage to the immediate family of the policy holder. This is unlike the instrument that allows dependent coverage to be given to any household member. As a result of this rule, imputed dependent coverage is biased downward in reference to the reported distribution. Currently, the Census Bureau is conducting an evaluation of a new imputation routine based on the results of Davern et al. (2007b). The preliminary expectation is that this new routine will be implemented beginning in 2011 for the 2010 estimates. Data Processing Differences To account for these data processing differences, we repeat the three-survey comparison using two alternative data sources. To account for the lack of a consistency edit in the 2008 ACS, we used ACS estimates 14 Available at: http://usa.ipums.org/usa [July 2010].
OCR for page 100
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary TABLE 9-4 Uninsured Rate for Children by Selected Characteristics in the ACS, CPSASEC, and NHIS, 2008 Universe: U.S. Civilian Noninstitutionalized Population ACS CPS ASECa NHIS Estimate SE Estimate SE ACS-CPS Difference Estimate SE ACS-NHIS Difference Percentage of Children Under Age 19 Uninsured 10.40 0.07 10.30 0.21 9.50 0.42 * Age Under 6 years 8.60 0.09 8.70 0.30 7.60 0.60 6 to 11 years 9.70 0.10 9.20 0.30 9.00 0.52 12 to 18 years 12.50 0.10 12.40 0.30 11.60 0.54 Race and Hispanic Originb White alone, NH 6.90 0.07 7.10 0.22 7.10 0.60 Black alone, NH 10.40 0.18 11.20 0.61 7.70 0.76 * Asian alone, NH 9.70 0.30 11.40 1.04 6.80 1.06 * AIAN alone, NH 27.20 1.09 16.30 2.09 * 13.60 4.85 * NHOPI alone, NH 11.80 1.94 12.50 5.09 n/a n/a n/a Some other race alone, NH 11.60 1.11 n/a n/a n/a n/a n/a n/a Two or more races, NH 7.40 0.29 6.80 0.82 8.00 1.78 Hispanic or Latino (of any race) 19.40 0.19 17.90 0.56 * 17.90 0.85
OCR for page 101
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Income Relative to the Federal Poverty Level 0-99% FPL 16.70 0.21 16.50 0.63 14.10 1.02 * 100-199% FPL 16.70 0.18 15.30 0.54 * 16.40 0.96 200% FPL 6.30 0.06 6.40 0.20 5.40 0.34 * Family Work Status No one working in family 13.30 0.28 13.50 0.65 8.90 1.31 * At least one worker in family 10.20 0.07 9.70 0.21 * 9.60 0.44 NOTES: *Statistically different from zero at the 95 percent confidence level. n/a = not available; FPL = federal poverty level; SE = standard error. aThe CPS ASEC asks about health insurance coverage over the prior calendar year, however, there is considerable uncertainty as to how respondents answer the health insurance questions in the survey (Congressional Budget Office, 2003; Davern et al., 2007a). It appears that the CPS ASEC, which purports to be a measure of all year uninsured, is closer to a measure of uninsurance at a point in time. The ACS and the NHIS estimates are measures of point-in-time uninsured. bAIAN = American Indian and Alaska Native, NH = not Hispanic or Latino, NHOPI = Native Hawaiian and Other Pacific Islander. SOURCE: Data from U.S. Census Bureau, 2008 American Community Survey, and the 2009 Annual Social and Economic Supplement to the Current Population Survey public use files; 2008 National Health Interview Survey public-use files.
OCR for page 102
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary from Lynch et al. (2010), which are based on 2008 ACS data with the consistency edit. To account for the imputation bias in the CPS ASEC, we used the SHADAC-enhanced CPS ASEC data set.15 These estimates were produced by removing the fully imputed cases (roughly 10 percent of the entire sample), weighting up the remaining cases to population controls, and adjusting for changes to the health insurance question (State Health Access Data Assistance Center, 2009; Ziegenfuss and Davern, 2010). While this method for correcting the imputation bias in the CPS ASEC may be different from reengineering the imputation routine, we believe it is a close approximation to what would be expected based on the new routine. In order to understand the contribution of these data processing factors in the differences observed and to provide estimates that may be a better representation of future data, we compare edited ACS, SHADAC-enhanced CPS ASEC, and NHIS data in Table 9-5. After applying the consistency edits, the ACS uninsured estimate was 14.6 percent compared with 15.1 percent prior to the edits (from Table 9-3). This estimate was nearly identical to the NHIS uninsured estimate of 14.8 percent. The SHADAC-enhanced CPS ASEC uninsured estimate was 14.8 percent, or 0.6 percentage points lower than the CPS ASEC estimate (15.4 percent, from Table 9-3) and identical to the NHIS uninsured estimate. The ACS (with the consistency edit) had a lower percentage of uninsured non-Hispanic blacks, 17.2 percent, than the SHADAC-enhanced CPS ASEC rate of 18.5 percent but was not statistically different from the NHIS rate of 16.4 percent. Although the uninsured rate was still statistically different for non-Hispanic blacks between the ACS and SHADAC-enhanced CPS ASEC, the estimates were moving closer together. The ACS uninsured rate for non-Hispanic blacks was 18.0 percent prior to the edits and the rate for the CPS ASEC was 19.0 percent (from Table 9-3). This pattern of the estimates moving closer together was seen for most of the other key subgroups. These results suggest the differences observed are driven in part by the data processing regimens of the alternate surveys. With the application of the consistency edits in the 2009 ACS and the possible implementation of a revised edit routine in the CPS ASEC for 2010 estimates, one would expect to see smaller differences among the surveys in future years. 15 Enhanced data are available from SHADAC’s Data Center at: http://www.shadac.org/datacenter [October 2010] and from the Minnesota Population Centers Integrated Public Use Microdata Series (IPUMS) at: http://usa.ipums.org/usa [October 2010].
OCR for page 103
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Comparison of Direct Purchase Estimates The largest anomaly apparent in Table 9-2 is the relatively high level of direct purchase in the ACS. While these results do not conclusively suggest that the ACS estimate is biased, previous research suggests that the CPS ASEC overcounts the level of direct purchased coverage (Cantor et al., 2007). By extrapolation, the ACS direct purchase estimate is likely to be upwardly biased as well. The possible misclassification of direct purchase in the ACS could be of two sorts. Respondents could indicate direct purchase in addition to another (and presumably accurate) coverage source. This would mean that misreporting of direct purchase had no effect on the accuracy of other coverage levels. Alternatively, direct purchase could be reported instead of the correct source of coverage. This would result in an undercount of the correct coverage source in addition to the overcount of direct purchase. The source of the direct purchase error in the ACS is currently poorly understood. Four hypotheses are currently under consideration: (1) the absence of state public program names could result in public program enrollees mistakenly reporting direct purchase; (2) the absence of a qualifier in the response option that explicitly states that direct purchase is not employer-sponsored coverage (as is done in the CPS ASEC) could result in group plan enrollees reporting direct purchase; (3) people with single service plans, such as dental coverage, could be reporting direct purchase; and (4) the person-level roster processing and/or the order of direct purchase in the response list could result in overreporting. Researchers at the Census Bureau in consultation with SHADAC and other interested parties are currently exploring these hypotheses. A recent study conducted by Urban Institute and SHADAC analysts found larger than expected levels of direct purchase across age, employment, and income distributions, suggesting that the problem is multifaceted and not limited to a single population segment (Lynch and Boudreaux, 2010). The analysis also found that a large portion of the direct purchase population had sociodemographic characteristics that would be consistent with coverage by employer-sponsored insurance or means-tested programs. The researchers logically removed direct purchase coverage from observations that had Medicaid and were eligible for such a benefit, that had military coverage, or that had employer-provided coverage and reported familial employment patterns consistent with employer-provided coverage. Such edits were more conservative than creating a hierarchical variable that assigned direct purchase only to those who reported direct purchase alone. These edits reduced direct purchase coverage among people aged 0 to 64 from 10.5 to 6.6 percent.
OCR for page 104
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary TABLE 9-5 Uninsured Rate by Selected Characteristics in the ACS (consistency edits), Enhanced CPS ASEC, and NHIS, 2008 Universe: U.S. Civilian Noninstitutionalized Population ACS with Consistency Edits SHADAC-Enhanced CPS ASECa NHIS Estimate SE Estimate SE ACS-CPS Difference Estimate SE ACS-NHIS Difference Percentage of People Uninsured 14.60 0.05 14.80 0.14 14.80 0.25 Age Under 6 years 8.00 0.07 7.80 0.30 7.60 0.60 6 to 18 years 10.60 0.07 10.30 0.25 10.50 0.44 19 to 64 years 19.50 0.06 19.80 0.19 20.00 0.32 65 years and over 0.90 0.02 1.70 0.11 * 0.60 0.08 * Sex Male 16.10 0.06 16.70 0.19 * 16.40 0.30 Female 13.20 0.04 13.10 0.15 13.30 0.28 Race and Hispanic Originb White alone, NH 10.30 0.04 10.10 0.14 10.60 0.28 Black alone, NH 17.20 0.10 18.50 0.44 * 16.40 0.50 Asian alone, NH 14.20 0.16 16.80 0.73 * 12.40 0.86 * AIAN alone, NH 29.70 0.47 26.40 1.61 * 22.80 4.69 NHOPI alone, NH 14.90 0.84 13.60 2.41 n/a n/a Some other race alone, NH 20.30 0.70 n/a n/a n/a n/a Two or more races, NH 13.10 0.22 12.60 0.85 14.60 1.58 Hispanic or Latino (of any race) 30.70 0.13 31.00 0.47 31.60 0.72
OCR for page 105
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Income Relative to the Federal Poverty Level 0-99% FPL 27.60 0.11 30.00 0.49 * 26.50 0.78 100-199% FPL 25.10 0.10 24.10 0.38 * 25.70 0.58 200% FPL 9.60 0.04 9.30 0.14 * 9.50 0.22 Citizenship Status U.S. citizen 12.30 0.04 12.50 0.13 12.30 0.24 Not a U.S. citizen 45.50 0.15 46.30 0.72 46.90 1.12 Marital status Not married 17.10 0.06 17.90 0.19 * 16.80 0.30 Married 10.90 0.04 10.60 0.17 12.10 0.30 * NOTES: *Statistically different from zero at the 95 percent confidence level. n/a = not available; FPL = federal poverty level; SE = standard error. aThe CPS ASEC asks about health insurance coverage over the prior calendar year, however, there is considerable uncertainty as to how respondents answer the health insurance questions in the survey (Congressional Budget Office, 2003; Davern et al., 2007a). It appears that the CPS ASEC, which purports to be a measure of all year uninsured, is closer to a measure of uninsurance at a point in time. The ACS and the NHIS estimates are measures of point-in-time uninsured. bAIAN = American Indian and Alaska Native, NH = not Hispanic or Latino, NHOPI = Native Hawaiian and Other Pacific Islander. SOURCE: Data from U.S. Census Bureau and 2008 American Community Survey from Lynch et al. (2010); 2009 SHADAC-Enhanced CPS ASEC; 2008 National Health Interview Survey public-use files.
OCR for page 106
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Given the substantial changes in the individual health insurance market as part of health care reform, improving the ACS instrument or developing postcollection adjustments to the direct purchase estimate will require a close partnership between survey methodologists and policy experts. Under health care reform, it is no longer clear what is meant by direct purchase on a conceptual level. Should people who purchase insurance with premium assistance from a government program or those purchasing coverage through an exchange be counted as direct purchase enrollees? If not, what categories describe them? Once these questions are answered, survey methodologists can begin to craft instruments that are better suited for the postreform environment. FINAL REMARKS The ACS, which added a question on health insurance coverage in 2008, is a powerful new resource that can be used both to provide guidance to the implementation of health care reform and to evaluate the impacts of health care reform at the national, state, and local levels. The survey’s very large sample size, combined with a sample frame that is representative of all areas of the United States, will support estimates for narrow population subgroups (e.g., young children, adolescents, teenagers transitioning to adulthood) and small geographic areas (e.g., states, counties, communities) that are not possible using other available data sources. This paper shows that the ACS estimates of health insurance coverage are remarkably consistent with estimates from the other national surveys that are often used to track health insurance coverage—the CPS ASEC and the NHIS. We have found few meaningful differences in estimates of the uninsurance rate in the ACS relative to the CPS ASEC or the NHIS for the overall population or for key population subgroups, including children. This is particularly true after making adjustments for data processing differences across the three surveys. To the extent that such differences are addressed in future rounds of the surveys (e.g., through expected processing enhancements to the ACS and possible improvements in the CPS ASEC imputation routine), we would expect fewer differences in the estimates from the three surveys in the future, ensuring greater consistency in the estimates over time. Notwithstanding the general consistency of the uninsurance estimates from the ACS with other national surveys, additional work is needed to assess its strengths and weaknesses as more years of data are available over time. For example, more work is needed to understand the relatively high estimate of direct purchase coverage in the ACS relative to other surveys that we have described here. Such work will be critical to establish-
OCR for page 107
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary ing the long-term value of the ACS as a resource for policy development, analysis and evaluation, and determining the relative value of the ACS, the CPS ASEC, and the NHIS for alternate uses. The ACS is particularly promising for state and local analysts and policy makers as they begin to implement health care reform, as it will support estimates for key population subgroups and small geographic areas. The ACS provides data for many states that have not had access to such detailed health insurance coverage information before, as well as for states that may no longer be able to conduct their own household and population surveys in the face of ongoing budget limitations. Ensuring that the ACS fulfills its promise as a state and local resource requires addressing the capacity of state and local analysts to use the ACS data files. It is likely that many states lack the hardware and software capacity to analyze the very large ACS data files. To address this constraint, SHADAC has taken the initiative to provide summary coverage estimates of the ACS on their Data Center online table generator.16 It is hoped that this user-friendly access point, along with detailed technical documentation and technical assistance, if needed, will help state and local analysts overcome the learning curve related to using the ACS and facilitate rapid policy analysis as states begin addressing the implementation of health care reform. ACKNOWLEDGMENTS Much of the work presented here was prepared by the University of Minnesota’s State Health Access Data Assistance Center (SHADAC) under contract to the U.S. Census Bureau and presented in the working paper “A Preliminary Evaluation of Health Insurance Coverage in the 2008 American Community Survey,” released September 22, 2009. That working paper was led by Joanna Turner while she was at the U.S. Census Bureau and coauthored by Michel Boudreaux (SHADAC) and Victoria Lynch (The Urban Institute) and is available at: http://www.census.gov/hhes/www/hlthins/data/acs/2008/2008ACS_healthins.pdf. Turner joined SHADAC in March 2010. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. 16 Available: http://www.shadac.org/datacenter [October 2010] and http://www.shadac.org/content/acs-info-and-resources [October 2010].
OCR for page 108
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary REFERENCES Cantor, J., Monheit, A.C., Brownlee, S., and Schneider, C. (2007). The adequacy of household survey data for evaluating the nongroup health insurance market. Health Services Research, 42(4), 1739-1757. Congressional Budget Office. (2003). How Many People Lack Health Insurance and for How Long? Washington DC: U.S. Congress. Available: http://www.cbo.gov/doc.cfm?index=4210 [October 2010]. Czajka, J., and Denmead, G. (2008). Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys Final Report. Report from Mathematica Policy Research, Inc. Available: http://www.mathematica-mpr.com/publications/PDFs/incomedata.pdf [October 2010]. Davern, M., Davidson, G., Ziegenfuss, J., Jarosek, S., Lee, B., Yu, T., Beebe, T.J., Call, K.T., and Blewett, L.A. (2007a). A comparison of the health insurance coverage estimates from four national surveys and six state surveys: A discussion of measurement issues and policy implications. Minneapolis: University of Minnesota. Available: http://www.shadac.org/files/shadac/publications/ASPE_FinalRpt_Dec2007_Task7_2_rev.pdf [October 2010]. Davern, M., Rodin, H., Blewett, L.A., and Call, K.T. (2007b). Are the current population survey uninsurance estimates too high? An examination of the imputation process. Health Services Research, 42(5), 2038-2055. Eberly, T., Pohl, M.B., and Davis, S. (2009). Undercounting Medicaid enrollment in Maryland: Testing the accuracy of the Current Population Survey. Population Research and Policy Review, 28(2), 221-236. Lynch, V., and Boudreaux, M. (2010). Health Insurance Estimates from the ACS: An Analysis of Directly Purchased Coverage. Presentation at the 2010 Joint Meetings of the American Statistical Association, August 3, Miami Beach, FL. Lynch, V., Boudreaux, M., and Davern, M. (2010). Applying and Evaluating Logical Coverage Edits to Health Insurance Coverage in the American Community Survey. U.S. Census Bureau, housing and household economic statistics division. Available: http://www.census.gov/hhes/www/hlthins/publications/coverage_edits_final.pdf [October 2010]. State Health Access Data Assistance Center. (2005). Reclassifying Health Insurance Coverage for the Indian Health Service in the Current Population Survey: Impact on State Uninsurance Estimates. Issue Brief #11. Minneapolis: University of Minnesota. Available: http://www.shadac.org/files/IssueBrief11.pdf [October 2010]. State Health Access Data Assistance Center. (2009). Historical Changes in Current Population Survey Health Insurance Coverage Items for Survey Years 1988 Through 2009. Issue Brief #19. Minneapolis: University of Minnesota. Available: http://www.shadac.org/files/shadac/publications/IssueBrief19.pdf [October 2010]. Turner, J., Boudreaux, M., and Lynch, V. (2009). A Preliminary Evaluation of Health Insurance Coverage in yhe 2008 American Community Survey. U.S. Census Bureau, housing and household economic statistics division. Available: http://www.census.gov/hhes/www/hlthins/data/acs/2008/2008ACS_healthins.pdf [October 2010]. Ziegenfuss, J., and Davern, M. (2010). Twenty years of coverage: An enhanced Current Population Survey—1989-2008. Health Services Research, doi:10.1111/j.1475-6773.2010.01171. x. Available: http://onlinelibrary.wiley.com/doi/10.1111/j.1475-6773.2010.01171.x/full[October 2010].