National Academies Press: OpenBook

Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary (2010)

Chapter: 13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder

« Previous: 12 The Massachusetts Experience: Using Survey Data to Evaluate State Health Care Reform--Sharon K. Long
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

13
Small-Domain Estimation of Health Insurance Coverage

Brett O’Hara and Mark Bauder

U.S. Census Bureau


The Census Bureau’s Small Area Health Insurance Estimates (SAHIE) Program uses small-area techniques to estimate health insurance coverage. These model-based estimates generally have lower variances than survey-only (i.e., nonmodel-based) estimates because the modeling incorporates covariates that are related to health insurance coverage. The SAHIE program is an outgrowth of its sister program, Small Area Income and Poverty Estimates (SAIPE). The primary purpose of SAIPE is to make model-based estimates of the number of poor, school-age children in a school district. For over a decade, the U.S. Department of Education has used these estimates as the major component of its allocation formula for dispersing funds from Title I of the Elementary and Secondary Education Act. From SAIPE, there was a knowledge base that the SAHIE program has benefited from and used.

In 2005, the SAHIE program published health insurance coverage estimates for the first time. This first release of data was to determine the feasibility of producing substate estimates when a limited set of estimates was published for only two age cohorts. The survey data used came from the Annual Social and Economic Supplement of the Current Population Survey (CPS ASEC). At the time, the CPS ASEC was the only survey that could produce health insurance estimates for states. State estimates were designed to be reliable when 3 years of CPS ASEC were combined. However, the survey could not produce reliable estimates for all counties. Not all counties are in the survey, and most counties in the survey have small

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

sample sizes. The SAHIE team established that it was feasible to produce estimates for all counties by using a methodology similar to SAIPE.

In 2007, the SAHIE program published state estimates of the prevalence of uninsured low-income women by age, race, and Hispanic origin. Low income is defined in terms of family income divided by the poverty threshold. The subpopulations of interest were determined by a sponsor of the SAHIE program, the Centers for Disease Control and Prevention’s (CDC’s) National Breast and Cervical Cancer Early Detection Program (NBCCEDP). Its funding of the SAHIE program made it possible to produce estimates of income-eligible women by age group (18-64, 40-64, and 50-64). Income eligibility for this benefit varies by state. Most state programs chose income eligibility at 0-200 or 0-250 percent of the poverty threshold. From these estimates, the SAHIE team evaluated whether the state model could meet the CDC requirements. State and county estimates of health insurance coverage were published in 2008 that met the requirements. The SAHIE model was later modified so that it had the functionality to support many more income categories and more age categories. However, the 3-year average of the CPS ASEC health insurance coverage data could not support model-based estimates of more income categories or additional age groups (Small Area Health Insurance Estimates Team, 2008).

Currently, the SAHIE program publishes state health insurance coverage estimates by age, sex, race, Hispanic origin (i.e., demographic characteristics), and by income categories (both 0-200 percent and 0-250 percent of the poverty threshold and the total poverty universe). For counties, SAHIE produces estimates by age, sex, and income categories (either 0-200 percent or 0-250 percent of the poverty threshold and the total poverty universe). Estimates from this model were released in 2009 with no significant changes from the year before. Before 2009, all of the SAHIE estimates were labeled “experimental.” For 2009, the SAHIE program had evolved to use a production model and production health insurance coverage estimates.

One of the SAHIE program goals is to accelerate its production process to publish estimates in a timelier manner. Currently, the modeled estimates are released in the summer, based on the most current 3 years of the CPS ASEC. The program goal is to release estimates based on current data in the early winter.

Evaluation of various model forms to produce SAHIE estimates is ongoing. The model-based estimates had smaller confidence intervals than the survey-only estimates for uninsured low-income children under age 18. In addition, the two estimates for a given state were close (O’Hara, 2008). The SAHIE numbers can also be used as a low-variance denominator for participation (or nonparticipation) rates for the uninsured popu-

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

lations. This empirical application estimated participation rates using administrative data from NBCCEDP (the numerator) and uninsured low-income women ages 40-64 at the state and county levels (the denominator) (O’Hara, Tangka, and Bauder, in review).

The SAHIE program is currently exploring how to take best advantage of the American Community Survey (ACS) data. Full production of the ACS in 2006 allowed it to replace the decennial census sample data (i.e., the long form). Beginning in 2008, the ACS included a health insurance question. As a result, there are annual ACS health insurance estimates for all geographic areas with a population of 65,000 or more (U.S. Census Bureau, 2009). The ACS contains approximately 30 times the number of addresses as a single-year CPS ASEC. Direct survey estimates, based on a 5-year accumulation from the ACS, for geographic areas with population of 20,000 or less will not be available until 2013.

This paper assesses an ACS-based SAHIE model in two ways. First, we compare the variances of state estimates of a CPS-based SAHIE model and an ACS-based SAHIE model. Second, the model is modified to obtain ACS model-based estimates for more income categories. The gains from model-based state estimates increase as the number of domains increases (e.g., the income categories). As yet, county-level estimates have not been explored. These potential changes to the model, based on the strengths of the ACS, are intended to create more useful or refined estimates of the uninsured populations for policy makers and other stakeholders.

BACKGROUND

Uninsured children are likely to use emergency room visits as a form of “free” primary care and to put off any use of medical services until a need must be met (Grumbach, Keane, and Bindman, 1993). Even when an uninsured child uses other care, access to and utilization of the care are lower than that of children with health insurance coverage (Newacheck, Hughes, and Stoddard, 1996). Children without health insurance coverage are more likely to have unmet medical needs that affect long-term health status compared with insured children (Newacheck et al., 2000). Being uninsured increases the risk of having high out-of-pocket expenses for medical services. As a result, families with uninsured children are more likely to become impoverished when the child has a health event (O’Hara, 2004).

The first nationwide means-tested health insurance program was the Medicaid program. Created in 1965, Medicaid provides full and comprehensive health insurance coverage (as well as partial benefits). The target populations are poor and near-poor children, the disabled, and the elderly (Centers for Medicare & Medicaid Services, 2010). In 1997, a new means-

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

tested health insurance program was enacted—the Children’s Health Insurance Program (CHIP). It targets uninsured children who do not qualify for Medicaid but whose family income is too low to pay insurance premiums (Dubay et al., 2007). Of the children whose family income fell between 133 and less than or equal to 200 percent of poverty, 22.4 percent were uninsured in 1996 and 14.6 percent were uninsured in 2008 (authors’ own calculation using the CPS).

The federal CHIP allocation to the states was dependent on several factors, one of which was an estimate of the state’s number of uninsured children under age 19 who are at or below 200 percent of poverty. Until recently, the only survey that produced those estimates was the CPS; however, it was ill suited for making state allocations for CHIP because the survey had a sample size too small to produce accurate estimates of the target population (Czajka and Jabine, 2002; National Research Council, 2002). The Medicare, Medicaid, and S-CHIP Balanced Budget Refinement Act of 1999 provided new funding to increase the sample size of the CPS in order to improve the reliability of the estimates (P.L. 106-113, 1999). The expansion of the CPS was implemented in collection year 2001 (U.S. Census Bureau, 2002).

Even with the CPS expansion, hereafter referred to as the CPS ASEC, much of the states’ year-to-year fluctuations in the estimates were still large, probably due to sampling error and not real change. Only 10 states have a sample size that results in estimates of more than 100 uninsured children at or below 200 percent of poverty (Blewett and Davern, 2007). Because of this small sample size, one suggestion has been to use model-based estimates of the target population that combine the ACS and CPS information (Blewett and Davern, 2007). In the Children’s Health Insurance Program Reauthorization Act of 2009 (CHIPRA), the allocation formulas are based on population estimates, not uninsured or low-income estimates (P.L. 111-3, 2009). It is possible that this decision was acknowledgment of the sampling error in the CPS ASEC. Under CHIPRA, the Census Bureau is required to rank the states in terms of their rates of uninsured low-income children and continue to produce population estimates of children under age 19. This part of the law is to determine “high-performing States,” that is, states with high insured rates. The Census Bureau was also required to determine whether the ACS or the CPS ASEC gives the best population estimates.

In September 2009, the ACS released its first set of health insurance estimates. Sample size is not a constraint for producing ACS estimates for states and many substate geographic areas. In the Patient Protection and Affordable Care Act of 2010 (PPACA), there are provisions that the Census Bureau will provide state-level estimates of the uninsured using the ACS and provide “data at the smallest geographic level” (P.L. 111-148, 2010).

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

PPACA made a correction to CHIPRA concerning population estimates; the law was changed to strike out “population estimates” and replace the phrase with “a high-performing State.” The Health Care and Education Reconciliation Act of 2010 (HCERA) requires the Census Bureau to rank the states in terms of their rates of uninsured people (P.L. 111-152, 2010).

Although it is not a requirement for the Census Bureau, the PPACA has new Medicaid requirements (all people at or below 133 percent of poverty are income-eligible), and establishes state-sponsored health insurance exchanges (all people between 133 percent and less than or equal to 400 percent of poverty are eligible for premium assistance). The level of premium assistance varies by the income-to-poverty ratio. For example, if the family income is between 150 and less than or equal to 200 percent of poverty, the family’s share of the premium would be less than or equal to 6.3 percent of the family’s income (Kaiser, 2010).

The SAHIE program has been responsive to stakeholders’ needs. It has always emphasized uninsured populations for the allocation of funds, outreach, and programmatic evaluations. For example, the NBCCEDP uses the SAHIE estimates to direct outreach funds to underserved counties. SAHIE state and county estimates of uninsured children can be used in the same way, as well as providing “data at the smallest geographic level.” The focus of the SAHIE program has changed somewhat given CHIPRA, the PPACA, and the HCERA. To be responsive to stakeholders’ needs, the SAHIE program is researching the feasibility of providing policy-relevant income-to-poverty ratios. Using the ACS, this research is possible.

DATA

The SAHIE program constructs statistical models that relate surveys, population estimates, and administrative records to produce health insurance estimates. The CPS ASEC provides annual national and state estimates based on a sample of about 100,000 addresses. All health insurance coverage and income questions refer to the previous year. Of the more than 3,100 counties in the United States, about 1,200 counties are in the sample. For the CPS-based SAHIE model, a 3-year average is used (Small Area Health Insurance Estimates, 2009). The CPS ASEC used in this evaluation are the collection years of 2007, 2008, and 2009. The ACS provides annual national, state, and substate estimates based on a sample of about 3 million addresses. Health insurance questions refer to coverage that day. Income questions refer to income over the last 12 months, and annual estimates cover a period of 23 months. A single year of survey data is used for the ACS-based SAHIE model; the 2008 ACS was used for this evaluation. For both surveys, family size is determined at the time of the interview.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

Dividing family income by the appropriate poverty threshold determines a family’s income group. For the ACS, both family income and the poverty threshold are adjusted monthly for inflation (U.S. Census Bureau, 2009). The survey data are tabulated by the number of uninsured by sex, age, race, and Hispanic origin categories (non-Hispanic white, non-Hispanic black, non-Hispanic people of other races or multiracial, Hispanic), and relevant income-to-poverty ratios. Both the CPS ASEC and the ACS are tabulated from microdata and are not published state estimates.

The covariates of income-to-poverty ratios are different from those for health insurance coverage. Covariates for income-to-poverty ratios include the estimates from the 2000 census sample data tabulated for the same categories as the survey data as well as other characteristics reported in the census sample data. The number of federal tax exemptions is calculated by age groups (children, nonelderly adults, and elderly adults) in various income-to-poverty ratios. The number of Supplemental Nutrition Assistance Program (SNAP) participants is also a strong predictor of income groups. The number of Medicaid and CHIP participants (for categories defined by age and sex) and the number of people employed in different-sized firms as shown in administrative records (the County Business Patterns) are strong predictors of health insurance. These predictors are aggregated to the county and state levels (O’Hara, 2008).

METHODS

The SAHIE program has developed a statistical model for producing estimates of numbers and proportions of insured and uninsured within income groups. For the geographic areas and domains of interest, survey-only estimates of the numbers in income groups, as well as numbers of insured, are often not adequate for users’ needs. Thus, the SAHIE model produces model-based estimates of both the numbers in income groups within geographic and demographic groups and the proportions and numbers insured within those income groups. Survey-only estimates of numbers in the income groups and the numbers insured and uninsured are modeled in terms of the quantities to be estimated. The model treats data from some of the administrative data in a way similar to survey estimates. They are modeled as random with distributions depending on the quantities to be estimated. Other data, in particular data from the County Business Patterns, are in the model as fixed predictors.

The model is a combination of two hierarchical models, corresponding to the proportions in income groups and proportions insured. The model is estimated as a fully Bayesian model. Vague prior distributions are assigned to unknown parameters. Point estimates are posterior means,

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

and posterior variances are used to construct confidence intervals. The model is estimated using Markov chain Monte Carlo methods.

Model Details

The current SAHIE model is similar to common small-area models. A basic and well-known example is the Fay-Herriot model (Fay and Herriot, 1979). Suppose that for area, i,Yi is the variable being estimated, and Ŷi is its survey estimate. In the Fay-Herriot model, given some predictors and parameters, Y follows a normal linear model and, conditional on Yi and parameters, and Ŷi is unbiased and normally distributed. The Fay-Herriot model can be written:

The variances and are model and sampling variances. θ contains any parameters on which these variances depend. Common simplifying assumptions are that the Yi’s are independent given the xi’s and parameters, the Ŷi’s are independent given the Yi’s and parameters, and that the model variances are the same for all i. It is also common to plug in survey or model-based estimates of the sampling variances and treat them as known.

The SAHIE model shares the hierarchical form of the Fay-Herriot model. But there are several differences between the current SAHIE model and the basic Fay-Herriot model described above:

  • There are two parts to the SAHIE model, each of which has a structure similar to a Fay-Herriot model. One part is for estimating proportions in income categories, the other is for estimating proportions insured.

  • Transformations of the proportions in the income categories and the proportions insured follow linear models, given predictors and parameters, whereas direct estimates of the untransformed proportions or numbers are unbiased and normal.

  • The two parts of the SAHIE model are “multivariate.” In addition to the survey estimates, other data are treated as random and are modeled by their distributions conditional on the true numbers and proportions.

  • Sampling variances are modeled, rather than treated as known.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Basic SAHIE model

We describe a basic version of the SAHIE model below, using the following notation:

  • a indexes state/age/race/sex or county/age/sex, and i indexes income group.

  • is the proportion of those in domain a who are in income group i.

  • is the proportion with health insurance within domain a, income group i.

  • and are the corresponding number in the income group and number insured.

  • POPa is the demographic population estimate for domain a, assumed known.

The proportions and numbers are related by

A basic version of the SAHIE model is as follows:

  • Survey estimates of the proportions in the income group and of the proportions with health insurance, given the true proportions, are normal, unbiased, and have variances that depend on parameters to be estimated. Let p denote either a proportion in an income group or a proportion insured, with corresponding survey estimate .

  • Census 2000 estimates, tax exemptions, SNAP participation, and combined Medicaid and CHIP participation are random. Each is normally distributed, given the true numbers in income groups and numbers insured. Except for 2000 census data, these data are not at the level of the survey estimates or the pai’s. Instead, they correspond to aggregates over base-level domains. Each has an associated parametric function for its mean and variance. Let Z be data from one of these four sources, and let t index its records. In a typical case, we have

Here, is a sum of numbers in income groups or numbers insured over a set of indices a, i that correspond to record t.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
  • Transformations of the proportions in the income groups and of the proportions insured follow normal linear models. For each of the two proportions estimated, there is a matrix of predictors, x, a vector of regression coefficients, β, a model variance νm, and a link function L.

Production and test models differ in details from the basic model described above. Parameters (the λ’s, α’s, and η’s above) can vary by demographic group. Models may have nonlinear functions for means and may assume distributions other than normal. There can be random effects in the regressions and mean functions. For specifics about production SAHIE models, see Bauder et al. (2008).

Model Evaluation

There is no “gold standard” relative to which the accuracy of SAHIE model estimates can be checked. There is no population for which the true numbers insured and uninsured are known, to which we could apply and test the model. At best, we can test the model assumptions by how well the model fits the observed data.

We use several techniques to assess the fit of the model. In addition to checking the plausibility of parameter estimates, we rely strongly on residual analysis and posterior predictive checks. In the process of estimating the model, we generate predictions of means and variances of the survey and administrative data. We can then check whether the observed data are consistent with these predicted means and variances. We also check for patterns in the residuals. Posterior predictive checks involve generating replicate data from the estimated distributions of observed data. We then check whether these replicate data resemble the observed data.

Model Weaknesses

There are several weaknesses in the modeling strategy that we hope to address in future research.

  • Assumptions of independence of observations. It is common to assume independence among observations at the level of the data that are modeled. There are practical reasons for making these assumptions, since estimation can become much more complex and difficult without them. It may be possible to relax these

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

assumptions. At the least, the effect of such assumptions should be investigated.

  • Assumptions about unobserved variables. The income and insurance parts of the SAHIE model are hierarchical, with the variables of interest occupying a different level than observed data in the hierarchy. Because these variables of interest are unobserved and inferable only by their “effect,” assumptions about their distributions are harder to justify and harder to verify than, say, assumptions about survey data. For example, we assume that a transformation of the proportion insured, given some predictors and parameters, follows a regression model with errors that are independent and have the same variance. This kind of assumption is common in small-area models but is worth investigating.

RESULTS

The SAHIE program has produced valuable estimates, using the CPS ASEC as the survey data. In this section, we report results from analyses of state-level estimates using the ACS 2008 data. We investigate whether we expect SAHIE estimates to be better using 1 year of the ACS data rather than 3 years of the CPS ASEC data and whether modeling for smaller domains is feasible using the ACS data.

Comparison of Models

Here, we compare the reliability of estimates from a CPS-based and an ACS-based model. We use models that are as similar as possible, except for the survey data. We use estimates from a preproduction estimation of the model using the CPS ASEC for collection years 2007, 2008, 2009 and from the corresponding ACS model for 2008.

We concentrate on one group of interest, uninsured children under age 19 at or below 200 percent of poverty by health insurance coverage. Variances or standard deviations of estimates are measures of uncertainty, so they are a reasonable basis for comparing estimates from different models. We calculated the ratios of posterior standard deviations from an ACS-based model over those from a CPS-based model. In Figure 13-1, we plot these ratios as the ACS-based standard deviation over the corresponding CPS-based one. The ratio of standard deviations is approximately the ratio of corresponding confidence intervals. The ACS-based estimates are generally better. At the median ratio of 0.78, a confidence interval from the ACS-based model will be approximately 20 percent shorter than an equivalent one from the CPS-based model. In only three

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
FIGURE 13-1 Ratios of posterior standard deviations from ACS-based and CPS-based models.

FIGURE 13-1 Ratios of posterior standard deviations from ACS-based and CPS-based models.

NOTES: For the number of uninsured children under 19 with family income less than or equal to 200 percent of poverty. The standard deviation from the ACS-based model is the numerator. The axes are on log scale.

SOURCE: Estimates are from the 2008 ACS-based and the 2007 CPS-based SAHIE models.

states, Idaho, Rhode Island, and Wyoming, do the ACS-based estimates have larger standard deviations.

Estimating More Income Groups

An anticipated advantage of using the ACS rather than the CPS ASEC data is that we will be able to produce reliable model-based estimates for smaller domains. We report here results from some early analyses to assess this possibility.

We modeled data and estimated the numbers and proportions insured and uninsured for the full cross-classification of six income groups: 0-133, 133-150, 150-200, 200-250, 250-400, and over 400 percent of poverty; four age groups: 0-18, 19-39, 40-49, and 50-64; four race/ethnicity groups: non-Hispanic white, non-Hispanic black, non-Hispanic people of other races or multiracial, and Hispanic; and both sexes.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

To illustrate the results, we consider a small income group—family income between 150 and less than or equal to 200 percent of poverty. We again consider children under age 19. The model-based estimates are aggregations over the race/ethnicity groups and the two sexes. We assess the value of model-based estimates by looking at the improvement in standard deviations compared with those of survey-only estimates.

We first confirm that the model estimates are consistent with the ACS direct estimates at this moderate level of aggregation. The estimates should be similar if each estimate is measuring the truth and the populations being estimated are large enough that the survey estimate is fairly reliable. A proper significance test is not possible here because the modeled estimates are based on the ACS direct estimates, so they are correlated, and we do not know the correlation. We can informally check that the estimates are close. Figure 13-2 shows the ACS estimates plotted against the model-based estimate for states for the number of uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty. The estimates are close. We also note that

FIGURE 13-2 Plot of the ACS direct estimates versus the ACS model-based estimates, for states, of the number of uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty.

FIGURE 13-2 Plot of the ACS direct estimates versus the ACS model-based estimates, for states, of the number of uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty.

SOURCES: Estimates are from the 2008 ACS survey and the 2008 ACS-based SAHIE model.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

the largest percentage differences are for South Dakota (71 percent), Louisiana (36 percent), and New Hampshire (36 percent). For only three states is the difference larger in magnitude than twice the ACS standard error.

Figure 13-3 shows the ratios of model estimate standard deviations to estimates of ACS-only standard deviations for state estimates of uninsured children under age 19 in the income group of 150-200 percent of poverty. All ratios are less than 1, so the model-based estimates are always “better.” As expected, there is less advantage to the model-based estimates as the population (and presumably the sample size) increases, because the survey estimates become more reliable. The maximum ratio, representing the smallest improvement, is 0.82 for California. The minimum ratio, representing the largest improvement, is 0.20 for South Dakota. Thus, for this group of estimates, confidence intervals from the model-based esti

FIGURE 13-3 Ratios of standard deviations of model-based estimates to estimates of standard deviations of survey-only estimates.

FIGURE 13-3 Ratios of standard deviations of model-based estimates to estimates of standard deviations of survey-only estimates.

NOTES: Standard deviations of the state estimates of the number uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty. The horizontal line is at the median of 0.54. Axes are on log scale.

SOURCES: Estimates are from the 2008 ACS survey and the 2008 ACS-based SAHIE model.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

mates are approximately one-fifth to four-fifths the length of those from the survey-only estimates.

A common measure of the goodness of an estimate is the coefficient of variation (CV), defined as the estimate’s standard deviation divided by its mean. We estimate the CV of an ACS estimate by dividing the estimate of its standard deviation by the estimate itself. Figure 13-4 shows the estimated CVs for the model estimates and the ACS estimates, for state estimates of the number of uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty. The states are ordered by increasing population under age 19. When CVs are high, estimates are likely to fluctuate relatively more over time because of sampling error than when they are low.

Table 13-1 shows summary statistics for CVs for these SAHIE model estimates of the number uninsured at the level of the data for the domains at the detailed modeling. The table also contains the same statistics for States in Order of Increasing Population Under 19

FIGURE 13-4 Estimated CVs by state for the ACS direct and the ACS model-based estimates of the number of uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty.

FIGURE 13-4 Estimated CVs by state for the ACS direct and the ACS model-based estimates of the number of uninsured children under age 19 with family income between 150 and less than or equal to 200 percent of poverty.

NOTE: States are ordered by increasing population under age 19.

SOURCES: Estimates are from the 2008 ACS survey and the 2008 ACS-based SAHIE model.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

TABLE 13-1 Percentiles of Coefficients of Variation for SAHIE ACS Model-Based Estimates of the Number Uninsured and Corresponding ACS Direct Estimates

Income

 

Percentile

25

50

75

90

95

99

000-133

model

0.11

0.14

0.17

0.19

0.20

0.22

 

direct

0.11

0.20

0.36

0.61

0.83

1.11

133-150

model

0.15

0.18

0.20

0.22

0.23

0.25

 

direct

0.25

0.41

0.64

0.92

1.02

1.12

150-200

model

0.12

0.16

0.19

0.21

0.22

0.25

 

direct

0.18

0.29

0.49

0.79

1.00

1.14

200-250

model

0.13

0.17

0.19

0.21

0.22

0.25

 

direct

0.19

0.32

0.54

0.79

1.00

1.16

250-400

model

0.12

0.16

0.19

0.21

0.22

0.24

 

direct

0.15

0.26

0.45

0.71

0.95

1.14

400+

model

0.15

0.18

0.20

0.22

0.23

0.25

 

direct

0.17

0.32

0.55

0.85

1.01

1.17

NOTES: Observations for which ACS does not have a variance estimate have been dropped. There are approximately 1,500 state/age/race/sex estimates for each income group.

SOURCE: Estimates are from the 2008 ACS survey and the 2008 ACS-based SAHIE model.

ACS direct estimates. A somewhat common, if informal, criterion has been that estimates with CVs greater than .30 should be used with caution. In fact, the Census Bureau had adopted the standard that, for data to be released, the majority of key statistics should have CVs below 30 percent (U.S Census Bureau, 2007). The actual usefulness of estimates can be judged only in the context in which they will be used. Nevertheless, Table 13-1 shows that over 99 percent of the SAHIE estimates for these domains fall below that threshold.

FINAL REMARKS

This research shows that an ACS-based SAHIE model generally has lower measures of uncertainty compared with a CPS-based SAHIE model. This is not surprising. Even if the SAHIE program did not implement any changes in the model, the estimates would be more reliable and timely because 1 year of ACS data is needed as opposed to using a 3-year average of CPS ASEC data. The second goal of the paper was to see if the ACS survey data could support modeling more income-to-poverty ratios. For most small domains (state/age/race/sex estimates for each income group), ACS model-based estimates offer a large improvement over using survey-only estimates. With ACS direct estimates, many of the domains at the state-level have CVs that should be used with caution or not at all.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

The ACS survey requires 5 years of data for estimates of all counties (U.S. Census Bureau, 2009). The first 5-year estimate of health insurance coverage will be the 2012 ACS to be released in fall 2013. Many of the small domains discussed in this paper should have a reliable 5-year health insurance coverage estimate at the state level. If a 5-year estimate of health insurance coverage is adequate for policy makers, the SAHIE state estimates should be viewed as a stopgap measure until 2013. It is unlikely that the ACS direct estimates will provide reliable county information for small domains (county/age/sex for each income group), even with multiyear data. It is likely that SAHIE county estimates for small domains will be reliable using underlying survey data of one year.

SAHIE estimates of uninsured people could be a valuable tool for policy makers to evaluate and administer means-tested programs, such as Medicaid and CHIP. These estimates can be used to target funds for outreach to specific uninsured and underserved demographic groups. States may want to use the estimates of the number of uninsured people, by income group, to approximate the total cost of subsidizing premiums for the health insurance exchanges.

FUTURE RESEARCH

Research on the SAHIE modeling is continuing. Some planned areas of research include the following:

  • Using the ACS direct variance estimates. We can produce replicate-based estimates of the variances of the ACS estimates. In the past, we have not used such variance estimates. We have instead predicted variances of survey estimates within the model. We intend to investigate ways to use the information in the replicate-based variance estimates.

  • Relaxing model assumptions. We intend to investigate alternatives to assumptions of independence and of constant variances.

  • Predictors for insurance coverage. We have many predictors for income status but fewer that relate to insurance coverage that are not also linked to income status.

  • Testing the model to incorporate more income groups (e.g., 250-300 and 300-400 percent of poverty) and age groups (e.g., 19-25-year-olds).

  • Multiyear ACS estimates. One of the administrative sources is income data from the 2000 census. This income data will not be available from the 2010 census. We intend to investigate replacing the census data with the multiyear ACS.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

ACKNOWLEDGMENT

This paper is released to inform interested parties of ongoing research and to encourage discussion of work in progress. Any views expressed on statistical, methodological, technical, or operational issues are those of the authors and not necessarily those of the U.S. Census Bureau.

REFERENCES

Bauder, M., Riesz, S., and Luery, D. (2008). Further Developments in a Hierarchical Bayes Approach to Small Area Estimation of Health Insurance Coverage: State-Level Estimates for Demographic Groups. Working Paper. Available: http://www.census.gov/did/www/sahie/publications/files/samb_jsm2008_paper_8.pdf [August 2010]. [August 2010].

Blewett, L., and Davern, M. (2007). Distributing state children’s health insurance funds: A critical review of the design and implementation of the funding formula. Journal of Health Politics, Policy and Law, 32(3), 415-455.

Centers for Medicare & Medicaid Services. (2010). Medicaid Program—General Information —Technical Summary. Available: http://www.cms.gov/MedicaidGenInfo/ [August 2010].

Czajka, J., and Jabine, T. (2002). Using survey data to allocate federal funds for the State Children Health Insurance Program (SCHIP). Journal of Official Statistics, 18(3), 409-427.

Dubay, L., Guyer, J., Mann, C., and Odeh, M. (2007). Medicaid at the ten-year anniversary of SCHIP: Looking back and moving forward. Health Affairs, 26(2), 370-381.

Fay, R., and Herriot, R. (1979). Estimates of income for small places: An application of James-Stein procedures to census data. Journal of the American Statistical Association, 74, 269-277.

Grumbach, K., Keane, D., and Bindman, A. (1993). Primary care and public emergency department overcrowding. American Journal of Public Health, 83(3), 372-378.

Kaiser. (2010). Explaining Health Care Reform: Questions About Health Insurance Subsidies. Working Paper #7962-02. Available: http://www.kff.org/healthreform/upload/7962-02.pdf [August 2010].

National Research Council. (2002). Data Needs for the State Children’s Health Insurance Program. Panel for the Workshop on the State Children’s Health Insurance Program. Editors A. Leibowitz and E. Pollack. Washington, DC: The National Academies Press.

Newacheck, P., Hughes, D., and Stoddard, J. (1996). Children’s access to primary care: Differences by race, income, and insurance status. Pediatrics, 97(1), 26-32.

Newacheck, P., Hughes, D., Wong, S., and Stoddard, J. (2000). The unmet health needs of America’s children. Pediatrics, 105(4), 989-997.

O’Hara, B. (2004). Do medical out-of-pocket expenses thrust families into poverty? Journal of Health Care for the Poor and Underserved, 15(1), 63-75.

O’Hara, B. (2008). Experimental health insurance estimates for low-income and demographic groups by state. Health Services Research, 43(5 Pt 1), 1693-1707.

O’Hara, B., Tangka, F. and Bauder, M. (in review). New Health Insurance Estimates Improve Measuring Program Participation in the National Breast and Cervical Cancer Early Detection Program.

P.L. 106-113. (1999). Medicare, Medicaid, and SCHIP Balanced Budget Refinement Act of 1999.

P.L. 111-3. (2009). Children’s Health Insurance Program Reauthorization Act of 2009.

P.L. 111-148. (2010). Patient Protection and Affordable Care Act of 2010.

P.L. 111-152. (2010). Health Care and Education Reconciliation Act of 2010.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×

Small Area Health Insurance Estimates. (2009). Model-Based Small Area Health Insurance Estimates (SAHIE) for Counties and States. Working Paper. Available: http://www.census.gov//did/www/sahie/ [August 2010].

Small Area Health Insurance Estimates Team. (2008). The Feasibility of Publishing County-level Estimates of the Number of Women Eligible for the CDC’s NBCCEDP. Working Paper. Available: http://www.census.gov/did/www/sahie/publications/files/cdc_feasibility_report_oct2008.pdf [August 2010].

U.S. Census Bureau. (2002). Current Population Survey Design and Methodology Technical Paper 63rv. Washington, DC: U.S. Government Printing Office.

U.S. Census Bureau. (2007). Quality Requirements for Releasing Data Products. Version 1.0. Washington, DC: U.S. Government Printing Office.

U.S. Census Bureau. (2009). Design and Methodology American Community Survey. Washington, DC: U.S. Government Printing Office.

Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 165
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 166
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 167
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 168
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 169
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 170
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 171
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 172
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 173
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 174
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 175
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 176
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 177
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 178
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 179
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 180
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 181
Suggested Citation:"13 Small-Domain Estimation of Health Insurance Coverage--Brett O'Hara and Mark Bauder." National Research Council. 2010. Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13024.
×
Page 182
Next: Appendix A: Workshop Agenda and Participants »
Databases for Estimating Health Insurance Coverage for Children: A Workshop Summary Get This Book
×
Buy Paperback | $62.00 Buy Ebook | $49.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report summarizes the proceedings of a workshop convened in June 2010 to critically examine the various databases that could provide national and state-level estimates of low-income uninsured children and could be effectively used as criteria for monitoring children's health insurance coverage.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!