In its 1995 report, Measuring Poverty: A New Approach, the National Research Council (NRC) Panel on Poverty and Family Assistance recommended that the federal government revise its decades-old methodology for measuring poverty by updating the thresholds used to define basic needs; replacing money income with disposable income (which subtracts taxes and the costs incurred in going to work) as a measure of the resources available to meet these needs; recognizing the role of federal and state assistance programs in helping low-income families address basic needs by including the cash value of noncash benefits in these resources; and expanding the family unit over which these thresholds and income are calculated (National Research Council, 1995). The panel could not resolve how to handle the growing but widely varied expenditures for medical care and recommended the creation of a separate medical care risk index (MCRI) to be produced as a companion to a new measure of poverty.
Neither the proposed poverty measure nor the more vaguely defined MCRI could be estimated with data that were collected by any single survey, if at all, and data availability has continued to be an issue. Researchers at the Census Bureau, the Bureau of Labor Statistics, and other institutions have cobbled together a variety of experimental poverty measures over the years (see, e.g., Short, 2001, 2010), using imputation and statistical match-
1 The views expressed in this paper are those of the author and do not necessarily reflect the views or conclusions of the National Research Council, the Institute of Medicine, the study panel, or the sponsor.
ing to combine data from multiple sources, but no one has produced an experimental MCRI. Recently, an interagency working group was established by the U.S. Office of Management and Budget and charged with developing the guidelines under which the Census Bureau would cooperate with the Bureau of Labor Statistics to produce a Supplemental Poverty Measure (SPM) on an annual basis, beginning in 2011. This new measure would not replace the current, official poverty measure, but its release on a formal basis in conjunction with the official measure would ensure that it received greater attention—and use—than previous experimental measures. Also unlike the official measure, the methods used to create the SPM would be modified over time as researchers inside and outside government proposed improvements supported by research or developing consensus.
With the SPM about to become reality, attention has refocused on the MCRI, and the charge to the Panel on Measuring Medical Care Risk in Conjunction with the New Supplemental Income Poverty Measure is to examine the state of the science in developing a measure of medical care risk that is feasible to produce and can be used to track changes in medical care economic risk as the implementation of health care reform progresses.
Data issues loom large, compounded by complex conceptual issues. This paper examines the data sources that might be used to construct a measure of medical care economic risk. Although the design of an MCRI need not be constrained by the data that are available at present, the reality is that, if an MCRI is produced in the next few years, it will have to be based almost exclusively on data that are being collected currently. The addition of a modest number of new items to an existing survey is possible, and the Census Bureau has done exactly that for the SPM. However, none of the federal agencies likely to be involved in the development and production of an MCRI has the budgetary resources to support significant additions to any of the key surveys. The Census Bureau, in fact, has not received the funding that was included in the president’s budget to support production of the SPM. Therefore, the existing data sources will largely define what is possible to include in an MCRI.
Multiple surveys could be considered as candidates to host the MCRI, each of them offering some unique advantage, but the sponsor of the study panel, the Assistant Secretary for Planning and Evaluation (ASPE), has indicated that the MCRI should be constructed from variables that are available in the Census Bureau’s Current Population Survey Annual Social and Economic Supplement (CPS ASEC). This will make it possible to compare a family’s medical care risk with its poverty status as reflected in the new SPM, which is also based on the CPS ASEC. ASPE has also indicated that
a second survey, the Medical Expenditure Panel Survey (MEPS), which is designed and sponsored by the Agency for Healthcare Research and Quality (AHRQ)—also in HHS—should serve as the source of data for modeling medical care risk, with the results of that modeling to be translated to the CPS ASEC through variables that are common to the two surveys.
MEPS could in fact stand alone as home to the MCRI. Unlike the CPS ASEC, MEPS collects essentially all of the variables that are likely to be needed to construct the MCRI. Although the study panel can recommend a different approach using different data, the argument that the MCRI should be measured from the same data as the SPM is compelling—at least until the two measures are firmly established and their relationships to each other are thoroughly understood. Users will want to know how the two measures compare and how they differ for the same family or individual.
Implications of Alternative Design Options
Two fundamental decisions regarding the design of an MCRI have important implications for its data requirements. The first is whether medical care risk is to be defined retrospectively or prospectively. With a retrospective definition, the principal data need is for out-of-pocket expenditures for medical care during a specified accounting period. For segments of the population that may have forgone care because of limited insurance coverage and an inability to pay for care out-of-pocket, actual expenditures are a poor measure of medical care risk and must be supplemented with other measures. For most of the population, however, actual expenditures may be sufficient to measure risk as a retrospective measure. With a prospective measure, which the 1995 NRC panel recommended, actual expenditures in the recent past, though not unimportant, become less important than measures of current health status on the one hand and the limits of insurance coverage on the other—both of which relate to the likelihood of incurring medical expenditures in excess of what an individual or family can afford to pay. In assessing data availability, I consider all of these characteristics.
The second decision is whether family resources will include only income or also assets. Doyle (1997), in a thoughtful discussion of issues related to defining and constructing an MCRI 2 years after the release of the 1995 NRC report, assumed that some component of assets would be included in the resources measure. She also considered ready access to loans as an alternative resource for covering unexpected medical costs, although she noted that access to such loans was generally restricted to families with significant assets as collateral. The inclusion of assets has important implications for the measurement of medical care risk among the elderly, whose income flows are diminished by retirement but who have had an entire
working life (and more) to accumulate assets. Again, I consider available measures of both income and assets.
Development Versus Production
In addition to these design considerations, it is important to distinguish between the data that are available for developing and evaluating an MCRI and the data available for production of a measure, as the data needs are different. In addition to the variables needed to construct the measure, production requires timely data that are representative of the entire U.S. population. Neither trait is critical for development and evaluation of an index, but the data needs are more extensive. Furthermore, both development and evaluation would be enhanced by longitudinal data that would allow examination of the consequences for persons flagged as high risk.
CPS ASEC and MEPS
Measures of Resources
The CPS ASEC is the official source for estimates of income and poverty for the U.S. population and will also be used to construct the SPM. The available data on income, then, include the official measure of money income—which is also used to estimate poverty—and the measure of disposable income that will be used for the SPM. This latter measure of income includes the cash value of noncash benefits (such as the Supplemental Nutrition Assistance Program, formerly the Food Stamp Program) while it subtracts taxes (which for some low-income families implies the addition of the refundable portion of a negative income tax), work-related expenses, and medical out-of-pocket expenditures (including premiums).
The potential inclusion of assets in the measure of resources is significant because the CPS collects no asset data. If, as expected, the CPS ASEC serves as the base data set for the MCRI, measures of liquid or near-liquid assets would have to be added to the survey or imputed from an external source. Imputation is a decidedly second-best option, because the point of including assets in the resource measure is that some people—particularly among the elderly—with relatively low income may nevertheless have sufficient assets to weather unexpected medical expenses. Income will not be a strong covariate of asset holdings among such persons, and it is not apparent that the CPS provides other strong covariates of asset accumulation. It should be assumed, therefore, that imputed assets will provide less value-added to an MCRI than directly measured assets. At the same time, adding new questions to the CPS to measure financial assets (property assets would probably not be needed for an MCRI as they are not very fungible, although a credit line based on home equity is a readily available resource
used by many consumers to cover needs for cash) is not necessarily straightforward. The Survey of Income and Program Participation (SIPP) collects a wide range of asset data in topical modules that are administered annually, but comparisons with the Federal Reserve Board’s Survey of Consumer Finances (SCF), which focuses almost exclusively on the collection of income, assets, and debts, show serious weaknesses (Czajka, Jacobson, and Cody, 2003). For example, aggregate SIPP estimates of financial assets and total assets in 1998 were 55 percent of the assets measured by the SCF. Excluding the wealthiest families increased this fraction to 74 percent, but it is clear from this that adding measures of assets without field-testing, which the Census Bureau currently lacks the resources to support, is risky.
MEPS collects data on multiple sources of income that, in the aggregate, correspond closely to the CPS concept of money income. Potential differences exist because the MEPS income questions follow the federal tax form and include capital gains, state tax refunds, and lump sum (as opposed to regular) withdrawals from retirement accounts, which are not counted in CPS money income. In addition, respondents who refer to their tax returns—who may be only a small fraction of all MEPS respondents— would omit those portions of their earnings that are excluded from taxation (and not reported on the tax return). They might also report only taxable rather than total Social Security benefits. Like the CPS, MEPS would require imputation and modeling to convert this money income to the concept of disposable income used for the SPM. MEPS collects fewer of the expenses that differentiate money income from disposable income; specifically, MEPS does not collect work-related expenses, which were added to the CPS ASEC in 2010. Like the CPS, however, MEPS does not capture taxes paid (or earned income tax credits received), which must be modeled.
Unlike the CPS ASEC, however, MEPS collects data on assets. MEPS obtains balances for retirement accounts (collectively), bank accounts, and other financial assets and requests the estimated value and debt for the family home, all vehicles, and all other nonfinancial assets. MEPS also requests the total amount of all additional debt (for example, loans and credit card balances). To my knowledge, the MEPS asset data have not been subjected to the same, detailed evaluations as the asset data from SIPP, the Panel Study of Income Dynamics, and the Health and Retirement Study, which makes them something of an unknown. Evaluations of asset data collected in these other surveys have shown that asset questions are subject to high item nonresponse and significant reporting error. Nevertheless, MEPS is well ahead of the CPS in having asset data at all.
Variables to Measure Medical Care Economic Risk
As part of its development of the SPM, the Census Bureau added a measure of medical out-of-pocket expenditures to the CPS ASEC in 2010
(this variable and the other new items are not included in the public use data for that year). Surprisingly, CPS ASEC estimates of medical out-of-pocket expenditures compare favorably to estimates from MEPS and SIPP, despite the more extensive measurement in these latter surveys (Caswell and O’Hara, 2010). The CPS ASEC also includes measures of health insurance coverage during the prior year, but the CPS does not collect any information on what was included in such coverage. This deficiency becomes critically important if medical care risk is defined prospectively.
The CPS ASEC identifies deafness or blindness and several types of activity limitations, including difficulty in concentrating, remembering, or making decisions; walking or climbing stairs; dressing or bathing; and doing errands. Separately, the survey identifies persons with work limitations and ascertains each household member’s general health (excellent, very good, good, fair, or poor). These items together with the reported receipt of one or more sources of disability income provide the only indication that a person has health issues that increase the risk of excessive expenditures for medical care in the near future. I note, however, that the items collected in the CPS are similar to what Short and Banthin (1995) used to assign the privately insured to either of two risk groups as part of their work to identify the underinsured. Other variables that were instrumental to that work are not captured in the CPS ASEC, however.
MEPS collects extensive data on health conditions, health status, the use of medical services, charges and payments, access to care, and health insurance—all of which are important in constructing a prospective measure of medical risk. In its initial year, 1996, MEPS also collected and abstracted detailed information from the health insurance plan booklets for sample members covered by private insurance. Similar data collected as a supplement to the 1987 National Medical Expenditure Survey provided a critical input to Short and Banthin’s (1995) estimates of the nonelderly underinsured. If such data were available today, they would very likely be the most central element in a prospective measure of medical risk.
Limited information on the quality of selected sets of relevant variables in the CPS ASEC and MEPS is available—not enough to make an overall assessment but worth reviewing for the perspective it may provide. Although the CPS accounts for more income overall and for most sources than does the Census Bureau’s nominal income survey, SIPP (Czajka and Denmead 2008; Roemer, 2000), the CPS falls short of SIPP in the measurement of retirement income (Czajka and Denmead, 2011). This limitation is notable because the elderly have disproportionately high medical expenditures and
would presumably account for a disproportionate share of those who are identified as at risk by the MCRI. Underreporting of retirement income will upwardly bias the MCRI among the elderly. In addition, recent research suggests that the CPS may understate annual SNAP benefits by close to one-half (Meyer, Mok, and Sullivan, 2009), which means that the SPM will overstate poverty and the MCRI will overstate medical risk among the nearly 15 percent of the population currently participating in SNAP.
The limitations of the CPS ASEC measure of health insurance coverage are well known and thoroughly documented. Briefly, the CPS ASEC asks respondents about their health insurance coverage in the past year, but the survey’s estimates of the uninsured compare more closely to other surveys’ estimates of people uninsured at a point in time (that is, at the time of the survey or in a particular month) rather than people uninsured for an entire year, which are about half as high. Consequently, users often reinterpret the CPS ASEC measure of health insurance coverage as indicating how many people have coverage (or a lack thereof) at a point in time. If respondents are in fact answering the health insurance questions as if they were asking about their coverage at the time of the survey, then this poses no problem. If, instead, respondents are doing a poor job of answering what they correctly hear as questions about their coverage in the prior year, then the resemblance to point-in-time coverage may be merely coincidental and the responses may not exhibit appropriate covariation with other variables in the survey—or do not do so consistently over time (Davern, 2010). Findings from research using Medicaid enrollment data linked to CPS ASEC data are more consistent with the latter interpretation (see Klerman et al., 2009). For present purposes, the implication is that, despite its widespread use, health insurance coverage as measured in the CPS ASEC may not be as good a predictor of medical care risk as measures of health insurance coverage collected in other major federal surveys.
The measures of private health plan content, medical service use and medical out-of-pocket expenditures collected in MEPS are unique in their detail. One could say that they provide the standard against which the data collected in other surveys are evaluated—if there were such data collected in other surveys. The strength of the MEPS measures of health insurance coverage is more ambiguous. At least in part by design, MEPS estimates of the uninsured tend to run higher than other surveys, but Davern (2010) identified divergent trends in health insurance coverage between MEPS and several other surveys in the middle of the past decade. Between 2006 and 2008, MEPS uninsured rates turned upward, whereas CPS and National Health Interview Survey (NHIS) uninsured rates remained flat or declined. The difference was especially pronounced among children. AHRQ staff reviewed the MEPS data in detail but found no clear cause.
Although this review of available data focuses on the two surveys that are preordained to play central roles in the development and production of an MCRI, other surveys have been mentioned as candidates in the past or more recently and, for this reason, merit brief discussion.
When the NRC Panel on Poverty and Family Assistance recommended major changes in the measurement of poverty in the United States, SIPP was the survey of choice. SIPP, after all, had been designed expressly as a vehicle to support policy analysis. SIPP collected far more detailed data on income than any other federal survey, and the quality of these data was almost uniformly high. Furthermore, SIPP’s design, with the collection of substantial core data in every wave and supplemental topical modules whose content varied from wave to wave, was well suited to a new poverty measure that would require new data elements but not necessarily every wave. By the time the NRC convened a workshop to review and update the recommendations in the 1995 report, SIPP’s star had fallen (National Research Council, 2005). With a redesign in 1996 that replaced annual, overlapping panels with abutted panels, SIPP could no longer provide consistently representative data. Compounding this problem, evidence began to emerge that the quality of SIPP’s income and asset data had deteriorated. In addition, SIPP continued to use an antiquated processing system that contributed to a decline in timeliness, and an established pattern of budget cutbacks and unpredictable sample reductions had made it clear that SIPP lacked the stability desired to support a key national indicator.
As if to underscore this last point, SIPP was terminated in 2007 and then brought back to life, but only after scores of users voiced their dismay. Although the 2004 panel was extended—with a sample cut of about one-half—and a new panel was initiated in late 2008, the Census Bureau launched a new redesign—a reengineering of the survey to collect in one annual interview what was previously collected in three waves and thereby reduce the survey’s rising costs by about two-thirds. To achieve this goal, the survey will use event history calendar methods to collect monthly data with a 12-month recall. Most of SIPP’s core content is being retained, and key items from annual topical modules—such as assets and both medical and work-related expenditures—will be added to the annual interviews. The new survey is scheduled to be fielded with its first round of annual interviews in early 2014, collecting data on calendar year 2013 (Fields, 2011).
To monitor the implementation of health care reform, an MCRI must be in production before the first new SIPP data are available. Thus the timing of the new design presents a serious problem for its use in either the development or initial production of an MCRI. Furthermore, while initial, small sample tests of the new design are encouraging, one cannot fully assess
the new design as yet. Another drawback, independent of the quality of the data, is that the new survey’s nonoverlapping panels, if maintained, do not address the declining representativeness of individual panels over time—a limitation present in the current design since 1996. On top of these considerations, SIPP’s funding history and the current budget climate raise concerns about sustained funding for the survey over time. However, the current SIPP, with panels longer than MEPS, could play a role in evaluating a prospective MCRI. In particular, such data could be useful in determining whether or not the subsequent experience of subpopulations matches their estimated risk.
The American Community Survey (ACS), which has replaced the decennial census long form, is attractive because of its exceedingly large sample size. Data are collected from 2 million households each year, and the sample can support estimates for levels of geography well below the state. The ACS would add a dimension of geographic detail to an MCRI that no other survey could match. However, in most respects the data collected in the ACS are more limited than what is collected in the CPS ASEC. The areas in which ACS data are richer than the CPS ASEC are not relevant to an MCRI. Moreover, the ACS questionnaire will not be open to revision for several years, ruling out for the near term any addition of items that would improve the survey’s ability to support an MCRI, and its mandatory nature severely restricts the content that can be included. Thus the ACS does not provide a viable option for either developing or producing an MCRI.
The NHIS, which serves as the sampling frame for MEPS, is larger than MEPS, and most of its content is released on a more timely basis. The NHIS collects more detailed information on health status, which could help to enrich a prospective measure of medical care risk. On most other components of an MCRI, however, NHIS data are more limited or nonexistent. Furthermore, because the MEPS sample is drawn from the NHIS sample, the data collected in the NHIS can be linked to MEPS records. In this sense, then, NHIS would add nothing in the way of content to what MEPS already provides, although the health data collected in the NHIS would be more current if used directly from the survey rather than through a linkage to MEPS, where it is 1 to 3 years older than the items collected in MEPS. The NHIS, then, is off the table as a resource for developing or producing the MCRI.
To summarize, questions about the data available to produce an MCRI come down to what data are collected in two surveys: MEPS and the CPS ASEC. MEPS collects essentially all of the data elements that would be needed to construct alternative versions of an MCRI whereas the CPS ASEC is missing critical variables for certain variants on an MCRI. However, the
CPS ASEC will be used to produce the new SPM, to which the MCRI is intended as a companion measure. Producing both measures from the same survey would enable researchers to compare and contrast how families and individuals are classified by the two measures. Such comparisons may be particularly helpful in establishing the value added to a poverty measure by the MCRI.
The CPS ASEC does have other advantages over the MEPS as the base for an MCRI. Depending on how it is defined and constructed, an MCRI based on the CPS ASEC could be released at the same time or shortly after the SPM, or 6-7 months after the completion of data collection (and 10-11 months after the end of the survey reference period). Given current production schedules, a MEPS-based measure would require an additional year.
There is a wrinkle in this assessment, however. A prospective MCRI would depend critically on data collected in MEPS, so releasing a CPS-based MCRI at the same time as the SPM would require using MEPS data from the previous year. The other significant advantage of the CPS ASEC is its sample size, which is five times that of the largest recent MEPS samples. The greater CPS ASEC sample size would support more precise estimates generally while allowing more extensive subgroup analysis. Finally, the CPS ASEC sample consists of independent, representative samples of the 50 states and the District of Columbia and, as such, can support state-level estimates, although not with satisfactory precision in every case. If the MCRI is to play an important role in monitoring the implementation of the Patient Protection and Affordable Care Act, this property of the CPS ASEC could be invaluable.
Caswell, K.J., and O’Hara, B. (2010). Medical Out-of-Pocket Expenses, Poverty, and the Uninsured. SEHSD Working Paper 2010-17. Washington, DC: U.S. Census Bureau, Social, Economic, and Housing Division. Available: http://www.census.gov/hhes/povmeas/methodology/supplemental/research/Caswell-OHara-SGE2011.pdf.
Czajka, J.L., and Denmead, G. (2008). Income Data for Policy Analysis: A Comparative Assessment of Eight Surveys. Final report to U.S. Department of Health and Human Services. Washington, DC: Mathematica Policy Research.
Czajka, J.L., and Denmead, G. (2011, August). Second Interim Memorandum: Retirement Income. Washington, DC: Mathematica Policy Research.
Czajka, J.L., Jacobson, J.E., and Cody, S. (2003, August). Survey Estimates of Wealth: A Comparative Analysis and Review of the Survey of Income and Program Participation. Washington, DC: Mathematica Policy Research.
Davern, M. (2010, June). Unstable Ground: Comparing Health Insurance Estimates from National Surveys. Paper prepared for the Workshop on Evaluating Databases for Estimating Health Insurance Coverage for Children, Committee on National Statistics, National Research Council, Washington, DC.
Doyle, P. (1997). Who’s at Risk?: Designing a Medical Care Risk Index. Working Paper. Washington, DC: U.S. Census Bureau. Available: http://www.census.gov/hhes/povmeas/publications/medical/doyle2.html.
Fields, J. (2011, June). Re-Engineering the SIPP: Creating the SIPP-EHC. Paper presented at the Household Survey Producers Workshop, June, National Research Council, Washington, DC.
Interagency Technical Working Group on Developing a Supplemental Poverty Measure. (2010). Observations from the Interagency Technical Working Group on Developing a Supplemental Poverty Measure. Washington, DC: U.S. Census Bureau.
Klerman, J.A., Davern, M., Call, K.T., Lynch, V., and Ringel, J.D. (2009, November/December). Understanding the Current Population Survey’s insurance estimates and the Medicaid undercount. Health Affairs 28(6):w991-w1001.
Meyer, B.D., Mok, K.C., and Sullivan, J.X. (2009, February). The Under-Reporting of Transfers in Household Surveys: Its Nature and Consequences. Working Paper No. 09.03. The Harris School of Public Policy Studies, University of Chicago. Available: http://harrisschool.uchicago.edu/About/publications/working-papers/pdf/wp_09_03.pdf.
National Research Council. (1995). Measuring Poverty: A New Approach. C.F. Citro and R. Michael (Eds.). Panel on Poverty and Family Assistance: Concepts, Information Needs, and Measurement Methods. Committee on National Statistics, Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
National Research Council. (2005). Experimental Poverty Measures: Summary of a Workshop. J. Iceland, Rapporteur. Group for the Workshop to Assess the Current Status of Actions Taken in Response to Measuring Poverty: A New Approach. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
Roemer, M.I. (2000, June). Assessing the Quality of the March Current Population Survey and the Survey of Income and Program Participation Income Estimates, 1990-1996. HHES Working Paper, Income Survey Branch, Housing and Household Economic Statistics Division. Washington, DC: U.S. Census Bureau. Available: http://www.census.gov/hhes/www/income/publications/assess1.pdf.
Short, K. (2001). Experimental Poverty Measures: 1999. Current Population Reports, Consumer Income, P60-216, U.S. Census Bureau. Washington, DC: U.S. Government Printing Office.
Short, K. (2010, January). Experimental Modern Poverty Measures 2007. Paper presented at the Allied Social Science Association Meetings, Atlanta, GA. Available: http://www.census.gov/hhes/www/povmeas/papers.html.
Short, P.F., and Banthin, J.S. (1995, October). New estimates of the underinsured younger than 65 years. Journal of the American Medical Association 274(16):1,302.