Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 191
6
Implementation Issues
The previous two sessions focused on the research and issues related
to the concepts of thresholds and resources in a measure of medical care
economic risk (MCER). This session addresses issues related to data needs
associated with the implementation of such a measure: what data are
available now, what relevant data will become available under health care
reform, data quality and timeliness concerns, and whether the measure of
MCER could be released at the same time as the Supplemental Poverty
Measure (SPM).
MEASURING MCER: AN ASSESSMENT OF DATA SOURCES
John Czajka (Mathematica Policy Research) presented an overview of
the background paper he prepared for the workshop (see Part III of this
volume), explaining that he restricted himself to using federal data sets,
because it is the federal agencies that are going to produce the estimates.
Although the design of a measure of MCER need not be constrained by
currently available data, any such measure produced in the next few years
will need to be based almost exclusively on data collected currently. It may
be possible to add a modest number of new items to an existing survey,
but funding to support significant revisions or additions is not available.
The sponsor of the study panel, the Office of the Assistant Secretary
for Planning and Evaluation (ASPE), has asked for the development and
implementation of a new measure of medical risk as a companion measure
to the new supplemental income poverty measure. With regard to data, the
Medical Expenditure Panel Survey (MEPS), which became continuous in
191
OCR for page 192
192 MEDICAL CARE ECONOMIC RISK
1996, will presumably be the bedrock source for estimating the parameters
of a medical care risk index. However, that estimation should use variables
on risk factors and insurance coverage that are available in the Current
Population Survey Annual Social and Economic Supplement (CPS ASEC),
so that a family’s poverty status can be compared with its medical care risk.
Czajka said that he thought that any new measure, if it is intended to
be a companion to the SPM, should be estimated from the same data as the
SPM for at least a few years, so that users can see how the two measures
work together. After that one can think about whether there is some good
reason to separate them.
Factors Affecting Data Requirements
Alternative design choices have a big impact on the data requirements
for a medical care economic risk measure. There are two main choices.
The first is between a retrospective and a prospective measurement of risk.
The first workshop session focused on prospective measures. Retrospective
measures got attention in the second session. The second choice is whether
resources should include assets or be limited to only income.
It is also important to distinguish between data used to develop a
measure and data used to produce the measure that is disseminated to
the public. Development of a measure requires data that, ideally, will sup-
port alternative measures and enable evaluation. Longitudinal data would
be valuable for evaluation and validation—especially with a prospective
measure. Production of a measure requires data to support only one, not
multiple measures; however, timeliness, representativeness, and statistical
precision become more important.
Measures of Resources
Regarding measures of resources, the CPS ASEC is the official source
of estimates of income and poverty for the United States. It is also being
used to construct the SPM, as it has been used to construct a number of
experimental measures over the years. It includes the official measure of
money income, which is what is used to estimate poverty. And it also does
or will provide the measure of disposable income that goes into the SPM.
It is notable that some of the components of the SPM, and even in the
CPS, have to be imputed. Not all items are collected, and taxes are the big
item that has to be modeled and applied through the model or through
imputation. Information on taxes is not really collected in any of these
surveys.
The CPS added the measure of medical out-of-pocket expenditures
and a few other components in 2010, to help support the new SPM. The
OCR for page 193
IMPLEMENTATION ISSUES 193
assessment of the out-of-pocket medical expenditures data was surprisingly
good, considering that this survey has had trouble measuring health insur-
ance coverage.
With regard to resources, the CPS does not include everything that one
may think of as income. It does not include capital gains, which are picked
up in taxes and which MEPS picks up. It does not include a lot of what
people withdraw from their retirement accounts, which again is something
that is reported on the tax return and taxed, and MEPS picks up some of
those items. These are differences that can be important, especially for the
elderly, who have a lot of these types of resources.
A critical consideration with regard to the CPS ASEC is that it col-
lects no asset data of any kind and never has. To deal with that deficiency,
Czajka thought that, rather than trying to impute assets, it would be better
to add a small number of items to the survey. But quality of data cannot
be ensured without careful testing. The difficulty with imputation for fi-
nancial assets is that, in including assets, one is really interested in what
people have that is not reflected in their income. But the likelihood is that
the imputations will be driven heavily by income. And that does not get
the orthogonal piece that is needed. At the same time, it is very challenging
to collect asset data, and the idea that one could write a question and get
the answer wanted is optimistic. That is a particular problem with the CPS.
MEPS collects sources of income that correspond reasonably closely
to the CPS concept of money income. MEPS income questions follow the
federal tax form and include capital gains and state tax refunds, which
are not counted in CPS money income. Respondents who refer to their
tax returns would omit portions of earnings and possibly Social Security
benefits excluded from taxation. MEPS collects fewer of the expenses that
differentiate money income from disposable income. Like the CPS, however,
MEPS does not capture taxes paid (or earned income tax credit received).
Unlike the CPS, MEPS collects data on assets. Assets are divided into six
broad categories, and amounts are collected for all six types.
One potential concern about MEPS is that, in following the tax return,
if people really do pull out their returns, they would be reporting items that
do not correspond conceptually to what is picked up in the CPS, which
looks at gross income.
Measures of MCER
The CPS ASEC contains a fairly small set of items relating to measur-
ing medical care risk. It added medical out-of-pocket expenditures in 2010.
Data compare favorably to MEPS and the Survey of Income and Program
Participation (SIPP), despite the more detailed measurement in these other
surveys. The CPS ASEC collects sources of health insurance coverage in the
OCR for page 194
194 MEDICAL CARE ECONOMIC RISK
“past year” but no additional information on what expenditures are cov-
ered. It also collects basic work and activity limitations and general health
status—potentially useful in defining risk groups and matching to MEPS.
MEPS, in contrast, collects very extensive data on health conditions,
health status, the use of medical services, charges and payments, access to
care, and health insurance coverage over time. What it lacks is information
on what is actually covered by each sample member’s health insurance.
Nevertheless, MEPS can support both prospective and retrospective mea-
sures of medical care risk.
Data Quality
Limited information on data quality for these various components sug-
gests some areas in which improvements would be desirable.
With respect to income, CPS is the official measure and comparisons
show that it does collect more total income than the other major surveys.
Despite its overall strength, CPS ASEC income data have notable weak-
nesses. Reporting of all components of retirement income other than Social
Security is well below SIPP, which falls short of the CPS ASEC on most
other income sources. This suggests that a data area on which one would
rely heavily in evaluating the MCER of the elderly is one in which the CPS
would need to be improved.
Supplemental Nutrition Assistance Program (SNAP, formerly the Food
Stamp Program) benefits are currently received by 15 percent of the popu-
lation. The latest estimates comparing what CPS captures with what is
actually paid out suggest that the CPS is missing nearly half of total SNAP
benefits. Because SNAP benefits are part of what gets added to disposable
income to create the SPM, the new measure will not reflect the status of
low-income persons as accurately as it would if SNAP benefits were mea-
sured more effectively.
Nonresponse to income questions is high; 30 percent of total income is
imputed to respondents. A note of caution with the imputation methods is
that the medical care risk index is a new measure, and the CPS imputation
procedures may not take account of a lot of the components that may be
part of this new measure and that may introduce certain kinds of error into
the resulting index. If imputation procedures do not account for covariates
of medical care risk, the index is weakened. Finally, limitations of CPS
ASEC health insurance measures are well known.
Data collected in MEPS on measures of medical service use and medical
out-of-pocket expenditures are unique in their detail. MEPS data set the
standard, but there is little out there to compare to them. Because of the
MEPS panel design, attrition may be the principal concern. The first MEPS
interview is actually the second interview with the MEPS sample, because
OCR for page 195
IMPLEMENTATION ISSUES 195
the sample is drawn from National Health Interview Survey (NHIS) re-
spondents. Are persons with high MCER overrepresented among attriters?
One of the strengths of MEPS is that it does have overlapping panels,
so that there is consistent representativeness over time. This is a very im-
portant feature for a measure that will be used to track change over time.
Czajka noted that, after tracking the estimates from the NHIS and the
CPS ASEC for most of the past decade, MEPS uninsured rates for adults
and children rose sharply in 2007 and 2008, when the other major surveys
showed stable or declining rates.
Other Surveys
Czajka then discussed briefly three other surveys.
Survey of Income and Program Participation
The Panel on Poverty and Family Assistance, authors of the 1995
National Research Council (NRC) report Measuring Poverty: A New Ap-
proach, viewed SIPP as the survey of choice for a new poverty measure. It
had the advantage that it was designed expressly to support policy analysis;
it collected more detailed income data than any other federal survey, and
the quality of these data was almost uniformly high. SIPP’s design, with
collection of substantial core data in every wave and supplemental topical
modules with varying content, was ideally suited to a new poverty measure
that would require new data but not in every wave.
A decade later, the view was different. Some of the reasons were that
the 1996 redesign replaced overlapping panels, critical to consistent cross-
sectional representativeness; evidence of deterioration in income and asset
data emerged; and timeliness issues and repeated budget/sample cuts de-
tracted from the stability needed to support a key national indicator.
SIPP was terminated in 2006 but then restored in response to objec-
tions from users. A new panel started in 2008 will continue until replaced
by a reengineered SIPP to be fielded in early 2014. Under the reengineered
SIPP, annual interviews will replace the 4-month interviews; event history
calendar methods will be used to collect monthly data with 12-month
recall; most of SIPP core content will be retained; and key items from
annual topical modules—such as assets and medical and work-related
e
xpenditures—will be added to annual interviews.
There are other issues, however, in using SIPP for development or
initial production of the medical care risk index. To monitor implementa-
tion of health care reform, the index must be in place before the first new
SIPP data will be available. Initial, small sample tests of the new design
are encouraging, but it is not possible to fully assess the survey yet. SIPP’s
OCR for page 196
196 MEDICAL CARE ECONOMIC RISK
funding history and current budget climate raise concerns about sustained
funding.
However, the current SIPP, with panels longer than the MEPS panel,
could play a role in evaluating a prospective medical care risk index. Does
the subsequent experience of subpopulations match their estimated risk?
Where and why do deviations occur?
American Community Survey
The American Community Survey (ACS) has also been mentioned as
a potential source of data for an MCER measure. The ACS is attractive
because of its large sample size; 2 million households interviewed each year
would offer unmatched geographic detail. It captures similar content to
the CPS ASEC but is more limited in depth. The areas in which ACS data
are richer than the CPS ASEC tend not to be relevant to MCER. Another
problem is that the ACS questionnaire tends to be fixed for long periods
of time because the survey is designed to allow estimates to be cumulated
over periods of 3 to 5 years, to produce very small geographic detail. Also,
much of the content of the ACS is mandated by law. The bottom line: ACS
does not provide a viable option for developing or producing a medical
care risk index.
NHIS
The NHIS provides the sampling frame for MEPS. It is larger, and
most of the content is released on a more timely basis. It collects detailed
information on health status, which could enrich a prospective measure of
risk. But on most of the other potential components of the MCER measure,
the NHIS data are more limited than MEPS or nonexistent. In addition,
because NHIS provides the sampling frame for MEPS, NHIS data can be
linked to MEPS sample records; thus NHIS would add no new content.
NHIS by itself is therefore not an option for developing or producing a
medical care risk index.
Conclusion
In conclusion, Czajka reiterated that questions about data source are
reduced to what is collected in two surveys: the CPS and MEPS. MEPS col-
lects essentially all data elements needed to construct alternative versions
of the medical care risk index, whereas the CPS is missing critical variables
for certain variants on these measures. Yet the CPS ASEC will be used to
produce the new SPM to which MCER is intended to be a companion
measure. Having both measures in the same survey would allow researchers
OCR for page 197
IMPLEMENTATION ISSUES 197
to compare and contrast how families and individuals are classified by the
two measures. Such comparisons may be helpful in establishing the value
added by a measure of MCER.
Czajka noted other advantages of the CPS ASEC. A CPS-based index
could be released concurrently or shortly after the SPM or 10-11 months
after the end of the survey reference period, which is the prior calendar
year; MEPS would require an additional year. The CPS ASEC sample size is
five times the largest recent MEPS samples. In addition to its size, the CPS
ASEC sample combines independent, representative samples of the 50 states
and DC. State estimates, although lacking in precision for individual years,
could be important for state comparisons in monitoring implementation of
the Affordable Care Act.
However, a prospective index would depend on data collected in MEPS;
these data would have to lag a year, or release of the index would have to
be delayed a year.
Finally, whatever survey is chosen, it is important to reassess both data
and methodology within a few years of implementation.
FLOOR DISCUSSION
Participants expressed their views on the various issues flowing from
the presentation. Kenneth Finegold suggested that the study panel should
consider using the Transfer Income Model, version 3 (TRIM3) as part of
the discussion about developing the medical care risk index. TRIM3 is
ASPE’s model based on the CPS, but it has modules that do a lot of different
things, including aligning Medicaid participation to match administrative
totals in response to underreporting.
Given the limitations of the data sources that Czajka suggested, he
said, one way to go would be to match TRIM3 to MEPS. Finegold was not
sure that has been done, but TRIM3 was matched to NHIS for a number
of years. Matching with MEPS would help to get at assets and medical
conditions, information that is not in the CPS. The TRIM3 model has been
used, he thought, under contract to the NRC, in development of the SPM
over the years.
In response to a question about the currency of TRIM3, Laura
Wheaton (Urban Institute), who is on the TRIM3 microsimulation project,
responded. The work on this project is conducted for ASPE. When the
analysts get each year’s CPS data, over the course probably of the following
year, they do the corrections for underreporting that Finegold was talking
about. Depending on different project priorities, that schedule sometimes
slips. So it is certainly not as timely as the CPS, she said. Various things
often delay the baselines. For example, the project incorporates imputations
of immigrant status to identify undocumented aliens, an important issue
OCR for page 198
198 MEDICAL CARE ECONOMIC RISK
for modeling eligibility for these various programs that adds some delay. If
TRIM3 were to be used, it would certainly take more time than the release
of the CPS. It would be at least 4 months after the release of the CPS before
anything would be available.
Barbara Wolfe had two questions: first, is there any way of knowing
more about the underreporting of income and assets by quintile? The rel-
evance of the underreporting of SNAP clearly applies to the low-income
population, but her impression had been that some of the assets are really
underreported at the highest levels. She asked whether, for this particular
population group, if it is possible to think more narrowly about the under-
reporting, particularly in the CPS, but in both data sets.
Second, is there any way that MEPS could in fact be available earlier, so
that there would not be a year’s delay for outside people to work with it?
Maybe there is some way that in fact that could be speeded up so it would
not be that long a delay.
Steven Cohen explained the method by which MEPS produces its
expenditure estimates. After the household interview, the team conducts
a Medical Provider Survey to get detail on all sources of payment. For
this type of analysis, it actually produces an annual file on utilization and
insurance coverage about 5 to 6 months after the period from the CPS
turnaround. If the Agency for Healthcare Research and Quality (AHRQ)
was just getting the out-of-pocket information to add to the file rather than
depending on the Medical Provider Survey, which includes all the different
aspects of expenditures, and it is also getting income data, there could be a
way of accelerating that sort of a component of MEPS that would then be
updated with purer estimates from the medical provider survey.
If one is depending just on the out-of-pocket information, perhaps the
premium information that is paid out-of-pocket and the income informa-
tion that is available before the Medical Provider Survey is conducted
would be sufficient. The out-of-pocket information does get corrected, but
it is not much of a departure. So there are some options in moving forward
with these potential data resource needs for AHRQ to consider.
Czajka responded to the question about asset and income data. High-
end assets are a big problem for all surveys. But the Survey of Consumer
Finances (SCF) has a sample of tax returns, high income, and the ability
to poststratify those people because it has data on who had what income.
Comparisons between SIPP and the SCF show, however, that below the
really high end there are big differences by type of asset. People do a very
good job consistently across surveys in reporting the value of their home
and the debt on their home, so there is a good estimate of home equity. That
may be because people get a statement every month that tells them what
they owe on their house. But that is not true of most of the other types of
assets. Information on business assets is really bad in SIPP. It is not some-
OCR for page 199
IMPLEMENTATION ISSUES 199
thing that is evident from looking at the question. There are differences
across different components, so that is why he hesitates to say one can just
pop in some questions and get the results that one wants.
With respect to income, it is not really possible to look at comparisons
with administrative records by quintile in quite the same way because the fo-
cus is family income, and one cannot put together those pieces. He reported
working a few years ago on comparing several federal surveys and compar-
ing by quintile across the surveys. For example, although the CPS did much
better on earnings than the other surveys for the most part, if one looked
at the low end, SIPP was getting more earnings. Also, the ACS was getting
more earnings. With SIPP one can expect that a lot of the problem at the low
end is that people are changing jobs. They may have a number of jobs, and
if asked about this at the end of a year, they are likely to forget. With SIPP
they are asked every 4 months about this employment. Although a survey
may be the best income survey, it can still have some serious deficiencies.
Peter Cunningham commented that he was pleasantly surprised about
the CPS out-of-pocket estimates. But not having seen a comparison, he was
still somewhat skeptical. He wanted to know if the CPS asked about both
out-of-pocket expenditures on premiums as well as on services, and how
much detail there is.
Kyle Caswell responded that on the 2010 survey there are just two sum-
mary questions, and basically the same questions that they took from SIPP.
One question asks people retrospectively how much did they spend in the
last 12 months on health insurance premiums, at the individual level. The
second question asks about nonpremium medical out-of-pocket expenses.
It is very simple. But the comparison across the most available data at the
time—MEPS, SIPP, and the CPS—did look good. The main issue with the
CPS data is that it had a lot of people who reported a zero amount for ei-
ther premium or nonpremium out-of-pocket expenditures compared with
the other sources. Other components of the data, or different percentiles,
looked surprisingly good, considering how much less effort was made than
with MEPS.
Cunningham asked if they had looked separately at people with high
expenditures, the tails of the distribution, as well, because he thought
that when people are reporting from recall without any referring back to
records, the underreporting tends to be severe. Caswell responded that dif-
ferent points of the distribution were not wildly different. That is the main
story, he said, especially compared with the differences in data collection
across the surveys.
Caswell commented that, given the conceptual model that was pre-
sented and thinking about risk in a prospective way, he thought that model-
ing health status and particular health conditions would be very important.
She asked what the study panel would think about using a data source that
OCR for page 200
200 MEDICAL CARE ECONOMIC RISK
does not collect that information. For example, to use MEPS and then glue
that health information onto the CPS, what is involved?
Czajka responded that this general approach is what is used for the
SPM and the experimental measures, and recently only the CPS has started
collecting these other components. The approach has relied on these kinds
of methods, imputing for the expenditures largely from MEPS and also
maybe SIPP. It is done in microsimulation models and clearly has risks. One
is trying to match, and making assumptions that the variables that one is
not matching on are lining up with the variables that one is matching. It is
not perfect. He said he thought that, down the road, they would probably
have to think about expanding the CPS, if that is the source, to add other
measures, and hope for the same success that the Bureau has had with the
medical out-of-pocket expenditures.
Pamela Short asked if the CPS has a scale for excellent, very good,
good, fair, and poor health. Bringing that into the statistical matching
would generate a lot more confidence than without that variable.
Sarah Meier commented that, with respect to the conceptual model, the
risk adjustment models explain only a relatively small amount of the varia-
tion in health expenditures. So working with a very complex model that in-
cludes ICD-9 diagnosis codes and all sorts of other information, is actually
not going to be a big improvement from working with just a scale of poor,
good, and very good health status. Stepping back from the idealized version
of what one might want to do if there were no data limitations, the types of
variables in terms of health status that are available in the CPS would be a
reasonable base for the health characteristic cells of interest here.
The bigger issue would be the expenditure information, how strong
and solid that is in modeling of the actual expenditure distributions. So
one could think of modeling those in MEPS and then attaching that to the
CPS. The big issue with implementing the full type of model is the lack of
insurance characteristics in the CPS.
Banthin cautioned that she would be concerned about matching MEPS
to the CPS. The out-of-pocket spending measure is just out-of-pocket, it is
not total, she said. And there are those extra zeros. It involves the entire
distribution.
And just like the orthogonality of assets to income, it is important to
preserve the orthogonality of expenditures, even the distribution of medi-
cal expenditures to income or to out-of-pocket. It is a different dimension.
Also, a scale for excellent, very good, fair, and poor health is a great
predictor, but additional data are needed. Although she has built many a
simulation model in which expenditures are matched to others, that was
for simulating changes in policy. This is for the construction of a medical
care risk index. So it does concern her that one of the key variables would
be imputed with only a limited set of matching covariates.
OCR for page 201
IMPLEMENTATION ISSUES 201
Jennifer Madans (National Center for Health Statistics) stated that a
scale of excellent, good, fair, and poor health is good at the extremes, but
it is not very good in the middle. To know if someone is going to go into
long-term care or die or will have a lot of expenditures, it is probably good
at one end, and excellent is really good. But it is really bad about differ-
entiating that big group of people in the middle. She has never been a big
proponent of counting conditions either, because of the big variability in
what that means. So there needs to be something else added. She said that
some of the disability work is going in that direction, to really get a better
composite measure of health that will be a better predictor of the use of
resources. She did not think any of the surveys have that yet, but they are
moving toward it, and some composite of that is going to be needed.
OCR for page 202
