Appendix C
Summary of Empirical Modeling Methodology
The committee commissioned a body of empirical analyses to examine geographic variation in spending, utilization, and quality using public and commercial datasets. The goals of the analyses were to characterize and account for the presence and magnitude of geographic variation across different geographic units, payers, and clinical condition cohorts. The population-specific studies conducted by Acumen, LLC, The Lewin Group, and Harvard University were carried out using the research framework outlined in Table C-1.1 Precision Health Economics’ methodological approach in synthesizing these results and evaluating geographic variation in total health spending is then summarized. The complete methodological details are available in the subcontractor reports.2
_________________
1This table only presents the methodology for the Medicare 2007-2009 analysis and does not show the Medicaid 2007-2009 analysis, the Medicare 1992-2010 growth analysis, or the Medicare Advantage 2007-2009 analysis, all of which use variations on this methodological approach.
2In addition to the studies summarized in Appendix C, the committee also commissioned reports from the University of Pittsburgh and the Dartmouth Institute for Health Policy and Clinical Practice. All papers can be accessed through the Institute of Medicine website via the following link: http://www.iom.edu/geovariationmaterials.
TABLE C-1
Acumen, Lewin, and Harvard’s Study Approach and Methodology in the Medicare and Commercial Analyses
Acumen | Lewin | Harvard | |
Data Source | Medicare Analysis: (Parts A, B, and D) Claims and Enrollment data *Medicare Advantage (MA) (Part C) was analyzed separately | Optum De-identified Normative Health Information (dNHI) Database | Thomson Reuters MarketScan Commercial Claims and Encounters database |
Years of Analysis | 2007–2009 | 2007–2009 | 2007–2009 |
Study Population |
• 100% sample of all Medicare fee-for-service beneficiaries • Majority of sample are over age 65. The clinical condition cohort analyses were limited to ages 18 and older. • Excludes costs for beneficiaries in the months that they are enrolled in Medicare Advantage (Part C). Excludes all beneficiaries who have any third-party payment in the observation window. |
• Included enrollees between the ages 0–64, with a small sample over age 65. • The clinical condition cohort analyses were limited to ages 18–64. |
• Included enrollees between the ages 0–64. • The clinical condition cohort analyses were limited to ages 18–64. |
Treatment of “Capitated Claims” |
• Not applicable |
• Excluded observations (0.3% of population) |
• Imputed value (6% of population) |
Measurement of Spending |
• Total all-cause spending includes all costs incurred by Medicare and the patient in covering inpatient, outpatient, hospice, home health, skilled nursing, carrier, durable medical equipment and Part D claim types. • Medicare analysis follows input-price standardization methodology developed with CMS as |
• Total all-cause spending includes all costs of all facility, provider and prescription drug costs incurred by payer, secondary payer and patient. • See Harvard input-price adjustment memo (Appendix E) |
• Total all-cause spending includes all costs of all medical and prescription drug costs incurred by payer, secondary payer, and patient. • See Harvard input-price adjustment memo (Appendix E) |
part of the Hospital Value Based Purchasing (HVBP) program. • The inpatient claims exclude indirect medical education (IME) and disproportionate share (DSH). |
|||
Measurement of Utilization | Measured in two ways: | Measured in two ways: | Measured in two ways: |
• Counts per service • Input price-standardized cost (separate input and output price adjustment is unnecessary in this analysis, as Medicare sets final prices accounting for regional variation). |
• Counts per service • Output-price standardized cost (Harvard output-price adjustment memo, Appendix E.3) |
• Counts per service • Output-price standardized cost (Harvard output-price adjustment memo, Appendix E.3) |
|
Measurement of Quality |
• Follows the Agency for Healthcare Research and Quality methodology for analyzing quality • The aggregate quality composites include 8 PSI, 6 IQI, and 12 PQI measures. • Separate quality measures computed for 13 of the clinical condition cohorts. |
• Aggregate analyses included AHRQ patient safety indicator (PSI) #90, pediatric quality indicator (PDI) #19, inpatient quality indicator (IQI) #91 and prevention quality indicator (PQI) #90. • Cohort quality analyses were limited those with adequate sample sizes, namely coronary heart disease, diabetes, and low back pain. |
• Created original composite measures: 4 domains of PQI, PSI, process measures, and readmis-sions within 30 days of discharge. • Excluded analysis of PDI as these were rare in the dataset. |
Multiple Regression Analyses | |||
Dependent Variables of Spending and Utilization |
• OLS Regression without area fixed effects; estimated area level effects as average of residuals from first model estimation • Did not “shrink” estimates |
• OLS Regression with area fixed effects: |
• OLS Regression without area fixed effects; estimated area level effects as average of residuals from first model estimation (Sensitivity analysis showed 0.98 correlation to fixed effects model) • Empirical Bayes framework of “shrinking” estimates used to correct for small sample size variation |
Acumen | Lewin | Harvard | |
Market Level Analysis |
• Used a 2-stage regression method: • Method comparable to Harvard; see Appendix F: Harvard Market Level Analysis Methodology Memorandum (11.21.12). |
• Used Harvard-developed market level measures (Appendix F.1). • Used a 2-stage regression method: • Used different set of market predictors. |
• Used a 2-stage regression method: Step 1: OLS regression without fixed effects. • Method comparable to Acumen; see Appendix F: Harvard Market Level Analysis Methodology Memorandum (11.21.12). |
Quality Analyses |
• Conducted logistic regression for the IQI and PSI, and OLS regression for the PQI composites. |
• Conducted logistic regressions for the PSI and PQIs, predicted the outcomes at an individual level and then averaged at area level to produce a rate. • Risk adjusted based on covariates from cluster regressions rather than covariates used in the AHRQ methodology (PHE, p. 12). |
• Used logistic models for “rare” quality outcomes, and linear models for all other outcomes. • The rate is risk adjusted by multiplying the ratio of observed to expected outcomes by a reference rate. |
Model Specification Clusters | Refer to Appendix D for the complete list of independent variables used by each subcontractor. | ||
Correlation Analyses | |||
“Within” Analysis |
• Examines the distribution of the ratios (10th percentile, 90th percentile, min, max, etc.) of the highest-spending to lowest-spending HSAs by HRR |
• OLS regression specifications followed. • Wald Test on HSA fixed effects within HRR. Statistical significance indicated intra-regional variation |
• Examines the distribution of the ratios (10th percentile, 90th percentile, min, max, etc.) of the highest-spending to lowest-spending HSAs within by HRR |
• Also performs an OLS regression |
in spending at HSA level not captured by HRR dummy variables • Intra-regional variation examined using CV for HSA PMPM spending. |
• Also Performs an OLS regression of HSA average spending on HRR indicator variables, weighted by beneficiary months in each HSA. |
|
“Between” Analysis |
Reported Pearson correlation of: • Medicare beneficiary utilization across clinical condition cohorts • Medicare beneficiary utilization and quality, across condition cohorts and in aggregate population |
Reported Pearson and Spearman correlations of: • HRR spending and rankings across clinical condition cohorts • HRR quality and rankings across clinical condition cohorts • Correlation of spending and quality measures |
Examines correlations of: • HRR spending and rankings across clinical condition cohorts • Correlation of spending and quality measures • Correlation of quality across cohorts |
Precision Health Economics Study Approach and Methodology Summary:
The Precision Health Economics (PHE) report first synthesized and summarized the results from spending, utilization, and quality regression analyses of the population-specific studies conducted by Acumen, Lewin, and Harvard, allowing for easy comparison of findings across public and private payers. In order to examine variation “within” HRRs, PHE conducted a random effects regression of spending at utilization at the HSA level, with the random effects at the HRR level.
Additionally, PHE created a measure of total health care spending, attempting to account for the total United States population by including spending for Medicare, Medicaid, commercially insured, and uninsured populations. This measure was created using the following steps:
1. Obtained spending estimates for Medicare, Medicare Advantage (or Medicare managed care), Medicaid, and commercially insured populations from the empirical analyses conducted by Acumen, Lewin, and Harvard.
2. Estimated spending for the uninsured and Medicaid managed care by HRR.
3. Created payer-specific weights to estimate unadjusted, total health care spending. The OptumInsight and MarketScan spending data were alternately used as “proxies” for commercial spending.
4. Created two measures of total PMPM spending by HRR, first unadjusted and then adjusted for input prices. Both estimates were adjusted for age, sex, and health status.
PHE conducted OLS regression analysis of total health care spending following methods used by other subcontractors in the individual studies.
• Note, for reasons of parsimony, PHE created an index of “health status” rather than using the complete set of HCCs used in the Acumen studies of Medicare and Medicaid.
• The market level analysis was also conducted using a reduced set of market covariates, selected according to several criteria: policy relevance, lack of redundancy, effect size in the population-specific studies, and, finally, the availability of consistent measurement of the predictors across payers.
• Regressions were also weighted by the population in HRRs. The health status predictors were additionally weighted by that population’s share of the total HRR population.