Geographic Adjustment in Medicare Payment: Phase II: Implications for Access, Quality, and Efficiency (2012)

Appendix B


Methods for the Analysis of Associations of
Quality Measures with Payments in Chapter 3

DATA AND VARIABLES

The analyses reported in Chapter 3 used data from two sources to generate the 16 measures listed in Table 3-2 of the body of the report. Medicare Consumer Assessment of Healthcare Providers and Systems (CAHPS) data were drawn from the fee-for-service (traditional Medicare) arm of the 2010 Medicare CAHPS survey. The sample of 275,000 noninstitutionalized beneficiaries with at least 6 months of continuous enrollment in traditional Medicare was stratified by state. Sampled beneficiaries were sent a prenotice letter followed by a mail survey, and initial nonrespondents were followed up with a replacement survey and up to 20 telephone contact attempts, with a final response rate of 58 percent. The content, goals, and use of this survey have been described elsewhere (Goldstein et al., 2001).

CAHPS measures in our analyses included 10 individual items from this survey and one composite of four highly correlated items on doctor communication. These are listed in Table 3-2, ordered as access/timeliness measures, experiences with care, and clinical quality. All items are top-coded so the mean represents the fraction selecting the most favorable response option, when there are more than two options (for all items, in the composite), with the exception of “seen within 15 minutes,” which combines the top two categories owing to its lower distribution. Results are presented in terms of percentages. Most of these items are among those used in public reporting of CAHPS data. The number of usable responses varies across items due to the use of screeners and skip patterns to ensure that only beneficiaries who had relevant experience during the 6-month reference period of the survey answered each item; access items were screened by items asking whether a service was ever needed during that period, and quality items by screeners for having used the service in question.

In addition to these measures, seven CAHPS items were used as adjusters for individual characteristics that are typically associated with measures, consistent with models used in comparative public reporting of CAHPS data (Zaslavsky et al., 2001). These were self-reported education, general and mental health status, and assistance by a proxy in completing the questionnaire,

as well as age, Medicaid dual eligibility, and low-income supplement eligibility from Centers for Medicare & Medicaid Services (CMS) databases. These categorical variables contributed 22 dichotomous variables to the models.

Five additional measures assessed provision of guideline-recommended clinical care, following specifications of the Health Plan Employer Data and Information System (HEDIS). These were constructed from a 20 percent sample of fee-for-service Medicare claims for 2009, and included breast cancer screening and recommended testing for cardiac patients and diabetics. These measures were adjusted only for the patient’s sex.

The key predictors of interest were defined at the county level. Rurality was represented by the 2003 version of the Rural—Urban Continuum Code (RUCC) with levels from 0 (central cities of largest metropolitan areas) to 9 (least densely populated rural areas), with odd numbers representing areas not adjacent to a more urban area.¹ The Health Professional Shortage Area (HPSA) designation is used to adjust Medicare physician fees at the ZIP code level.² For this analysis, the committee coded this information by ZIP code into a five-category county-level variable that captures the percentage of the county’s population in HPSA ZIP codes: none (0 percent); between 0 and 20 percent; from 20 to 80 percent; over 80 but not 100 percent; or 100 percent (full-county HPSA). Both the physician practice geographical adjustment factor (GAF) and the physician work geographical practice cost index (GPCI) were used in separate models. The GAF drives geographical variation in total payments to practices for a given service mix, while the work GPCI is the component of the GAF specifically addressing the cost of physician labor; thus the two measures encompass the role of physicians both as operators of businesses and as workers. In most models including GAF or GPCI, the current CMS value for 2010 (excluding frontier floors) is used, on the assumption that this is best related (among current measures) to the historical experience of geographical adjustments potentially affecting the counties. The proposed Institute of Medicine (IOM) factors are used only in calculating the differences representing the potential effects on payment of shifting to the IOM committee’s method.

MODEL SPECIFICATION

For each of 16 measures, 13 models were fitted. A baseline model included only the individual-level adjuster variables as regressors. Four models each added a single county-level variable (RUCC, HPSA, GAF, or GPCI) to the model, to assess their distinct associations with the measures. Four additional models combined a descriptive geographical variable (RUCC or HPSA) with a payment factor (GAF or GPCI) to assess whether one acted as a mediator for the other. Finally, four models entered the difference between proposed IOM and current CMS factors (either GAF or GPCI, with or without additional control for RUCC) to assess the relative impact of the change in method on payment in higher- and lower-performing areas.

All models were specified as multilevel random-effects linear models, reflecting geographical clustering of quality variations. We compared models with a single level of clustering (county) and two levels (county nested within state). For most models there was significant evidence (deviance >3.84) in favor of the latter model, which accounts for clustering at larger as well as smaller scales. We therefore used this specification for all models. Significance of fixed effects was assessed with Wald tests using the robust Huber variance estimator. Across all 143 CAHPS models (13 model specifications for 11 measures), case-mix effects for the two health variables

______________

¹See http://www.ers.usda.gov/briefing/rurality/ruralurbcon/priordescription.htm.

²Also see discussion in Appendix A regarding translation of ZIP codes to county-level data.

were significant in every model, and effects of age, education, proxy response were significant in between 104 and 130 models (F test, P <.05).

Univariate Models

Adjusted associations of the GAF with all of the CAHPS measures were negative (although only two were significant), as were 8 out of 11 associations of GPCI with CAHPS measures, all significant (Table B-5, models PFC, PIC). Associations of both GAF and GPCI with cholesterol screening for both cardiac patients and diabetics were positive, although significant only for GAF. These coefficients should be interpreted in relation to the range of these adjustment factors. Multiplying the largest estimated coefficient (39.75 for wait <15 minutes) by the interquartile range (IQR) of county GPCI yields a predicted difference of 39.75 × (1.017 − 0.973) = 1.75 percentage points; similarly, multiplying the largest GAF coefficient (32.98, also for wait <15 minutes) by the IQR yields a predicted difference of 32.98 × (1.032 − 0.938) = 3.10 percentage points. Effects extrapolated over the full range of the payment factors would be about four times as large. Thus, higher payment indices were generally associated with worse scores for patient-reported measures, but with better cholesterol screening.

Global F tests of single geographic variables, relative to the base model, were significant (P <.05) for five measures for HPSA status (have a personal doctor, wait <15 minutes, both immunization measures, breast cancer screening) (Table B-1). The most consistent HPSA effects were for comparison of the 80-100 percent HPSA counties to the (baseline) 0-20 percent HPSA counties, with a mean 1 percentage point lower score for the former group, while non-HPSA (0 percent) counties had slightly higher scores than the non-HPSA group.

Patterns for RUCC were more variable and complex given the greater number of categories. Significant variation across the 10 categories was indicated by the F test for 12 out of 16 measures (Table B-2), including the five measures with significant variation by HPSA category and also specialist appointment, rating of care, doctor communication composite, and the remaining HEDIS items. We used a linear regression of coefficients for each measure, and the mean across measures, on the RUCC (for RUCC 1 to 9) as an ad hoc summary of the trend in beneficiary-reported quality from the most urbanized to the most rural areas. This summary for the mean suggested a weak trend toward lower quality in the more rural areas (Table B-4, first column). The trend in the mean, however, conceals opposite trends in ratings of care and doctor, timely routine care, and doctor communication, which are slightly higher in the more rural areas, and in the other measures, most notably getting needed care and immunizations, which trend lower in more rural counties.

ASSESSING THE MEDIATING ROLE OF PAYMENT RATES

We assessed mediation by adding controls for GPCI or GAF to models for RUCC or HPSA effects. If quality differences among areas along the urbanicity/rurality dimension were due to the effects of differences in the corresponding payment factors, we would expect that controlling for payment factors would reduce the variations across the corresponding categories; a similar argument could be made for HPSA categories.

HPSA coefficients without payment factor controls are compared to those with either GAF or GPCI controls in Table B-3, which shows the standard deviation of coefficients (representing adjusted differences). This measure increases and decreases with GAF or GPCI control for

approximately equal numbers of measures, but on the average is almost the same after GPCI control for either of the adjustment factor variables; this is opposite to the effect that might be expected under the hypothesis that adjustment factors mediated HPSA effects.

RUCC coefficients without payment factor controls are compared to those with either GAF or GPCI controls in Table B-4. The left-hand columns compare the slopes across RUCC categories in the three models, to examine ordered trends. These do not show a reduction in trends relating measures to rurality; on the contrary, the results are quite mixed, but the magnitude of the changes is generally small. The right-hand columns quantify the variation among RUCC categories, without regard to order, by the standard deviation of the 10 RUCC coefficients. The magnitude of variation increases for the majority of measures when the GAF or GPCI variable is added to the model. As with the HPSA comparisons, these are opposite the effects that might be predicted under a mediation hypothesis.

Looking at mediation from the opposite perspective, we considered whether RUCC or HPSA is a mediator of the paradoxical GAF and GPCI effects. As shown in Table B-5, controlling for RUCC or HPSA status does not remove the negative associations of GAF or GPCI with quality reports. Indeed, more often than not it strengthened the associations, especially for GPCI. After control for RUCC, GPCI was significantly and negatively associated with 14 of our 16 measures. Effect sizes for a one-IQR difference in GPCI, relative to total county variation, exceeded one- half for three measures (ratings of doctor and care, and pneumonia immunization). Like our previous results for HPSA and RUCC coefficients, this was opposite to predictions that might be expected under a mediation hypothesis.

ASSSOCIATIONS OF QUALITY MEASURES WITH
PROPOSED CHANGES IN GAF OR GPCI

We supplemented our impact analysis by examining associations of the proposed changes in payment factors with beneficiary-reported quality assessments. As shown in Table B-6, these coefficients are mixed in sign and mostly not significant. Indeed, for the clinical measures (immunizations and HEDIS), the coefficients are mostly opposite in sign for GAF (all positive) and GPCI (mostly negative). After controlling for RUCC, only four coefficients for GAF and none for GPCI remain significant. Furthermore, the IQR of the GAF or GPCI changes is only about one-third that of current levels, further attenuating estimates of the differences in current quality associated with projected changes in the cost index.

Thus, this analysis provides little evidence to suggest that the committee’s proposed revisions would systematically favor areas now experiencing either superior or inferior performance.

REFERENCES

Goldstein, E., P. D. Cleary, K. M. Langwell, A. M. Zaslavsky, and A. Heller. 2001. Medicare managed care CAHPS: A tool for performance improvement. Health Care Financing Review 22(3):101-107.

Zaslavsky, A. M., L. B. Zaborski, L. Ding, J. A. Shaul, M. J. Cioffi, and P. D. Cleary. 2001. Adjusting performance measures to ensure equitable plan comparisons. Health Care Financing Review22(3):109-126.

TABLE B-1 Coefficients of HPSA Category Dummies

NOTES: Coefficients are in percentage point units and are relative to baseline category, HPSA coverage >0 percent but <20 percent. Model PH includes only casemix adjusters and HPSA dummies. Significance indications: *= p <.05; + = p <.01; # = p <.001. HPSA = Health Professional Shortage Area

TABLE B-2 Coefficients of RUCC Category Dummies

NOTES: Coefficients are in percentage point units and are relative to baseline category, HPSA coverage >0 percent but <20 percent. Model PR includes only case-mix adjusters and RUCC dummies. Significance indications: * = p <.05; + = p <.01; # = p <.001. RUCC = Rural—Urban Continuum Code.

TABLE B-3 Comparison of Variation Among HPSA Category Coefficients Without and With Control for GAF or GPCI

NOTES: Entries summarize the variation among HPSA categories by the standard deviation of category coefficients, in percentage point units. GAF = geographic adjustment factor; GPCI = geographic practice cost index; HPSA = Health Professional Shortage Area.

TABLE B-4 Comparison of Variation Among RUCC Category Coefficients Without and With Control for GAF or GPCI

NOTES: Entries summarize the variation among RUCC categories by the slope of the trend line across categories 1 to 9 (left 3 columns) and the standard deviation of category coefficients (right three columns), on a scale of percentage points. GAF = geographic adjustment factor; GPCI = geographic practice cost index; RUCC = Rural—Urban Continuum Code.

TABLE B-5 Coefficients of GAF and GPCI, Without Controls and With Controls for HPSA or RUCC

NOTES: On scale of percentage points. Significance indications: * = p <.05; + = p <.01; # = p <.001. GAF = geographic adjustment factor; GPCI = geographic practice cost index; HPSA = Health Professional Shortage Area; RUCC = Rural- Urban Continuum Code.

TABLE B-6 Coefficients of Difference Between Current (CMS) and Proposed (IOM) Factors (as IOM-CMS), With and Without Control for RUCC

NOTE: On scale of percentage points. Significance indications: * = p <.05; + = p <.01; # = p <.001. CMS = Centers for Medicare & Medicaid Services; GAF = geographic adjustment factor; GPCI = geographic practice cost index; IOM = Institute of Medicine; RUCC = Rural—Urban Continuum Code.