2

Criteria for Selecting Social Risk Factors for Application in Medicare Quality Measurement and Payment

RATIONALE FOR ACCOUNTING FOR SOCIAL RISK FACTORS IN QUALITY MEASUREMENT AND PAYMENT

Performance measurement and public reporting are critical building blocks for quality improvement and attainment of a high-performing health system. Public reporting provides information to payers, administrators, and regulators to help identify which providers offer high-quality care and which require improvement—or worse, merit sanctions (IOM, 2006). By enabling comparisons of provider performance, public reporting can also incentivize quality improvement and inform consumer decision making (IOM, 2006). As Medicare and the U.S. health care system at large shift their focus from providing individual treatments to providing preventive care and improving population health, payment systems have correspondingly reflected a shift from paying for the provision of services (fee-for-service or volume-based payment) to paying for the production of good health care outcomes (value-based payment, or VBP) (McGinnis, 2016). These VBP models tie payment to performance, which shifts greater financial risk to providers, thereby raising the stakes of performance measurement and public reporting.

Underlying the assumption that both public reporting and VBP will motivate improvement is the reasoning that performance measurement and comparisons of provider performance will help identify the drivers of variation, which will in turn inform how subpar performance can be improved. At the same time, meaningful comparisons aim to reflect provider performance with all else equal by minimizing the effect of other factors such as patient characteristics that may affect health care quality or outcomes independently of provider influence (Ash et al., 2013). In epidemiology, these other factors are considered confounders.

A confounder is a third variable that is associated with both an exposure (independent variable) and the outcome of interest (dependent variable), but does not mediate the effect of the exposure on the outcome (the confounder is not in the causal pathway). In performance measurement, the exposure is the reporting unit’s (e.g., hospital, health plan) performance and the outcome is the measure of health care use, health care outcomes, or resource use. The committee embraced measures from these domains in its expanded definition of “health outcomes” in its first report. Health care use captures measures of health care utilization and

clinical processes of care; health care outcomes include health outcomes, but also measures of patient safety and patient experiences of care; and resource use captures cost measures. In this report, the committee groups these domains and measures into the overarching category of performance indicators used in VBP.

If confounders are unevenly distributed across comparison groups, this can lead to bias (systematic error) in performance measurement, which in turn leads to a distortion of the true association between the exposure and outcome (Aschengrau and Seage, 2008). In experimental studies (such as a randomized controlled trial), researchers can minimize the effect of confounders by randomizing known (and unknown) confounders across treatment groups to ensure the groups are comparable. However, in the real world (and thus observational data), confounders are typically unevenly distributed across groups. To minimize this bias, those assessing provider performances can identify specific factors as confounders and account for them statistically in an effort to more accurately measure the true association. When comparing health system performance, the principal method to account for differences in patient characteristics in quality and outcomes measurement is risk adjustment (also known as case-mix adjustment), although it is only one of several potential methods. (Methods are discussed in Chapter 4.)

Clinical risk factors are patient characteristics that may influence performance indicators used in VBP and may also be unevenly distributed across providers. For this reason, quality measures and payment models currently account for underlying and systematic differences in clinical risk factors known to independently drive variation in performance (NASEM, 2016a). For example, one health plan may have sicker patients than another health plan. Risk adjustment for clinical risk factors accounts for this selection (Kunkel and Powell, 1981; Pope et al., 2004). Accounting for social risk factors extends the rationale of accounting for clinical risk factors to also include social risk factors as characteristics that may impede accurate comparisons across health care providers. To the extent that social risk factors influence performance indicators used in VBP independently of provider actions and these social risk factors are unevenly distributed across providers, when providers are held accountable for their performance on these measures, accounting for underlying differences in social risk factors may be appropriate. Importantly, identifying a third variable as a confounder is primarily a conceptual exercise that can be supported by empirical data. However, a third variable can also have other conceptual relationships between an exposure and an outcome (for example, mediation and moderation). When factors have other relationships with an outcome of interest, it may be desirable to account for the factor in a different way or not account for the factor at all.

GUIDING PRINCIPLES

The committee developed selection criteria for social risk factors to support the methods for accounting for social risk factors in Medicare payment, particularly payment tied to performance indicators. Underlying the committee’s approach to accounting for social risk factors is a commitment to achieving health equity. Health equity is an ethical value that broadly refers to the elimination of unfair inequalities in health status by power, wealth, or prestige that may exist across social groupings by virtue of factors such as race, income, or sex (Braveman and Gruskin, 2003). When applied to health care, equity represents the commitment of providers, health systems, or payers to achieving a universally high standard of health care quality for all patients. To the greatest extent possible, a commitment to equity requires the

elimination of disparate outcomes that arise across otherwise similar patients because of their social risk factors. However, complete equality of outcomes may not be attainable within a health system because some of the factors contributing to disparities are not modifiable by providers, health systems, or payers. Additionally, achieving health equity requires more than equitable health care, or providing the same type and quality of health care to all patients regardless of social risk, because this may not be sufficient to reduce health disparities. Some subpopulations (such as those with greater levels of social risk factors) may require more intensive care to achieve the same health outcomes that can be achieved in the general population with less intensive care and at lower cost. A system of accounting for social risk factors in VBP achieves the ethical goal of equity when it appropriately recognizes the challenges of caring for populations with social risk factors, while creating incentives that are likely to lead to the improvement of care for socially at-risk patients.

The translation of this ideal into policies to account for social risk factors is complex for both philosophical and practical reasons. Philosophically, the concept of health equity is open to multiple possible interpretations, with different implications for resource distribution and measurement (Culyer, 2007). For example, achieving a fair distribution of resources across populations with different social risk factors is likely to require balancing resources across subgroups defined by level of disease burden, socioeconomic status, race, and other social risk factors. Establishing an equitable allocation of resources in these cases requires policy makers to define reasonable trade-offs between worthwhile societal goals, such as eliminating health disparities and improving overall health. Relatedly, there are significant trade-offs between health equity, efficiency, and other values.

The committee does not take a position on how to resolve the complex trade-offs inherent in promoting health equity, a task that exceeds the scope of this report. However, it is assumed that some conception of equity is always guiding choices of different methods for accounting for social risk factors, as well as the choices of measures used to represent health care quality and equity. Accordingly, it is important that any selection of methods to account for social risk factors be justified relative to a particular conception of equity, and trade-offs in resource allocation that arise from different alternative conceptions of equity be made transparent by policy makers and subjected to ethical deliberation among affected stakeholders.

Even after adopting a conception of equity to account for social risk factors, there is a tension around the best way to get there, because, as described above, equity is open to multiple possible interpretations. As described in Chapter 1 and in the committee’s first report, critics of accounting for social risk factors in VBP programs are concerned about the potential to institutionalize a poorer standard of care and to reduce incentives to improve care for socially at-risk populations. Proponents are concerned about incentives for providers to avoid socially at-risk populations, further reducing already limited resources among providers disproportionately serving socially at-risk populations, and, consequently, increasing health disparities. Such differences may arise from differential weight these opposing views place on the potential harms of accounting for social risk factors in VBP and the harms of not accounting for social risk factors (i.e. the status quo). (Potential harms of the status quo compared to accounting for social risk factors are discussed in Chapter 4.)

Underlying these arguments is a fundamental disagreement about the interpretation of observed differences as well as the ability of providers to address social risk factors that may influence observed differences in performance. As described in the committee’s second report, the lower average performance among providers disproportionately serving socially at-risk

populations and the poorer health and health care outcomes among socially at-risk populations is likely neither wholly attributable to factors out of the control of providers nor wholly the result of poorer care on the part of providers (NASEM, 2016b). On the one hand, some opponents believe that because observed differences in performance indicators used in VBP could reflect actual differences in health care quality as well as the influence of social risk factors, given that it is not possible to determine whether or to what extent the poorer performance is due to real differences, risk adjustment could obscure real disparities and thereby reduce incentives to improve care and reduce health disparities. Proponents might counter that if the difference in outcomes affecting the disadvantaged groups is found consistently across the health care system, then providers treating more patients from those groups should not bear the entire penalty for those disparities. Opponents might argue that these providers should be held responsible for providing services in a manner that compensates for social risk factors, while proponents might view social risk factors as less easily addressed through provider actions compared to opponents. They may also believe the costs of addressing social risk factors to be high and thus must be accounted for in the payment system even if it is appropriate to expect providers to address social risk factors.

The primary goal of the criteria is, therefore, to guide selection of social risk factors that could be accounted for in VBP so providers or health plans are rewarded for delivering quality and value independent of whether they serve patients with relatively low or high levels of social risk factors. As described in the previous section, VBP aims to incentivize quality improvement for all patients by tying payment to performance. Under VBP, providers who care for patients who would score lower on the measures of performance as a result of factors outside of the providers’ control (such as certain social risk factors), rather than as a result of the quality of care delivered, should not be penalized because of the influence of these non-modifiable factors. The measures should reflect quality; the effect of other factors, such as patient characteristics, should be minimized. In sum, the criteria should guide identification of social risk factors that could be accounted for in performance indicators used in VBP to promote accuracy in reporting.

COMMITTEE PROCESS

The criteria put forth by this committee adhere closely to the guidelines for selecting risk factors developed by the National Quality Forum (NQF) in their 2014 report Risk Adjustment for Socioeconomic Status or Other Sociodemographic Factors. However, the committee made several changes to reflect their priorities. Like NQF, the committee’s criteria explicitly focuses on selecting risk factors that will be applied to adjustment of measures used for VBP. However, as will be discussed in detail in Chapter 4, there are multiple methods to account for social risk factors in VBP, including some models that adjust payment directly. The committee’s criteria reflect the need for selected social risk factors to apply to this broader range of methods.

The NQF guidelines drew on guidelines previously developed for the Centers for Medicare & Medicaid Services (CMS) hierarchical condition categories (HCCs) risk adjustment model (Pope et al., 2004), which the committee also reviewed. Whereas the NQF guidelines and the committee’s criteria reflect the increased need to account for social risk in addition to clinical risk as Medicare moves towards a payment model tied to performance, prevention, and population health (Burwell, 2015), the CMS-HCC model aims to predict medical expenditure risk. In developing the criteria, the committee also reviewed criteria developed to guide whether to include a specific HCC in the Department of Health and Human Services (HHS) HCC risk

adjustment model for individual and small group markets (Kautter et al., 2014) as well as criteria to identify case-mix adjustors for the Consumer Assessment of Health Care Providers and Systems (CAHPS) surveys case-mix adjustment model (Elliott et al., 2009; O’Malley et al., 2005). The criteria the committee reviewed and drew upon in developing their criteria are listed in Appendix A.

CRITERIA FOR SELECTING SOCIAL RISK FACTORS

1. The social risk factor is related to the outcome.
   1. The social risk factor has a conceptual relationship with the outcome of interest.
   2. The social risk factor has an empirical association with the outcome of interest.
2. The social risk factor precedes care delivery and is not a consequence of the quality of care.
   3. The social risk factor is present at the start of care.
   4. The social risk factor is not modifiable through provider actions.
3. The social risk factor is not something the provider can manipulate.
   5. The social risk factor is resistant to manipulation or gaming.

These criteria along with their rationale, potential limitations, and practical considerations for applying the criteria are described in detail in the subsequent sections.

Criteria Category A: The Social Risk Factor Is Related to the Outcome

This category is the most basic pair of criteria for a social risk factor—that there be both a plausible and valid reason why the risk factor would be associated with the outcome and empirical evidence that such a relationship holds in practice. Together these criteria lay the foundation for the validity and practical importance of the risk factor.

Criterion 1: Conceptual Relationship with the Outcome of Interest

A conceptual relationship between the social risk factor and the outcome of interest (i.e., performance indicators used in VBP) ensures there is a reasonable link that might explain an association between the factor and the outcome. Conceptual relationships can be direct or indirect; a risk factor may also be a marker or proxy for otherwise unmeasured factors. To meet this criterion, the conceptual relationship may follow any of these pathways—the factor may be in a direct causal pathway, mediate the causal pathway, or be associated with an otherwise unmeasured confounder. In other words, while this criterion requires some conceptual relationship, it does not require a well-established, direct, causal relationship.

Establishing a unique causal effect can be difficult. In particular, identifying causal mechanisms can be challenging when risk factors operate through multiple pathways, as many social risk factors do (NASEM, 2016a). The main rationale for including social risk factors for which the precise causal mechanism is not well established is the fundamental cause theory (Link and Phelan, 1995). From this perspective, the causal mechanisms are expected to change over time, but the effect of fundamental causes like social risk factors on health outcomes are expected to persist. For example, it is well established that race and ethnicity may influence health. (See the committee’s first report [NASEM, 2016a] or Chapter 3 for a more thorough discussion.) However, the precise mechanism by which race and ethnicity affect a given health care outcome in a particular setting may be less well established and may change over time. When the aim is prediction and accounting for differences in underlying risk between providers, rather than explanation, how a social risk factor like race/ethnicity affects health, is less important than that factor’s predictive power (i.e., the strength of the relationship between race/ethnicity and the outcome of interest) (Elliott et al., 2009; O’Malley et al., 2005). Thus, an association between a social risk factor and outcome that persisted despite changes in the underlying causal mechanism would meet this criterion. Note that, because the goal is not to identify causes of poor quality, but rather to allow providers to see their performance with and without the influence of social risk factors, the existence of a conceptual relationship does not necessarily imply that outcomes would improve through interventions targeted at social risk factors or at ameliorating the effects of social risk factors.

Establishing the conceptual relationship to meet this criterion can be informed by the academic literature such as theories from the epidemiologic or social sciences literature, or by clinical expertise. For example, Link and Phelan use their fundamental cause theory to establish a relationship between race and health outcomes, while Epstein and colleagues used clinical criteria to establish an association between race and renal transplant utilization (Epstein et al., 2000; Phelan and Link, 2015). If the only conceptual rationale is that the social risk factor is correlated with an otherwise unmeasured factor, it would be preferable to have a direct measure for that factor. However, if the unmeasured factors cannot be measured, then, while not ideal, it may be justifiable to include the confounded factor as a proxy. Establishing a conceptual relationship takes into account several practical considerations.

One consideration is that some factors that are conceptually related to the outcome might nonetheless be unacceptable for inclusion in risk adjustment or alternative methods because of concerns regarding face validity or acceptability. For example, if people who profess a particular religious affiliation had systematically lower-quality scores, it might be unacceptable for a hospital’s payment to be in any way influenced by its patients’ religious affiliations. Similarly, it would be problematic if all the social risk factors included in a model were selected because of social norms or political considerations rather than an established conceptual relationship. Another practical issue would be to consider whether and how the construct works in a population of interest. This may be particularly relevant when identifying the appropriate indicator and measure used to assess a given social risk factor. For example, current occupation would not be a good indicator of SEP among beneficiaries who qualify for Medicare based on disability, because not working is an eligibility requirement.

Criterion 2: Empirical Association with the Outcome of Interest

An empirical relationship means that there is a statistical association of a meaningful magnitude between the social risk factor and the set of outcomes of interest (i.e., performance

indicators used in VBP) that is unlikely to be caused by chance. Empirical support for a conceptual relationship between a social risk factor and an outcome can come in part from the literature.

A common method to identify an empirical relationship is to assess the association or correlation between the two variables. For example, mortality is higher for those with lower income. The correlation in this example is known as a bivariate relationship, as it refers to the association of two variables (mortality and income). The committee considers a bivariate relationship the minimum standard necessary to meet this criterion. A multivariate association is one that considers more than two variables at a time. For example, in a bivariate association, low-income older adults have higher Medicare expenditures than high-income older adults. However, low income is also associated with health status. Specifically, low-income older adults are sicker than high-income older adults. Thus, if the third variable of health status is added to create a multivariate model, after the association between income and expenditures is adjusted for underlying differences in health status, low-income older adults may have expenditures that do not differ from higher-income older adults. As seen in this example, assessment of the empirical relationship using the bivariate relationship can lead to a different conclusion than assessment in a multivariate context. In general, evidence of multivariate associations may be considered stronger evidence of an empirical association as multivariate evidence is closer to establishing a unique association between the social risk factor under consideration and the outcome. Such multivariate evidence also rules out the possibility that the additional variables in the model completely explain the bivariate association.

This discussion raises one of several practical considerations for implementation that are specific to the setting under consideration. First, when relying on the literature to establish an empirical relationship, the setting of the empirical association in the literature may not be generalizable to the particular setting to which it is being applied with respect to VBP. Relatedly, evidence of bivariate or multivariate associations of a social risk factor with an outcome is more compelling and relevant if it has been established within different reporting units within the setting in question (Elliott et al., 2001; Jha and Zaslavsky, 2014; Zaslavsky et al., 2001). For example, one might consider education as a social risk factor for flu immunization in Medicare health plans. Suppose that flu immunization is lower overall for those with less education. If no such association exists after controlling for the health plan in which a Medicare beneficiary is enrolled, then there is no within-plan association—the initial overall association would be entirely “between-plans.” In this scenario, the observed association between education and immunization rates reflects only differences in immunization rates between plans that differ in their members’ education. In other words, the observed association between education and immunization may be capturing the unmeasured influence of a provider (health plan) characteristic linked to overall quality as a proxy and may reflect an ecological fallacy (incorrectly drawing inferences about individuals within a group from inferences about the group to which those individuals belong). Therefore, the between-plan association provides only weak evidence of the effect of education itself on immunization rates.

If, however, beneficiaries with lower education were less likely to receive flu immunizations than beneficiaries with more education in the same health plans, this would be evidence of a within-plan association of education and flu immunization. Here, the observed association between education and immunization rates reflects differences in patient characteristics rather than health plan characteristics. Thus, this within-plan association supports the hypothesis that the health plans achieve worse outcomes with patients with low education

compared to patients with high education on average. Given that this criterion is meant to confirm the hypothesized conceptual relationship, this within-plan association therefore provides stronger evidence of the hypothesized relationship between patient education and flu immunization than only an overall or between-plan association in the context of health plans.

This raises a related consideration. A particular multivariate model in the literature may reduce the association of interest due to a variable that might not be included in the model being built for Medicare payment adjustment. Thus, it may be important to statistically assess the empirical association of a specific social risk factor within the particular multivariate setting in which it is being applied. In particular, social risk factors should contribute unique variation in the outcome of interest. Social risk factors will not affect scores or payment unless they both (a) vary across units, and (b) predict performance indicators used in VBP within reporting units after considering other retained social risk factors. For example, gender might not vary much across health plans (situation a), whereas one measure of socioeconomic position (SEP) might not predict any performance measures strongly if another measure of SEP strongly correlated with it were already included (situation b). Thus, some social risk factors might be “redundant” for some or all performance measures in a given setting at a given time. To the extent that the goal is prediction/adjustment rather than explanation/causal inference, which the committee believes to be the case (as described in the first criterion regarding establishment of a conceptual relationship), prediction/adjustment is not harmed by the inclusion of redundant social risk factors. To the extent that the data collection of a particular risk factor is costly or burdensome to collect, the government may choose not to measure a risk factor that appears to explain little of the variation in relevant outcomes, but this criterion should be applied weakly and should not be used to exclude social risk factors that are important to some performance measures. Moreover, if a disparity associated with a social risk factor has been eliminated, consideration should be given to continuing to include the factor to preserve incentives, in addition to monitoring disparities associated with the social risk factor.

Finally, when applying this criterion, it will be important to consider whether accurate data on the social risk factor is feasible to collect and use. This consideration should take into account privacy laws (such as those relating to the Health Insurance Portability and Accountability Act of 1996, also known as HIPAA) and privacy concerns regarding data collection, use, and disclosure (IOM, 2014). This may be especially relevant when accounting for social risk factors, because social risk factors touch upon numerous sensitive issues and because individual-level data is needed to identify within-plan differences (IOM, 2014). This consideration should also take into account data that is both currently available and used, available but underutilized data, as well as future sources of data. It is important to note that current literature can only provide evidence of descriptive (i.e., observed) relationships. It should be noted that these descriptive relationships do not represent necessary relationships, and as noted, these relationships may change over time.

It is critical to consider whether the data exist in data sources that are underused. For example, electronic health records (EHRs) collect substantial amounts of data about individuals that are not used in performance measurement for VBP. Additionally, CMS already conducts data collection through CAHPS surveys that could be applied to other outcomes at an aggregated level. For example, Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) collects data on a sample of patients at a given hospital, and thus could be a source of hospital-level data that could be used to account for differing populations with different levels of social risk factors. The valid use of this approach may currently be very limited, as it cannot

be used to establish within-hospital associations or to develop within-hospital adjustments. However, in future, if HCAHPS data could be linked to other patient data and there remains sufficient sample size at the intersection of these data sets, it might be possible to use this data to account for social risk factors.

With respect to future sources of data, collecting accurate data is important and should be balanced against data burden and feasibility. Other considerations include whether the missing data elements regarding social risk factors may be issues that are more subjective and therefore potentially subject to manipulation. Additionally, providers may have the ability to gather information on risk factors, but the desire to collect data providers report should be balanced against the need for objective sources of data. Data sources are the topic of the committee’s next and fourth report.

Criteria Category B: The Social Risk Factor Precedes Care Delivery and Is Not a Consequence of the Quality of Care

Factors that reflect a model of care delivery (e.g., nurse staffing levels in a hospital), a treatment decision, or the direct consequences of care or treatment decision are not appropriate adjustors, as they reflect true differences in quality of care or other outcomes. In addition, adjusting for some modifiable risk factors may discourage some means of quality improvement and disparity reduction. To achieve goals of VBP as stated above, it is critical to consider whether risk factors are the consequence of provider efforts. If a factor can be influenced by the provider, then accounting for it may diminish incentives to improve that risk factor. For example, if health literacy improves health care outcomes and can be improved by providers or health plans, using the level of health literacy in risk adjustment would diminish incentives to do so. At the same time, it may still be appropriate to account for health literacy in other ways if improving health literacy or if aligning the demands of the health system to patients’ skills and abilities to facilitate their access, understanding, and use of health information and services is costly and/or low-literacy patients require more resources (e.g., the use of navigators) to achieve the same health care outcomes.

Criterion 3: Risk Factor Is Present at the Start of Care

While not a guarantee of avoiding selection of social risk factors that are a consequence of the care provided, identifying factors that are present at the start of care are unlikely to be affected by the care they are about to receive. Note that for some criteria, it is possible to have reasonable confidence that a factor was present at the start of care even if measured later—such as educational attainment for Medicare beneficiaries. This criterion may be inadequate for highlighting some risk factors that are the consequence of care in the context of repeated use of the same provider or arrangements that hold providers accountable for population health. In these contexts, the “start of care” may also be the “end of care” one period earlier.

Often the timing of a risk factor must be carefully considered in a particular application. If one adjusts for a factor as it was present at the start of care, one is accounting for its influence on or associations with outcomes for the duration of a particular hospitalization, a particular year of outpatient care from a plan, or some other limited period. One would have to acknowledge that if a patient sees the same provider for many years, such an adjustment does not control for any effects that provider had on the factor before the start of the episode of care in question. For this reason, it may be helpful to prioritize slowly-changing factors over rapidly-changing factors.

This may be particularly challenging when a risk factor can change quickly in response to the circumstance under consideration. For example, marital status and living alone are indicators of social support. Both indicators can change quickly, especially in older persons. For example, if a husband and wife live together and the husband dies, the woman would suddenly be widowed and living alone. In that case, the measured risk factor could change. Social risk factors that are subject to rapid change may be more likely to be a consequence of the care provided. Similarly, it is important to consider whether a factor represents a cumulative life cycle effect or a transient effect. For example, poverty has a cumulative effect over a lifetime, whereas transportation unavailability might be transient.

Criterion 4: Risk Factor Is Not Modifiable Through Provider Actions

To avoid selecting factors that are the consequence of the quality of care, in addition to avoiding factors that are not present at the start of care, it is important to critically assess whether that factor is something that a provider can modify either directly or indirectly. Although such factors are often highly correlated with outcomes, accounting for such factors contravenes the goal to account for factors that are either largely beyond a provider’s control or only modifiable at great expense and with great difficulty. The absence of air conditioning in the patient’s home, for example, could be an indicator of a patient’s environmental context as a social risk factor for poor outcomes that can be present at the start of care. At the same time, if purchasing air conditioners for their high-risk patients were somehow to become an indicator of high-quality care, absence of air conditioning could be a characteristic of the care provided. If air conditioning were then to be adjusted for in quality measurement, providers with more patients without air conditioning would receive a higher payment. At the same time, providers that purchased air conditioners for their high risk patients to improve quality would not fully benefit in terms of VBP because, although outcomes may improve, these providers would also have lower risks and thus risk adjustment would lower their payment. Applying this criterion together with the preceding one (the risk factor is present at the start of care) could help avoid the challenge of identifying factors that may be present at the start of care, but can also be a characteristic of the care provided.

It is important to distinguish between factors that can themselves be modified or influenced and those that are unmodifiable themselves, but can be addressed by appropriately tailored approaches that improve outcomes without changing the underlying disadvantage. Unmodifiable factors include race and ethnicity, nativity, and gender. Although themselves unmodifiable, these factors and indeed all efforts at disparity reduction and quality improvement for socially at-risk patients are predicated on the assumption that tailored, appropriate care for those with any particular risk factors is possible. Other factors, like income, wealth, occupation, language, housing, and transportation are potentially modifiable, but doing so likely requires substantial effort and cost. Health care providers can advocate for the inclusion of health in all policies to address underlying social conditions as root causes of health care outcomes, but the responsibility to improve transportation and education systems, reduce poverty, teach English, and ensure a living wage and sufficient affordable housing stock lies outside of the health care system. At the same time, the committee acknowledges that health care providers are increasingly held responsible for addressing social risk factors by, for example, partnering with social service agencies, public health agencies, and community-based organizations. Examples of these interventions are described in detail in the committee’s second report (NASEM, 2016b). Although such interventions mitigate the effects of social risk factors on certain health care

outcomes, they do not change the underlying social conditions. Both unmodifiable factors and factors that are potentially modifiable but beyond the purview of the health care system would meet this criterion. Methods of adjusting or otherwise accounting for social risk factors, as described in Chapter 4, can account for unmodifiable risk factors while rewarding providers who provide better, appropriate, tailored care that minimize the impact of social risk factors on certain health care outcomes.

The critical challenge of applying this criterion is that it can be difficult to identify the extent to which care provision might affect a particular risk factor in practice. While it is fairly easy to determine whether a risk factor is present at the start of care, a great deal of judgment may come into play when deciding whether a risk factor could be altered by higher-quality care particularly as care providers become responsible for population health and may engage in interventions that are not contained within the provider’s health care setting.

Criteria Category C: The Social Risk Factor Is Not Something That the Provider Can Manipulate

Criterion 5: The Risk Factor Is Resistant to Manipulation or Gaming

Gaming is unproductive behavior by the agent being evaluated (e.g., the provider) that distorts measurement to improve the agent’s performance measure score (better measured care or outcomes) without a corresponding improvement in the intended care or outcome for which the performance measure is capturing (better actual care or outcomes) (Bevan and Hood, 2006). In this case, the focus is not on gaming a performance measure but rather on gaming a measure of a social risk factor that will in some way change the interpretation of the performance measure inappropriately. Gaming the measure of a social risk factor results in obscuring rather than clarifying true performance.

The rationale for this criterion is to protect against including social risk factors that might create perverse incentives for providers to engage in unproductive behavior or deliver suboptimal care for the purpose of the payment system rather than for the purpose of quality of care. Such a perverse incentive might include the possibility that a target could be achieved by reducing the quality of care for patients with low social risk rather than improving the quality of care for socially at-risk patients and overall. Any provider or health plan should be expected to maximize payment within legal limits (McGuire, 2000). This has been documented extensively in terms of coding patient diagnoses more completely in order to ensure higher payments (Dafny, 2003; Kronick and Welch, 2014). Selecting the most advantageous patients within any given group has been documented as well (Newhouse et al., 2012). Note that this criterion to address potential gaming is not an ethical judgment about health care providers. Rather, these providers operate within systems in which quality improvement and disparities reduction goals are not aligned with the payment system. Consequently, these systems may not optimally allocate resources to improve quality and reduce health disparities and may therefore embed perverse incentives for providers.

Gaming the measurement of social risk factors may be less likely if measures are externally collected and reported. Gaming may be more likely if measures used for accounting purposes were based on provider reporting. For example, if hospitals reported patients who were referred to receive meal delivery as a measure of food insecurity and indicator of financial stress, hospitals might “over-refer” patients who do not need this type of support. Additionally, if the

indicator (or measure) were based on a sample of patients, the sample could be taken in a way that was not representative of the provider’s entire pool of patients.

Gaming can be practically addressed in several ways. First, continuous measures are preferable to discrete ones because there are no large gains from threshold level changes. For example, if adjustment only accounted for the population below 100 percent of the federal poverty level (FPL), then systems might have a lot to gain from capturing income at a particular low point that would increase the number of people falling below 100 percent of FPL than if adjustment were continuous at all levels of income. Second, one could prioritize specific coding over vague coding. In diagnostic criteria, measure developers limited the potential for gaming by giving greater weight, for example, to a diagnosis of cancer that could be verified by a biopsy result in a chart review, than to a symptom report of fatigue. Similarly, more weight was given to an acute myocardial infarction than to a non-specific chest pain diagnosis. These principles can be generalized and applied to social risk factors. For example, one could consider a measure that indicates enrollment in specific nutritional assistance programs that are means-tested rather than using a subjective measure of food insecurity captured at the time of hospital readmission. Third, one could pay particular attention to potentially influential (high-leverage) risk factors. Risk factors that are not prevalent but highly predictive of outcomes can be particularly influential. As such, they may be particularly tempting with regard to gaming if such gaming is possible. Such measures therefore require particular scrutiny with regard to gaming. Nonetheless, such factors may be particularly important if gaming can be avoided, as a failure to account for such factors could greatly disadvantage providers who care for large proportions of patients with high levels of social risk factors.

The committee’s criteria along with the rationale and potential challenges of each criterion are summarized in Table 2-1.

TABLE 2-1 Criteria for Selecting Social Risk Factors for Application in Medicare Quality Measurement and Payment, Rationale, and Potential Challenges

GUIDANCE REGARDING THE PROCESS FOR USING SELECTION CRITERIA

Given that there is more than one appropriate way of accomplishing the same ideal objective, the committee does not offer a formula for using the criteria specified above. However, the committee does offer guidance on how to apply these criteria. With respect to applying criteria to individual social risk factors, indicators, or measures, the committee proposes:

• The rationale for selecting a factor, as well as alternatives considered, is transparent.
• Trade-offs with respect to how well a factor meets the criteria are discussed and weighed in the final determination of whether to include a factor.
• Although a social risk factor may meet all five criteria, it may still be excluded from inclusion due to practical considerations—for example, if it lacked face validity or due to data limitations.

The committee also offers guidance on developing a set of social risk factors that could be used together for accounting purposes:

• Priority should be given to how a set of indicators perform together over the performance of each indicator on its own.
• When criteria are met, common measures across outcomes are preferred as this would be more practical and may improve the face validity of the models.
• Where possible and appropriate, it is preferred to use a common set of adjusters across the different measures in a given setting.

For both individual social risk factors (and respective indicators and measures) and also a set of social risk factors, the committee offers the following considerations:

• In terms of how measures are implemented and concerns of misuse:
  • Stakeholders such as providers and patient advocates have an opportunity to weigh in on the measures
  • The opportunities for gaming and misuse are discussed and considered. Often, potential stakeholders may have more insights into this process than modelers.
• It is critical to monitor the application of the selected risk factors.
  • How and whether a social risk factor meets the criteria may require continuous evaluation and reassessment of criteria; these criteria are applied in a dynamic setting. As health care evolves and health care disparities get addressed, the justification for certain measures may lapse and others may become more important.
  • Monitoring is also necessary to assess whether the use of social risk factors in Medicare payment strategies is appropriately incentivizing both improved quality and reduction in health disparities. Yet the criteria themselves are meant to be stable and reapplied to allow for an adaptive system.

The next chapter returns to the social risk factor framework presented by the committee in its first report and applies the criteria outlined above to specific measures of social risk.

REFERENCES

NASEM (The National Academies of Sciences, Engineering, and Medicine). 2016a. Accounting for social risk factors in medicare payment: Identifying social risk factors. Washington, DC: National Academies Press.

NASEM. 2016b. Systems practices for the care of socially at-risk populations. Washington, DC: National Academies Press.

NQF (National Quality Forum). 2014. Risk adjustment for socioeconomic status or other sociodemographic factors. Washington, DC: National Quality Forum.

Newhouse, J. P., M. Price, J. Huang, J. M. McWilliams, and J. Hsu. 2012. Steps to reduce favorable risk selection in Medicare Advantage largely succeeded, boding well for health insurance exchanges. Health Affairs 31(12):2618-2628.

O’Malley, A. J., A. M. Zaslavsky, M. N. Elliott, L. Zaborski, and P. D. Cleary. 2005. Case-mix adjustment of the cahps hospital survey. Health Services Research Journal 40(6 Pt 2):2162-2181.

Phelan, J. C., and B. G. Link. 2015. Is racism a fundamental cause of inequalities in health? Annual Review of Sociology 41:311-330.

Pope, G. C., J. Kautter, R. P. Ellis, A. S. Ash, J. Z. Ayanian, M. J. Ingber, J. M. Levy, and J. Robst. 2004. Risk adjustment of medicare capitation payments using the CMS-HCC model.

Zaslavsky, A. M., L. B. Zaborski, L. Ding, J. A. Shaul, M. J. Cioffi, and P. D. Cleary. 2001. Adjusting performance measures to ensure equitable plan comparisons. Health Care Financing Review 22(3):109-126.