5
Recommendations for Regulatory Cost-Effectiveness Analysis

This report responds to a charge from a consortium of federal agencies to make recommendations for conducting cost-effectiveness analysis (CEA) to assess regulatory interventions affecting human health and safety. In particular, the Committee was asked to consider the theoretical soundness, feasibility, and ethical implications of health-adjusted life-year (HALY) measures in making our recommendations. The previous chapters review a number of issues related to the use of these measures in regulatory analysis, including current federal guidelines and agency practices (Chapter 2), various HALY measures and strategies for applying them (Chapter 3), and ethical and contextual considerations related to the use of these measures in decision making (Chapter 4). This final chapter presents the Committee’s conclusions based on the review and analysis described in the previous chapters, and its recommendations for conducting CEA in the regulatory setting.

The Committee drew on a variety of sources for insights, information, and evidence in determining how CEA could best inform regulatory decision making. These sources include:

  • Interviews with policy and analytic staff at federal agencies about how they currently assess the economic costs and benefits of environmental, health, and safety regulations;

  • Federal Executive Office of the President guidance on regulatory development, analysis, and reporting;

  • Regulatory impact analyses for proposed and final regulations from



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis 5 Recommendations for Regulatory Cost-Effectiveness Analysis This report responds to a charge from a consortium of federal agencies to make recommendations for conducting cost-effectiveness analysis (CEA) to assess regulatory interventions affecting human health and safety. In particular, the Committee was asked to consider the theoretical soundness, feasibility, and ethical implications of health-adjusted life-year (HALY) measures in making our recommendations. The previous chapters review a number of issues related to the use of these measures in regulatory analysis, including current federal guidelines and agency practices (Chapter 2), various HALY measures and strategies for applying them (Chapter 3), and ethical and contextual considerations related to the use of these measures in decision making (Chapter 4). This final chapter presents the Committee’s conclusions based on the review and analysis described in the previous chapters, and its recommendations for conducting CEA in the regulatory setting. The Committee drew on a variety of sources for insights, information, and evidence in determining how CEA could best inform regulatory decision making. These sources include: Interviews with policy and analytic staff at federal agencies about how they currently assess the economic costs and benefits of environmental, health, and safety regulations; Federal Executive Office of the President guidance on regulatory development, analysis, and reporting; Regulatory impact analyses for proposed and final regulations from

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis federal agencies, including the Environmental Protection Agency, the Food and Drug Administration, the Food Safety Inspection Service, the National Highway Traffic Safety Administration, the Federal Motor Carrier Safety Administration, the Occupational Safety and Health Administration, and the Consumer Product Safety Commission; Public workshop presentations by developers of health-related quality of life (HRQL) survey instruments and indexes, researchers in the fields of HRQL measurement and CEA, federal survey research officials, and ethicists; Three CEA case studies developed by the Committee in collaboration with federal agency staff, based on published regulatory impact analyses for final rules governing air quality, food safety, and children’s car seat restraints; and Reviews of the peer-reviewed literature on the performance of HRQL measures and methods, methodological research on CEA using health-related effectiveness measures, and empirical and theoretical ethical analyses of the use of HRQL indexes and HALYs in CEA. The Committee’s investigations, analyses, and deliberations led us to the following overarching conclusions: CEA, like benefit–cost analysis (BCA), offers a useful tool for the development and assessment of regulatory interventions to promote human health and safety. Different measures of effectiveness, including single-dimension measures such as life years and integrated metrics that combine estimates of HRQL and longevity such as HALYs, each provide useful and distinctive perspectives on regulatory impacts. As in the case of BCA, the results of CEA for regulatory interventions are not by themselves sufficient for informed regulatory decisions. The results of economic analyses are routinely supplemented with other types of analysis, and with information from the public, to provide a more comprehensive assessment of the advantages and disadvantages of different regulatory strategies. These other sources of information are a necessary part of the decision-making process because it is not possible to quantify all of the impacts of concern. It is feasible to apply CEA to regulatory interventions today, but additional data and improvements in the methods for measuring HRQL would make it more useful and reliable. Federal regulatory agencies analyze disparate data and contemplate widely varying interventions and types of impacts from their actions. They use diverse approaches to value health-related benefits, partly because of these differences in data sources and types of impact, but also for reasons related to institutional history and precedent. Greater consistency in the

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis reporting of assumptions, data elements, analytic methods, and in the resulting estimates of costs, effectiveness, and benefits would increase the transparency and comparability of the results. Presentations of cost-effectiveness ratios for diverse interventions can be misleading if they do not include information that highlights differences in methods, unmeasured effects, and distributional impacts across interventions. Our recommendations for the use of CEA in regulatory analysis fall into four areas: Selecting integrated measures of effectiveness; Constructing and reporting cost-effectiveness ratios; Providing additional information needed for decision making; and Pursuing data collection and research necessary to improve HRQL measurement and regulatory CEA. The recommendations are discussed in the following section and the chapter concludes with a brief summary. RECOMMENDATIONS Selecting Integrated Measures of Effectiveness Because different effectiveness measures (e.g., deaths averted, life years saved, QALYs gained) have particular advantages and limitations, all regulatory CEAs should report more than one measure of effectiveness. Reporting a variety of measures provides decision makers with a richer understanding of the impacts of different regulatory choices and responds to different questions. The Committee’s criteria for selecting integrated measures for use in regulatory CEA are summarized in Box 5-1. Recommendation 1: Regulatory CEAs that integrate morbidity and mortality impacts in a single effectiveness measure should use the QALY to represent net health effects. QALY estimates should be based, to the greatest possible extent, on research that considers the risk characteristics addressed and the population affected by the regulatory intervention. The index values estimated for health conditions or health states of interest should be based on information from the population affected by the costs, benefits, or other impacts of the regulatory intervention,

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis BOX 5-1 Criteria for Selecting Integrated Effectiveness Measures for Health-Related CEA Choices Among Health-Adjusted Life Year (HALY) Measures Because the requirements for regulatory CEA are already in effect and analysts need tools that are ready for application, the Committee’s criteria for selecting among these HALY measures in the near term are largely practical ones: The HALY metric should be widely used, and methods for estimating the index values as well as estimates for specific health states should be available in the literature. The metric should be easy to understand and interpret. The metric should be comparatively inexpensive to use, in terms of both providing immediately applicable methods and values and facilitating the development of values for additional health states. In addition to these practical considerations, measures must also provide valid and reliable estimates of the relative value of different health states. Choices Among Generic Health-Related Quality of Life (HRQL) Indexes Because generic indexes are well established and easy to use, the Committee expects that they will often be applied in regulatory analysis in the near term. The Committee’s criteria for choosing among these indices focus on their suitability for regulatory analysis as well as the reliability and validity of the resulting estimates. The HRQL instrument must be applicable to the range of health-related effects being evaluated. The instrument should be sensitive enough to distinguish among health endpoints. The instrument should reflect the values or preferences for health of the population of interest. The instrument must be acceptable to and understandable by survey respondents, policy makers, and the general public. The instrument should be as inexpensive to use as is compatible with the other objectives. which for most economically significant regulations will be best represented by the general U.S. population. In the absence of direct preference elicitation for health conditions of interest from the affected population, QALY estimates should be based on well-developed, generally accepted, and widely used generic HRQL indexes whose valuation is based on general population samples.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis The characterization of the health states or conditions of interest using generic HRQL indexes should be based on information obtained from people who are familiar with the conditions, such as patients. The QALY is the best measure at present on which to standardize HALY estimation because of its widespread use, flexibility, and relative simplicity. As discussed in Chapter 3, alternatives to the QALY measure, including the healthy year equivalent, the saved young life equivalent, and the disability-adjusted life year, are either less feasible, have not been used extensively or evaluated, or incorporate features that render the measure not comparable with QALY results. For regulatory analysis, the QALY is best thought of in practical terms, as a measure of health improvement or production that allows analysts and decision makers to compare the impacts of different interventions. In short, we recommend the QALY as a useful construct on which to standardize the accounting of changes in health and longevity. QALY estimates of regulatory effects may be based either on newly collected information or on previously conducted research. In both cases, the QALY estimates should address the same health states (i.e., the specific types of disease or injury) as identified in the risk assessment for the regulatory analysis. For example, the health states should be similar in terms of the severity and duration of the symptoms and of the effects of treatment. For chronic or long-term impacts, it may be desirable to separately assess different phases to reflect the variation in the HRQL impacts at different points in time. The QALY estimate also should reflect, to the extent possible, the effect of the health state on the particular population affected by the regulatory intervention in terms of characteristics such as age, preexisting conditions, income, and geographic location. If the HRQL estimate reflects a health state or scenario that differs from the regulatory health endpoint (i.e., addresses a somewhat different condition or affected population), these differences should, to the extent possible, be discussed and addressed in the uncertainty analysis. (See Recommendation 6.) Chapter 3 discusses alternative strategies for developing QALY estimates for use in regulatory analysis. Briefly, these include: Eliciting preferences directly for the health states of interest, through a new valuation survey of the population that bears the costs and receives the benefits of the proposed regulation. Using generic health indexes to characterize and value the health states of interest. Health states may be characterized through surveys of patients or physicians knowledgeable about the condition of interest. This descriptive step is separate from the valuation of these health states, which

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis ideally should be derived from a survey of a representative sample of the population affected by the costs and benefits of the rule. Using previously published health state and/or condition values. Such previously published values may have been either directly elicited or estimated with generic indexes. Due to time and budget constraints, the Committee recognizes that, in the near term, regulatory analysts are likely to rely on published research and to adopt relatively simple approaches to assess health-related impacts in regulatory CEAs. Over the long term, the Committee hopes that investment in additional research will improve the information available for these assessments. (See Recommendations 11 and 12.) Different approaches in different regulatory analyses may be required to pursue the dual objectives of (1) ensuring that the health states and populations addressed in the QALY analysis match those identified in the risk assessment and affected by the intervention, and (2) using ethically sound and robust societal valuations of health states. The best approach will depend on the time and resources available and the extent and quality of existing valuation research for the conditions of interest. Because these factors vary, it is not possible to specify one standard approach for QALY-based CEA that would apply in all cases. Regulatory analysts must exercise judgment in weighing the importance of different factors in choosing an approach to QALY estimation. The following discussion summarizes the Committee’s conclusions from Chapter 3 and offers guidance regarding the use of existing HRQL research and generic indexes for QALY-based regulatory analysis. Valuation. Health states can be valued directly in surveys of patients or the general population using elicitation techniques such as the standard gamble, time trade-off, category rating (e.g., visual analogue scale), and person trade-off. When a generic index is used, health states are described by locating their attributes within the functional categories or domains of the index. These domain attributes are then valued using a statistical model or algebraic formula based on a separate valuation survey that employed one or more elicitation methods. The underlying valuation surveys for the various generic indexes are based on general population surveys that differ in size and in the extent to which they represent the U.S. population as a whole (see Table 3-4). For regulatory analysis, the population valuing different health states should include both those who will benefit from the intervention and those who will bear its costs. In the case of economically significant regulations that have relatively large costs and/or benefits, the affected population

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis whose valuations are of interest is usually best represented by the U.S. population. Elicitation methods that include an explicit choice, such as the standard gamble and time trade-off, would be preferred to other methods of preference assessment if they were more comprehensible and more easily administered. However, standard gamble and time trade-off methods are difficult for many respondents to understand, and often lead to inconsistent or poorly reproducible responses. Although category rating and visual analogue scale methods do not imply a direct trade-off between years of life and HRQL (a trade-off that is implicit in QALY-based CEA), they are generally easy to administer. Particularly if they are calibrated against tradeoff methods to ensure that the same numerical rating means the same thing for both types of methods, rating scale elicitation methods can also play an important role. Methods that elicit individual preferences for health have dominated QALY-based CEA. However, alternatives (such as the person trade-off method) that aim to elicit societal values for investments in health improvements more directly merit further development, as discussed in Recommendation 12. The aggregation of individual preferences for one’s own health is but one approach to determining societal preferences for improved health, and evidence suggests that values for health states elicited in the standard way may not be well correlated with societal health resource allocation choices (Ubel et al., 1996). Generic indexes. The Committee reviewed, and applied in its case studies, several generic HRQL indexes. These included the Quality of Well-Being Scale (QWB), the Health Utilities Index (HUI) Marks 2 and 3, the EuroQol-5D (EQ-5D), and the SF-6D. As discussed in Chapter 3, we concluded that no generic instrument is superior in all respects to the alternatives. All four of these instruments have well-developed and widely tested survey formats. Health state values based on the QWB, HUI, and EQ-5D are well represented in the published literature and, because the SF-6D can be calculated from SF-36 and SF-12 health profile data, it has the potential for extensive application. Recent research suggests that these four generic instruments rank-order health states consistently, although the absolute values of individual health states differ depending on the instrument used (Franks, 2004; Franks et al., 2006). Furthermore, a growing research literature offers statistical conversions or translations of values from one instrument to another; see Table 3-7 for a summary of these studies. These instruments do vary, however, in the representativeness of their underlying valuation surveys and in specific aspects of their methodology.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis The QWB’s valuation survey dates back to the mid-1970s, was based on a San Diego community sample, and uses a rating-scale methodology. The HUI attributes were valued by a sample from a metropolitan population in Ontario, Canada. The SF-6D values were derived from a U.K. population survey. The EQ-5D is the only generic instrument with a nationally representative U.S. valuation survey underlying its index values. In addition, some of the instruments (e.g., the HUI) require licensing fees, and others involve fees to access health profile data (e.g., as would be useful with the SF-6D). Given the current state of the art of HRQL measurement, the Committee recommends that agencies consider using the EQ-5D in their primary estimates for regulatory analyses at this time. Characterizing health states with generic indexes. Using generic indexes to measure HRQL involves characterizing or locating the health states of interest according to the specific functional categories or domains of the index. In contrast with direct preference elicitation surveys, with generic indexes this characterization of health states is separate from the valuation of the generically described health states, where the latter is based on general population samples. As described in Chapter 3, health states can be characterized using generic indexes either by patients or clinical experts who are familiar with the condition of interest. Published studies are one source of patient-based characterization or description of health states using these indexes. If original data collection for a regulatory analysis is contemplated, direct elicitation of health state index values for the conditions of interest, rather than patient characterization using a generic health index, should be considered. If neither of these approaches is feasible, then clinical experts could be asked to characterize the health conditions of interest using a generic index, similar to the approach used in the Committee’s case studies. Good practices for expert assessment are discussed in Chapter 3. HRQL measurement quality. Finally, the Committee recommends that sources of HRQL values for QALY-based CEA should be evaluated with specific and consistent criteria regarding: The quality of underlying valuation surveys; and The precision and reliability with which health states of interest are captured or located by direct elicitation or generic indexes, respectively. Although it is not possible to develop absolute standards for assessing an existing study’s quality and applicability for regulatory CEA, greater specification and standardization of quality review criteria in HRQL measurement will help analysts to (1) weigh the limitations of a study against the

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis value of using it, and (2) determine when available valuation studies should not be used as a source of health state values in the context of a particular analysis. There is a growing body of evidence on the quality of different valuation techniques and surveys, and formal criteria are being developed for judging quality in survey-based valuation research that addresses both willingness-to-pay and QALY estimation.1 Such criteria for evaluating study quality should be further developed and applied to HRQL valuation research. For example, more attention should be given to internal consistency tests. Chapter 3 provides additional information on the research underlying generic indexes and these criteria. Constructing and Reporting Cost-Effectiveness Ratios The overarching objective of the Committee’s recommendations is to improve the quality and comprehensiveness of the information available for regulatory decision making. We believe this objective can be advanced by standardizing, to the extent practical, the structure of CEAs within and across agencies, thus increasing the transparency of the presentation of analytic assumptions, methods, and results in regulatory analyses. Recommendation 2: Regulatory analyses should report four measures of cost-effectiveness: Compliance cost per death averted using the net number of deaths averted as the outcome measure. Compliance cost per life year gained using the net change in years of preventable mortality as the outcome measure. A health-benefits-only ratio using the net change in QALYs as the outcome measure. Costs would include those associated with compliance, offset by estimates of the net changes in health care treatment costs associated with the outcomes included in the QALY measure. A comprehensive ratio using QALYs as the outcome measure and incorporating the value of other benefits as offsets to compliance costs. The cost measure would incorporate both net changes in health care treatment costs and the value of any monetized nonhealth benefits as offsets. The components of these four ratios are illustrated in Table 5-1. 1   See Freeman (2003, Chapter 6) and OMB (2003a) regarding criteria for willingness-to-pay studies and Chapman et al. (2000), Neumann et al. (2000), and AHRQ (2005) for criteria for QALY-based CEA.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis TABLE 5-1 Components of Cost-Effectiveness Ratios   Compliance Costs Per Death Averted Compliance Costs Per Life Year Gained Health-Benefits-Only Ratio Comprehensive Ratio Included in net costs Regulatory compliance costs • • • • Health care treatment cost savings     • • Value of nonhealth benefits   • Included in net effects Fatal effects: Deaths averted •   Fatal effects: Years of life gained   • • • Fatal effects: Quality-adjusted life years gained     • • Nonfatal effects: Quality-adjusted life years gained     • • Ratios need not be reported if they do not provide additional information for decision making. For example, compliance cost per life year gained need not be presented when a regulation would have negligible impacts on longevity, and the comprehensive ratio need not be presented for regulations that do not provide monetized nonhealth benefits. These ratios should be calculated over a time period selected to reflect the effects of the full implementation of the regulation. In addition, annualized impacts should also be reported and used to estimate expected cost-effectiveness on a yearly basis. The time periods within which the costs and savings and the health-related effects accrue should be reported using a time line to indicate the undiscounted impacts expected in each year. In addition, the present value of the impacts should be calculated using the same discount rate for both costs and effects (life years or QALYs). As discussed in Recommendations 8 and 9, agencies should also highlight information on distributional impacts and ethical considerations, on uncertainty in the estimates, and on any regulatory impacts not included in the cost-effectiveness measure. A simplified example of the four recommended cost-effectiveness ratios, as well as the results of an accompanying BCA, is provided in Box 5-2. The Committee recommends reporting all four cost-effectiveness ratios because no single formulation will be ideal in all circumstances. Different audiences will find different formulations more informative, more readily interpretable, or more comparable to other analyses. Furthermore, differ-

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis BOX 5-2 Example of Cost-Effectiveness Ratios Values used in example: Regulatory compliance costs: $100 million Benefits values (willingness to pay) for BCA: $60 million for averted mortality $30 million for averted morbidity $40 million for averted ecological impacts Benefits values for CEA: 400 QALYs gained or 100 life years saved or 10 premature deaths averted Health treatment cost savings associated with reduced morbidity and mortality: $20 million Net benefits: Net benefits = ($60 million + $30 million + $40 million) − $100 million = $30 million Cost-effectiveness ratios: Compliance costs per premature death averted = $100 million / 10 lives = $10 million/death averted Compliance costs per life year gained = $100 million/100 life years = $1 million/life year Costs per QALY, health benefits only = ($100 million − $20 million) / 400 QALYs = $200,000/QALY Costs per QALY, comprehensive = ($100 million − $20 million − $40 million) / 400 QALYs = $100,000 / QALY NOTES: For simplicity, this example provides the results for a single year and ignores the need to address the timing of the impacts. It also does not provide information on the uncertainty in the estimates. For simplicity we assume that the willingness-to-pay estimates used in the BCA calculation of net benefits encompass health treatment cost savings. See Chapter 2 and OMB, 2003a (Appendix C), for more discussion of this issue. ences among potential interventions in the relative size or ranking of the four measures can highlight important aspects of the impact of alternative interventions. While agencies should report these four ratios at a minimum as relevant, they may also, at their discretion, provide additional comparisons that incorporate alternative perspectives if such comparisons are useful for decision making.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis Pollutant/Type of Impact Nonquantified Effects HC health Cancer (benzene, 1,3-butadiene, formaldehyde, acetaldehyde). Anemia (benzene). Disruption of production of blood components (benzene). Reduction in the number of blood platelets (benzene). Excessive bone marrow formation (benzene). Depression of lymphocyte counts (benzene). Reproductive and developmental effects (1,3-butadiene). Irritation of eyes and mucus membranes (formaldehyde). Respiratory irritation (formaldehyde). Asthma attacks in asthmatics (formaldehyde). Asthma-like symptoms in nonasthmatics (formaldehyde). Irritation of the eyes, skin, and respiratory tract (acetaldehyde). Upper respiratory tract irritation and congestion (acrolein). HC welfare Direct toxic effects to animals. Bioaccumulation in the food chain. Damage to ecosystem function. Odor. NOTES: PM = particulate matter; CO = carbon monoxide; HC = hydrocarbons. SOURCE: EPA (2004b, p. 39139, Table VI.E-6). analysis make a number of recommendations for treating uncertainty in all economically significant rules, noting the need for qualitative discussion as well as quantitative assessment using sensitivity analysis or probabilistic modeling. This guidance now requires agencies to conduct formal probabilistic uncertainty analysis for all rules with impacts that exceed $1 billion annually (OMB, 2003a). Recommendation 10 of this report addresses the need for better information on the health effects of regulatory interventions, which is one major source of uncertainty. At present there is also uncertainty about the correct health state index values to use in calculating QALYs due to several factors: (1) the lack of agreement on the concept of HRQL, (2) how it should be measured, and (3) measurement error. Different generic instruments and elicitation methods produce different results without a clear consensus on the theoretical or empirical superiority of one particular approach or model. Measurement error in estimating health state index values should be reported as credible intervals around point estimates and examined in the uncertainty analysis.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis Recommendation 7: Regulatory analyses should not assign monetary values to estimates of health-adjusted life years as a method for valuing health states. In the existing literature, monetary values have been applied to HALYs for two reasons. First, as discussed in Chapter 2, agencies often use monetized HALYs in their BCAs, apparently because suitable, high-quality willingness-to-pay estimates are lacking for many nonfatal health effects of concern. Health state index values are more plentiful, and address the shortcomings associated with reliance on other proxy measures (such as cost-of-illness estimates) for valuation. Although the Committee recognizes that in the short term, regulatory agencies might continue this practice of using monetized HALY values in BCAs due to the lack of willingness-to-pay estimates for morbidity effects, we disapprove of and discourage this practice. As discussed in Chapter 1, willingness-to-pay and HRQL valuation and measurement have developed out of distinct disciplinary and methodological traditions. Given their different theoretical underpinnings and the different types of trade-offs they consider, it is misleading to combine them. The second reason for assigning monetary values to HALYs is to provide a threshold for determining whether an intervention should be pursued. As discussed in Chapters 1 and 3, neither theoretical justification for nor evidence of a consensus about any particular threshold value exists. In the absence of any compelling rationale, the Committee concludes that the use of thresholds is inappropriate. Information Needed for Regulatory Decision Making Regardless of whether economic analysis takes the form of BCA or CEA, economic analysis is only one of many inputs into the policy-making process. Statutory requirements, judicial decisions, executive orders, and agency guidance all stress the importance of considering the distribution of a regulatory intervention’s impacts, the ethical implications of different options, and the implications of nonquantifiable effects, some of which may be related to health and others not. The Committee endorses this multifaceted approach to decision making, and believes that the results of CEA should continue to be but one element in a deliberative policy development process that takes full account of both quantified and qualitative information. Recommendation 8: The regulatory decision-making process should explicitly address and incorporate the distributional, ethical, and other implications of a proposed intervention along with the quantified results of BCA

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis and CEA. Comparisons of different interventions should highlight these distinctive features of the interventions and also any methodological differences, both in the case of cost-effectiveness ratios and of estimates of net benefits. CEA plays an important role in regulatory development, providing a useful approach for collecting and organizing information, as well as for reporting the quantifiable results in summary form. Decision makers should, however, recognize the limitations of this and other approaches to economic analysis. Both BCA and CEA must be supplemented by other types of information. Because both BCA and CEA focus on economic efficiency, they should be accompanied by a discussion that highlights distributional and ethical considerations that are not fully incorporated into the quantitative results. These concerns may relate to the disproportionate adverse effects of baseline (preregulatory) or postregulatory health conditions on subgroups of particular concern (e.g., very young and very old people, minority and/or low-income groups, or individuals with preexisting conditions). They also may relate to the characteristics of the risk itself, such as the extent to which those experiencing the risk do so voluntarily or involuntarily. The Committee proposes that regulatory analysts and decision makers use a structured and systematic approach in the consideration of distributional and other ethical considerations raised by a particular regulatory action. By design, BCA and CEA aggregate benefits across the population. Their summary forms thus obscure important distributional effects that should be considered explicitly in the policy development process. In addition, they generally focus on the physiological consequences of the risk, and do not consider other characteristics of risk that may lead to different values. For example, the societal value placed on addressing two causes of preventable mortality—air pollution and car accidents, for example—may vary even if the probability of fatality is the same. A 1-in-100,000 chance of death may be valued differently depending on whether the risk is perceived as controllable or voluntary. To help ensure that important concerns are not omitted, we have identified (in Box 5-3) features of a specific risk or regulatory intervention that should be considered, if applicable, in the regulatory impact analysis and in any summary comparison of CEAs. This itemization is not intended merely as a checklist, but rather as a framework for organizing important considerations not likely to be emphasized sufficiently in the CEA itself. Such distributional and other ethical considerations could be highlighted, for example, by presenting disaggregated quantified information about regulatory effects or by conducting sensitivity analyses with alternative valuation assumptions that reflect some of these considerations. As discussed in Chapter 4,

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis BOX 5-3 Distributional and Other Aspects of Risk and Regulation Distribution of Impacts Do the baseline (preregulatory) or postregulatory costs or risks disproportionately affect certain segments of the population? The unborn or future generations Infants and young children Elderly people Persons with disabilities or preexisting health conditions Those particularly vulnerable to the risks of concern Members of minority groups Members of low-income groups Those residing in particular geographic locations Characteristics of Risks Do the risks have attributes that affect their value, but that are not reflected in the quantified valuation measures? Are the risks not subject to significant personal control? Are the risks particularly dreaded? Are the risks undetectable by the senses? Are the effects of the risks delayed, rather than immediate? Are the risks not well understood? agencies may want to conduct alternative quantitative analyses that use measures other than standard QALYs in uncertainty analyses. Such quantification of certain distributional effects could highlight the implications of particular value commitments. Although comparisons of CEA ratios across different types of regulatory interventions can provide useful information on the relative impacts of different programs or policies, those using or reviewing these comparisons should recognize their limitations. Both policy makers and scholars are often interested in the relative effectiveness of different governmental or nongovernmental interventions aimed at achieving particular outcomes, such as the relative effectiveness of different programs for reducing preventable mortality. Those developing or using such comparisons, however, should recognize that economic evaluations in their summary forms (i.e., cost-effectiveness ratios and net benefits) are incomplete and may not be fully comparable due to differences in methodology as well as differences in the types of effects considered.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis Different approaches to estimating risks or valuing effects may lead to differing estimates; the ranking of interventions may be affected by the lack of a standardized methodology rather than by real differences in effectiveness. Interventions that appear to differ significantly based on central tendency estimates may in fact be indistinguishable when the uncertainty in the estimates is considered. Thus it is important that such presentations of comparative information across interventions, such as relative cost-effectiveness or net benefit estimates, highlight these types of factors. The Committee endorses the addition of QALY-based CEA to the other requirements for regulatory impact analysis. At the same time, we are concerned that presenting summary measures such as cost-effectiveness ratios in simple tables without reference to limitations of such comparisons or to their ethical and distributional implications could hinder—rather than help—the development of sound regulatory policy. Recommendation 9: Because of the many value dimensions encompassed by societal decisions regarding the mitigation of risks to health and safety and the far-ranging impacts of such decisions, policy makers and program administrators should work to ensure the substantive involvement of a broad range of individuals and groups at all stages of policy development for regulating risks. Regulatory agencies are, by definition, part of a political system designed to involve the public in decision making and balance competing views. Public outreach is mandated by several statutes and administrative orders and is a standard component of the regulatory development process. The Committee is concerned about the need to ensure that this outreach encourages widespread involvement and allows adequate consideration of the concerns voiced by diverse parties. Numbers can be very powerful in policy contexts; thus it is important that decision makers are presented not only with the results of economic analyses, but also have an opportunity to engage in deliberations with all constituencies and affected parties. Although we did not evaluate the role of public participation in the regulatory development process in depth, we suggest that these activities merit further review and study. In particular, an effective deliberative process is needed to ensure that the appropriate weight is placed on those ethical, distributional, and nonquantifiable factors that are not included in the quantitative analysis. Data Collection and Research Needed to Improve HRQL Measurement and CEA for Regulatory Decision Making Although useful for regulatory analysis, the data and methods currently available for measuring and valuing health impacts in CEA have limitations

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis that should be addressed by a long-term research agenda. As regulations become more stringent, or simply more costly, the importance of sound methods and accurate information for assessing societal priorities and values grows. Additional data collection and research to support regulatory analysis are costly undertakings that must be considered in the context of federal agencies’ overall missions and priorities. Better planning and coordination among the relevant agencies could improve the cost-effectiveness of their investments in improved information. For example, population HRQL norms for one or more generic indexes broken out by demographic characteristics (age, gender, race and ethnicity) have not been available in the past, and are not a standard component of periodic national health surveys. In addition, QALY-based CEAs may use any one of several generic HRQL indexes, and estimates based on different survey instruments are not readily combined or compared, because the relationships among the estimates produced by different instruments are not well understood. Perhaps most importantly, the data collection efforts for the risk assessments and epidemiological studies that underlie the economic analyses of regulations have not been designed with QALY-based analyses in mind, and the data are often inadequate for estimating HRQL impacts. The areas where additional routine data collection and research are most needed and likely to be fruitful include the following. Recommendation 10: A high research priority should be improving the data used to assess the health risks (effects on incidence of particular types of illness, injuries, and deaths, and the duration and latency of effects) addressed by regulatory actions. One significant source of uncertainty in estimating the economic impact of regulations is the information and modeling underlying the estimation of the type and magnitude of health-related effects on a population, that is, the risk assessment itself. Risk assessment is of fundamental importance because it supports BCA and CEA alike, as well as other aspects of the regulatory development process. Comparative risk assessment also helps set regulatory priorities. Greater precision and detail in the estimation of health effects would particularly improve QALY-based CEA because it provides more extensive information on the impact of the risk and its abatement on health status over time that are needed for this kind of analysis. As did the recent National Research Council Committee to Estimate the Public Health Benefits of Proposed Air Pollution Regulations, we recommend that federal agencies set as a high research priority improving the epidemiological and health status data used to model health and safety risks and the effects of interventions for reducing these impacts (NRC, 2002). For a discussion of methodological issues related to the calculation

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis of attributable risk and population attributable fractions, see the Institute of Medicine workshop summary, Estimating the Contributions of Lifestyle-Related Factors to Preventable Death (IOM, 2005). Recommendation 11: The Department of Health and Human Services (DHHS) and other federal agencies should collect HRQL information through routinely administered population health surveys and other major studies and data collection efforts related to risk assessment and monitoring. DHHS should ensure that at least one population health survey—such as the National Health Interview Survey or the Medical Expenditure Panel Survey—incorporates on a periodic basis (e.g., every 3–4 years) at least one complete HRQL survey instrument that supports a preference-based measure in order to provide age- and sex-specific population HRQL norms or baselines. Survey questions regarding specific health conditions should be developed in consultation with regulatory agencies so that conditions that are common health endpoints for regulatory analyses or that are anticipated to be the targets of future regulatory action can be included. DHHS and its statistical agencies and programs should consult with regulatory agencies to identify these information needs and to reserve a portion of questionnaires and surveys for these purposes. The sampling frames of these national health surveys are not ideally constructed for the collection of nationally representative HRQL information. People who reside in institutions, including those with severe mental and physical disabling conditions, and homeless people are systematically excluded from these household-based surveys. These exclusions may skew the statistical characterization of HRQL for the population overall and inhibit the ability of the agencies to assess HRQL impacts on these subgroups. Regulatory agencies should consider including HRQL measures, as well as individual-level diagnostic and health profile information, in major data collection efforts and epidemiological studies undertaken as part of their risk monitoring systems and risk assessment research. All federally supported research that includes HRQL measures and applications of any such measures should produce public access data sets. DHHS should support the refinement and expansion of a catalogue of health state values derived from information in population health surveys, building on recent work to create a catalogue of preference-based chronic disease index values (see Sullivan et al., 2005). Such research should give special attention to the documentation of co-morbid conditions and the development of HRQL values for health states involving multiple impairments.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis Recommendation 12: DHHS should coordinate, with the involvement of federal regulatory offices and agencies, the development of an integrated research agenda to improve the quality, applicability, and breadth of HRQL measures for use in regulatory CEA. The Committee identifies the following areas as priorities for research: Current elicitation methods such as the standard gamble and time tradeoff, while theoretically well founded, may be difficult for respondents to understand and prone to generate inconsistent responses. Research to facilitate improved methods is needed. In addition, methods for eliciting societal values for investments in health (in contrast to individual preferences for health states), such as person trade-off techniques, should also be investigated. Despite widespread acceptance and endorsement by economists and decision theorists in the health field, preference elicitation methods based in utility theory have been criticized by behavioral scientists and survey researchers who have focused on the cognitive challenges, artificiality of choices posed, and susceptibility of responses to the framing of the choices (Fischhoff, 1991; Kahneman and Tversky, 2000). Research to refine survey practices for eliciting preferences for health states through standard gamble and time trade-off methods could improve the reliability, precision, and validity of survey results. A more fundamental critique of using aggregate individual preferences to represent societal values in health policy has been offered by ethicists and political philosophers, among others (Nord, 1999; Ubel et al., 2000; Hausman, 2004). One approach to value elicitation that has attempted to capture judgments about societal investments in health has been the person trade-off method, either conducted individually or in consensus group settings. This method is still in a developmental phase. Although the person trade-off method will doubtless be tested and refined further to improve reliability and interpretability of results, other approaches to social valuation beyond the simple aggregation of individual preferences should also be explored. Methods for measuring children’s HRQL, including characterization of the impact of illness and injury and the valuation of these impacts, need continued development and refinement. The Committee is particularly concerned about the adequacy of current metrics and methods for valuing health-related effects in children in two respects. First, the impacts of illness and injury on children are not well understood, due to limitations in the underlying health science research and in the methods used to describe these effects with HRQL instruments.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis Existing instruments are limited in their capacity to capture important aspects of HRQL for children, such as impacts on cognitive abilities and social interactions. Second, the surveys used to assess societal values for changes in HRQL generally address impacts on adults. In reality, those affected by costs and benefits of economically significant regulations may assign a higher value to improving the HRQL and longevity of children than of adults. In addition, there are questions about when it may be desirable to include children’s valuation of their own health states as one component of the total societal value of these effects. Because children may lack the maturity and experience to evaluate their own health, and especially to provide informed responses to choice-based valuation questions, it may be appropriate to substitute the judgments of parents or other proxy respondents. Other questions include how to adapt preference elicitation techniques for use with children and young adolescents and how to include the effects of children’s health on the well-being of parents and caretakers, effects that are not captured by individual-level HRQL measures. There appears to be no consensus on best practices for the measurement of children’s HRQL and the conduct of CEA for pediatric interventions, and a concerted research and consensus development initiative on this topic is warranted. (See Griebsch et al., 2005, for a literature survey of pediatric CEAs and variability in valuation methods.) Methods to correlate QALY estimates based on different generic HRQL indexes should be developed so that estimates from different underlying valuation studies are consistent and can be used in the same analysis. As noted in Chapter 3, a federally supported survey effort with a nationally representative sample of noninstitutionalized adults is under way to collect HRQL information using several generic indexes, so that the relationships among the estimates produced by different indexes can be documented and conversion formulae developed. The results of this major data collection effort should make it possible to combine HRQL information based on different generic indexes, and could obviate the kinds of problems the Committee encountered in the air quality case study using published health state index values based on different generic instruments. (See Box 3-7 and Appendix A for discussions of this case study.) SUMMARY Regulatory decisions are, and should continue to be, based on a public and transparent deliberative process that includes consideration of a wide range of factors, including but not limited to the results of economic analyses using BCA and CEA. BCA and CEA are complementary tools in the development of major health and safety rules because they offer distinct

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis perspectives and different types of information about regulatory impacts and effects. Both kinds of analyses provide a structured framework for collecting, analyzing, and presenting information on regulatory impacts. They do not, however, provide a complete accounting of all the effects and consequences that are important for policy-making purposes. Given the substantial impact of major health and safety regulations on the national economy and societal welfare, it is imperative that related decisions be based on high-quality policy analysis, the results and limitations of which are clearly communicated in a form that is understandable by a wide variety of audiences. Because these rules vary significantly in the type of intervention, the characteristics of the affected population, and the characteristics of the risks addressed, benefits measures are needed that can be applied to a broad range of health scenarios. These measures should be supplemented by discussion of any attributes of the scenarios that cannot be fully captured in the quantitative measures. Furthermore, the substantial uncertainty that accompanies the risk analysis underlying the calculation of health-related effects, along with uncertainty about the preference weighting of QALYs due to alternative HRQL concepts and constructs and variability in measurement, should be conveyed in uncertainty analyses. Finally, the process of developing and issuing regulations should: Be publicly accessible; Be based on information (including that used in BCA and CEA) that can be interpreted for and communicated to a wide audience; Facilitate the involvement of affected individuals, populations, and organizations in deliberations about health and safety risks and proposed interventions; and Be accountable for the policy choices made with reasons that are available to all participants and observers.

OCR for page 159
Valuing Health for Regulatory Cost-Effectiveness Analysis This page intentionally left blank.