Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 191
Valuing Health for Regulatory Cost-Effectiveness Analysis Appendixes
OCR for page 192
Valuing Health for Regulatory Cost-Effectiveness Analysis This page intentionally left blank.
OCR for page 193
Valuing Health for Regulatory Cost-Effectiveness Analysis A Summary of Case Studies PURPOSE AND SCOPE As part of the charge from its sponsors, the Institute of Medicine (IOM) Committee to Evaluate Measures of Health Benefits for Environmental, Health, and Safety Regulation was asked to conduct case studies that applied data from completed economic analyses to assess the impacts using different measures of effectiveness. The Committee chose to conduct three case studies that reflect the data and analytic approaches applied by different regulatory agencies as well as the diverse health impacts addressed. This appendix summarizes the case studies, which are described in more detail in three separate reports (Robinson et al., 2005a,b,c). The implications of these case studies for our deliberations are discussed in the main text of this report; some of the key conclusions are also summarized at the end of this appendix. The case studies were a learning exercise for the Committee. They allowed us to examine in detail the data and methods currently applied by federal agencies when estimating the value of health and safety benefits. These case studies also permitted us to apply alternative quality-adjusted life-year (QALY) methods in the context of regulatory analysis and to examine the outcomes. Because the case studies were completed with limited resources and largely in advance of the Committee’s deliberations, the case studies do not reflect in every respect the best practices ultimately recommended by the Committee, nor were they designed to replicate the
OCR for page 194
Valuing Health for Regulatory Cost-Effectiveness Analysis complexity of a full regulatory analysis.1 They do, however, provide a starting point for researchers interested in conducting more sophisticated versions of these types of analyses. The Committee identified candidates for these case studies as part of a review of all major federal health and safety regulations finalized in recent years (Robinson, 2004). This review focused on those economically significant regulations that were supported by quantitative assessment of both costs and health or safety-related impacts, that is, the types of rules for which new Office of Management and Budget (OMB) guidance (2003a) now requires cost-effectiveness analysis (CEA) in addition to benefit–cost analysis (BCA). Based on this review and discussions with agency staff, we determined that the three rules listed below appeared to best illustrate the range of types of regulations, current practices, and health and safety impacts most likely to be significantly affected by the Committee’s recommendations. The Food and Drug Administration’s (FDA’s) January 2001 juice processing rule: This food safety regulation provides an example of FDA’s use of monetized QALYs to value the impacts of acute and chronic illness in BCA. The health outcomes considered include acute gastrointestinal effects associated with exposure to four foodborne pathogens as well as chronic conditions stemming from these infections. Few cases of mortality were associated with these pathogens. The National Highway Traffic Safety Administration’s (NHTSA’s) March 1999 child restraint rule: Because more recent rules were undergoing revision, we chose a somewhat older rule for the NHTSA case study. However, the data sources and analytic approach are similar to those currently used by NHTSA. NHTSA’s approach to CEA involves converting nonfatal injuries to “equivalent lives saved” (ELS) based on the ratio of their costs to the value of a fatality; these costs include both expenditures and monetized QALY impacts. (See Chapter 2 and Box 2-4 for further detail on the ELS approach.) The health effects addressed by this rule include a variety of fatal and nonfatal crash-related injuries to children. The U.S. Environmental Protection Agency’s (EPA’s) June 2004 nonroad diesel rule: Air pollution regulations account for a substantial 1 One of the most important differences between these case studies and the Committee’s recommendations is the limited information they provide on the range of possible values and associated uncertainties. We rely largely on mean or median estimates to assess QALY impacts, and also do not report uncertainties in each agency’s characterization of the health effects averted by the regulations nor in their estimation of regulatory costs. The case studies also do not include detailed information on the distribution and equity of the impacts. In Chapter 4 of this report, however, we use the case studies to illustrate distributive and other concerns.
OCR for page 195
Valuing Health for Regulatory Cost-Effectiveness Analysis proportion of all major health and safety regulations finalized in recent years; this was the most recent of these rules. In its BCA, EPA used estimates of willingness to pay (WTP) to value benefits, supplemented by cost-of-illness estimates when suitable WTP values were not available. This case study provided an example of a rule that had several health-related impacts that could not be quantified, as well as both quantified and nonquantified nonhealth effects (e.g., on visibility, crop yields, and other ecosystem functions). The key health effects of concern include preventable mortality and a number of acute and chronic cardiovascular and respiratory conditions. The following sections provide an overview of the general analytic approach for these case studies. We then discuss the details of the approaches applied in each case and report our results and conclusions. The final section summarizes the major lessons learned from these analyses. GENERAL APPROACH To estimate the QALY impacts of the regulations addressed by the case studies, we followed a three-part process.2 First, we described each type of injury or illness averted by the rule, based (to the extent possible) on the materials the agency used to support its regulatory analysis. Second, we used several different approaches to estimate the impact of each condition on health-related quality of life (HRQL) over the affected individuals’ lifespans. The methods used varied; each case study involved the application of three or four different approaches. Third, we determined the QALY losses averted by the regulation. This step involved estimating the change in HRQL attributable to the injury or illness under two scenarios: a base case analysis that assumed that affected individuals would be in average health (adjusted for age) over their remaining life expectancy in the absence of the condition of concern, and a sensitivity analysis that assumed that they would be in perfect or optimal health. For nonfatal effects, we then multiplied the resulting decrement by the expected duration of each illness. For preventable mortality, we estimated the change in life expectancy based on the average age of the affected individuals. This process is illustrated in Figure A-1. 2 The acknowledgments at the end of this appendix provide a complete list of those involved in each case study analysis.
OCR for page 196
Valuing Health for Regulatory Cost-Effectiveness Analysis FIGURE A-1 Case Study Process In these case studies, we focused on annual impacts for simplicity and comparability, assessing the change in disease or injury incidence attributable to a single year of the regulatory intervention. If the health effect is chronic or long-lived, however, the new cases of injury or illness prevented each year will have longer term impacts. We take these future year impacts into account and assess the lifetime effects of such cases, calculating the results both discounted and undiscounted. (We follow the discounting guidance in OMB, 2003a, as discussed in the main text of this report.) Agencies’ regulatory analyses generally take a longer view and assess the impacts of the rulemakings over a multiyear period. We believe that this multiyear focus is appropriate; although the presentation of annualized impacts can provide useful information, it should be provided only as a supplement to an analysis that considers the implementation of the rule over a longer time horizon. Below, we provide an overview of the methods we applied across all three case studies, focusing on the process used to describe the health endpoints and to compare HRQL with and without the condition of concern. In the health care field, “without condition” health (i.e., the health status of an individual in the absence of a particular illness or injury of concern) is often referred to as “baseline” health. We avoid this term because baseline means something different in regulatory analysis; it refers to the situation in the absence of the rule, which is equivalent to “with condition” health status.
OCR for page 197
Valuing Health for Regulatory Cost-Effectiveness Analysis Describing Health Endpoints The first step in the case study analysis involved describing the health endpoints so that they could be valued under alternative HRQL approaches. To increase our understanding of the information typically available to regulatory agencies and for consistency with the agency analyses, we based these descriptions on the same information used by the agency in its risk assessment to the maximum extent possible. Because the original FDA analysis used an HRQL index in its BCA, it supplied most of the information needed for the case study. In contrast, the approach used in the NHTSA rule relied on broad standardized injury classifications that were not adequate for estimating HRQL impacts. Thus we used a different data set to develop descriptions of the injuries averted. For the EPA rule, we relied on a combination of the information provided in the agency’s regulatory analysis and in a separate EPA analysis of the QALY impacts of air pollution-related health effects. In each case study, we used at least one approach that involved expert assignment of the HRQL attributes for the illnesses or injuries of concern. Developing descriptions for these expert assignments involved several challenges. First, we needed to determine the appropriate level of detail. Our goal was to provide enough information so that medical experts could understand and distinguish between different health endpoints, without overwhelming them with unnecessary information. Our schedule precluded formal pretesting; instead, we consulted informally with individuals with relevant expertise to develop these descriptions. Second, we wanted to avoid using language in the descriptions that could prejudice the assignment of the attribute levels included in each index (e.g., “little” or “no” difficulty in self-care; “moderate” pain). It was difficult to avoid this language completely, however; in some cases such terminology was part of the description used by the agencies to distinguish between different endpoints. For example, FDA distinguished between different types of long-term reactive arthritis based in part on the degree of pain experienced. Finally, the agency regulatory assessments of the health endpoints were for predicted risks (or statistical cases) rather than for individual, identifiable patients, and cover time periods over which HRQL impacts may vary. In theory we could have developed longitudinal models that identified distinct phases of each condition, the duration of each phase, and its probability of occurrence. Such models are difficult to develop, however, and require substantially more time and resources than were available. Instead, we encouraged the experts to consider the average or typical patient with each illness or injury and to assess the expected average HRQL impact over the course of the condition. In some cases, we divided the health conditions
OCR for page 198
Valuing Health for Regulatory Cost-Effectiveness Analysis into different phases. For the child restraints analysis, for example, we asked the experts to estimate the duration of the acute, rehabilitation, and long-term phases and to assign attribute levels separately to each phase. In the air quality case study, we split the cardiovascular disease endpoints into subcategories (based on age at incidence, severity, and disease progression), to better distinguish different health states. Estimating “With Condition” HRQL To estimate the HRQL impacts of each health condition averted by these regulations, we relied on several commonly used generic indexes: the EuroQol (EQ)-5D, the Health Utilities Index (HUI) Mark 2 and Mark 3, the SF-6D, and the Quality of Well-Being Scale (QWB).3 In addition, for the NHTSA case study, we applied an instrument which is now being created specifically to assess the longer term impacts of traumatic injury, the Functional Capacity Index (FCI). Chapter 3 and Appendix B of this report provide detailed information on each of these indexes. Applying these indexes entails two steps. First, the characteristics of each health condition are matched to (or assigned) attribute levels under each domain of each index. For example, for the EQ-5D, this process involves determining whether the disease or injury leads to “severe,” “moderate,” or “no” impairments within five domains—mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Second, the resulting attribute responses are weighted to reflect the value placed on different levels of impairment. Each generic index relies on a particular scoring algorithm to develop relative values for particular health states; this algorithm is based on statistical analyses of the results of a valuation survey developed especially for the classification system of that index. These valuation surveys are described in Chapter 3; see especially Table 3-4. In each case study, at least one of the HRQL approaches involved expert assignment of the attributes defined under a particular generic index. Although it is generally preferable to ask patients to complete this step, expert judgment is often used to provide a faster and less costly assessment. For expediency, we followed a simple expert judgment process that was not fully consistent with the best practices described in Chapter 3. For example, we recruited volunteer experts through our informal professional networks based largely on their availability. Consequently, the resulting groups may not represent the full range of subspecialties or types of patients relevant to 3 As discussed in Chapter 3 and shown in Appendix B, the HUI-2 and -3 include some differences in domains, in part because the HUI-2 was originally developed to assess health states among children.
OCR for page 199
Valuing Health for Regulatory Cost-Effectiveness Analysis the assessment.4 A more sophisticated approach could use specific selection criteria to ensure a broad range of relevant expertise and experience as well as geographic stratification, and could involve asking specialty societies for nominations. We also did not work with the experts to ensure that they had a thorough or common understanding of the materials describing the health endpoints, the domain attributes, and the task itself. Nor did we attempt to resolve any inconsistencies either within the responses of an individual expert or across the responses from different experts. We used simple decision rules to fill in any missing data. In a few cases, we relied on patient data from the available research literature rather than expert judgment. For the NHTSA study, we used QWB values from a study of trauma patients (Holbrook et al., 1999). For the EPA case study, we used preliminary condition-specific EQ-5D values estimated from the Medical Panel Expenditure Survey (MEPS) (Sullivan et al., 2005). In the EPA case study, we also transferred values from two patient studies selected from the Harvard School of Public Health’s CEA Registry (http://www.hsph.harvard.edu/cearegistry/), based on a review by Brauer and Neumann (2005). The case study approaches are summarized in Table A-1. Comparing to “Without Condition” HRQL To represent likely HRQL in the absence of the conditions of concern (i.e., once the regulation has been implemented), we used estimates of average population health broken out by age from major national population health surveys that included the relevant generic index questionnaire. This approach is equivalent to assuming that, in the absence of the hazard addressed by the regulation, affected individuals on average would have the same health status as the average member of the U.S. population in the same age group. In sensitivity analysis, we also compared the “with condition” HRQL estimates to a value of 1.0. This latter comparison is equivalent to assuming that, in the absence of the illness or injury, the affected individuals would be in perfect or optimal health.5 These age-adjusted estimates of average population health use the same underlying community-based valuation survey for each index (as discussed in Chapter 3) and were based on unpublished analyses prepared for the 4 The original construction and valuation of the FCI provides an example of a more formal expert judgment process in which multiple relevant clinical specialties and perspectives were represented. 5 We provide this comparison because it is often found in the literature; however, the Committee does not recommend this approach. See Fryback and Lawrence (1997), for a discussion of the problems with calculating changes from optimal health (1.0).
OCR for page 200
Valuing Health for Regulatory Cost-Effectiveness Analysis TABLE A-1 Approaches for Determining “With Condition” HRQL Rule Approach Indexes Data Source FDA Juice Processing Expert assignment EQ-5D, HUI-3, QWB, SF-6D Analysis of data provided by medical experts contacted by case study team NHTSA Child Restraints Expert assignment EQ-5D, HUI-2 Analysis of data provided by medical experts contacted by case study team Trauma patient survey QWB Analysis of patient data provided by Troy Holbrook, University of California, San Diego Expert judgment FCI Expert data and weighting formula provided by Ellen MacKenzie, Johns Hopkins University EPA Nonroad Diesel Emissions Expert assignment EQ-5D Analysis of data provided by medical experts contacted by case study team Population survey (MEPS) EQ-5D Preliminary analysis of self-reported HRQL provided by Patrick Sullivan, University of Colorado Transfer from Harvard Registry studies EQ-5D, HUI-3 Analysis of patient data from Oostenbrink et al. (2001) and Torrance et al. (1999) Committee’s use in the case studies.6 The estimates were provided by age and gender, and generally broken into 10-year age groups. These population averages were missing estimates for very young and very old individuals. We assumed that, for ages 0 through 9 years, average health would equal perfect health (a value of 1.0); for ages 10 through 19, average health would be the midpoint between perfect health and the values estimated for ages 20 through 29; and for those older than the reported age 6 EQ-5D estimated based on 2001 MEPS data by Dr. William Lawrence, Agency for Healthcare Research and Quality. HUI-3 estimated based on 2002 Joint U.S.–Canada Survey of Health data by Barbara Altman, National Center for Health Statistics. (We used the HUI-3 estimates for the HUI-2 analysis, since the general populations averages are expected to be similar.) SF-6D estimated based on 2001 MEPS data by Janel Hanmer, University of Wisconsin-Madison. QWB estimated based on 2001 U.S. National Health Interview Survey data by John Anderson, University of California, San Diego. Updated estimates for the EQ-5D, SF-6D, and QWB are available in Hanmer et al. (2006). Population averages were not available for the FCI.
OCR for page 201
Valuing Health for Regulatory Cost-Effectiveness Analysis TABLE A-2 Without-Condition HRQL Age 20 Age 40 Age 60 Age 80 Mean Population Index Value (base case) EQ-5D 0.92 0.88 0.83 0.75 HUI-2 and 3 0.91 0.88 0.82 0.69 SF-6D 0.84 0.82 0.79 0.72 QWB 0.82 0.80 0.74 0.65 Perfect Health (sensitivity analysis) All indices 1.0 1.0 1.0 1.0 NOTES: See Hanmer et al. (2006) for updated estimates and information on uncertainty. Table presents results rounded to two significant figures for selected age groups. Unrounded estimates for each year of age are used in all calculations. SOURCES: EQ-5D: Unpublished analysis by William Lawrence, November 9, 2004. HUI-3: Unpublished analysis by Barbara Altman, January 7, 2005. SF-6D: Unpublished analysis by Janel Hanmer, January 24, 2005. QWB: Unpublished analysis by John Anderson, April 21, 2005. ranges, average health would remain constant at the value reported for the eldest age group. This approach means that the HRQL impacts for young children will be the same regardless of whether the comparison is to perfect or average health, since a value of 1.0 is used for “without condition” HRQL in both cases.7 Table A-2 presents the estimates of average population health used in this analysis for selected ages, for males and females combined. These estimates are provided for illustrative purposes; the case study calculations used the full range of estimates available for each age group. As is evident from the table, the estimates of average population health vary. This variation reflects several factors, including the differences in (1) the population surveyed to determine their health-related attributes; (2) the underlying valuation survey; and (3) the construction of indexes themselves. In combination, these factors generally lead to the highest average HRQL estimates under the EQ-5D and the lowest under the QWB. As expected, average HRQL declines with age under each index. The comparison of HRQL with and without the conditions of concern is complicated by the assumptions that underlie the approach used to assign and value attributes under each index. In these comparisons, we adjusted the values depending on the source of the “with condition” esti- 7 The assumption that average HRQL for infants and children is close to optimal and can be approximated by an index value of 1.0 may not be well founded, however. Some surveys of children’s self-reported HRQL have reported lower values (Hennessy and Kind, 2002).
OCR for page 234
Valuing Health for Regulatory Cost-Effectiveness Analysis various combinations; and our benefits transfer from the CEA Registry studies used one estimate for chronic bronchitis and one estimate for all post-AMI conditions. In the expert assignment, we found that the results did not always vary across the severity categories. The EQ-5D allows a choice of three attribute levels within each domain. In some cases, individual experts assigned the same attribute levels to cases of differing severities. The assignments also indicated that the experts disagreed about whether certain conditions would impose no, moderate, or severe problems in a particular domain. Where the estimates varied across endpoints, they generally followed the expected pattern, showing increasing problems for cases with increasing severity. Mild cases resulted in median HRQL values close to 1.0, indicating a negligible effect on the quality of life. In contrast, the most severe form of congestive heart failure led to HRQL values close to zero, with median estimates of 0.05 or less. In general, the median values were identical for the two age groups specified in the AMI scenarios (those above and below 65 years). The QALY estimates varied across the three approaches. In Table A-14, we provide the results for the average age at incidence under each approach, in comparison to both average and perfect health. (The adjustments made in these comparisons are described in the “General Approach” section, above.) While these adjustments seem sensible within the context of each approach, they lead to inconsistencies in the relationships across the results. As illustrated by the table, for the expert assessment, the “with condition” values (and the decrement from normal health) are consistently lower under the average health scenario than under the perfect health scenario; we applied the same percentage reduction to a lower value (average “without condition” HRQL is less than perfect HRQL). For the MEPS-based EQ-5D catalogue, the “with condition” values are the same under both scenarios, but the decrement is larger under the perfect health scenario and increases with age (because we add the difference between average and optimal health, which grows with age). For the values taken from the CEA Registry studies, which scenario results in larger estimates depended on age, because we anchored the percentage reduction from average population health at the average age of the underlying study samples. The average age in the chronic bronchitis study is 55 years, slightly higher than the average age at incidence used in our analysis (Torrance et al., 1999). For the AMI study, the average age of the study sample is 69 years (Oostenbrink et al., 2001). We multiplied the estimates of decrements from “without condition” health by duration (taking life expectancy into account) to determine the QALY losses associated with each nonfatal endpoint as well as with pre-
OCR for page 235
Valuing Health for Regulatory Cost-Effectiveness Analysis TABLE A-14 Nonroad Diesel Emissions Case Study: HRQL with Illness, at Average Age of Incidence Endpoint Average Age at Incidence Base Case, Compared to Average Health Sensitivity Analysis, Compared to Perfect Health Without Conditiona With Condition Without Conditiona With Condition EQ-5D Expert Assessmentb EQ-5D MEPS Cataloguec Transfer from Selected Studiesd EQ-5D Expert Assessmentb EQ-5D MEPS Cataloguec Transfer from Selected Studiesd Nonfatal chronic bronchitis 49 0.88 0.34–0.88 0.81 0.82 1.00 0.39–1.00 0.81 0.78 Nonfatal acute myocardial infarction 53 0.85 0.03–0.85 0.70–0.81 0.60 1.00 0.03–1.00 0.70–0.81 0.58 78 0.78 0.02–0.78 0.63–0.74 0.55 1.00 0.03–1.00 0.63–0.74 0.58 Preventable mortality—adults 74 0.78 0.00 0.00 0.00 1.00 0.00 0.00 0.00 Preventable mortality—infants 0 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 NOTES: Ranges reflect the results for the different health state subcategories assessed for each endpoint. aWithout condition values for average health are based on the EQ-5D, except for the values for chronic bronchitis under the Harvard Registry approach, which are based on the HUI-3. At age 49, the value for average population health is 0.88 under both indices. bFor the expert assignment, “with condition” health is assumed to be the same fraction of average health as of perfect health for all years of age affected. cFor the EQ-5D MEPS catalogue, numerical decrements from average health are assumed to be constant across all years of age, and the difference between “without condition” average and perfect health is added to this decrement for the perfect health comparison. dFor the transfer from the CEA Registry studies, “with condition” health is assumed to be a constant fraction of “without condition” health; this fraction is calculated based on the average age of the samples used in each study. SOURCES: Case study team analysis of data from the following sources. Expert assignment: Data provided February to April, 2005. MEPS data: preliminary results provided by Patrick Sullivan, April 4, 2005. CEA Registry: Torrance et al. (1999) and Oostenbrink et al. (2001).
OCR for page 236
Valuing Health for Regulatory Cost-Effectiveness Analysis ventable mortality. We report the results of these calculations in Table A-15. The results reflect the losses for all cases, assuming that the health status of affected individuals would be the same as the population average for individuals of the same age in the absence of the pollution-related health effects. These estimates represent the lifetime losses for all cases averted by the annual reduction in pollution levels as of the year 2030; using discounting to reflect the future year impacts of the new cases, i.e., their lifetime effects. Undiscounted, the results range from 160,000 to 170,000 QALYs. Without adjustment for HRQL, the life-year losses associated with the cases of preventable mortality (including fatalities for 12,000 adults and 22 infants) total 130,000 life years undiscounted; 93,000 life years if discounted at 3 percent; and 64,000 life years discounted at 7 percent. As shown in Table A-15, the three approaches to estimating HRQL impacts yield differing results. Because the estimates for mortality are identical under all three approaches, these differences are driven by the approaches used to value the nonfatal endpoints. The expert assignment yields values for chronic bronchitis that are more than twice as large as the estimates from the EQ-5D MEPS catalogue or CEA Registry studies. For TABLE A-15 Nonroad Diesel Emissions Case Study: QALY Losses, All Cases HRQL Approach/Endpoint 3% Discount Rate 7% Discount Rate Expert Assignment of EQ-5D Attributes Nonfatal chronic bronchitis 16,245 9,966 Nonfatal AMI 10,259 7,823 Preventable mortality 92,852 63,605 Total 119,356 81,395 EQ-5D MEPS Catalogue Nonfatal chronic bronchitis 7,136 4,341 Nonfatal AMI 8,848 6,402 Preventable mortality 92,852 63,605 Total 108,837 74,349 Transfer from Selected Studies Nonfatal chronic bronchitis 6,028 3,699 Nonfatal AMI 15,246 10,782 Preventable mortality 92,852 63,605 Total 114,126 78,086 NOTES: Assumes that, in the absence of illness, health status would equal the average for the U.S. population in the same age group. Represents HRQL decrement per case multiplied by duration and by number of new cases averted annually. Detail may not add to total due to rounding.
OCR for page 237
Valuing Health for Regulatory Cost-Effectiveness Analysis the AMI endpoints, the CEA Registry studies lead to estimates of QALY losses that are greater than the results under the expert assessment or the EQ-5D catalogue, possibly because that study addressed more severe cases than the average post-AMI population. The estimates of the number of cases avoided, age at incidence, and life expectancy are constant across all three approaches; hence these results reflect the differing estimates of the HRQL decrement associated with each condition. Table A-16 provides the estimates of QALY losses that result when the “with condition” HRQL is compared to perfect health rather than to average age-adjusted HRQL. As noted earlier and illustrated in Table A-14, the approach that produces the largest estimates of “with condition” HRQL varies due to the differing adjustments used in these comparisons. As expected, the results are larger in the perfect health comparison because perfect health is represented by a constant value of 1.0 across all years of age, while average health declines with age. Cost-Effectiveness Ratios Our final step involved reporting the four cost-effectiveness ratios discussed in Chapter 5 of this report, based on the data available for this case study. In these calculations, we use EPA’s estimates of annualized regulatory costs, which are reported as $2.0 billion per year regardless of whether a 3 or 7 percent discount rate is used. For health care treatment costs, we use the per-case medical cost estimates for treatment of chronic bronchitis and nonfatal AMIs provided in Hubbell (2004), which round to $1.1 billion regardless of which discount rate is applied. This estimate will understate total health care cost savings because it excludes other types of costs (such as health care-related time losses) associated with treatment of the conditions. TABLE A-16 Nonroad Diesel Emissions Case Study: Sensitivity Analysis for QALY Losses Scenario Discount Rate EQ-5D Expert Assignment EQ-5D MEPS Catalogue Transfer from Selected Studies Total QALY losses compared to average age-adjusted health 3% 119,356 108,837 114,126 7% 81,395 74,349 78,086 Total QALY losses compared to perfect health 3% 154,447 186,785 173,160 7% 104,666 125,292 116,638
OCR for page 238
Valuing Health for Regulatory Cost-Effectiveness Analysis In the comprehensive ratio, we net out the value of the benefits not addressed in the effectiveness measure; i.e., the short-lived health impacts and the environmental effects. According to EPA’s analysis, the total value of these additional benefits is about $2.3 billion annually as of the year 2030. In other words, the combined value of these other benefits exceeds the costs of the regulations. Thus netting these benefit values out of the regulatory costs led to negative costs, or savings. In Table A-17, we report the results for each of the ratios recommended by the Committee. The costs per QALY are less than the costs per life year saved in part because the estimate of costs in the former ratio is lower due to the netting out of medical cost savings. The ratios are within the same order of magnitude across the different approaches used to assess HRQL, and in some cases appear indistinguishable. For the comprehensive ratio, we do not report the results of the calculations because the netting out of other benefits leads to cost savings. All of the cost per QALY estimates would be lower if we used the results of our sensitivity analysis, since the comparison to perfect health yields larger estimates of QALY losses. Again, this case study does not fully reflect certain of the Committee’s recommendations. While we did not fully assess the distributional or ethical implications of this regulation, Chapter 4 provides an example of a summary of these impacts, and EPA’s analysis provides more detailed information on related topics. In addition, our analysis relies on mean or median values and provides only limited assessment of uncertainty. More extensive uncertainty analysis is required by both the Committee’s recommendations and the existing government-wide guidance. EPA’s BCA provides substantial discussion of this issue, including various assessments of the degree of uncertainty in both the cost and benefit estimates. In this case study, the experts involved in determining the EQ-5D attributes raised several issues similar to those raised by the experts involved in the FDA and NHTSA studies. These concerns related to the relationship between the disease descriptions and the attribute descriptions, the differences between expert and patient judgments about disease impacts, and the difficulties inherent in considering an “average” or “typical” case rather than an individual patient. As noted earlier, there are a number of steps that analysts can take to develop a more thorough assessment process; e.g., pretesting the approach, working with the experts to ensure that they have a common understanding of the health conditions, index attributes, and the task itself, and following the initial assignment with a process for resolving (or better understanding) any inconsistencies in the results. Relying on patient, rather than expert, assignments was not possible given the time and resources available for this case study, but could significantly alter the findings. For the other two approaches used in this case study, related uncertain-
OCR for page 239
Valuing Health for Regulatory Cost-Effectiveness Analysis TABLE A-17 Nonroad Diesel Emissions Case Study: Cost-Effectiveness Ratios 3% Discount Rate 7% Discount Rate Averted deaths 12,000 deaths 12,000 deaths Averted life-year losses 93,000 years 64,000 years Regulatory compliance costs $2.0 billion $2.0 billion Compliance cost per fatality averted $170,000 $170,000 Compliance cost per life-year gained $22,000 $31,000 EQ-5D Expert Assignment EQ-5D MEPS Catalogue Transfer from Selected Studies EQ-5D Expert Assignment EQ-5D MEPS Catalogue Transfer from Selected Studies Averted QALY losses 120,000 QALYs 109,000 QALYs 114,000 QALYs 81,000 QALYs 74,000 QALYs 78,000 QALYs Regulatory compliance costs, net of health treatment savings $ 0.9 billion Regulatory compliance costs, net of health treatment savings and value of additional benefits ($1.4 billion) Health-benefits-only ratio $7,500 per QALY $8,300 per QALY $7,900 per QALY $11,000 per QALY $12,000 per QALY $12,000 per QALY Comprehensive ratio Cost-Saving NOTES: Reflects new incidence averted in the year 2030; 2000 dollars. Assumes that, without the pollution-related illness, health status would be the same as the average for the U.S. population in the same age group. Rounded to two significant figures, numbers in parentheses are negative, calculations are based on unrounded results.
OCR for page 240
Valuing Health for Regulatory Cost-Effectiveness Analysis ties are discussed in the background documents. The MEPS-based EQ-5D analysis (Sullivan et. al, 2005) includes a variety of data that could be used in more formal, quantitative analysis of uncertainty. In applying estimates from the CEA Registry studies, we rely on a single study for each endpoint. However, other studies report varying results for similarly defined health conditions (see Brauer and Neumann, 2005). A more comprehensive approach would consider the full range of values reported; similar, for example, to the approach used in Hubbell (2004). CONCLUSION These case studies demonstrated that it is possible to apply a number of approaches to assess the cost-effectiveness of economically significant health and safety regulations. While the Committee was not able to conduct new primary research on the HRQL impacts of the health effects considered, we were able to examine the consequences of applying expert judgment processes and information from different types of existing studies. Although more sophisticated application of these approaches is desirable in the context of actual regulatory analyses, all appear feasible and provide information of interest for decision making. The case studies also aided us in identifying areas where more research would be useful. For example, the experts involved in the assignment process noted that the generic indexes did not always provide attribute descriptions that were applicable to the health conditions being characterized, and better tailored approaches might be desirable. This was particularly true when the indexes were applied to children. In addition, our review of existing studies in the CEA Registry indicated gaps and inconsistencies in the HRQL values currently available for application to regulatory analysis. Meta-analysis or other approaches that combine results of different studies, as well as additional analysis of uncertainties, also could be helpful. In addition, further development of criteria and best practices for transferring estimates from existing studies would be desirable. We also found that the MEPS catalogue used in the EPA case study was quite useful for this sort of analysis; it provides U.S. population health state index values for a variety of conditions encountered in many regulatory analyses. The case studies suggested that the types of health risk information available to regulatory analysts pose challenges not necessarily present in clinical outcomes studies or medical technology assessments. In particular, regulatory agencies generally work with risk estimates that reflect small changes in the probability of injury, illness, or death spread throughout a large population. This focus on expected or statistical cases often may require assessing HRQL and longevity impacts for an average or typical
OCR for page 241
Valuing Health for Regulatory Cost-Effectiveness Analysis case (or range of cases) of each condition averted by a rule. While some of the health risk information needed to implement a QALY-based CEA is not needed for a BCA, many agencies have developed this additional data in the context of implementing their own approaches to CEA. We faced the most significant data constraints in the NHTSA case study because of the broad injury categories used by that agency. More detailed data on the injuries averted by a particular rule would allow more accurate assessment of HRQL impacts. The cost-of-illness estimates currently used by the agencies are not entirely compatible with the definition of health treatment costs developed for the reference case by the U.S. Panel on Cost Effectiveness in Health and Medicine (Gold et al., 1996b) and discussed in the Committee’s recommendations. In many cases, these estimates only include direct medical costs. When lost productivity estimates were available, they addressed the long-term impacts of the health condition, not solely the impacts of medical treatment. As noted in the main text of this report, such estimates of lost productivity are likely to double count impacts included in the effectiveness measure, and hence are not suitable for this type of analysis. Development of standard estimating practices for the health care treatment costs to be used in CEA would be useful. The case studies also provide examples of the implications of a number of the Committee’s recommendations. For instance, the FDA and EPA rules differ significantly in terms of the importance of preventable mortality to the results. For the EPA rule, which averts a relatively large number of deaths, the cost per life year and cost per QALY gained are much more similar than in the case of the FDA rule, which prevents very few deaths. The EPA rule also illustrates the potential for significant changes in the cost-effectiveness measure when other benefits are considered in a comprehensive ratio. Furthermore, the analyses show the importance of comparing “with condition” values to measures of expected actual “without condition” health; comparisons to perfect health lead to estimates of QALY losses that are misleadingly large in some cases. Finally, we were not able to assess whether alternative HRQL approaches would change regulatory decisions. The final rules used in these case studies lacked information on the impacts of the wide range of regulatory options required by the OMB guidance, so we could not compare the results of different HRQL approaches across regulatory options. However, the cost-per-QALY estimates appear relatively similar across the different HRQL approaches used in the case studies. For example, using a 3 percent discount rate, the range for the health-benefits-only ratio was $13,000 to $18,000 per QALY in the FDA case study, and $7,500 to $8,300 in the EPA case study.
OCR for page 242
Valuing Health for Regulatory Cost-Effectiveness Analysis ACKNOWLEDGMENTS This appendix represents the collaborative efforts of the IOM Committee members, advisers, consultants, and staff with federal agency staff and consultants. The case studies could not have been completed without the exceptional efforts of a great many people. The goal of these studies was to enhance the Committee’s understanding of current practices and of the issues that arise in applying different measures of benefits in a regulatory context, and they were an important source of information and insights that contributed significantly to our deliberations. In a very real sense, everyone who contributed to these case studies was a volunteer. The scope of effort to produce these analyses exceeded the time and money originally budgeted for the task, and the case study teams worked beyond all original expectations. The Committee is indebted to all those who have contributed their time and expertise to gather information, explain agency policies and practices, and complete a daunting array of analytic tasks. The Committee thanks the following individuals for their advice, generosity, and hard work. Juice Processing Regulation Case Study Lead authors: Lisa A. Robinson, Independent Consultant; Wilhelmine Miller, Institute of Medicine; Robert Black, Independent Consultant. IOM Committee advisers: Alan Garber (lead); Judith Wagner. Other advisers: Clark Nardinelli, Food and Drug Administration; Sajal Chattopadhyay, Centers for Disease Control and Prevention. Contributors: John Anderson, University of California, San Diego; Barbara Altman, National Center for Health Statistics; Fred Angulo, Centers for Disease Control and Prevention; Lawrence Deyton, M.D., Veteran’s Administration; Sherine Gabriel, M.D., Mayo Clinic; Janel Hanmer, University of Wisconsin-Madison; William Lawrence, M.D., Agency for Healthcare Research and Quality; Gwen Wanger, M.D., Beth Israel Deaconess Medical Center. Expert application of generic indexes: Infectious disease—Claire Panosian, M.D., David Geffen School of Medicine, University of California, Los Angeles (UCLA); David A. Pegues, M.D., David Geffen School of Medicine, UCLA; Matthew Leibowitz, M.D., David Geffen School of Medicine, UCLA; Glenn Mathisen, M.D., Olive View-UCLA Medical Center; Sherwood L. Gorbach, M.D., Tufts New England Medical Center; David R. Snydman, M.D., Tufts New England Medical Center; Mark Holodniy, M.D., Veteran’s Administration Palo Alto Health Care System; Victoria
OCR for page 243
Valuing Health for Regulatory Cost-Effectiveness Analysis Davey, R.N., M.P.H., U.S. Department of Veterans Affairs. Rheumatology—Lenore Buckley, M.D., Virginia Commonwealth University School of Medicine; Gene G. Hunder, M.D., Mayo Clinic (retired); Eric L. Matteson, M.D., Mayo Clinic College of Medicine; Daniel H. Solomon, M.D., Harvard Medical School; Elizabeth A. Tindall, M.D., Oregon Health and Science University. Child Restraints Regulation Case Study Lead authors: Lisa A. Robinson, Independent Consultant; Phaedra Corso, Centers for Disease Control and Prevention; Xiangming Fang, Centers for Disease Control and Prevention; Robert Black, Independent Consultant; Wilhelmine Miller, Institute of Medicine. IOM Committee advisers: Emmett Keeler (lead); Henry Anderson; Lisa Iezzoni; Alan Krupnick. Other advisers: Larry Blincoe, National Highway Traffic Safety Administration; Jim Simons, National Highway Traffic Safety Administration; Carmen Brauer, M.D., Harvard School of Public Health. Contributors: John Anderson, University of California, San Diego; Barbara Altman, National Center for Health Statistics; Nancy Bondy, National Highway Traffic Safety Administration; David Feeny, Kaiser Permanente; Janel Hanmer, University of Wisconsin-Madison; Troy Holbrook, University of California, San Diego; Robert Kaplan, University of California, Los Angeles; William Lawrence, M.D., Agency for Healthcare Research and Quality; Ellen MacKenzie, Ph.D., Johns Hopkins University; Bryce Mason, Rand Corporation; Ted Miller, Pacific Institutes for Research and Evaluation; Ryan Palugod, Institute of Medicine; William Rhoads, Centers for Disease Control and Prevention; Jon Walker, National Highway Traffic Safety Administration. Expert application of generic indexes: Carmen Brauer, M.D., Harvard School of Public Health; Kristine Campbell, M.D., Children’s Hospital of Pittsburgh; Tim Davis, M.D., Centers for Disease Control and Prevention; Arlene Greenspan, Ph.D., Centers for Disease Control and Prevention; David Mooney, M.D., Children’s Hospital, Boston. Nonroad Engine Air Emissions Regulation Case Study Lead authors: Lisa A. Robinson, Independent Consultant; Wilhelmine Miller, Institute of Medicine; Robert Black, Independent Consultant.
OCR for page 244
Valuing Health for Regulatory Cost-Effectiveness Analysis IOM Committee advisers: Maureen Cropper (lead); Richard Burnett; James Hammitt; Alan Krupnick. Other advisers: Carmen Brauer, M.D., Harvard School of Public Health; Bryan Hubbell, U.S. Environmental Protection Agency; Tursynbek Nurmagambetov, Centers for Disease Control and Prevention; Seymour Williams, Centers for Disease Control and Prevention. Contributors: Adam Atherly, Centers for Disease Control and Prevention; Sarah Brennan, Industrial Economics Incorporated; Jim DeMocker, U.S. Environmental Protection Agency; Chris Dockins, U.S. Environmental Protection Agency; Janel Hanmer, University of Wisconsin-Madison; Fernando Holguin, Centers for Disease Control and Prevention; William Lawrence, M.D., Agency for Healthcare Research and Quality; Darwin LaBarthe, Centers for Disease Control and Prevention; Jim Neumann, Industrial Economics, Incorporated; Peter Neumann, Harvard School of Public Health; Nathalie Simon, U.S. Environmental Protection Agency; Patrick Sullivan, University of Colorado. Expert application of generic indexes: Respiratory disease—David M. Mannino, M.D., University of Kentucky School of Medicine; Peter Barkin, M.D., Emerson Hospital; R. Graham Barr, M.D., Presbyterian Hospital, Columbia University; Scott D. Ramsey, M.D., Fred Hutchinson Cancer Research Center; Mark J. Utell, M.D., University of Rochester; Roger Yusen, M.D., Washington University School of Medicine. Cardiovascular disease—Harlan M. Krumholz, M.D., Yale Medical School; Russell V. Luepker, M.D., Mayo Clinic; John Rumsfeld, M.D., University of Colorado; Douglas D. Schocken, M.D., University of South Florida; John Spertus, M.D., University of Missouri-Kansas City.
Representative terms from entire chapter: