2
Characteristics of Major Regulations and Current Analytic Practices
The U.S. Congress has granted the executive branch broad authority to develop regulations addressing health and safety risks. At the federal level, the Office of Management and Budget (OMB) has primary responsibility for coordinating and reviewing the economic analyses that support these regulations, and has developed detailed guidelines for these analyses. This chapter builds on the information provided in Chapter 1 to discuss the general approaches to health and safety regulations authorized by Congress and summarizes current practices for analyzing their impacts. It provides background and context for the Committee’s recommendations.
TYPES OF RISK REGULATIONS
The starting point for the issuance of federal regulations is authorizing legislation developed by Congress. This legislation generally establishes a legal basis or standard for the regulations and lays out the considerations to be taken into account by the agency in developing the regulations. Although the government issues thousands of different types of regulations each year, only a few are economically significant health and safety regulations subject to the OMB requirements for benefit–cost analysis (BCA) and cost-effectiveness analysis (CEA). If the relevant statute prohibits the consideration of costs or BCA, then the executive branch cannot countermand the statute in developing the regulation. (See Whitman v. American trucking (U.S. S. Ct. 2001).) The use of economic analysis in developing regula-
tions has been subject to judicial challenge and may be upheld or rejected depending on the wording of the specific statute.
Statutory Standards
Under the U.S. Constitution, Congress has wide authority, upheld by the courts, to enact legislation to protect the public’s health and safety. The U.S. Supreme Court has generally permitted Congress to delegate this discretion to the federal regulatory agencies. When Congress authorizes individual agencies to examine particular risk exposures, such as those associated with air or water pollution, it may establish an approach based on health alone (prohibiting consideration of cost); on the best available technology (including the consideration of cost); on feasibility (which depends on cost but not on BCA); on the least burdensome alternative to achieve a given goal (implying CEA); or on reasonableness (implying BCA). At times the legislation will be silent as to the type of standard or will take a mixed approach. Many of these authorizing statutes have both health- and nonhealth-related goals; for example, they may require that agencies address impacts on the natural environment as well as on human health. In addition, some statutes establish other factors that must be considered, such as the impact on sensitive populations.
The following discussion sketches the statutory and regulatory context for the economic analyses of regulations that are the subject of this report and of the Committee’s recommendations. We consider legislative standards and procedural requirements for agency deliberations and policy determinations, supplementing the discussion of the regulatory development process in Chapter 1.
Health-Based Requirements
Health-based statutes establish a general goal of reducing the risks of death, illness, and injury. In Section 3(8) of the Occupational Safety and Health Act of 1970, for example, Congress gave the Secretary of Labor broad regulatory authority to adopt standards “reasonably necessary or appropriate to provide safe or healthful employment.” Section 6(b)(5), for toxics, requires the Occupational Safety and Health Administration (OSHA) to “set the standard which most adequately assures, to the extent feasible, on the basis of the best available evidence, that no employee will suffer material impairment of health or functional capacity….” Immediately following OSHA’s benzene and cotton dust regulations, industry groups argued that the statutory language required OSHA to use BCA in promul-
gating regulations. However, after a series of cases, the Supreme Court concluded that the statute did not require OSHA to use BCA and that the agency could instead rely on epidemiological and other scientific findings to establish safe levels of exposure.
Even though nothing in a health-based statute may prohibit an agency from taking costs into account, the courts expect regulatory actions to be based on sound scientific evidence that establishes an appropriate level of public safety. Indeed, one concern about a health-based standard is whether it amounts to zero tolerance. Except for a now-repealed amendment that explicitly enacted a zero-tolerance standard for new food additives, courts have provided agencies with considerable discretion to issue regulations that focus on more significant risks to human health and safety.
On occasion, Congress has specified a threshold of risk exposure. For example, Section 112 of the Clean Air Act set a threshold that certain carcinogenic emissions must be controlled if they produce an incremental risk of cancer of 1 in 1 million over a lifetime. For the most part, the risk exposure thresholds are established by the agencies as part of the regulatory development process. For example, the Environmental Protection Agency (EPA) explicitly considers health risk in hazardous waste listing decisions under the Resource Conservation and Recovery Act. Where the individual incremental cancer risk exceeds 1 in 100,000 over a lifetime, a wastestream will be considered for listing, and if the cancer risk exceeds 1 in 10,000, there is a presumption that listing is required (Hazardous Waste Management System, 59 Fed. Reg. 66,072, 66,077 (1994)). OSHA uses a 1 in 1,000 threshold for occupational exposures that create fatality risks (Adler, 2003).
Section 112 of the Clean Air Act, which regulates hazardous air pollutants, directs the Administrator to set an emission standard “at the level which in his judgment provides an ample margin of safety to protect the public health.” In NRDC v. EPA, 824 F.2d 1146, 1164 (D.C. Circuit 1987), the court interpreted what constitutes an ample margin of safety by noting that:
This language permits the Administrator to take into account scientific uncertainty and to use expert discretion to determine what action should be taken in light of that uncertainty…. Once “safety” is assured, the Administrator should be free to diminish as much of the statistically determined risk as possible by setting the standard at the lowest feasible level.
Other statutes also focus on reducing unreasonable risks to health, such as the Federal Insecticide, Fungicide, and Rodenticide Act; and the Consumer Product Safety Act. In general, such statutes may also require agencies to take other factors into account, such as harm to natural resources or ecological risks.
Technology-Based Requirements
Statutes may require that agencies apply the “best available technology” (BAT) in setting regulatory standards. The BAT approach allows the regulatory agency to consider a range of factors, including costs, but it usually does not require industry to develop new technologies. Rather, it requires industry to adopt available technologies to reduce emissions or toxic exposures. But stating such requirements is much easier than implementing them. As noted in NRDC v. EPA, 863 F.2d 1420, 1427 (9th Cir. 1988):
Technology-based limitations under BAT must be both technologically available and economically achievable. To be technologically available, it is sufficient that the best operating facilities can achieve the limitation. To demonstrate economic achievability, no formal balancing of costs and benefits is required; BAT should represent a commitment of the maximum resources economically possible…. EPA has considerable discretion in weighing the costs of BAT.
Hybrid Requirements
The above types of standard-setting requirements may be combined as well as supplemented with other factors for consideration by regulators. For example, Section 112 of the Clean Air Act first requires industries to use maximum achievable control technology standards to reduce toxic emissions, but then requires EPA to promulgate more stringent regulations if excess cancer risks exceed a threshold of 1 in 1 million. Thus, for example, the nonroad diesel rule (the subject of one of the Committee’s case studies) was authorized by two separate provisions of the Clean Air Act, which differ in focus. Section 213 is BAT based, instructing EPA to set standards that achieve the largest emissions reduction achievable through the use of available technology and allowing the EPA Administrator to consider the cost, lead time, noise, energy, and safety factors associated with the application of such technology. In contrast, Section 211(c) requires EPA to regulate fuels as needed to reduce adverse effects on human health or welfare, as well as to prevent impairment of emissions control devices.
Another example of a hybrid approach is the Safe Drinking Water Act. This statute requires EPA to set standards that are as close as “feasible” to the level at which there are no known or anticipated adverse health effects associated with exposure to the contaminant, taking into account an adequate margin of safety and considering the effects on sensitive subpopulations. “Feasible” is defined as the use of the best technology and treatment techniques examined for efficacy under field conditions, taking cost into consideration (Section 1412(b)(4)(D)). However, EPA can, at its discretion, establish an alternative standard that “maximizes health risk reduction
benefits at a cost that is justified by the benefits” (Section 1412(b)(6)(A)), with certain exceptions for small water systems.
This balancing of costs and benefits has been upheld by some courts, which have required agencies to consider the costs of a regulation as well as its benefits. For example, the Fifth Circuit has stated with respect to both the Occupational Safety and Health Act and the Toxic Substances Control Act (TSCA) that the government must show “… substantial evidence that the benefits to be achieved [by a regulation] … bear a reasonable relationship to the costs imposed by the reduction [before it can] … show that the standard is reasonably necessary to provide safe or healthful workplaces” (American Petroleum Institute v. OSHA, 581 F.2d 493, 504 (5th Cir. 1978); Corrosion Proof Fittings v. EPA, 947 F.2d 1201, 1223 (5th Cir. 1991)).
Several statutes identify other types of factors that an agency must take into account in developing regulations. Some require consideration of particular aspects of the health risks addressed. For example, the Food Quality Protection Act requires consideration of the susceptibility of infants and children to pesticide residues (Section 408(b)(2)(C)).
Others require certain types of analysis. In the case of SDWA, the 1996 amendments require the assessment of costs and benefits and identify factors that must be addressed. In particular, EPA must consider:
-
the quantifiable and nonquantifiable health risk reductions associated with controlling the contaminant of concern and any co-occurring contaminants;
-
the costs of compliance with the control requirements;
-
the incremental costs and benefits associated with each alternative under consideration;
-
the effects of the contaminant on the general population and on groups within the population that are likely to be at greater risk of adverse health effects from drinking water contaminants, “such as infants, children, pregnant women, the elderly, and individuals with a history of serious illness”;
-
the increased health risks, if any, that may result from compliance with the proposed standard, including risks associated with co-occurring contaminants; and
-
other relevant factors, including the quality of the available information supporting the analysis, the uncertainties in the analysis, and factors relating to the degree and nature of the identified risks (Section 1412(b)(3)(C)(i)).
Thus it is important for regulatory agencies to generate scientific bases for rules and to consider the least restrictive alternatives (i.e., those least
burdensome to industry), but to also balance costs and benefits, usually through BCA or CEA. In reviewing challenges to regulations, courts will not prescribe which general methodology an agency should use to estimate costs and benefits. As noted later, however, courts will not hesitate to question the particular methods and reasoning that agencies use in constructing their analyses.
Examples of Recent Regulations
As noted in Chapter 1, the federal requirements for CEA that are the subject of this report apply mainly to economically significant health and safety regulations. To identify the types of regulations that may be subject to these requirements, the Committee commissioned a review focusing on those agencies that recently issued economically significant regulations with quantified health and safety impacts, and/or that were in the process of doing so (Robinson, 2004). This review identified seven such agencies: EPA, the Food and Drug Administration (FDA), the Food Safety and Inspection Service (FSIS), OSHA, the National Highway Traffic Safety Administration (NHTSA), the Federal Motor Carrier Safety Administration, and the Consumer Product Safety Commission (CPSC).1
From January 2000 through June 2004, these agencies finalized 18 economically significant regulations with quantified health and safety impacts, listed in Table 2-1. Although they represent a relatively small proportion of all regulations issued each year, these regulations have substantial effects. For example, in fiscal 2003, an OMB summary indicates that only 3 of the 4,312 final rules published were economically significant rules with quantified health and safety benefits (OMB, 2003b, 2004). However, the 18 rules listed in Table 2-1 had estimated net benefits totaling close to $200 billion annually. If only costs are considered, six of these rules resulted in annual costs estimated to be greater than $1 billion. In terms of benefits, eight of the rules had estimated gross annual benefits exceeding $1 billion and, of these, three had estimated gross annual benefits exceeding $10 billion.
The majority of the quantified benefits addressed in these regulations were those attributable to EPA requirements for control of air pollution. Related benefits include increased life expectancy (the avoidance of premature mortality or preventable deaths), as well as reduced morbidity related to respiratory and cardiovascular effects. The other regulations addressed a wide variety of acute and chronic conditions as well as various types of injuries, which led to varying degrees to preventable mortality.
The Committee’s review also identified several economically significant
TABLE 2-1 Summary of Recent Major Health and Safety Rulemakings (January 1, 2000–June 30, 2004)
Agency |
Authorizing Statute(s) |
Rulemakings |
EPA: Office of Air and Radiation |
Clean Air Act |
|
EPA: Other offices |
Toxic Substances Control Act; Safe Drinking Water Act |
|
FDA |
Federal Food, Drug, and Cosmetic Act; Public Health Service Act |
|
FSIS |
Federal Meat Inspection Act; Poultry Product Inspection Act |
|
OSHA |
Occupational Safety and Health Act; Construction Safety Act |
|
NHTSA |
Transportation Equity Act for the 21st Century; Transportation Recall Enhancement, Accountability, and Documentation Act |
|
FMCSA |
Interstate Commerce Commission Termination Act |
|
SOURCE: Robinson (2004). |
regulations that the agencies were developing that were likely to include quantified health and safety impacts. The types of health impacts that the agencies expected to assess for these rules are listed in Table 2-2. No other agencies appear to be planning to develop such regulations in the near term.
Judicial Review of Regulatory Analyses2
The use of CEA and BCA to assess the impacts of these types of regulations has been subject to judicial scrutiny. In general, the courts defer to
TABLE 2-2 Health Effects Likely to Be Quantified in Forthcoming Major Health and Safety Rulemakings
Agency |
Potential Quantified Health Impacts |
EPA |
|
FDA |
|
FSIS |
|
OSHA |
|
NHTSA |
|
CPSC |
|
SOURCE: Robinson (2004). |
regulatory agency expertise, but agency decisions must be well reasoned and not “arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with a law” (Administrative Procedure Act, 5 U.S.C. 706(2)(A)). In addition, agency actions must be supported by substantial evidence when the record is viewed as a whole, and agencies must explain the rationale and factual basis underlying their decisions. Within this framework, agencies have a great amount of discretion when constructing economic analyses. The scope of judicial review over agency decisions is narrow, and courts must not substitute their own judgment for that of the agency, particularly in matters requiring technical expertise. Generally, courts defer to the agencies on issues related to the preparation of the regulatory analyses that support their rulemakings.
However, many courts have not hesitated to question the methods and the reasoning agencies used in constructing BCAs or CEAs. In Corrosion Proof Fittings v. EPA, 947 F.2d 1201 (5th Cir. 1991), the court was highly critical of the BCA methodology EPA used to justify a complete regulatory ban of asbestos. The court was “troubled” by EPA’s strategy of discounting future calculated costs while failing to discount future calculated benefits, thus significantly “skewing” the analysis and calling its validity into question. The court also found that EPA violated TSCA by failing to consider
less burdensome alternatives between a total ban and no action, and by failing to assess the risk of substitute products such as vinyl chloride in pipes and nonasbestos automobile brake linings.
Some courts have gone even further in arguing that regulatory activity must take into account the possibility that regulations may improve safety in one area, but reduce it in another. Known conceptually as “risk–risk” analysis, the notion is that regulations imposed to save lives can also have the effect of costing lives through the substitution of less safe products or other changes. A related argument, “health–health” analysis, is that by increasing the costs of production, regulation results in lost jobs and income, pricing some consumers out of the market for safer products (“richer is safer”).3
Take, for example, challenges to fuel economy standards.4 In Competitive Enterprise Institute v. NHTSA, 956 F.2d 321 (D.C. Cir. 1992), a group of national automobile lobbyists petitioned NHTSA to relax the fuel economy standards for model year 1990 cars. NHTSA had the authority to relax the standards but declined to do so, based in part on an agency BCA indicating that the more stringent fuel economy standards produced a total net benefit. The plaintiff filed suit, claiming the agency had failed to assess the impact of additional automobile accident fatalities that were being caused by downsizing cars in response to the stricter standards. The court was critical of NHTSA’s reasoning throughout its rulemaking process, but was most concerned about the agency’s failure to include the additional fatalities in its BCA, stating:
Even if the 27.5 mpg standard for model year 1990 kills “only” several dozen people a year, NHTSA must exercise its discretion; that means conducting a serious analysis of the data and deciding whether the associated fuel savings are worth the lives lost. When the government regulates in a way that prices many of its citizens out of access to large-car safety, it owes them reasonable candor. If it provides that, the affected citizens at least know that the government has faced up to the meaning of its choice. The requirement of reasoned decision-making ensures this result and prevents officials from cowering behind bureaucratic mumbo-jumbo (at p. 27).
Following remand to NHTSA, the agency considered the safety implications of more stringent fuel economy standards. A different three-judge panel upheld NHTSA, ruling that the agency’s action was adequately supported by the record (Competitive Enterprise Institute v. National Highway Traffic Safety Administration, 45 F.3d 481 (D.C. Cir. 1995)).
CURRENT PRACTICES FOR REGULATORY ANALYSIS
In recent years, OMB has issued a series of guidance documents to improve the economic analysis of the types of regulations discussed above. Initially, these documents focused on using BCA to estimate social costs and benefits, noting that analysts should also report the distribution of impacts across subgroups of concern. Until recently OMB paid relatively little attention to CEA, indicating only that it can be useful in certain cases where benefits are difficult to value in monetary terms.
These guidance documents are based on the requirements of Executive Order 12866, as discussed in the first chapter. They provide requirements that OMB expects agencies to follow as well as information on preferred or “best” practices, recognizing that agencies’ ability to implement the recommended approaches may be constrained, for example, by limitations in the available research base. The most recent version of these guidelines was published by the Bush Administration in September 2003 as OMB Circular A-4, Regulatory Analysis, after extensive public comment, interagency review, and independent peer review. The new guidelines became effective in January 2004 for proposed rules and January 2005 for final rules.
OMB Circular A-4 is intended to help analysts define good regulatory analysis as well as to standardize the way benefits and costs are measured and reported. Figure 2-1 illustrates in simplified form the process specified in the Circular, which is included in full in Appendix C.
Although it is similar to earlier guidelines, the Circular provides substantially more detailed information on the criteria for high-quality analysis, imposes certain new requirements, and alters the details of some of the
previous guidance. Chief among the new requirements are those related to the use of CEA. The key analytic requirements of the Circular are summarized in Table 2-3.
These requirements are designed to support decisions regarding the appropriate approach for addressing a particular policy problem. OMB
TABLE 2-3 Key Analytic Requirements of 2003 Office of Management and Budget Guidelines
Requirement |
Guidance |
Type of analysis |
Both BCA and CEA |
Monetary valuation of morbidity |
Prefer estimates of willingness to pay from stated or revealed preference studies plus any additional economic costs of illness, may use health utility studies |
Monetary valuation of mortality |
Agency discretion in selecting value of statistical life estimates, may adjust for income growth or time lag but not age, caution on use of value of statistical life year |
Effectiveness measures for health and safety |
Use integrated measures that combine consideration of morbidity and mortality where appropriate, report more than one measure as well as estimates of physical impacts |
Effects on children and the elderly |
Avoid measures that place lower values on benefits accruing to these subpopulations, apply CEA when children are affected |
Cost estimates |
Include costs and savings related to private-sector compliance, government administration, losses in consumers’ or producers’ surplus, discomfort or inconvenience, or loss of time in work, leisure, commuting, or travel |
Discounting |
Present costs and benefits undiscounted and discounted at both 3 and 7 percent; may consider other rates; intergenerational impacts require special consideration |
Uncertainty analysis |
Discuss qualitatively, present sensitivity analysis, and complete probabilistic analysis as appropriate; probabilistic analysis required if impact is greater than $1 billion annually |
Nonquantified or nonmonetized effects |
Highlight in presentation of impacts |
Distributional impacts |
Quantify impact on different segments of the population when important, including both transfers and total social costs and benefits |
SOURCE: OMB (2003a). |
encourages agencies to consider a wide range of regulatory and nonregulatory options that vary, for example, in terms of stringency or the nature of the requirements (e.g., providing information, creating market-based approaches, or establishing performance standards). In assessing these options, OMB instructs agencies to compare the options to a baseline (sometimes referred to as the “no action” alternative) that represents current and potential future conditions in the absence of the regulation.5 When statutory language limits the consideration of desirable alternatives, OMB requires the agency to discuss these constraints and estimate their impacts.
The following briefly summarizes those aspects of the 2003 guidance that are most relevant to the discussion in this report, and then provides examples of current valuation practices. This section concludes with a summary of cross-cutting OMB requirements that affect both CEA and BCA.
Benefits Analysis Guidelines
OMB devotes a significant portion of the contents of Circular A-4 to the valuation of benefits. It discusses the monetary valuation of morbidity, mortality, and other impacts (e.g., ecological effects) in BCA, as well as approaches for constructing effectiveness measures in CEA. It also provides guidance related to judging the quality of the studies used for valuation.
This guidance on measuring effectiveness in CEA has implications for BCA as well. OMB supports the use of monetized health-adjusted life year (HALY) measures in BCA when suitable willingness-to-pay estimates are not available, and many agencies follow this practice, as discussed in Box 2-1.
OMB highlights three areas of concern related to the application of integrated HALY measures (OMB, 2003a):
-
HALY measures may lead to CEA results that differ from the results of BCA because HALYs must meet certain restrictive assumptions to be a valid representation of individual preferences.
-
HALY measures may affect the perceived fairness of the analytic approach, for example, if lower values are used for life-extending interventions affecting persons with disabilities. Rather than using different estimates of life expectancy or health-related quality of life (HRQL) for different subgroups, OMB recommends that analysts use general population averages.
BOX 2-1 In discussing approaches for assigning dollar values to morbidity impacts in BCA, the OMB notes that if suitable willingness-to-pay studies are not available, agencies may combine HALY studies “typically … based on the standard gamble, the time tradeoff or the rating scale methods … with known monetary values for well-defined health states … to estimate monetary values” (OMB, 2003a, p. 29). The Committee’s review of current practices found that several agencies, most notably the FDA and the NHTSA, use monetized HRQL indexes in some form when valuing nonfatal illnesses and injuries (Robinson, 2004). Although the details of the approaches vary, each agency monetizes the HRQL values based on annualized estimates of the value of statistical life.* In addition, each agency adds the economic costs of illness to these monetized HRQL measures when estimating total benefits, although the types of costs included vary across agencies. This reliance on monetized HRQL estimates appears to stem largely from the shortage of stated or revealed preference studies of willingness to pay that address the health effects of concern to these agencies. |
-
Different HALY measures may yield varying results and provide different perspectives; hence OMB recommends that agencies apply more than one such measure in their analyses. OMB instructs agencies to disclose the underlying data used in their calculations so that OMB and the public can, if desired, recalculate the results using alternate measures and compare the findings across different rulemakings.
Circular A-4 also provides detailed information on the use of stated preference and other methods for valuing benefits, and notes that the criteria for evaluating stated preference studies for use in BCA also apply to the HALY studies used in CEA. Although some of these criteria (listed in Box 2-2) refer specifically to monetization, most are relevant to both types of research.
OMB also discusses the use of benefit transfer; that is, the practice of taking estimates from a prior study and applying them to a rulemaking. While noting that use of such transfers can be expedient, Circular A-4 advises that they should be considered a last-resort option and not used
BOX 2-2 When you are designing or evaluating a stated-preference study, the following principles should be considered:
SOURCE: OMB (2003a, p. 23). |
without explicit justification if the scenario considered in the study differs in significant ways from the regulatory scenario. The Circular describes the steps analysts should follow in conducting such transfers as well as the criteria for selecting studies, and indicates that the biases and uncertainties that result from the transfer should be acknowledged.
Current Valuation Practices
The Committee’s commissioned review of federal agency practices revealed diverse approaches to the valuation of health and safety improvements (Robinson, 2004). While the OMB guidance provides a general framework for assessing the impacts of regulations, it allows the agencies substantial leeway in determining the details of the approach to valuation for both BCA and CEA.
The Committee’s review suggests that regulatory analyses are usually completed under tight time frames with limited staff and budgetary resources. Analysts often must make many decisions about the analytic approach early in the rule development process, when they face substantial uncertainty regarding the regulatory requirements to be assessed, the overall schedule for the rulemaking, and the resources available for completing the analyses. The rulemaking schedule is usually too tight to allow time for new primary research on benefit values; analysts generally rely on preexisting studies, analytic methods, and quantitative models, as exemplified in the rules that were the basis of the Committee’s case studies (see Appendix A).
These available analytic resources in some cases resulted from longer term efforts to develop valuation methods for overall program evaluation, separate from particular rulemaking efforts. For example, NHTSA’s approach is based on its periodic studies of the national costs of motor vehicle accidents (NHTSA, 1996, 2002a), and EPA’s approach has evolved as a result of its prospective and retrospective studies of the Clean Air Act (EPA, 1997, 1999).
Most of the analyses reviewed by the Committee provided a range of measures of regulatory impact, including estimates of the numbers of cases of preventable mortality and nonfatal illness and injuries averted as well as information on uncertainty and nonquantified impacts. All of the analyses included estimates of the costs of regulatory compliance, focusing on those costs expected to be most significant in the context of individual rulemakings. These costs often consisted primarily of direct compliance costs (e.g., the costs associated with implementing pollution controls or administering a new program) and any resulting savings. In some cases, the agencies also assessed the market impacts of price changes associated with the rulemaking; such impacts were generally small.
TABLE 2-4 Quantified Benefits of EPA’s Nonroad Diesel Rule (primary estimate for the year 2030)
Agency analyses of distributional issues tended to focus on the impacts of compliance costs, particularly on small businesses, reflecting the need to comply with related statutory requirements. However, some analyses provided information on the distribution of health impacts, for example, by reporting separate estimates of impacts on children or on individuals with preexisting health conditions.
Examples of the types of health effects examined are provided in Tables 2-4, 2-5, and 2-6. We present these tables to illustrate the ranges of benefits assessed; the source documents cited provide detailed information on the derivation of the estimates and the uncertainty surrounding them, as well as on the importance of nonquantified effects. Additional information on these estimates is available in the Committee’s case studies and review of current practices (Robinson, 2004).
All of these agencies are in the process of developing methods for CEA to implement the new guidelines in Circular A-4. Adapting to the new guidance is relatively straightforward for several agencies; five of the seven agencies studied already use HALY-based measures in some form in their BCAs. In general, these approaches involve either transfers of estimates
TABLE 2-5 Quantified Benefits of Food and Drug Administration’s Juice Processing Rule
from the available literature or the application of generic HRQL indexes using expert judgment.
For example, FDA has traditionally used monetized quality-of-life measures in its BCAs and now reports the results as both costs per quality-adjusted life year (QALY) and net benefits as discussed in Box 2-3. FDA’s approach varies depending on the health effects assessed, and includes transferring QALY weights from an online, open-access database of health-related CEAs (see Box 3-6 for a description), using expert judgment to
TABLE 2-6 Maximum Abbreviated Injury Scale (MAIS) Categories Used in NHTSA Analyses
Injury Severity Category (based on risk to life) |
Examples |
MAIS 1: Minor Injury |
Whiplash, bruise, broken tooth |
MAIS 2: Moderate Injury |
Closed leg fracture, finger crush |
MAIS 3: Serious Injury |
Open leg fracture, amputated arm, major nerve laceration |
MAIS 4: Severe Injury |
Partial spinal cord severance, concussion with neurological signs (unconscious less than 24 hours) |
MAIS 5: Critical Injury |
Complete spinal cord severance, concussion with neurological signs (unconscious more than 24 hours) |
MAIS 6: Immediately Fatal |
N/A |
SOURCE: Examples provided in Miller et al. (1991); the Abbreviated Injury Scale was originally developed by the Association for the Advancement of Automotive Medicine. |
apply existing indexes (e.g., the Quality of Well-Being Scale), and calculating condition-specific QALY weights based on an approach developed by Cutler and Richardson (1997; see also Scharff and Jessup, 2001). Within the Department of Agriculture, the FSIS reports that it is considering approaches similar to those used by FDA.
NHTSA has historically conducted CEA based on estimates of “equivalent lives saved,” which represent the ratio of the dollar value of injuries (including monetized HRQL impacts and economic costs) to the dollar value of fatalities. For example, NHTSA estimates that the total value (economic and quality of life combined) of an injury in the least severe category is $10,396 (excluding noninjury costs). Because $10,396 is 0.31 percent of the per-fatality value ($3.4 million), NHTSA assumes that each injury in this category is equivalent to 0.31 percent of a life saved. NHTSA now uses the values that underlie this approach to calculate net benefits as well as cost-effectiveness ratios. This approach is summarized in Box 2-4.
EPA and OSHA are the two agencies that did not previously use HRQL measures in their BCAs. OSHA’s plans are uncertain, but EPA recently completed a pilot CEA that transfers estimates of HRQL impacts from the existing literature for its Clean Air Interstate Rule (EPA, 2005a, Appendix G), as summarized in Box 2-5. EPA labeled its approach the Morbidity-
BOX 2-3 The FDA has used monetized QALYs in its BCAs for many years, and is now applying the same approaches to assess QALY gains in its CEAs. These approaches are evolving as a result of the agency’s ongoing research and vary from rule to rule depending on the nature of the health impacts. FDA’s juice processing rule, which was the subject of one of the Committee’s case studies, illustrates one of the approaches used. (See Robinson, 2004, for others.) The juice processing analysis is based on a 1998 assessment of the HRQL impacts of mild, moderate, and severe infections associated with exposure to four pathogens as well as resulting cases of reactive arthritis. To determine the per-case value of averting these health effects, FDA completed the following steps:
SOURCES: FDA (1998, 2001). |
Inclusive Life Year (MILY). This approach adds an estimate of the HRQL impacts of nonfatal cases of myocardial infarction and chronic bronchitis to an estimate of the number of life years lost to preventable mortality. These life years lost are not adjusted to reflect the HRQL expected in the absence of air pollution-related mortality.
BOX 2-4 For many years, the NHTSA has reported the results of its regulatory analyses as costs per equivalent lives saved (ELS). This approach converts injuries to “equivalent lives” based on the relative monetary value of different injury categories. These values are calculated for a given year for different injury categories based on data from all motor vehicle crashes that occurred in that year. The fractional ELS values that result are then used in subsequent rulemakings. The most recent estimates were calculated as follows:
|
BOX 2-5 The EPA’s MILY approach sums unadjusted life years gained from averted premature mortality and QALY adjusted gains from averted morbidity. To implement this approach, EPA first searched the literature for estimates of the HRQL impacts of cardiac disease following nonfatal myocardial infarction and of chronic bronchitis. For each health state, EPA then developed a distribution of values that reflected the varying estimates found in the literature. These values were multiplied by a range of estimates of duration, taking life expectancy into account. The resulting range of QALY estimates were then combined with a range of estimates of life years gained to determine the total MILY gains attributable to the rule, using a probabilistic model. To estimate net costs, EPA subtracted two items from the costs of regulatory compliance: the avoided costs of illness (including medical costs and lost earnings |
The “injury only” values are designed for use in assessing interventions that avert injuries, but not the crash itself (e.g., by requiring protective measures such as air bags); the noninjury values are added to the injury values when the analysis addresses interventions that would avert the crash entirely (e.g., by reducing alcohol-related problems). SOURCES: Miller et al. (1991); NHTSA (2002a). |
for chronic bronchitis and myocardial infarction, but not for premature mortality), and the monetized value of those health and nonhealth impacts not captured in the MILY measure. These other benefits are valued based on willingness-to-pay (WTP) estimates to the extent possible; cost-of-illness estimates are used to value certain of the health endpoints for which suitable WTP estimates were not available. In addition to reporting the resulting range of costs per MILY, EPA separately reported each component used to construct the cost-effectiveness measure, including the estimates of life years gained from mortality risk reductions, the estimates of QALY gains for each of the morbidity endpoints, and the sum of these values (i.e., the total MILYs gained). Each of the estimates is presented using both 3 and 7 percent discount rates, and is accompanied by the estimates that bound the 95 confidence interval. SOURCES: Hubbell (2004); EPA (2005a). |
EPA’s approach is equivalent to comparing HRQL with the pollution-related health conditions to perfect health for preventable mortality and to average health for the nonfatal endpoints. EPA notes that
[t]his measure may be preferred to existing QALY aggregation approaches because it does not devalue life extensions in individuals with preexisting illnesses that reduce HRQL. However, the MILY measure is still based on life years and thus still inherently gives more weight to interventions that reduce mortality and morbidity impacts for younger populations with higher remaining life expectancy (EPA, 2005a, p. G-2).
EPA developed this approach in response to OMB Circular A-4 guidance, which suggests that life years lost to preventable mortality should be based on population averages and not adjusted for disabling or other conditions.
Other Guidelines Relevant to CEA
OMB’s 2003 guidelines also discuss the estimation of costs in CEA, and include a number of general provisions that affect the analysis of costs and benefits in both BCA and CEA. Below, we summarize several key requirements that are most relevant to the Committee’s deliberations.
Estimating Costs
In regulatory analysis, OMB notes that the valuation of both costs and benefits should be based on the concept of “opportunity cost,” consistent with the general framework of welfare economics as introduced in Chapter 1. This concept recognizes that, because resources are limited, any decision to use them for one purpose means that they cannot be used for other purposes. Hence the value of a resource can be determined based on the value of its best alternative use.
OMB’s discussion of the application of this concept to the analysis of regulatory costs is relatively brief, although this topic is addressed in detail elsewhere (see especially EPA, 2000a). Circular A-4 directs that the assessment of costs should generally follow the same guidance as the assessment of benefits, and notes that the analysis should address both costs and savings related to private-sector compliance and government administration, as well as changes in consumer or producer surpluses, discomfort or inconvenience, and time spent in work, leisure, commuting, or travel. Changes in technology or innovation that may affect the baseline and the impacts of the regulations over time should also be considered.
For BCA, OMB indicates that countervailing costs and benefits can be included in either the “cost” or “benefit” side of the analysis as long as they are not double counted, because the end result is the calculation of net
benefits (benefits minus costs). For CEA, which involves the calculation of a ratio, the appropriate categorization of benefits and costs requires more attention. OMB advises that both public and private costs should be considered, and that
[t]he numerator in the cost-effectiveness ratio should reflect net costs, defined as the gross cost incurred to comply with the requirements (sometimes called ‘total’ costs) minus any cost savings. You should be careful to avoid double-counting effects in both the numerator and the denominator of the cost-effectiveness ratios (OMB, 2003a, pp. 11–12).
Furthermore, OMB recommends that benefits not included in the effectiveness measure should be subtracted from the cost estimate before calculating the cost-effectiveness ratio if these excluded benefits can be measured in monetary terms. If the value of some of the ancillary benefits cannot be estimated, the analysis should note this so that the cost-effectiveness ratio can be properly interpreted as likely overstating costs relative to benefits.
Discounting Impacts over Time
The OMB guidance requires the use of discounting to reflect the timing of impacts that accrue during different periods or are distributed unevenly over time.6 The Circular indicates that the same rate should be used to discount both costs and benefits, and that benefits should be discounted regardless of whether they are presented in monetary terms or as physical or HALY impacts. OMB requires that agencies present information on the time periods within which the undiscounted impacts are likely to occur.
OMB recommends that agencies estimate the net present value of benefits and costs using both 3 and 7 percent discount rates. These rates reflect the ongoing debate regarding the extent to which the economic impacts of regulations primarily affect investment or consumption. The 7 percent rate represents the opportunity cost of capital, that is, the real (net of inflation) before-tax rate of return on incremental private investment. The 3 percent rate represents the social rate of time preference, sometimes referred to as the consumption rate. The Circular also discusses cases where other discount rates may be considered, and considers several issues related to the appropriate treatment of intergenerational effects.7
6 |
For example, most individuals generally would prefer to receive money today rather than at a later date because they can invest it and earn interest. Discounting involves adjusting numerical values to account for these types of time preferences. |
7 |
See Chapter 4 of this report for the Committee’s discussion of accounting for intergenerational impacts. |
Using Ratios
In addressing the interpretation of the results of economic analyses, OMB notes that ratios can be deceptive and that the net benefits (not the ratio of benefits to costs) is the correct BCA measure to consider in decision making. In CEA, OMB notes that ratios based on averages can be problematic, and instructs analysts to determine the cost-effectiveness of each option incrementally in comparison with the baseline and with each successively more stringent set of requirements.
Assessing Uncertainty and Nonquantified Effects
The OMB guidelines advise that, as appropriate, agencies should discuss qualitatively the main uncertainties in the calculations; use sensitivity analysis to assess the effects of changes in the approach on the resulting estimates; and develop formal probabilistic analyses of uncertainty using simulation models and/or expert judgment. Formal probabilistic analysis is required for all rules with impacts that exceed $1 billion annually.
Because a net benefit or cost-effectiveness estimate may be misleading if important impacts cannot be measured in monetary terms, OMB also emphasizes the importance of providing information on impacts that cannot be quantified or that can be quantified in physical terms but not assigned a monetary value (in BCA) or included in the effectiveness measure (in CEA). Hence analysts are required to clearly specify any nonquantified effects that should be considered in the regulatory decision.
Determining the Distribution of Impacts
In addition to estimating the total national impacts of the regulatory options, agencies are directed to describe distributional effects, that is, to report how benefits and costs affect subpopulations of particular concern. In this assessment, OMB indicates that analysts should consider the allocation both of total social costs and benefits (from the national BCA) and of impacts that represent transfers between different subgroups. The Circular defines distributional effects as the impact across gender, income and racial groups, industrial sectors, and geographic regions, as well as impacts that occur over time or across generations.
Communicating the Methods and Results
OMB emphasizes the need for clear communication of the regulatory options and analytic steps, including information on important assumptions and the sensitivity of the results to these assumptions. The Circular
also discusses standards for information quality, instructing agencies to document that the analysis rests on the best obtainable scientific, technical, and economic information.
SUMMARY
The authorizing statutes for regulatory programs vary in the types of factors that they require agencies to consider. These may include the need to maximize risk reductions, avoid excessive costs, and/or apply the best available technologies. In addition, some statutes require agencies to consider impacts on particular groups of concern, such as children or sensitive populations, as well as to pursue goals other than improving health, such as reducing ecological effects.
The responsibility for implementing these programs is delegated largely to executive branch agencies, which develop many regulations each year. Economically significant health and safety regulations that are subject to OMB’s requirements for economic analysis are a very small proportion of this total. However, these regulations have broad national impacts. For example, the 18 regulations included in the Committee’s review produce roughly $200 billion in net benefits each year and include several individual regulations with national impacts well in excess of $1 billion annually.
The Circular A-4 guidance requiring agencies to begin conducting CEA in addition to BCA is now in force. Agencies have made significant progress in determining how to implement this guidance; however, the dollar resources available for related research are generally limited (Robinson, 2004). In addition, statutory and judicial deadlines, political pressures, and the desire to address health and safety risk in a timely manner often mean that these rules must be developed within a short time frame that does not allow for significant new primary research. As a result, the agencies frequently rely on valuation approaches that do not require a substantial investment of time or funding. These agencies generally transfer estimates from available studies or apply expert judgment rather than conduct new survey research. In some cases, the agency’s approach has resulted from long-term projects that develop new methods or data for valuation.
Review of agency practices indicates that some agencies use monetized HALY measures for valuation in BCA, as permitted under current OMB guidance. Use of these measures in part reflects significant gaps in the WTP literature, which includes relatively few studies that address the health effects of concern in regulatory analysis. Although a detailed review of practices for monetizing HALY measures is outside the scope of the Committee’s charge, it is clear that such approaches mix valuation measures from two differing, and not entirely compatible, frameworks. As discussed in Chapter 1, these approaches are based to varying degrees on the tenets of
utility theory, and represent differing research practices and types of trade-offs.
The magnitude of the impacts of regulations for which economic analyses are required is great. The significance of these public interventions argues for careful attention to the development of high-quality, unbiased analyses that include thorough documentation of their limitations. Such analyses must be rigorous and conform to accepted professional standards for best practices. Data, methods, results, uncertainties, and limitations must be clearly communicated.
The rest of this report elaborates on these objectives for regulatory analysis and policy development. Subsequent chapters consider and make recommendations about the use of HALY measures in regulatory CEA, ethical and other nonquantified information to be considered in developing regulatory policies, and the construction and presentation of CEAs using health-related effectiveness measures. Importantly, the conclusions and recommendations presented throughout address the use of CEA specifically for public policy analysis of interventions affecting the environment, public health, and safety. Different characteristics of measures and criteria for use of a particular measure may be of greater relevance in other contexts than the criteria proposed here.