Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Appendix G Methods and Limitations of Regulatory Assessment United States federal agencies must satisfy specific requirements for analyzing proposed major or economically significant regulatory actions, such as those expected to have an annual effect on the economy of $100 million or more.231, 236 As required under Executive Order 12866 and other executive branch authorities,43, 284 OMB sets out a hierarchy of methods: â¢ benefit cost analysis (BCA), in the ideal, monetizes all the benefits and costs of a proposed regulatory action to arrive at a net balance; â¢ cost-effectiveness analysis monetizes the cost of meeting a policy objective, most typically stated as a numerical target, and can be used when costs are known and important categories of benefit can be quantified, but not monetized; and â¢ break-even or threshold analysis, asks how much value society must place on the benefits or costs of an action for the benefits to outweigh the costs, and can be used when benefits or costs cannot be monetized or quantified. As noted below, DHS has employed break-even analysis in much of its recent analyses of major regulatory actions to address terrorism threats. BCA, cost-effectiveness analysis, and break-even analysis have gained acceptance in state and local policy making and among non-governmental entities, but all three methods have limitations. They do not, for example, address concerns about equity, the distribution of benefits and costs, and the possibility that some members of society might gain while others would not. OMB states, âregulatory analysis should provide a separate description of distributional effects (i.e., how both benefits and costs are distributed among sub-populations of particular concern) so that decision makers can properly consider them along with the effects on economic efficiency.â Moreover, for regulations intended to confer benefits under circumstances of extreme uncertainty, the analytical requirements pose special challenges, not limited to data deficiencies. Guidance on Regulatory Assessment OMB specifies that a good regulatory analysis, regardless of method, should include: â(1) a statement of the need for the proposed action, (2) an examination of alternative approaches, and (3) an evaluation of the benefits and costsâquantitative and qualitativeâof the proposed action and the main PREPUBLICATION COPY: UNCORRECTED PROOFS 159
160 Restricting Access to Explosive Precursor Chemicals alternatives identified by the analysis.â Before recommending federal regulatory action, an agency must demonstrate that the proposed action is necessary. Questions to consider include the following: â¢ Is the action intended to address a significant market failureâa situation in which the market for goods or services, alone, cannot provide a satisfactory outcomeâor to meet some other compelling public need? â¢ Could other types of actionsâfederal or notâbe as, or more, effective than the action? â¢ Could the compelling public need, identified above, be met through state or local regulation, voluntary programs (e.g., programs sponsored by trade associations), public-private partnerships, insurance, or tort liability? OMB further specifies that a prospective evaluation of benefits and costs should â¢ explain how the proposed requirements are linked to the expected benefits; â¢ identify a baseline, typically no action, against which to measure the effects of the proposed regulatory action, and draw comparisons to alternatives; and â¢ identify and capture the expected undesirable side effects and ancillary benefits of the proposed regulatory action and the alternatives. OMB also provides the policy community with context, warnings, and caveats. It explains that the motivation for the analysis is to ââ¦ learn whether the benefits of an action are likely to justify the costs or â¦ discover which of various possible alternatives would be the most cost-effective,â but it notes that efficiency might not be the only or overriding public policy objective. OMB also warns that good regulatory analysis is not strictly formulaic; rather, it requires competent professional judgment, transparency, and well-documented supporting evidence, including a discussion of the sensitivity of the results to embedded assumptions. OMB asks agencies to quantify anticipated benefits and costs to the extent possible, but acknowledges that quantification, particularly monetization, may not be feasible across the board. When it is not possible to monetize benefits or costs, an agency may calculate them in terms of physical units; when calculations of physical units are also impossible, an agency may, instead, proceed qualitatively. In particular, OMB recognizes that, It will not always be possible to express in monetary units all of the important benefits and costsâ¦. If the non-quantified benefits and costs are likely to be important, you should carry out a âthresholdâ analysis to evaluate their significance. Threshold or âbreak-evenâ analysis answers the question, âHow small could the value of the non-quantified benefits be (or how large would the value of the non-quantified costs need to be) before the rule would yield zero net benefits?â None of the methods specified by OMBâBCA, cost-effectiveness analysis, or break-even analysisâinherently addresses OMBâs concerns for either the links between regulatory actions and anticipated benefits or the possibility of unintended consequences and ancillary benefits, nor does it deal directly with matters of equity. The section that follows provides an overview of some of the challenges of assessing control strategies, particularly their benefits. PREPUBLICATION COPY: UNCORRECTED PROOFS
Appendix G 161 Analytical Challenges Assessing the benefits of actions to prevent or reduce losses from terrorist attacks (e.g., estimates of potential reductions in deaths and injuries, property damage, or various social effects) under extremely uncertain conditions poses substantial challenges.236,237, 285,286 Such an analysis would require an explicit characterization of the underlying risk of a terrorist attack and an understanding of the ways in which the regulations might affect that risk, which would, in turn, present analytical obstacles. In addition, a monetary estimate of benefits might involve evaluating societyâs willingness to pay to avoid terrorism- related damages.287,288 Farrow cites specific challenges of interdependencies among defenses or targets; choices among damage- and probability-reducing strategies; the avoidance of unintended consequences in complex systems; ambiguity about the likelihood of events and consequences; the existence of intelligent and adaptive adversaries; and the path-dependent and irreversible nature of some defenses and attack consequences.289 Two challenges stand out in this context: (1) a lack of data both for typifying scenarios, even those with a historical basis, and for projecting the nature and likelihood of future incidents; and (2) an inability to anticipate how terrorists will adapt to changes in the security environment. 236 245, 290,291 Relevant, reliable, and accurate data are scarceâor unavailableâbecause terrorist attacks in the United States and comparable settings are infrequent and highly variable and because sensitive information about attacks requires special handling or protection. Moreover, a change in policy that affects the security environment might lead to displacement; in particular, a terrorist could choose a different method of attack or redirect his or her efforts to other settings.292,293 If, for example, a terrorist finds that is has become too difficult to obtain precursor chemicals to construct IEDs in one location, he or she might choose to work with a vehicle, gun, or knife or operate elsewhere. The scarcity of data compounds the problem of characterizing potential adaptations and outcomes. Some recent regulatory assessments have used break-even analysis to circumvent the challenges of benefits estimation. For example, for the proposed Ammonium Nitrate Security Program, DHS compared the anticipated costs of the program to the costsâto people and propertyâof the bombing of the Murrah Building in Oklahoma City and then calculated the frequency with which similar events must happen to offset the cost of the security program. Members of the policy community have recognized DHSâs adoption of break-even analysis as a step forward in the evolution of analysis of terrorism security policy, but have called for further advancement.237 In a DHS-sponsored workshop held at the RAND Corporation in 2011, the agency, itself, sought to identify novel approaches to valuing benefits and, potentially, to move beyond break-even analysis.236 For example, DHS staff presented a fault tree approach that can be used to map existing security programs and policy mechanisms and estimate event failure probabilities, such as the failure of existing efforts to stop the entry of terrorists into the United States. In developing these models, expert judgment is essential. Quantitative judgmentsâincluding quantitative statements of a modelâs uncertaintyâcan be combined in these risk models to develop distributions (or ranges) of both baseline risks and changes in risks with the adoption of additional programs and policy mechanisms. Cost assessment tends to be less fraught because it presents greater certainty. Generally speaking, regulatory agencies propose specific measures that entail knownâor knowableâadministrative requirements and operational burdens that can be estimated with established accounting and engineering methods. Nevertheless, some elements of cost assessment, such as those that depend on evaluations of PREPUBLICATION COPY: U NCORRECTED PROOFS
162 Restricting Access to Explosive Precursor Chemicals businessesâ or consumersâ behavior or involve unintended consequences, can present uncertainties and pose analytical challenges. PREPUBLICATION COPY: UNCORRECTED PROOFS