National Academies Press: OpenBook

Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals (2018)

Chapter: Appendix G: Methods and Limitations of Regulatory Assessment

« Previous: Appendix F: Training Materials
Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×

Appendix G

Methods and Limitations of Regulatory Assessment

United States federal agencies must satisfy specific requirements for analyzing proposed major or economically significant regulatory actions, such as those expected to have an annual effect on the economy of $100 million or more.231,236 As required under Executive Order 12866 and other executive branch authorities,43,284 OMB sets out a hierarchy of methods:231

  • benefit-cost analysis (BCA), in the ideal, monetizes all the benefits and costs of a proposed regulatory action to arrive at a net balance;
  • cost-effectiveness analysis monetizes the cost of meeting a policy objective, most typically stated as a numerical target, and can be used when costs are known and important categories of benefit can be quantified, but not monetized; and
  • breakeven or threshold analysis asks how much value society must place on the benefits or costs of an action for the benefits to outweigh the costs, and can be used when benefits or costs cannot be monetized or quantified.

As noted below, DHS has employed breakeven analysis in much of its recent analyses of major regulatory actions to address terrorism threats.

BCA, cost-effectiveness analysis, and breakeven analysis have gained acceptance in state and local policy making and among nongovernmental entities, but all three methods have limitations. They do not, for example, address concerns about equity, the distribution of benefits and costs, and the possibility that some members of society might gain while others would not. OMB states, “Regulatory analysis should provide a separate description of distributional effects (i.e., how both benefits and costs are distributed among sub-populations of particular

Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×

concern) so that decision makers can properly consider them along with the effects on economic efficiency.” 231 Moreover, for regulations intended to confer benefits under circumstances of extreme uncertainty, the analytical requirements pose special challenges not limited to data deficiencies.

GUIDANCE ON REGULATORY ASSESSMENT

OMB specifies that a good regulatory analysis, regardless of method, should include “(1) a statement of the need for the proposed action, (2) an examination of alternative approaches, and (3) an evaluation of the benefits and costs—quantitative and qualitative—of the proposed action and the main alternatives identified by the analysis.”231 Before recommending federal regulatory action, an agency must demonstrate that the proposed action is necessary. Questions to consider include the following:

  • Is the action intended to address a significant market failure—a situation in which the market for goods or services, alone, cannot provide a satisfactory outcome—or to meet some other compelling public need?
  • Could other types of actions—federal or not—be as, or more, effective than the action?
  • Could the compelling public need, identified above, be met through state or local regulation, voluntary programs (e.g., programs sponsored by trade associations), public-private partnerships, insurance, or tort liability?

OMB further specifies that a prospective evaluation of benefits and costs should

  • explain how the proposed requirements are linked to the expected benefits;
  • identify a baseline, typically no action, against which to measure the effects of the proposed regulatory action, and draw comparisons to alternatives; and
  • identify and capture the expected undesirable side effects and ancillary benefits of the proposed regulatory action and the alternatives.

OMB also provides the policy community with context, warnings, and caveats. It explains that “the motivation for the analysis is to (1) learn whether the benefits of an action are likely to justify the costs or (2) discover which of various possible alternatives would be the most cost-effective,” 231 but it notes that efficiency might not be the only or overriding public policy objective. OMB also warns that good regulatory analysis is not strictly formulaic; rather, it requires competent professional judgment, transparency, and well-documented supporting evidence, including a discussion of the sensitivity of the results to embedded assumptions. OMB asks agencies to quantify anticipated benefits and costs to the extent possible, but acknowledges that quantification, particularly monetization,

Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×

may not be feasible across the board. When it is not possible to monetize benefits or costs, an agency may calculate them in terms of physical units; when calculations of physical units are also impossible, an agency may, instead, proceed qualitatively. In particular, OMB231 recognizes that,

It will not always be possible to express in monetary units all of the important benefits and costs. . . . If the non-quantified benefits and costs are likely to be important, you should carry out a “threshold” analysis to evaluate their significance. Threshold or “break-even” analysis answers the question, “How small could the value of the non-quantified benefits be (or how large would the value of the non-quantified costs need to be) before the rule would yield zero net benefits?”

None of the methods specified by OMB—BCA, cost-effectiveness analysis, or breakeven analysis—inherently addresses OMB’s concerns for either the links between regulatory actions and anticipated benefits or the possibility of unintended consequences and ancillary benefits, nor do any deal directly with matters of equity. The section that follows provides an overview of some of the challenges of assessing control strategies, particularly their benefits.

ANALYTICAL CHALLENGES

Assessing the benefits of actions to prevent or reduce losses from terrorist attacks (e.g., estimates of potential reductions in deaths and injuries, property damage, or various social effects) under extremely uncertain conditions poses substantial challenges.236,237,285,286 Such an analysis would require an explicit characterization of the underlying risk of a terrorist attack and an understanding of the ways in which the regulations might affect that risk, which would, in turn, present analytical obstacles. In addition, a monetary estimate of benefits might involve evaluating society’s willingness to pay to avoid terrorism-related damages.287,288 Farrow cites specific challenges of interdependencies among defenses or targets; choices among damage- and probability-reducing strategies; the avoidance of unintended consequences in complex systems; ambiguity about the likelihood of events and consequences; the existence of intelligent and adaptive adversaries; and the path-dependent and irreversible nature of some defenses and attack consequences.289

Two challenges stand out in this context: (1) a lack of data both for typifying scenarios, even those with a historical basis, and for projecting the nature and likelihood of future incidents; and (2) an inability to anticipate how terrorists will adapt to changes in the security environment.236,245,290,291 Relevant, reliable, and accurate data are scarce—or unavailable—because terrorist attacks in the United States and comparable settings are infrequent and highly variable and because sensitive information about attacks requires special handling or protection. Moreover, a change in policy that affects the security environment might lead to dis-

Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×

placement; in particular, a terrorist could choose a different method of attack or redirect his or her efforts to other settings.292,293 If, for example, a terrorist finds that is has become too difficult to obtain precursor chemicals to construct IEDs in one location, he or she might choose to work with a vehicle, gun, or knife or to operate elsewhere. The scarcity of data compounds the problem of characterizing potential adaptations and outcomes.

Some recent regulatory assessments have used break-even analysis to circumvent the challenges of benefits estimation. For example, for the proposed Ammonium Nitrate Security Program, DHS compared the anticipated costs of the program to the costs—to people and property—of the bombing of the Murrah Building in Oklahoma City and then calculated the frequency with which similar events must happen to offset the cost of the security program. Members of the policy community have recognized DHS’s adoption of break-even analysis as a step forward in the evolution of analysis of terrorism security policy, but have called for further advancement.237 In a DHS-sponsored workshop held at the RAND Corporation in 2011, the agency itself sought to identify novel approaches to valuing benefits and, potentially, to move beyond break-even analysis.236 For example, DHS staff presented a fault tree approach that can be used to map existing security programs and policy mechanisms and to estimate event failure probabilities, such as the failure of existing efforts to stop the entry of terrorists into the United States. In developing these models, expert judgment is essential. Quantitative judgments—including quantitative statements of a model’s uncertainty—can be combined in these risk models to develop distributions (or ranges) of both baseline risks and changes in risks with the adoption of additional programs and policy mechanisms.

Cost assessment tends to be less fraught because it presents greater certainty. Generally speaking, regulatory agencies propose specific measures that entail known—or knowable—administrative requirements and operational burdens that can be estimated with established accounting and engineering methods. Nevertheless, some elements of cost assessment, such as those that depend on evaluations of businesses’ or consumers’ behavior or involve unintended consequences, can present uncertainties and pose analytical challenges.

Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×
Page 179
Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×
Page 180
Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×
Page 181
Suggested Citation:"Appendix G: Methods and Limitations of Regulatory Assessment." National Academies of Sciences, Engineering, and Medicine. 2018. Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals. Washington, DC: The National Academies Press. doi: 10.17226/24862.
×
Page 182
Next: Appendix H: Examples of Retail-Level Control Strategies and Other Measures or Activities »
Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals Get This Book
×
Buy Paperback | $60.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Improvised explosive devices (IEDs) are a type of unconventional explosive weapon that can be deployed in a variety of ways, and can cause loss of life, injury, and property damage in both military and civilian environments. Terrorists, violent extremists, and criminals often choose IEDs because the ingredients, components, and instructions required to make IEDs are highly accessible. In many cases, precursor chemicals enable this criminal use of IEDs because they are used in the manufacture of homemade explosives (HMEs), which are often used as a component of IEDs.

Many precursor chemicals are frequently used in industrial manufacturing and may be available as commercial products for personal use. Guides for making HMEs and instructions for constructing IEDs are widely available and can be easily found on the internet. Other countries restrict access to precursor chemicals in an effort to reduce the opportunity for HMEs to be used in IEDs. Although IED attacks have been less frequent in the United States than in other countries, IEDs remain a persistent domestic threat. Restricting access to precursor chemicals might contribute to reducing the threat of IED attacks and in turn prevent potentially devastating bombings, save lives, and reduce financial impacts.

Reducing the Threat of Improvised Explosive Device Attacks by Restricting Access to Explosive Precursor Chemicals prioritizes precursor chemicals that can be used to make HMEs and analyzes the movement of those chemicals through United States commercial supply chains and identifies potential vulnerabilities. This report examines current United States and international regulation of the chemicals, and compares the economic, security, and other tradeoffs among potential control strategies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!