National Academies Press: OpenBook
« Previous: 4 Review of Candidate Alternative Technologies
Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.


Evaluation Criteria

This chapter describes the criteria that the committee uses in evaluating alternative technologies that may be used in place of open burning (OB) and open detonation (OD). These criteria establish a means for comparing OB/OD to alternative technologies and allow for a systematic evaluation (see Chapter 8). The ordering of the presentation of the criteria in this chapter does not reflect any prioritization. The committee made no judgment on which are more or less important. The four criteria mentioned in the statement of task—throughput, personnel safety, environmental impacts, and cost—are discussed before additional criteria that the committee adopted.

Four evaluation criteria (throughput capacity, personnel safety, environmental impacts, and cost) were explicitly specified in the legislative language that became the committee’s statement of task (National Defense Authorization Act for Fiscal Year 2017; SEC 1421, 2016). The committee applied five additional criteria: public health impacts, technology maturity, permitability or other approvals, monitorability, and public confidence. The committee has identified these as important criteria that also need to be considered when selecting alternative approaches to OB/OD. In selecting the additional five criteria, the committee considered criteria used in prior reviews of alternatives to OB/OD performed by the Army, the National Research Council (NRC), the Cease Fire! Campaign (see Appendix B), as well as the experience and judgment of the members of this committee.

The actual selection of alternative technologies for deployment would require a detailed engineering analysis based on the design and construction of each munition, which is outside the committee’s scope and ability to conduct within the limits of this study. Further, the committee has chosen not to weight these criteria in terms of importance, because the committee firmly believes that the application of the criteria is munition, technology, site, and community specific. Clearly some criteria, such as personnel safety, need to be addressed rigorously, while others, such as maturity, might be desirable but would probably carry less weight than safety in selecting a specific technology. However, in all cases there needs to be a high level of confidence that the technology employed will demilitarize the munitions, that it can be implemented, and that it will be protective of workers, the nearby community, and the environment.

Munition demilitarization can be achieved via a combination of different technologies for each demilitarization process. Criteria used in selecting a treatment train not only need to focus on how effective a process is for a given munition but also need to take into account the ease with which a single process in such a treatment train can be upgraded or substituted so that the full treatment process can be easily adapted to a different type of munition. Consideration also needs to be given to the inclusion of a new promising technology, either as an integrated whole-munition process or as a process or component within an existing treatment train that might not have matured enough for immediate implementation when the treatment train was originally selected.

The committee’s nine evaluation criteria are discussed in detail below. The committee provides a general narrative on how these criteria are applied to the alternative technologies in Chapter 8.


Throughput is the rate of material that can be processed in a given time and can affect treatment costs. When a technology in and of itself provides treatment for the whole munition, determining throughput may be straightforward. Where treatment of the whole munition is based on a treatment train, including pretreatment, treatment, and post-treatment activities, determining throughput is less precise. It is important to identify any potential impacts on throughput that arise in a specific treatment configuration. Some factors that affect throughput can be mitigated relatively easily by adding more low-cost units or components (e.g., band saws for increased size reduction capability), but in other cases the cost of additional treatment capability (e.g., more or larger detona-

Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.

tion chambers) may significantly increase costs. Ideally, the design of an overall treatment system would allow for the flexibility to process different munitions or to be reconfigured relatively easily for that purpose. Finally, it is important to recognize that whole-munition treatment systems may also have rate-limiting steps (i.e., loading, cooling, etc.) that affect their throughput.


The avoidance and prevention of possible injury or death to workers conducting demilitarization operations is paramount. There are risks to personnel when transporting or otherwise moving an item to the facility or location where it will be processed, and there are risks in any preprocessing step (i.e., size reduction for alternative technologies or positioning munitions for OB/OD), in the treatment processing step, and during post-processing operations. The risks in all of these steps need to be identified, evaluated, and compared when selecting technologies. The safety risks inherent to alternative technologies and many individual steps within a treatment train can be mitigated with fully automated systems that minimize potential injury to workers.

In addition to injuries from catastrophic events, personnel health and safety risks from cumulative and aggregate direct and indirect exposures to potential toxicants associated with munitions and their destruction, including noise, also need to be considered.


A technology’s ability to monitor, prevent, minimize, and control emissions of contaminants to all environmental media (water, air, soil), both during the demilitarization process and during process upsets, will determine its environmental impact. Furthermore, because various treatment technologies and pollution abatement systems result in the accumulation of secondary waste streams, it is important to consider the ability to meet the management and disposal requirements of these streams. Ideally, the treatment method is a complete solution with no long-term storage or disposal requirements for hazardous process waste streams.

Given the nature of the source material being treated, damage to equipment during treatment is possible. Hence, technologies for the demilitarization of conventional munitions need to have safety controls to prevent accidental releases of emissions, due to equipment damage, that have environmental or health concerns.

Other environmental impacts that need to be considered include those on ecosystems, including wildlife, marine, and aquatic receptors, as well as cultural, recreational, and commercial activities depending on such ecosystems.

Finally, the effects of vibration, noise and shock, visual plumes, and odor need to be considered, as they affect nearby communities.


The costs of demilitarization technologies include capital costs, startup costs, operational and maintenance costs, and closure costs. Alternative technologies will have higher capital costs, which include development, design, construction, site preparation, and installation costs. Site preparation may also include the installation of required utilities. In addition to the capital costs, the implementation of any technology needs to account for permitting and regulatory compliance costs, including state, federal, and internal approvals, including those required by the Department of Defense Explosives Safety Board (DDESB).

Operation and maintenance costs comprise the costs of staff and operators including their procedural and safety training, scheduled and emergency maintenance, environmental and health monitoring, and the ultimate disposal of secondary wastes generated during the demilitarization process. Given that some munitions may have to be transported to sites where the appropriate demilitarization technology is available, transportation costs might need to be accounted for. Recycling may generate income streams that can reduce the overall operational cost of a demilitarization activity.

The cost evaluation of any demilitarization technology also needs to include closure costs. Equipment and sites will have to be decontaminated after closure. Given the much larger land area affected by OB/OD operations, and lack of containment, their closure costs are expected to be highest. Closure costs are usually not considered in the cost of demilitarization activities but need to be considered in an overall cost evaluation A life cycle cost (LCC) analysis is required if a true cost comparison of alternative technologies to OB/OD is to be made. According to Department of Defense (DoD) guidance, LCC is defined “as the cost to the government of a program over its full life, including costs for research and development; testing; production; facilities; operations; maintenance; personnel; environmental compliance; and disposal.”1 However, due to the lack of complete information on costs, the committee was not able to conduct an LCC.


The potential impacts of different technologies on public health is an important consideration. Permitted OB/OD and alternative technologies are required by the Resource Conservation and Recovery Act (RCRA) to be protective of human health and the environment.

The potential risks from exposure to all of the substances likely to be emitted during demilitarization of particular munitions (cumulative exposure) need to be considered, as do all significant potential sources of direct exposure and indirect exposure (aggregate exposure). Sources of direct exposure include air, water, and soil that a person might come


1 For LCC definition, see

Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.

into direct contact with, while sources of indirect exposure include, for example, consumption of contaminated species such as fish or hunted game. Because susceptibility to the potential effects of exposure can vary with age and other factors, differences in susceptibility need to be evaluated during any risk assessment of alternative technologies.

Potential impacts on public health arising from all activities of a demilitarization process other than emissions would also need to be considered, such as the shifting of risk burdens among communities caused by transportation of munitions or munition components from one depot to another or to commercial facilities for treatment.


When selecting technologies for conventional munition demilitarization, it is important to assess how far the technology has progressed toward being able to be used in industrial operation and hence the level of confidence that the technology will operate successfully once implemented. The closer a technology or system is to being ready for industrial use, the more mature it is. There are alternative technologies at several stages of maturity, ranging from novel ideas that have not yet been tested at the bench scale to treatment systems that have a proven track record and for which data are available for the effectiveness, throughput, cost, safety, maintenance requirements and downtime, environmental impacts, and reliability of the full-scale process.

A good measure of technology maturity is the number of munitions that have been treated successfully so far with a specific technology. The Army method when estimating technology maturity during the acquisition process is to assign a Technology Readiness Level (TRL) to the technology. TRLs range from 1 to 9, with 9 being the most mature technology. A TRL of 6 indicates a “system/subsystem model or prototype demonstration in a relevant environment,” the lowest TRL considered by the committee (DoD, 2011). In assessing whether a technology has reached a certain maturity level, and when comparing the maturity of different technologies, one also needs to consider whether that specific technology is in operation under existing permits or other approvals. A history of successful permitting or approval may provide some confidence that the emplacement and operation of the technology at a different site will be successful.

Data from a demilitarization technology with a proven track record may also enable a more accurate cost estimate as well as an estimate of the throughput rate when processing a given type of munition.


Permitability is the ability, or expected ease, to permit an alternative technology under applicable federal, state, tribal, and local laws and regulations. This criterion also includes other approvals, as not all technologies will require permits. There are also instances where alternative technologies may be exempt from RCRA permitting based on the RCRA regulations applicable to a given treatment process (e.g., recycling or certain RCRA exemptions). However, a declaration of exemption from RCRA can also require an application and approval, and the facility may need other permits (e.g., Clean Air Act [CAA]).

The most significant law for the demilitarization of conventional munitions is RCRA (see Chapter 6). The RCRA regulations applicable to an alternative technology depend on the type of technology used to treat the conventional munition or munition component. RCRA regulations focus on protecting human health and the environment based on risks from exposures to contaminants in the air, surface and groundwater, or soil. In addition, there may be other permits required when permitting an alternative technology unit, including the CAA, the Clean Water Act, water use laws, or other cultural and natural resources laws.

Several factors can impact permitability or other approvals. First, as a practical matter, an alternative treatment technology is more likely to be permitted or otherwise approved in a more efficient manner if that technology has been permitted or approved at similar locations and for the treatment of the same or similar wastes. Similarly, the more mature the munition access, treatment, and pollution control technology, the more data are available and, again, the more likely the process will be permitted or otherwise approved in a more efficient manner.

Also, public opposition can impact permitability or approvals by lengthening the permitting process or by asking regulators to impose operational conditions, health risk studies, and other conditions on OB/OD or the alternative technology permits.


It is important that any treatment method can be monitored accurately for both emissions and completeness of treatment. In terms of monitoring emissions, treatment methods need to allow for the effective monitoring of pollutants of concern in all waste streams to determine, for example, if air pollution abatement systems are performing as designed, or if liquid or solid secondary wastes need specific treatment before their disposal. Furthermore, monitoring of surrounding soil, water, and air for pollutants that might be emitted during demilitarization activities may be necessary based on permit or other requirements. Monitoring of soil and groundwater can support rapid corrective action and significantly reduce closure costs. Monitoring for ambient air pollutants (associated with the demilitarization technology) is key to ascertain whether the demilitarization activity could affect public health.

Although costly and a negative impact on throughput, some technologies allow for holding a waste stream for monitoring or testing before it is released. This can be done

Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.

for solid, liquid, and gas streams. However, hold, test, and release is usually applied to treatment of chemical weapons due to the toxicity of the chemical warfare agents.

Last, any demilitarization technology would ideally allow for a post-treatment inspection and testing of the munition or treated residue to determine that explosives and or propellants have been sufficiently treated, to allow for disposal or recycling.


Experience has shown that building public confidence and trust via public consultation and participation in the technology selection process is instrumental (it will promote legitimacy); substantive (it will lead to better decisions); and normative (it is the right thing to do in our society; see Fiorino, 1989, and NRC, 2008). A lack of public confidence can impact schedules, cost, and ultimately, the implementation of technologies. Also, as noted earlier, public opposition can cause delays in the permitting process and influence the permit conditions imposed on alternative technology implementations.

Public confidence in a technology and related program management can be linked to core questions, such as “how safe is safe enough,” and cannot solely be answered by science nor technology; they are questions of human values (NRC, 1996). These values, along with social trust, are key factors shaping both management and public perceptions of technologies, including (of relevance to this study) the selection and prioritization of criteria that are used to evaluate them. For example, if the public conceives that some variables are weighted more heavily in the decision-making process than those that are a priority for them (e.g., public health or environmental impacts), public confidence could be undermined.

To evaluate public confidence in making decisions on alternative technologies, the committee considered the following factors:

  • The characteristics of a technology and associated risks (e.g., the potential for catastrophic releases, familiarity of technology and risks, types of secondary wastes generated, distribution of risks and benefits within and among communities);
  • The management of the technology (e.g., information is publicly available about how the technology and its pollution abatement system work, monitoring data are immediately available and accessible); and
  • The processes for making decisions (e.g., are they viewed as being fair, transparent, based on accepted and appropriate criteria).


DoD (Department of Defense). 2011. Technology Readiness Assessment (TRA) Guidance.

Fiorino, D.J. 1989. Technical and democratic values in risk analysis. Risk Analysis 9(31):293-299.

NRC (National Research Council). 1996. Understanding Risk: Informing Decisions in a Democratic Society. Washington, D.C.: National Academy Press.

NRC. 2008. Public Participation in Environmental Assessment and Decision Making. Washington, D.C.: The National Academies Press.

SEC. 1421. 2016. Alternative Technologies for Munitions Disposal, National Defense Authorization Act for Fiscal Year 2017. 2000; Public Law 114-328.

Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.
Page 67
Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.
Page 68
Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.
Page 69
Suggested Citation:"5 Evaluation Criteria." National Academies of Sciences, Engineering, and Medicine. 2019. Alternatives for the Demilitarization of Conventional Munitions. Washington, DC: The National Academies Press. doi: 10.17226/25140.
Page 70
Next: 6 Regulatory Requirements Applicable to Open Burning, Open Detonation, and Alternative Technologies »
Alternatives for the Demilitarization of Conventional Munitions Get This Book
Buy Paperback | $75.00 Buy Ebook | $59.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. military has a stockpile of approximately 400,000 tons of excess, obsolete, or unserviceable munitions. About 60,000 tons are added to the stockpile each year. Munitions include projectiles, bombs, rockets, landmines, and missiles. Open burning/open detonation (OB/OD) of these munitions has been a common disposal practice for decades, although it has decreased significantly since 2011.

OB/OD is relatively quick, procedurally straightforward, and inexpensive. However, the downside of OB and OD is that they release contaminants from the operation directly into the environment. Over time, a number of technology alternatives to OB/OD have become available and more are in research and development. Alternative technologies generally involve some type of contained destruction of the energetic materials, including contained burning or contained detonation as well as contained methods that forego combustion or detonation.

Alternatives for the Demilitarization of Conventional Munitions reviews the current conventional munitions demilitarization stockpile and analyzes existing and emerging disposal, treatment, and reuse technologies. This report identifies and evaluates any barriers to full-scale deployment of alternatives to OB/OD or non-closed loop incineration/combustion, and provides recommendations to overcome such barriers.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook,'s online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!