National Academies Press: OpenBook

Carbon Filtration for Reducing Emissions from Chemical Agent Incineration (1999)

Chapter: 5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process

« Previous: 4 Facility Design with a Carbon Filter System
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×

5
Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process

BACKGROUND

Carbon filters were first considered for the baseline incineration system in 1991, when the observation was made during an NRC workshop that "the use of redundant air pollution control could substantially enhance public confidence..." (NRC, 1991). In a 1992 report, the NRC recommended that "the Army should consider incorporating passive controls, such as activated carbon beds..." (NRC, 1992). In 1994, the Stock-pile Committee revisited the issue of carbon filters and made the following recommendation (NRC, 1994):

Recommendation 13. The application of activated charcoal filter beds to the discharge from baseline system incinerators should be evaluated in detail, including estimations of the magnitude and consequences of upsets, and site-specific estimates of benefits and risks. If warranted, in terms of site-specific advantages, such equipment should be installed.

Anticipating that carbon filters would offer a net benefit by reducing public risk or, at least, enhancing public confidence, the Army decided in 1994 to add carbon filters to the baseline incineration designs for facilities at Anniston, Alabama; Umatilla, Oregon; and Pine Bluff, Arkansas (U.S. Army, 1994d). JACADS was already built and operating in 1994, and the Army concluded that it would be extremely difficult to retrofit the facility without causing extensive delays in the destruction of the stockpile; the decision on whether to add carbon filters at the Tooele facility, which was then under construction, hinged on the results of a planned risk assessment of the Tooele PAS filter system (PFS).

In response to the NRC recommendation that the Army evaluate the effectiveness of adding the PFS, the Army issued contracts for analysis of the risks and benefits to the design and permitting of baseline incineration system facilities at Anniston, Umatilla, and Pine Bluff. Summary assessments of the risks associated with carbon filters at Anniston (U.S. Army, 1998a) and Umatilla (U.S. Army, 1998b) were provided to the Stockpile Committee; the report for the Pine Bluff facility was not available for review.

Focus of the Stockpile Committee's Review

The committee's review of the Army's risk-based decision making focused on the following issues:

  • the Army's risk evaluations of the PFS at Anniston and Umatilla

  • the Army's application of the CMP and public involvement in the PFS evaluation

  • the Army's decision to retain the PFS in the designs for Anniston and Umatilla and to continue operating Tooele without a PFS

  • what the Army should do before implementing its decision

Although no change is being proposed in the planned designs at Anniston and Umatilla, the Army decided to use its CMP to present the results of its analysis to the public (U.S. Army, 1998d). The Army's use of the CMP, as it relates to the PAS, is discussed below.

Documents Reviewed

The committee's evaluation is based on the following reports and documents:

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
  • Risk Assessment of the Pollution Abatement Filter System for the Anniston Chemical Agent Disposal Facility (ANCDF) (U.S. Army, 1998a)

  • Draft Risk Assessment of the Pollution Abatement Filter System for the Umatilla Chemical Agent Disposal Facility (UMCDF) (U.S. Army, 1998b)

  • Letter Report from the Program Manager for Chemical DemilitarizationImplementing the Change Management Process: Validation of the Anniston Chemical Agent Disposal Facility Pollution Abatement Filter System Implementation (U.S. Army, 1998d)

  • Evaluation of the Pollution Abatement Filter System for Chemical Agent Disposal Facilities: Methodology for Evaluating Risks (U.S. Army, 1998e)

  • PMCSD Change Management Process Plan (U.S. Army, 1998f)

  • Letter Report from the Program Manager for Chemical DemilitarizationUmatilla Chemical Agent Disposal Facility Pollution Abatement Filter System Summary of Risk Assessment Results (U.S. Army, 1998g)

  • The Change Management Process to Accompany the Guide to Risk Management Policy and Activities (May 1998) (U.S. Army, 1998h)

  • Public Involvement Plan for the Chemical Stock-pile Disposal Program Change Management Process (U.S. Army, 1998i)

  • Anniston Chemical Agent Disposal Facility Phase I Quantitative Risk Assessment (U.S. Army, 1997c)

  • Umatilla Chemical Agent Disposal Facility Phase 1 Quantitative Risk Assessment (U.S. Army, 1996b)

  • Tooele Chemical Agent Disposal Facility Quantitative Risk Assessment (U.S. Army, 1996a)

  • Review of Acute Human-Toxicity Estimates for Selected Chemical-Warfare Agents (NRC, 1997b)

  • PAS Filter System (PFS) Analysis (Hopkins, 1998a)

  • Bringing the PAS Filter Evaluation to Closure (Hopkins, 1998b)

RISK ASSESSMENTS FOR ANNISTON AND UMATILLA

The PFS (carbon filter system) risk assessments carried out for Anniston and Umatilla are summary-level extrapolations of (1) the results of the site-specific HRAs performed to meet regulatory requirements and (2) the results of the Phase 1 QRAs for off-site consequences of processing accidents involving agent. The assessments include risks to on-site workers, nearby residents, and the surrounding environment for acute and chronic effects of exposure, with and without the use of the carbon filters. Effects on facility construction schedules and costs are also addressed, as well as the risks of ongoing storage. The Army's decisions about PFSs are based on this information.

The Army used the site-specific, multipathway HRAs done to meet state regulatory requirements as a starting point for its PFS analysis. The HRA evaluation performed by the Army was a "bounding" analysis of the effects of adding or removing the PFS at Anniston and Umatilla. This assessment was based on an assumption of severe conditions or minimum effectiveness of the PFS to estimate the maximum risk.

The effects of the PFS on the HRA were evaluated by comparing results for alternative designs. The only effects considered were reductions in the flow rate and temperature of the stack gas from changes in operating conditions with the PFS. These analyses did not assume the removal of SOPCs from the flue gas by the PFS but examined the effects of the PFS that exhausts gas flow rate at 160°F and 39 percent relative humidity compared to a facility without a PFS that exhausts gas at 190°F and 100 percent relative humidity. Reducing the flow rate and temperature as a consequence of adding the PFS would reduce the effective height of the stack and, therefore, reduce the atmospheric dispersion of emissions. Therefore, the Army analyzed plume dispersion to assess how the changes would affect concentrations of contaminants or health risks.

The Army considered this analysis to be a "worst case" or "bounding" calculation to ensure that the addition of the PFS would not increase public health risks. For example, the concentrations of SOPCs used in the analysis were the highest value recorded for each measured substance in the JACADS trial burns. The detailed analysis of the trial-burn data provided in Chapter 2 shows that these estimated values are much higher than the anticipated actual values. However, the Army did not conduct comparative HRAs assuming that the PFS would reduce emitted concentrations to either the analytical detection limits or to levels below analytical detection limits based on estimates of carbon filter removal efficiency. Nor did the Army quantify reductions in the emissions of acid gases or metals.

Because no SOPC removal was assumed to occur in

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×

the PFS—an assumption that is conservative but contrary to field experience and engineering calculations (see Chapters 3 and 4)—potential reductions in public risk were not evaluated in an HRA. The Army did conduct a sensitivity analysis of HRA results based on the detection limits. However, it is difficult to quantify the potential public health risk benefits without a more complete HRA evaluation for comparison.

Risks to the environment and human health could have been determined by assuming that the PFS would reduce emissions of semivolatile organics, volatile metals, particulates and their associated SOPCs, as well as acid gases, to either their detection limits as demonstrated by tests conducted at carbon filter-equipped hazardous waste incinerators (Hartenstein, 1992) or to the concentrations indicated by engineering models (Mitretek Systems, 1997). The results of the HRA evaluations were augmented by the site-specific Phase 1 QRAs, which evaluated the off-site consequences of agent-related accidents. The Phase 2 QRA, which addresses worker risk associated with agent processing at the TOCDF, was used to provide insight into possible accident scenarios at Anniston and Umatilla, which are expected to have similar designs and operating practices. The QRA analysis carried out using the Phase 2 QRA from the TOCDF identified blockage of the exhaust gas flow by the PFS, coupled with loss of the induced draft (which maintains the pressure drop for the exhaust-gas flow), as the only upset condition that would result in increased risk from a release of agent caused by the PFS.

The frequency of two types of accident scenarios would substantially increase as a consequence of potential PFS blockage and loss of induced draft: (1) an agent-vapor explosion in the MPF if agent vapors accumulated, and (2) a natural-gas explosion in the MPF if the plant was shut down but natural gas continued to flow to the burner. The magnitude of the worker risk from a natural-gas explosion leading to an agent release was much less than the risk caused by the potential agent-vapor explosion. Both scenarios had negligible impact on risk to the public (≤ 1 × 10-12 individual risk of fatality or cancer over the facility life). However, the potential for an agent-vapor explosion caused by PFS blockage represents 3 percent of the total worker risk. In addition, the evaluation of risk from a potential agent-vapor explosion did not consider scenarios of poorly drained munitions being processed, which could significantly increase the amount of agent in the MPF.

The Phase 1 QRAs indicate that the PFS would not be associated with any accident scenario likely to expose off-site receptors to agent. However, individual worker fatality risk from agent due to upsets in the PAS are estimated at 3.3 × 10-5 with the PFS and 1.1 × 10-5 without the PFS over the life of the facility. This is in contrast to total worker risk from agent of 4.1 × 10-4 as estimated for the TOCDF. These findings also can be compared with the worker accidental death rates of 3 × 10-5per year for manufacturing industries and 1.5 × 10-4per year for construction industries during 1996 (National Safety Council, 1997). The risk from the PFS is within the range of the uncertainty for worker risk analysis at the facility but significant enough to warrant further evaluation.

Although mitigation measures have not yet been identified, they will be investigated as part of the Phase 2 QRAs and HEs (job safety hazard evaluation reviews) for Anniston and Umatilla. The increased risk might be mitigated through improved maintenance procedures and heightened operator awareness. In any case, the presence of the PFS is not expected to reduce worker risk. The Phase 2 QRAs and HEs for Anniston and Umatilla are scheduled to be carried out once facility designs have been finalized and operating procedures developed.

The Army's QRA deals only with the risk of agent release caused by accidents and the consequent effects of agent on the public and on workers. It is not intended to, and does not, take into account the usual industrial risks of building, operating, maintaining, and supplying a large industrial plant. (An example of increased worker risk as a consequence of increased nonagentrelated maintenance activities is the need for frequent replacement of the HEPA filters in the PFS, estimated to be required after every 100 hours of operation.) These are important management concerns, however, and they are given considerable attention in the HEs (U.S. Army, 1997a).

Based on the combined results of the HRA and QRA evaluations, the Army concluded that even a PFS with a zero removal efficiency would have off-site impacts and risks acceptable for issuing a RCRA permit. From these analyses, the Army concluded that the best course of action was to make "no change" in the current configurations (i.e., retaining the PFS at Anniston, Umatilla, and Pine Bluff but not adding a PFS to Tooele). This decision was based on the following rationale:

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
  • Delays in processing caused by a permit change would increase public risk from continued storage.

  • Delays would probably increase the overall risk to workers by increasing storage risk.

  • The reduction in public risk from processing emissions (that are already below regulatory levels) by the addition of a PFS at Tooele would be more than offset by the increased risk due to delay.

The Army's decision was made in the context that the original intent of the PFS was to increase public confidence through the presence of a redundant air pollution control system. However, the Army's analyses suggested that changes to risk would be small, that these changes could represent an improvement or a degradation depending on the uncertainties in the analysis, and that risk to the public and workers might differ. The Army concluded that its bounding analyses provided sufficient information for making a reasoned management decision and that more refined analyses were unnecessary.

The Stockpile Committee agrees that properly performed screening level analyses may demonstrate that an emission is far enough below the thresholds of regulatory concern and that more refined analyses are not needed in this case. The EPA often uses the same rationale to balance environmental protection and costs. However, the committee believes that the piecemeal approach taken in the Anniston and Umatilla PFS HRAs and the use of the TOCDF QRA as a surrogate are neither the simplest nor the clearest way to support risk management conclusions. The Army did not provide the committee with an integrated analysis that clearly indicates the environmental effects, the public health benefits, or the worker safety implications of the PFS. Nor did they provide quantification (or even clear identification) of the uncertainties associated with the analyses.

THE ANNISTON PFS RISK EVALUATIONS

The following results are from the Anniston summary PFS risk assessment (U.S. Army, 1998a):

  1. For all receptors (subsistence farmer, three subsistence fishers, adult resident, and child resident), a system configuration for the ANCDF without the PFS achieves the state-approved health risk thresholds. For example, the estimated excess cancer risk is 4E-06 for the subsistence farmer (i.e., an additional chance of about 4 in 1 million that an individual will contract cancer during his or her lifetime) and 3E06 for the adult resident. With the PFS, the estimated excess cancer risk for the subsistence farmer is 3E-06; for the adult resident it is 2E-06. For the ANCDF, the corresponding state-approved threshold for excess cancer risk is 1E-05 (an additional 10 in 1 million chance of contracting cancer).

  2. A sensitivity analysis shows that the calculated excess cancer risk values using the state-approved methodology are overly conservative. When certain parameter values are adjusted to more accurately reflect program factors or conditions, there is insignificant difference between the risk calculated for a facility ''with PFS'' and one "without PFS." The difference in risk values for the 2 facilities is on the order of 3E08.

  3. The PFS is not expected to affect the potential for PAS emissions to negatively impact ecological communities. Removal of the PFS will not change the screening level ERA [environmental risk assessment] findings that there is little potential for the SOPCs to impact vegetation, soil invertebrates, or aquatic and benthic species in the Anniston depot area.

  4. The PFS does not reduce the risk from accidents related to agent stack release. The QRA results show that the PFS is relatively risk neutral. While the PFS could reduce the potential for agent release from the stack, the PFS has no net effect on the overall individual or societal risk from stockpile disposal activities because the risk is dominated by external events—such as aircraft crashes—which are unaffected by the PFS. Furthermore, while there is an increase in the risk for worker fatality associated with the operation and maintenance of the PFS, the magnitude of the increased risk is relatively small (i.e., 2.8E-05 risk increase over a baseline worker risk of 2.2E-04).

  5. Any delay in the disposal schedule will result in an increase in the munition storage period and therefore an increase in risk. Facility operation with the filters is expected to result in a small delay due to additional downtime associated with filter maintenance activities. However, this is insignificant when compared to: 1) the potential delay associated with a permit change to remove the filters; or 2) the shutdown that would accompany a potential stack release on a design without filters.

  6. The hazard evaluation of solid waste generated from the PFS shows that there is some additional risk associated with solid waste disposal, but its overall impact on public and worker risk is insignificant. The PFS increases the amount of solid waste generated for the baseline system by about 0.3 percent. Hence, removal of the PFS has no significant effect on the risk from solid waste disposal.

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
  1. The total cost of implementing the PFS at the ANCDF is in the range of $58 million to $68 million; however, the cost of removing the PFS from the ANCDF could range from about $52 million to $102 million. The net effect is a cost saving of as much as $16 million ($68M-$52M) or an increase in cost of as much as $44 million ($102M-$58M). The largest uncertainty is in how long the destruction schedule could be delayed as a result of obtaining a RCRA permit modification. The increase in stockpile storage cost for destruction schedule delay is the main cost driver.

The draft Umatilla summary PFS risk assessment presents similar conclusions (U.S. Army, 1998b). Both assessments concluded that residents living near facilities with carbon filters would face essentially the same risks as residents living near facilities without carbon filters.

In neither the Anniston nor Umatilla summary PFS risk assessments has the Army made a clear presentation of the risk of adding or removing the PFS. The assessments were generated for the Army's internal use, and, therefore, the Army considered the audience for its PFS reports to be only Army management because its decision was to make "no change" (Hopkins, 1998a, 1998b). For a wider audience, the Army would have to explain the impact of the PFS on total risk at Anniston, which would require combining the results of the PFS risk assessment with information from the Anniston Phase 1 QRA, the Anniston HRA, the TOCDF Phase 2 QRA, the TOCDF QRA for the PFS, and the TOCDF HE for the PFS. The committee believes that the lack of a single document containing clear, graphical, and quantitative answers to basic questions about PFS risks is likely to interfere with effective decision making and is a crucial lapse in terms of enabling the public to follow the process or influence the results.

The Anniston Letter Report

The Army prepared a draft Anniston Letter Report (U.S. Army, 1998d) summarizing the results of the PFS risk assessment (U.S. Army, 1998a) to communicate the risks and benefits of carbon filters at Anniston to the public (U.S. Army, 1998f). The draft Anniston Letter Report shows that the Army has attempted to present the basis for its decision by explaining the difficulties in reconciling differing values, concerns, and perceptions in language more accessible to the public. The Anniston Letter Report also explains that the PMCD is ultimately responsible for the decision and outlines the process by which the PMCD intends to solicit opinions and comments from interested stakeholders.

The key findings in the Anniston summary PFS risk assessment (U.S. Army, 1998a) and the Anniston Letter Report (U.S. Army, 1998d) differ somewhat in content and presentation. The findings from the letter are reproduced verbatim in Table 5-1 (U.S. Army, 1998a, 1998d). The findings from the analogous Umatilla reports are essentially the same as those for Anniston (U.S. Army, 1998b, 1998g). The reports for the Pine Bluff facility were not available at the time this report was written.

The Anniston Letter Report cites the possibility of delays caused by permit modifications to remove the PFS and the consequent increase in storage-related risk as the main reasons for leaving the carbon filters in the current design at Anniston. However, the magnitude of the increased storage risk is dependent on the length of the delays. The increased risk from prolonged stock-pile storage has been estimated on a per year of storage basis (U.S. Army, 1998a). For the population 2 to 5 km from the Anniston Chemical Agent Disposal Facility, the individual public fatality risk is 1.4 × 10-5 per year; the societal public fatality risk is 2.6 × 10-2 per year. By contrast, the disposal processing risks for the same population is 3.8 × 10-8 per year (individual public fatality risk) and 1.8 × 10-5 (societal public fatality risk). Thus, the per year risk from storage is at least three orders of magnitude greater than the risk from disposal processing. Consequently, very short delays would increase public risks more than the total public risk from disposal. A delay of approximately one year would result in increased individual public risks of the same order of magnitude as estimated increases in individual worker fatality risk over the entire period of disposal processing.

CHANGE MANAGEMENT PROCESS

The PFS risk assessment reports state that the evaluations were conducted in accordance with the CMP (change management process), which is the Army's recently developed and evolving process for ensuring that community views are considered in risk management decisions (U.S. Army, 1998f). According to the CMP, the Army will include a public involvement step if a proposed change meets one or more of the following criteria:

  • significant physical modifications to furnaces or the PAS

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×

TABLE 5-1 Effect of the Carbon Filter System on Risk at the Anniston Facility

 

Area of Concern

Summary of Conclusions

1.

Public Health Risk from Emissions

The disposal facility emissions are low enough to meet regulatory approval even without the PFS.

2.

Public Risk from Accidental Agent Release

The total public risk of accidental agent release would be increased from any extension of stockpile storage. Delays associated with changing the current plan to operate ANCDF [Anniston Chemical Agent Disposal Facility] with carbon filters outweigh any delays caused by the presence of the filters.

 

 

The public fatality risk due to accidents during disposal is unchanged by the presence of the PFS.

3.

Worker Risk

There likely would be a small increase in worker risk if the carbon filters are included, but the risk likely could be controlled to be minimal and within acceptable limits.

4.

Environmental Risk

The PFS has no impact on the screening level ecological risk assessment.

5.

Schedule Risk

Overall, it is estimated that there could be a substantial delay if there is an attempt to change the current plan to build and operate ANCDF with the filters.

6.

Cost Risk

The PFS will cost over 60 million dollars in capital and operating costs. However, the removal of the PFS at this point could cost between 50 and 100 million dollars due to changes in disposal schedule, facility design and permit modifications.

 

Source: U.S. Army, 1998d.

  • increase in stack-emission limits specified by the permits

  • significant modifications to agent and explosive material handling operations within the two most external engineering control boundaries

  • significant changes in or to major process technologies, such as incineration, pollution abatement, or explosives handling

  • other physical or operational modifications determined to carry significant potential for affecting the results of the QRA or HRA

Because removing or adding carbon filters would meet several of these criteria, the Army had intended the assessment of carbon filters to be the first use of the CMP to evaluate a potential major design change. The site-specific PFS risk assessments were prepared for the public-participation phase of the CMP, when the results of each risk assessment would be openly reviewed and discussed with the facility stakeholders. Because attendance at the public meetings held to discuss the results of the PFS risk assessments was extremely poor, local stakeholders and permitting authorities are likely to be under the impression that the PFS would reduce risk. In fact, neither the potential reductions in risk to the public nor the potential increases in risk to workers from the PFS have been well characterized, and the broad uncertainties associated with the estimates of risk have not been presented.

Because one of the primary reasons for adding carbon filters was "enhancing public confidence" in baseline system technology (the other being the capture of brief increases in emissions, or puffs), the committee believes the Army should quantify and verify the risk estimates and share them with affected stakeholders. The Army should include information on the change in the composition of the flue gas with the PFS in its presentation.

The committee has consistently recommended that the Army increase its efforts at obtaining meaningful public involvement in important decisions (NRC, 1994, 1996, 1997a). In addition, in the Risk Assessment and Management report, the committee included several recommendations for integrating public involvement and emergency management into the Army's draft Guide and risk management plan (NRC, 1997a; U.S. Army, 1997a). For example, the committee made the following recommendations:

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×

Recommendation 5. The Army should develop a management plan (and include it in the Guide) that defines the integration of management roles, responsibilities, and communications across activities by risk management functions (e.g., operations safety, environmental protection, emergency preparedness, and public outreach).

Recommendation 6. The Army should review and expand the current draft risk management plan to include public involvement in appropriate areas beyond the management of the change [CMP] process.

Recommendation 7. The Army should institutionalize the management of change process developed in the Guide. The Army should track performance of the change process and document public involvement and public responses to decisions. The Army should use the experience to improve the change process.

Recommendation 8. The Army should expand implementation of the risk management program to ensure that workers understand the results of risk assessments and risk management decisions.

Significant portions of the PFS evaluation were completed before the CMP was fully developed. Nevertheless, the committee feels compelled to comment on the Army's progress in developing and implementing the CMP. In the Risk Assessment and Management report, the committee noted that the CMP was an attempt to break new ground, and the draft Guide, which was not yet completed, was still missing sections that were crucial to the Army's management of change, including a means of distinguishing matters of risk assessment from matters of policy (value judgments) and an approach to integrating them that would involve the public (NRC, 1997a). The committee encouraged the Army to complete the draft Guide, especially Chapter 7 on public involvement, so that it could become policy. The committee has continued to monitor the Army's efforts to complete the draft Guide and finalize the CMP as it relates to the issue of the PFS.

The first attempt to make the draft CMP operational and to obtain public input took place in July 1997 in Anniston, Tooele, and Umatilla. At the time, the proposed CMP involved only RCRA Class 2 and 3 permit changes.1 Prior to these meetings, neither the office personnel for local public outreach nor the public was involved in the development of the CMP (Campbell, 1998). All of the public meetings on the CMP were very poorly attended. Nevertheless, in part on the basis of these meetings, the Army and its contractor, Booz, Allen, and Hamilton, began the process of reformulating the CMP; several subsequent drafts were developed, and a document labeled "Final" was published in May 1998 (U.S. Army, 1998h). After several more drafts, the PMCSD [Project Manager for Chemical Stockpile Disposal] Change Management Process Plan was approved and published in final form in November 1998 (U.S. Army, 1998f).

The Army has been ambivalent about using the PFS evaluation to exercise the public involvement portion of the CMP. According to the PMCSD Change Management Process Plan, the CMP was supposed to ensure that "facility or operational changes with the potential for significant risk [are] publicly reviewed before PMCD [the Army] decides whether the change should be made" [italic emphasis added] (U.S. Army, 1998f). The Army's ambivalence about involving the public in the PFS decision was partly a reflection of the Army's belief that keeping the PFS would not entail significant risk (Evans and Hopkins, 1998). Nevertheless, the release of the Army's Anniston and Umatilla Letter Reports summarizing the risk assessment results for the PFS took place in September and October, respectively, and was closely followed by attempts to hold "public outreach availability sessions" (Hopkins, 1998c).

The Army had already made its position clear with regard to the PFS in the letter reports, and the public hearings were held only to implement the public involvement portion of the CMP (U.S. Army, 1998d; 1998g; and 1998i). According to the CMP, if significant public opposition had been expressed, the Army would have revisited the PFS issue. Otherwise, the Army's recommendation to retain the PFS would become a matter of record (Hopkins, 1998c).

Very few, if any, members of the public attended the "public outreach availability sessions" in Anniston or Umatilla (Hopkins, 1998c). The committee considers the Army's public outreach efforts in the PFS decisions a pro forma exercise of the public outreach component of the CMP. The exercise did, however, reveal several shortcomings of the CMP. First, public involvement must be initiated much earlier in the process for evaluating change. For example, public involvement could have been used to help frame the questions to be answered during the PFS risk evaluation process.

1  

 RCRA Class 3 permit changes involve major physical modifications to the facility; RCRA Class 2 permit changes pertain to less significant physical changes or operational changes.

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×

Second, the public involvement process should allow for public input prior to the Army reaching a decision on process changes, even if initial assessments indicate that the decision will be to make "no change." Once the Army had made the initial decision, significant public input during the evaluation process was no longer possible, and the Army was left in a quandary as to how public involvement and the CMP should be carried out. In effect, the CMP process was bypassed, and the decision was made by the Army, based on input from the Alabama Department of Environmental Management.

A credible public involvement process that engenders public trust requires clear guidelines for initiating the process; and the guidelines must be followed, not circumvented by executive decision. Despite the committee's recommendations and the Army's public statements acknowledging the importance of the CMP and public outreach, the Army made no meaningful attempt to use the CMP in the PFS decision.

FUTURE EVALUATIONS OF MAJOR DESIGN CHANGES

Future risk assessments of major design changes should be either probabilistic to capture the range of uncertainty associated with estimates or, if based on bounding estimates, should include both worst-case estimates and reasonable performance assumptions for reducing emissions of SOPCs. Assessments should include the incremental increase in risk to on-site workers from ordinary industrial and agent-related accidents. The following basic questions should be addressed:

  1. What is the baseline risk of storing chemical agents and munitions in the community?

  2. What is the risk, in terms of both probability and consequences, of incinerating chemical agents and munitions to residents and workers at the points of maximum acute and chronic exposure?

  3. How does the risk change with the proposed modification (e.g., addition or removal of the PFS)?

  4. How do the risks change if the proposed modification involves a change in the permitted configuration that causes a six-month delay in processing chemical agents and munitions? A year's delay?

  5. How do all of these risks compare with accepted thresholds and background risks?

  6. What are the risk-benefit trade-offs of the proposed modification in terms of worker risk, public risk, cost, and schedule?

  7. What are the major factors contributing to risks and can they be mitigated?

Answers to many of these questions can be found in the Anniston and Umatilla PFS risk assessments and their supporting documents. However, for these assessments to be useful to anyone except the most knowledgeable participants in the Chemical Stockpile Disposal Program, the key information must be available in an integrated presentation. Neither the public nor the Stockpile Committee has sufficient resources to extract the necessary information from the raw data. Hence, the technical analyses supporting results of the risk assessments were not scrutinized for this report. Risk evaluations should also include independent peer reviews, as the Stockpile Committee has recommended (NRC, 1997a):

Recommendation 2.... The Army should continue to obtain interactive, independent expert reviews of all site-specific risk assessments.

Given the potential risks and benefits from carbon filters, a more quantitative, peer-reviewed risk analysis should be done to support future decisions. Without a clear understanding of the nature and magnitude of the risks to human health and the environment, including a comprehensive evaluation of worker risks and how they can be controlled, no fully informed decision can be made about the overall value of the PFS. If the increased risk to on-site workers is found to be substantial, the Army should consider making design modifications, as long as they do not substantially increase overall worker risk or public risk, including the risk of storage during a prolonged delay.

The Army could consider installing the carbon filter units in accordance with current design and permitting requirements but not loading the filter elements. A PFS without the HEPA filters and activated carbon would be simply an elaborate piece of duct work that would minimize or eliminate the risks associated with operating the PFS, as well as the cost increases and schedule delays associated with removing the PFS. Leaving the PFS in place without the filter elements is one option that could be offered for consideration to the Citizens Advisory Commission at each site where permits have already been issued. The Army can still perform meaningful risk assessments and present the results to the public.

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×

SUMMARY

The concepts outlined in the recent CMP Plan, the CMP Public Involvement Plan, and the two Letter Reports on the PFS represent positive steps in the Army's outreach and risk management programs (U.S. Army, 1998f, 1998i, 1998d, 1998g). However, the Army has not demonstrated an effective means of involving the public in its risk management process and has not successfully implemented the CMP.

A successful public outreach program requires sharing timely, accurate, useful information with affected stakeholders as a basis for informed public review and comment. Several Army technical reports have been drafted on the estimated health effects to workers and residents of installing and operating carbon filters at Anniston, Umatilla, and Pine Bluff. The Stockpile Committee is hesitant, however, about commenting on the details and conclusions of these current risk assessment reports because they are not freestanding documents and are not complete. The Army will have to provide a careful assessment of worker risk, as well as an independent technical review of the underlying PFS HRA and QRA reports to meet scientific standards and present a defensible case to the public. A complete technical review would include all risk assessments completed to date (including the Tooele HRA, which was completed under contract to the State of Utah [Utah DSHW, 1996] and other assessments done to meet regulatory requirements). The Tooele QRA has already undergone extensive independent review and does not require further assessment, although new QRA reports should reflect any updated methodologies or data that have since become available.

Conceptually, the committee agrees with the Army' s decision to proceed with the current designs at Anniston and Umatilla and not to alter the operating configurations of JACADS and the TOCDF. Removing or adding carbon filters at this point would most likely cause delays that would increase the risk to workers and the public from prolonged storage of the stockpile. To mitigate the potential adverse consequences of adding carbon filters at Anniston and Umatilla, worker risk should be evaluated quickly and managed effectively.

Because of the incomplete and fragmented nature of the PFS reports and the lengthy period of evaluation prior to communicating the results to local stakeholders, it would be difficult for the Army to initiate a meaningful public involvement program on the issue now without causing delays in the processing of munitions. Proceeding without meaningful public involvement, however, would be contrary to the spirit and intent of the Army's stated public goals as expressed in the CMP (U.S. Army, 1998f) and the Anniston Letter Report (U.S. Army, 1998d).

If clear, technically adequate information can be made public quickly through the CMP or by other means, Citizens Advisory Commissions and other interested parties at each site would have an opportunity to make substantive comments while the technical data and results are undergoing review and revision. Consistent with the philosophy in the CMP, public concerns could then be formally addressed in the final PFS risk assessment reports. To further the overall goal of minimizing risk to nearby residents and workers, parallel technical and public reviews should be conducted to minimize delays that could increase the risk of continued storage. The Army can still perform meaningful risk assessments and present the results to the public.

The CMP highlights the difficult trade-offs that must be made in risk management decisions involving the disposal of chemical agents and munitions. The ongoing storage of agents and munitions poses the greatest overall risk to nearby residents and the environment; operating and maintenance requirements for the PFS, however, may add to the industrial risk to on-site workers.

Depending on the nature and extent of delays incurred by removing the carbon filters from the current designs and the resulting permit modifications, the PAS configurations for baseline incineration system facilities (with or without carbon filters) should not be changed unless (1) local stakeholders specifically request it, or worker risk is determined to be unacceptably high, and (2) the applicable regulatory agencies can expedite approval for permit modifications to minimize delays in processing the munitions. Accurate, integrated assessments of the worker and off-site risks associated with various PAS configurations would assure the public that prudent decisions are being made.

Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 40
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 41
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 42
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 43
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 44
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 45
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 46
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 47
Suggested Citation:"5 Risk Assessments and Change Management: An Evaluation of the Army's Decision-Making Process." National Research Council. 1999. Carbon Filtration for Reducing Emissions from Chemical Agent Incineration. Washington, DC: The National Academies Press. doi: 10.17226/9651.
×
Page 48
Next: 6 Findings and Recommendations »
Carbon Filtration for Reducing Emissions from Chemical Agent Incineration Get This Book
×
Buy Paperback | $51.00 Buy Ebook | $40.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report reviews the Army's evaluation of carbon filters for use in the baseline incineration PAS, as well as the Army's change management process (the Army's tool for evaluating major equipment and operational changes to disposal facilities). In preparing this report, members of the Stockpile Committee evaluated exhaust gas emissions testing at the two operating baseline incineration systems, JACADS and the TOCDF; evaluated the development of the dilute SOPC carbon filter simulation model; and evaluated the conceptual design of a modified PAS with an activated carbon filter. The two major risk assessments conducted for each continental disposal site that use the baseline system, namely, (1) the quantitative risk assessment, which evaluates the risks and consequences of accidental agent releases, and (2) the health risk assessment, which evaluates the potential effects of nonagent emissions on human health and the environment, were also examined.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!