Tooele Chemical Agent Disposal Facility: Update on National Research Council Recommendations (1999)

In keeping with the governing recommendation that the CSDP should proceed expeditiously and with technology that minimizes total risk to the public at each site [RC-1], the Stockpile Committee has continued to evaluate the risk assessment¹ and risk management² practices at the TOCDF and throughout the CSDP. The Risk Assessment and Management report provided a detailed overview of the status of risk evaluation and management as of September 1997 at the TOCDF (NRC, 1997). The present report concerns how the results of risk assessments; screening health, safety, and environmental evaluations; and other information have been used (from the Programmatic Lessons Learned [PLL] and other programs) to implement a sound risk management program. Recommendations are focused on the following areas: overall safety, risk assessment, and risk management. The recommendations in each of these areas are summarized below.³

The development and implementation of the overall safety program at the TOCDF must be given a high priority [S-1]. Safety and environmental goals should be given at least equal weight with production goals in establishing contractor award fees [S-2]. Applicable portions of the QRAs (quantitative risk assessments) must be completed for all safety-related concerns before the start of agent destruction campaigns [S-3]. High-quality, well staffed safety management systems must be completely implemented prior to the start of agent

operations [S-10]. The risk management plan (RMP) must be fully implemented during the first year of agent operations [S-14].

During the first year of agent operations, a comprehensive, integrated TOCDF risk assessment, including a full description of all significant, acute, and latent agent and nonagent risks (QRA and HRA) associated with disposal operations, as well as with the continued maintenance of the DCD stockpile, should be completed. A full explanation of the uncertainties associated with the various estimates should be included [S-15]. A system for tracking ''near-misses" during operation should be developed and integrated into a plan for continual safety improvements at the TOCDF [S-16]. In addition, the Army should update both the QRA and the HRA at the TOCDF whenever system or operational changes occur that could significantly affect the risk estimates and should document the changes in A Guide to Risk Management Policy and Activities (the draft Guide) [R-1]. The Army should continue the site-specific QRA and HRA processes at all PMCD sites and heed the lessons learned from development of the TOCDF QRA [R-2]. If the QRA methodology is changed, the methodology manual should be updated [R-3].

The Army should expand its draft Guide to encourage the establishment of a "safety culture," including industrial safety, in all groups involved in the program and develop a management program (and include it in the Guide) that defines the integration of management roles, responsibilities, and communications across activities by risk management functions (e.g., operations, safety, environmental protection, emergency preparedness, and public outreach) [R-4]. The CMP (change management process)⁴ developed in the draft Guide should be institutionalized and improved [R-7]. The Army should expand the implementation of the RMP to ensure that workers and emergency preparedness officials understand it and the QRA, as well as how their activities might affect risk [R-8]. The RMP should be implemented and updated as necessary to ensure that it reflects current practices and lessons learned [R-9].

Risk management at the DCD/TOCDF involves a number of activities intended to control the risks to the public and workers from potential releases of agent and products of incomplete combustion and to reduce the incidence of worker injuries during normal industrial operations (NRC, 1997). There are four steps to risk management:

• understanding the risk

• suggesting alternative ways to reduce risk

• evaluating risk-reduction alternatives

• selecting and implementing preferred alternatives

These steps must be tailored to address site-specific factors. A number of very diverse groups affected by DCD/TOCDF operations must be involved in the risk management process to ensure its effectiveness. Each group must understand the risk assessment process, the results of the assessments, and the significance of the results; each group must also participate in the process of resolving issues of interest. The Stockpile Committee has made several recommendations in previous NRC reports for improving risk management. Recommendations related to worker safety (industrial safety) are considered in Chapter 4. Recommendations related to public and community interactions are considered in Chapter 5. The recommendations related to chemical releases and general risk management policies are considered in this chapter.

In the Systemization and Risk Assessment and Management reports (NRC, 1996a, 1997), the committee reviewed the DCD/TOCDF risk assessments and risk management program. Findings in these reports indicated that the QRA was well done and that the HRA had satisfied most of the committee's previous recommendations (NRC, 1996a). These two reports also include extensive information about the risks at

DCD/TOCDF and the quality of the risk assessments. The committee concluded that certain aspects of risk assessment and risk management at DCD/TOCDF, and throughout the CSDP program, required refinement. Therefore, both reports also included additional risk-related recommendations.

In the Systemization [S] report (NRC, 1996a), the committee endorsed the approach developed for the DCD/TOCDF QRA and initial risk management activities. To ensure continued application of these analytic methods and further development of the risk management processes, the committee recommended that the QRA be completed for all campaigns [S-3] and that the approach to risk assessment and risk management be formally established [S-14, 15, 16] (see Appendix C). As the committee noted in the Risk Assessment and Management report, the TOCDF Phase 2 QRA ⁵ was completed before the start of agent operations at the TOCDF. To date, all safety-related concerns identified in the Phase 2 QRA have been addressed before the start of each campaign.

Several of the issues identified in the Systemization report have been the subject of discussions between representatives of the Army and the Stockpile Committee, and the committee commends the Army for its proactive response. Nevertheless, risk management continues to be an informal, albeit thorough, process. The committee is concerned that an informal process directed by key individuals in the PMCD could break down if there are changes in personnel or that the process might not be fully transferred to the specific sites. Therefore, the committee urges the PMCD to order that a formal RMP be established for QRA-identified safety issues, including a tracking mechanism for identifying and compiling new issues as they arise and for monitoring their resolution.

In the Risk Assessment and Management report, the committee described the Army's draft Guide and Chemical Agent Disposal Facility Risk Management Program Requirements (U.S. Army, 1996b), which were developed during the first year of agent operations at the TOCDF. Unfortunately, significant gaps in the draft Guide still must be resolved [R-4, 5, 6, 7].

The two studies, the QRA and the HRA, that make up the complete DCD/TOCDF risk assessment are based on different methodologies for reasons documented in the Risk Assessment and Management report. Both the QRA and HRA were completed before the start of agent operations. The QRA provided a full analysis of the uncertainties, while the HRA calculated only an upper limit of risk. Therefore, it would be extremely difficult to integrate their data. Consequently, although the Army has not developed a single integrated risk report as recommended by the committee in the Systemization report [S-15], the committee believes that the Army has met the functional requirements of the recommendation.

In several cases, the HRA emissions estimates turned out to be lower than actual emissions in the subsequent trial burns. Therefore, the Army should provide a brief update of the HRA as necessary to reflect the trial-burn results. As discussed in Chapter 2, the overall results and conclusions of the HRA are not expected to change because of these higher measured emissions.

The PMCD collects key information on problems encountered through the PLL and publishes the information on a regular basis in a newsletter distributed to all sites. The PMCD has also held regular program-wide meetings at which Army and contractor managers from each site can share information. Managers at individual sites are responsible for disseminating the information to site employees.

The PLL programs have gone a long way toward providing a system for documenting and tracking unexpected upsets, errors, failures, and other concerns during operation of the facilities. The PLL programs have also provided a means of disseminating this information with the aim of promoting continual safety improvements at the TOCDF, as the committee had recommended [S-16], and at all other CSDP sites. However, at the site level, implementation is informally directed by certain individuals. The committee believes the Army should make PLL programs formal requirements for all CSDP organizations to ensure that this information is disseminated to employees at all sites.

In the Risk Assessment and Management [R] report, the committee evaluated the QRA and HRA, as well as the independent Expert Panel review process for the QRA. The committee found that the QRA was performed to standards that met or exceeded the previous state of the art and appropriately modeled a wide variety of potential accidents involving the release of agent. The results and insights of the QRA were endorsed by the Expert Panel and the committee and were directly useful to PMCD and TOCDF personnel in managing the facility and developing practices to reduce risk to the surrounding population. The committee attributes the high quality of the assessment to the competence of the QRA team, the strength of the Expert Panel and other reviewers, and the responsiveness of the PMCD and the QRA team to comments and questions from reviewers.

The committee also evaluated the HRA in the Risk Assessment and Management report.

The EPA screening approach defines a plausible "worst-case" scenario that is evaluated using a point-estimate HRA. This is not the realistic, integrated analysis (including uncertainty parameters) that the committee had recommended. However, the state of Utah's HRA, a "screening risk assessment," found the public risk from routine operations (normal operations with defined worst-case emissions and conservative upset conditions)⁶ to be much lower than the risk from accidents determined by the QRA. Therefore, because the risk is dominated by the accidents modeled in the QRA, the committee agreed that Army funding of a more realistic analysis of the risk of routine operations was not necessary.

The data from the TOCDF trial burns showed that emissions of several compounds were actually higher than the estimated emissions in the HRA, indicating that some of the assumptions in the HRA were not as conservative as the state of Utah had intended (although the overall results and conclusions of the risk assessment are not expected to change). In subsequent reviews, the committee found that neither the most current risk assessment methods nor the air-dispersion and deposition models recommended by EPA at the time (e.g., guidance issued through December 1994) had been used in the HRA. The committee concluded that the Army should issue a brief update of HRA results using the measured trial-burn emission rates. To be comparable, the Army should follow the same guidance used in the original HRA.

For QRAs, continuing interactive review by an Expert Panel (whereby new methods were being developed as the TOCDF QRA progressed) may not be necessary. But before detailed analysis proceeds, the protocols, input data, calculations, and results should be reviewed.

In the Risk Assessment and Management report, the committee expressed its satisfaction with the risk assessment process at DCD/TOCDF. The committee included a number of recommendations related to risk management [R-1, 2, 3, 7, 8, 9, 10] to ensure that the lessons learned in the DCD/TOCDF risk assessment process would be applied at all CSDP sites and that the developing RMP would be strengthened. An analysis of the status of these recommendations follows.

The Army has the overall responsibility for the safe operation of the TOCDF and must be in compliance with regulatory requirements in order to operate. The HRA is vital for understanding potential off-site health effects and for meeting regulatory requirements. Thus, the HRA has at least two uses: (1) off-site risk management, and (2) permit acquisition. The characteristics of a "good" or "correct" assessment vary, depending on whether the HRA is considered a compliance instrument or a risk management tool. As a tool used to manage risk on a continuing basis, the models must be applicable to

the current conditions at the facility and, therefore, the HRA should have the following characteristics:

• It should be realistic and include a thorough exposition of uncertainty.

• It should be a living analysis maintained on site and continually updated and recalculated to guide risk management decisions.

• It should be done early enough for the results to affect design and operating decisions.

An HRA for regulatory compliance is designed to show one-time permitting compliance and, therefore, should have the following characteristics:

• It should conform to guidance provided by the permitting agency, which will necessitate it being conservatively biased. This is particularly true for screening-level HRAs.

• It should show that the maximum plausible risk for particular individuals at particular locations is less than regulatory thresholds, reflecting design or operational changes if necessary.

• It should be a one-time analysis, possibly supplemented with letter reports on particular issues (e.g., actual emissions data that are higher than estimates in the original HRA).

An HRA based on a conservative analysis acceptable for regulatory decision making, such as whether to grant a permit, lacks many essential details. If efforts to control risk are based on an HRA, they could mistakenly be focused on areas that have been artificially inflated in terms of frequency or consequences for the purposes of the conservative analysis. Problems that could arise from using an HRA performed for regulatory compliance in communicating with other interested parties are listed below:

• The HRA may be assumed to describe actual releases rather than upper-bound results. Thus, the Army could be accused of releasing more agent and products of incomplete combustion than are actually being released.

• Attempts to correct "conservative" assumptions could be interpreted as a cover-up.

• Risk management is likely to be focused on aspects of the HRA with the most pessimistic assumptions, rather than those with the most impact.

• The scenarios required for the HRA may not reflect the most serious facility risks.

Problems could also arise from using an HRA intended to be a risk management tool in communicating with other interested parties for the following reasons:

• It contains complex results that acknowledge uncertainties.

• It does not include simple worst-case scenarios based on point-estimate analyses, and results may be more difficult to interpret and explain.

• Because it is site-specific, it does not necessarily follow established generic screening guidance for compliance-oriented HRAs, which may compromise the credibility of the results.

The draft Guide requires that the QRA and HRA be updated to include significant changes to the facility (U.S. Army, 1997a). However, the process for updating analyses when plant changes are planned has not yet been incorporated into the Guide. All plant changes require some review of risk management. The procedure for determining the appropriate level of review, the review process, and whether or not the change may affect the QRA or HRA has not been described. For example, a change in paperwork that has no health or safety impacts may require minimal review. Major changes that could impact the QRA, or, in rare cases, the HRA, would require thorough review. Changes that may affect worker safety and health but do not involve any agent release would not be part of the QRA and might require an intermediate level of review.

The Guide should also outline the procedures for performing the health, safety, and environmental evaluations (e.g., hazard analyses, job activity analyses, training requirements) to assess whether a proposed change could affect worker safety, for reviewing options for mitigating increases in worker risk, and for deciding whether a change is justified. If PLL programs identify ways to reduce worker risk, mechanisms for incorporating these recommendations into the change management process should be described.

Another problem is that no requirements for updating the analyses based on new information (new emissions measurements or lessons learned) have been promulgated. It is particularly important that the QRA be updated because it can be significantly affected by plant changes. In addition, although Army plans call for

updating QRAs at each site on an ongoing basis, the QRA at the TOCDF has yet to be updated (Holmes, 1998b).

The Army has continued site-specific QRA processes at other CSDP sites and has issued Phase 1 QRA reports for the Anniston, Pine Bluff, and Umatilla sites. It is important that the Phase 2 QRAs be initiated while construction at these sites is in the early stages so that the results can be used to implement necessary changes to the design or operations. The Army has stated that it intends to continue independent expert reviews for all site-specific risk assessments and is incorporating the lessons learned from the TOCDF QRA, including the recommendations by the Expert Panel that were adopted for the TOCDF QRA. The ongoing QRAs for the other sites have not yet progressed far enough to determine whether other recommendations by the Expert Panel will be adopted. The independent reviews of the QRAs for these sites have not yet begun. The committee believes these reviews should be in progress by the time the Phase 2 QRA process begins. Otherwise, the kind of productive, interactive process that resulted at the TOCDF will be impossible.

The committee has not been asked to review HRA studies for the other sites, all of which are now completed and show that the HRA risks are largely secondary to QRA risks at each site. In accordance with current EPA guidelines, however, uncertainty analyses (as part of HRAs) at future sites may not be necessary for screening-level HRAs if the risks are well below regulatory thresholds.

The QRA methodology manual has not been revised. Extensive improvements to the methodology evolved during the DCD/TOCDF QRA. Although members of the QRA team are aware of the lessons learned, there is no guarantee that experienced individuals will not leave the team. In fact, several already have. The committee hopes the Army will capture their expertise while it still can.

Public input in the CMP was supposed to begin with a series of workshops to discuss and refine the process. After that, a revised draft of the Guide was expected to address the issues raised in the Risk Assessment and Management report. The revision to the Guide is not yet complete, and the entire process is far off schedule. (The public involvement aspects of this recommendation are discussed further in Chapter 5.)

The RMP at DCD/TOCDF has been effective in dealing with technical issues related to risk. The draft Guide was issued by CSDP managers at the PMCD, and, more recently, The Change Management Process to Accompany the Guide to Risk Management Policy and Activities was issued (U.S. Army, 1997a; 1998b). Together, they have begun to define the CSDP's overall approach to risk management. In addition, CSDP managers have provided briefings on the HRA and QRA (which are both publicly available) to the communities near the TOCDF. However, in discussions with DCD/TOCDF workers and the public, it has become apparent that neither has a real understanding of the risks portrayed in these analyses. The CSDP will have to redouble its efforts to ensure that the information is understood. (The aspects of this recommendation that deal with the Chemical Stockpile Emergency Preparedness Program [CSEPP] are discussed in Chapter 5.)

At the time of the Risk Assessment and Management report, the Army had implemented a successful ad hoc risk management approach for the TOCDF, established preliminary RMPs, and issued the draft Guide and its companion volume. However, the recommended updates to the Guide have not yet been completed.

The committee strongly believes that the Army should rapidly document and formalize the RMPs that are presently being used effectively on site-specific and programmatic levels. Cross-communication, cooperation, and learning between sites has greatly enhanced the entire program and should be continued.

The PAS carbon bed filter technology and risk management is the subject of another NRC report that was not available at the time of this writing (NRC, 1999).

The Stockpile Committee has been concerned with the CSDP's safety performance since its early evaluations of operational verification testing (OVT) at JACADS and has made many recommendations for improving safety, including the development of a well qualified and trained workforce and the establishment of a safety culture. This chapter revisits the recommendations discussed in Chapter 3 that pertain to worker safety and the implementation of sound industrial safety practices. These include: (1) putting a high priority on the development and implementation of the overall safety program [S-1]; (2) setting management goals for high levels of safety and environmental performance in all work areas and giving these goals at least equal weight with production goals [S-2]; and (3) developing strong safety management systems [S-10].

When Congress authorized the destruction of the chemical agent and munitions stockpile in 1985 (PL 99-145), the law specified that destruction should be accomplished in a way that ensured the safety of the public, workers, and the environment. As part of its oversight responsibility, the Stockpile Committee has expressed continuing concerns over safety and has made many observations and recommendations in several reports for improving safety at specific sites and at the programmatic level. A summary of relevant observations and recommendations follows.

In an NRC letter report, Evaluation of the Johnston Atoll Chemical Agent Disposal System Operational Verification Testing: Part I, issued in July 1993, the committee made the following recommendation:

In April 1994, the Stockpile Committee completed its evaluation of OVT at JACADS and issued Evaluation of the Johnston Atoll Chemical Agent Disposal System Operational Verification Testing: Part II, which contained the following findings and recommendations (NRC, 1994b):

In March 1996, the committee issued Review of Systemization of the Tooele Chemical Agent Disposal Facility, the continuation of several earlier NRC reports, with the intention of (1) reviewing the completion of testing of certain secondary systems that had not been completely tested at JACADS, (2) reviewing the changes implemented by the Army in response to earlier recommendations pertaining to the TOCDF, and (3) providing an overview of the status of the facility at the end of the systemization period (NRC, 1996a). The following excerpts from this report are related to safety issues: