Risk Assessment and Management at Deseret Chemical Depot and the Tooele Chemical Agent Disposal Facility (1997)

The Stockpile Committee has reviewed the PMCD and DCD/TOCDF risk management program. Specifically, the committee has examined the TOCDF risk assessments (QRA and HRA), the DCD QRA, the QRA Expert Panel's review report, the PMCD and TOCDF risk management policy and implementation documents, and the PMCD draft evaluation of charcoal filters for the PAS (U.S. Army, 1996c, 1996d, 1996e, 1996f; 1997c, 1997d; Utah DSHW, 1996; EG&G, 1996; MITRETEK Systems, 1996).

In the 1993 letter report (NRC, 1993b), the Stock-pile Committee recommended that a site-specific QRA that fully displays uncertainties be conducted using ''modem up-to-date methodologies. . . by organizations with recognized expertise in the field.'' The committee further stated that "independent peer reviews are an absolute requirement" and "that local representatives of neighboring communities must be involved early" so their concerns could be considered in the analytical process.

The site-specific TOCDF QRA meets the goals stated by the committee and is much improved over the earlier draft reviewed in the Systemization report (NRC, 1996b). The QRA is now both more thorough and more complete, the treatment of uncertainties is more rigorous and consistent, and the mechanistic models and supporting data are more convincing. An independent peer review panel conducted in-depth reviews as the work progressed and issued many insightful comments. The QRA team responded thoroughly to all comments, which led to significant improvements.

With respect to public involvement, early attempts were made to involve state officials and the Utah CAC. However, as the committee noted in the Systemization report, these efforts were not very productive. Plans are being developed for early public involvement at other sites and for expanding public involvement during operational (risk management) phases at the TOCDF. The Stockpile Committee supports these efforts, encourages their effective implementation, and reiterates its position that public involvement is a PMCD management function and not strictly a public affairs office function (NRC, 1996c).

The following findings are based on the Stockpile Committee' s general evaluation of the PMCD's risk management program, on knowledge and observation of the TOCDF baseline incineration system and the DCD stock-pile, on information provided by the Army, the Army contractor SAIC, and others, as well as on seven site visits to the TOCDF (November 1991, March 1993, May 1994, March 1995, March 1996, December 1996, and March 1997). Four subgroups of the committee also visited the TOCDF in the spring of 1995. The committee's community involvement subcommittee held meetings with the Utah CAC and other interested parties in Tooele County in March 1995 and again with the CAC in December 1996 and March 1997. The committee's risk analysis subcommittee sent representatives on five occasions to meetings of the QRA Expert Panel, SAIC, and PMCD personnel (February 1995, March 1995, May 1995, August 1995, and September 1995). In addition, committee representatives monitored a number of telephone conferences among members of the Expert Panel and the QRA team during 1996.

The Stockpile Committee has followed the DCD/TOCDF QRA project closely since its inception and has maintained oversight of the Expert Panel independent peer review process. The QRA has achieved the goals set out in the committee's 1993 letter report (NRC, 1993b) and the Recommendations report (NRC, 1994b). The success of the QRA was a direct result of a skilled SAIC technical team, firm support from the Army and TOCDF personnel, and frequent and positive interactions between the TOCDF field staff and the QRA team. The resulting QRA was significantly improved during the Expert Panel review. The findings of the QRA are consistent with the interim findings in the Systemization report (NRC, 1996b).

Members of the Expert Panel included internationally recognized experts in quantitative risk assessment, as well as experts in chemical process safety and an expert from the state of Utah in combustion and chemical engineering. The peer review was especially effective because it was interactive. The panel met or talked by telephone with the Tooele QRA team many times throughout the project. This enabled the QRA team to respond to questions and revise their approach as the analysis evolved, making effective, cost-effective corrections and changes in direction and allowing the panel to investigate analyses in greater depth than would have been possible after work had been completed.

Risk assessment experts from the Stockpile Committee observed most meetings of the Expert Panel and found the panel's review to be thorough and technically sound. The panel was equally thorough in examining theoretical issues, practical results, and the style of presentation. In addition, they searched for possible omissions.

The Stockpile Committee concurs with the Expert Panel's findings (MITRETEK Systems, 1996):

• The methodology was sound and has extended the state of the art in several areas.
• The methodology was well implemented.
• Despite some reservations concerning a few technical aspects of this QRA, the panel was reasonably satisfied that these did not affect the overall conclusions of the QRA.

The committee finds that the interactive independent review process was effective and that the Expert Panel played a significant role in ensuring that the QRA met state-of-the-art standards in all significant respects. The Expert Panel had a significant impact on key areas of the QRA:

• The treatment of uncertainty is now more clearly addressed.
• The seismic vulnerability analysis for the liquid propane gas tank has been improved.
• The model for workers donning masks after a strong motion earthquake is more realistic.
• The mechanistic modeling of munitions handling accidents is much improved.
• Significant improvements in the QRA methodology manual (EG&G, 1996) have been made.

The committee believes that the HRA performed by the Utah DSHW, which is based on many assumptions and follows EPA-mandated protocols, is appropriate at this stage of TOCDF operations because it approximates a worst case for all evaluated parameters. The greatest uncertainty in the HRA is about the magnitude and composition of actual TOCDF emissions (emissions in the HRA were based on adjusted data from JACADS). As actual TOCDF operating parameters are established and data on the nature and magnitude of actual emissions become available, they can be incorporated into the HRA. The HRA does not provide the more realistic and detailed estimation of risk sources, impacts, and distribution provided by the QRA. The HRA screens latent cancer risk to "maximally exposed" individuals, imposes an acceptability criterion (1 x 10-5 carcinogenic risk level over a 70-year lifetime), and infers that the exposure to multiple individuals at or below the screening level is acceptable.

The committee finds that the TOCDF risk management plan has progressed and that positive steps have been taken, e.g., the development and limited use of guidance and implementation documents. The Army's draft Guide on risk management provides an overview of the overall risk management program, incorporating references to subsidiary risk management documents and activities. The Guide defines interrelationships

among Army offices, contractor offices, and public entities that are or should be involved in risk management activities (U.S. Army, 1997a). The Guide is a significant step by the Army toward following NRC recommendations on risk management and public involvement (NRC, 1994b; 1996b, 1996c), particularly with regard to using risk analysis in the management of change process. The Guide, however, has not yet been completed and finalized. The committee's comments in this report reflect the expectation that the draft Guide will be improved, adopted, and implemented. The committee may evaluate its implementation in the future, but at present, the draft Guide falls short of expectations in several respects:

• The Guide does not describe the contributions of risk assessment to changes that led to the current Established Configuration, against which proposed future changes will be evaluated.
• The focus of the Guide is primarily on agent-related safety, rather than on developing and institutionalizing a comprehensive safety program (i.e., establishing a safety culture). The Guide should emphasize the importance of a safety culture and of adherence to the best established safety practices used by industrial process plants. In reviews of drafts of the risk management program requirements document, the QRA Expert Panel expressed concern "that the importance of establishing and maintaining a safety culture may not be addressed in the Army standards" (MITRETEK Systems, 1996).
• The Guide acknowledges that more must be done to shift the focus from public information to public involvement. The role of public involvement should be extended and integrated beyond the management of change process. This will require tracking public involvement activities and public response decisions and involving the public in additional activities, such as monitoring and emergency management. This integration should be implemented expeditiously.
• The Guide is not specific enough about how to ensure that workers, CSEPP personnel, and the public understand the risk analyses. Improved communications are essential and need to be part of risk communication plans.
• Although the Guide indicates that there are functional relationships among PMCD organizations with regard to risk management, it does not detail how management roles and communications across all the Army groups, contractors, subcontractors, and other agencies involved in the program will be integrated.
• The Guide does not indicate plans to track CSEPP responses to changes in risk or to document public involvement and Army responses to public input regarding risk.

On the positive side, the draft Guide presents a framework for managing changes in the configuration of a facility or changes in operations that may significantly affect risk levels. The framework allows for public input on significant changes through a comment process, which is followed by formal feedback to the public explaining the basis for a decision. The committee finds the proposed management of change process satisfactory and encourages its use. However, there may be opportunities to expand public participation further as the Army develops a more comprehensive public involvement program.

The committee finds the following with regard to the evaluation of adding carbon filters to the PAS:

• The proposed methodology, if well implemented, is appropriate for evaluating whether or not to install a PFS on a site-specific basis (U.S. Army, 1996d).
• The proposed methodology for the PFS evaluation is consistent with the Army's proposed management of change process, as described in the Guide.
• Carbon filters appear to be effective in reducing the levels of dioxins/furans to below the limits of detection and to have a useful life of at least one year. Because these levels cannot be measured, however, credit only for reducing them to the detection limit appears in the HRA.
• The QRA calculations for the PFS must account for a potential sudden release of accumulated agent (based on HRA-assumed emissions at the lower detection limit) in case of a PFS malfunction.

Recommendation 1

The Army should update both the QRA and HRA at the TOCDF whenever changes to system design or operations occur that could affect QRA or HRA calculations to ensure that estimates of risk are current and reflect changes in operating conditions and experience, assumptions, and program status (current Established Configuration). The process for updating the QRA and HRA should be included in the Guide.

Recommendation 2

The Army should continue the site-specific QRA and HRA processes at all PMCD sites. The development of assessments for sites other than the DCD will be greatly simplified because much of the methodology has already been established. The Army should continue to obtain interactive, independent expert reviews of all site-specific risk assessments. The Army should heed the lessons learned from development of the TOCDF QRA and should incorporate the changes recommended by the Expert Panel.

Recommendation 3

The QRA methodology manual should be updated to reflect the significant improvements that have been made.

Recommendation 4

The Army should expand its draft report on risk management policy, A Guide to Risk Management Policy and Activities, to encourage the establishment of a "safety culture" within the PMCD and its field offices and among contractors and other government agencies. The Guide should elucidate the Army's policy on industrial safety, including the responsibilities of individuals and managers in the field and the definitions of acceptable performance.

Recommendation 5

The Army should develop a management plan (and include it in the Guide) that defines the integration of management roles, responsibilities, and communications across activities by risk management functions (e.g., operations, safety, environmental protection, emergency preparedness, and public outreach).

Recommendation 6

The Army should review and expand the current draft risk management plan to include public involvement in appropriate areas beyond the management of change process.

Recommendation 7

The Army should institutionalize the management of change process developed in the Guide. The Army should track performance of the change process and document public involvement and public responses to decisions. The Army should use this experience to improve the change process.

Recommendation 8

The Army should expand implementation of the risk management program to ensure that workers understand the results of the risk assessments and risk management decisions. The Army should also ensure that CSEPP and other emergency preparedness officials understand the QRA and how their activities might affect risk. CSEPP activities should be tracked by the Army as part of their risk management program.

Recommendation 9

The Army should implement their risk management plans and update them whenever necessary to ensure that they reflect current practices and lessons learned.

Recommendation 10

The Army should proceed with the application of its proposed methodology for evaluating the use of PAS carbon filters on a site-specific basis. For consistency with the HRA assumptions, the QRA should take into account the possible sudden release of agent that may have accumulated on the filter at a gas concentration equal to the lower detection limit.