4

Process Modeling in Support of Washout Deletion

The Blue Grass Chemical Agent Destruction Pilot Plant (BGCAPP) is a one-of-a-kind facility that contains a number of first-of-a-kind (FOAK) pieces of equipment and procedures for the processing of nerve agents stored at the Blue Grass Army Depot. There is no standard of operation to compare against. BGCAPP is essentially a FOAK plant with a number of FOAK pieces of equipment. There is an inherent risk to any one-off chemical plant. In industry there is normally a pilot plant preceding the full-scale production plant. In this case, that is not really possible. At this writing, each part of the process has either been approved by the Kentucky Department for Environmental Protection in the current Research, Development, and Demonstration (RD&D) permit, or is being reviewed by the Kentucky Department for Environmental Protection (i.e., the design and process revisions presented in the current RD&D Permit Revision 5 Submission). Given the obvious risks involved with processing chemical warfare agents, it is important to use modeling to identify operational, management, maintenance, and safety issues rather than waiting to encounter them during actual operations. To this end, a discrete-event Monte Carlo simulation model was used by BGCAPP program staff to predict how long BGCAPP will be in use.

One should recognize that the BGCAPP model is a process model, not a model of the underlying chemical methodology of agent destruction. The process model assumptions describe the flow of material within the processing system. The system parameter values used in the BGCAPP model are based on estimates by BGCAPP personnel of chemical agent inputs and outcomes.

THE BGCAPP FACILITY MODEL

The BGCAPP process model was presented to the committee in block diagram flow format, shown in Figure 1-1. Comparing the structure of the diagram to the technical information presented to the committee at the initial meeting at BGCAPP, the model appears to represent the process well. Given the complexity of the BGCAPP facility, it was difficult to determine from the committee’s quick walk-through of the facility whether or not the structure of the model reliably represents the BGCAPP facility from a process-flow standpoint. It is clear, however, from presentations given to the committee and the ensuing discussion, that BGCAPP management supports the model from a structural standpoint and is fully invested in it. In fact, the model structure has been vetted periodically over time by BGCAPP personnel.¹ Given this, the committee’s two main questions were the following:

• Was the model exercised appropriately? and
• Was the model fed appropriate parametric data?

Discussions with BGCAPP staff during the first meeting and later by telephone revealed that some of the input data for the model was gleaned from historical data that was gathered in a way such that the true stochastic nature (or random behavior) of the data was not identified.^2,3 Some of the input data were estimated by gathering all of the subject-matter experts in the organization together to make educated, consensus estimates of the values to be used in the model. Thus, it appears that most of the stochastic nature of many important system parameters is not available to the model. This should not be construed as critical of the processes that BGCAPP has used to estimate the operational data to be fed to the model. The data used in the model were simply the best data available at the time. The average processing rates and times used in the model could be accurate in the mean

___________________

¹ John Barton, chief scientist, Bechtel Parsons Blue Grass (BPBG), conference call with committee member Thom Hodgson on September 30, 2015.

² Ibid.

³ John Barton, chief scientist, BPBG, Charles O’Classen, throughput engineer, BPBG/Bechtel Pueblo Team (BPT), Michael Noyes, and John Coyne, Bechtel Parsons Group, conference call with committee member Thom Hodgson on October 1, 2015.

(for example, the estimates of the time to drain the agent from weapons).

The proposed elimination of the washout functions will impact the operation of downstream treatment systems and thus the model input parameters. Two processes that will be impacted are presented as examples below: munition drain times and filter sock change-out activities.

Finding 4-1. While the process model explores the influence of variations in operating parameters on the performance of BGCAPP, the limited treatment of the stochastic nature of those parameters does not reflect operational experience.

Finding 4-2. The reliance on point estimates in the model data does raise concerns about the ability of the model to accurately forecast future facility operations in terms of the length of time to complete the processing of the chemical weapons and the risks involved in operating the facility.

Munition Drain Times

With washout deletion, there may be significant differences in munition drain times that the model is not able to reflect because of the non-stochastic nature of the data fed to it. In the case of the initial drainage amounts from the munitions, the committee spoke with individuals who had actually drained a number of similar munitions 10 or so years ago.⁴ Anecdotal accounts indicated considerable variance in the condition of the GB agent in the weapons. They commented on observing the state of the agent in some cases as gelled and/or crystallized. It is certainly unlikely that the condition of the munitions has improved over the past 10 years. This informal data might be used to estimate (at least) upper and lower bounds on drain time. Running the model using the upper and lower bounds would give a range on the actual time to complete agent processing. Were data available on the distribution of drain times for particular munition types and agent lots, the model predictions could gain even greater fidelity with actual plant operations.

Finding 4-3. The stochastic nature of the gelling or crystallization of the GB agent may still be partially retrievable. A formal debriefing of individuals who have drained munitions to capture the (informal and clearly anecdotal) nature of the condition of the agent in the weapons might be useful in developing more believable assumptions as to the condition and variability of the chemical agents in the weapons.

Recommendation 4-1. BGCAPP should retrieve and document historical (informal and anecdotal) data on munition drain times and run these data, complete with ranges of uncertainty, through the BGCAPP model.

Filter Sock Change-Out Frequency

The BGCAPP model currently assumes filter sock change-out once every 3 days.⁵ This was said to be a conservative estimate that is based on assumptions about how much gel will be in the munitions, how much of that gel will drain from the munition using gravity, and how much will be trapped in the filters. Other than scheduled and unscheduled maintenance of equipment, the filter change-out rate may be the most critical step in terms of impact to schedule. That is to say, it may be the most important “pinch point” in plant operations that can impact schedule.

The main take-away point is that the length of time required to actually complete munitions processing at BGCAPP may have been underestimated using the BGCAPP model. This may have a negative schedule impact on BGCAPP operations, with implications for budget, treaty compliance, and the timely reduction of storage risk by destroying the stockpile.

Finding 4-4. The actual filter sock change-out rate may be the most important rate-limiting factor in BGCAPP operations and may be underestimated.

EXPLORING SYSTEM SENSITIVITY TO THE INPUT PARAMETERS

Many operational issues cannot be fully known until the facility is actually in operation. Nothing was presented to the committee relative to the sensitivity of the performance of the system to the various input parameters values.⁶ It is clear that the BGCAPP staff has attempted to be conservative in all of their parameter estimates, but it is also clear that many of the parameters potentially have larger variances than expected.

The unavoidable deficiencies in the estimation of the system parameters used in the BGCAPP model, in and of themselves, argue for trying to estimate the sensitivity of the system to variations in the operating parameters. Considerable effort has been expended over time to validate the structure of the model.⁷ Given the effort put forth, it can be expected that the point estimates used for the parameters are at least in the ballpark. However, the stochastic nature of the BGCAPP processes is not well represented in the model due to the reliance on point estimates.

___________________

⁴ John Barton, chief scientist, BPBG, conference call with committee member Thom Hodgson on September 30, 2015.

⁵ John Barton, chief scientist, BPBG, “Rocket Handling System/Munitions Washout System (RHS/MWS) Process and Infrastructure Changes Due to Washout Deletion” presentation to the committee on September 9, 2015.

⁶ John Barton, chief scientist, BPBG, conference call with committee member Thom Hodgson on September 30, 2015.

⁷ John Barton, chief scientist, BPBG, Charles O’Classen, throughput engineer, BPBG/BPT, Michael Noyes, and John Coyne, conference call with committee member Thom Hodgson on October 1, 2015.

As noted above, the model apparently does represent the process flow of the facility.^8,9 Thus, in its present form, the model should be sufficient to determine estimates of the sensitivity of BGCAPP operations to variations in the operating parameters. Bottlenecks can be identified as a function of varying various parameters in model runs. In this way, the potential for excessive filter cleanouts or excessive munition drain times to impede system performance can be explored. Other issues of importance might also be identified, explored, quantified, and mitigated (e.g., the need to process fewer munition bodies on the trays going through the metal parts treater).

In order to perform a sensitivity exploration exercise, it would be necessary to design a series of runs of the model—that is, initially placing relevant parameters at an upper bound and then at a lower bound (i.e., the highest and lowest levels). The objective is to determine a model of system responses to changes in the parameters (i.e., a response surface). Regression could be a reasonable way to develop a response surface from the model output.

Finding 4-5. Analysis of the sensitivity of the BGCAPP operations to variations in model input parameters might expose potential operational issues, allowing them to be quantified and possibly mitigated prior to operations.

Recommendation 4-2. BGCAPP should design and execute a series of modeling experiments to determine the sensitivity of operations to variations in operating parameters, reflecting the stochastic nature of some processes. Examples of parameters include maintenance and repair times, added characterization steps, retreatment for batches not meeting destruction efficiency, and compounding problems such as long munitions drain times together with very frequent filter sock change-outs. The results of these experiments should be used to prepare for potential challenges and mitigate them ahead of time as much as possible.

OPERATIONAL DATA COLLECTION

It is clear that the accumulation of data to characterize the actual values of operating parameters is the best way to model plant performance. For many critical parameters, however, this may be only possible to do as the facility enters operations. In other words, there may be no way, at this point, to improve current estimates of many of the operating parameters prior to actually bringing the system into actual operation. However, BGCAPP management plans to bring the BGCAPP facility into operation slowly and to verify predicted chemical reactions, reaction rates, and thermodynamics. This will provide the opportunity to collect data on critical operational parameters and develop parameter baselines for later operations. This start-up effort is critical to the operation of the facility and to updating estimates of the time that will be required to complete munitions processing. BGCAPP has developed a comprehensive control center to gather, in real-time, all relevant operational parameters, which would make gathering the appropriate data straightforward.

Early operations will be the first time that these parameters will, in fact, have the potential to be accurately estimated from actual operational data. However, it is important to realize that these estimates still contain randomness. With improved estimates of the parameters and of their statistical distributions, continued real-time forecasts of system performance can be made with the model to tune the processes, to improve the model, and to aid in the management of the overall system. Note that doing this will allow modifications to the model, as many parameters that are now modeled as fixed parameters will actually be able to be modeled as stochastic in nature. For example, data on drain times of a particular munition may fall into multiple classes, with some munitions draining rapidly and completely and others, where gelation or crystallization has occurred, draining more slowly and less completely. Such data could be represented in the form of a probability density function that would then replace the point estimates used in the early modeling.

Finding 4-6. Point estimates of operational parameters are only a starting point. To fully understand the plant operation and, ultimately, to understand the plant timeline, one needs data on the distribution of parameter values that may be encountered during operation.

Recommendation 4-3. During start-up, and continuing through plant operations, BGCAPP should gather data for relevant model parameters with sufficient resolution to assess the probability density functions for these parameters.

The stability of system operation will be important to observe and control. The concepts of statistical quality control could be of use in the analysis of the data (Grant and Leavenworth, 1974). This is a tool used regularly in industry to help control processes and maintain process integrity. Essentially, it is a methodology that allows statistical analysis of operating parameters to detect if a parameter is within operational bounds and to determine if the processing system is operating as required. The committee notes that BGCAPP is going to do some of this during systemization and operations ramp-up.

Finding 4-7. Statistical quality control could be a useful management tool for understanding and identifying possible problems as they occur.

___________________

⁸ John Barton, chief scientist, BPBG, conference call with committee member Thom Hodgson on September 30, 2015.

⁹ John Barton, chief scientist, BPBG, Charles O’Classen, throughput engineer, BPBG/BPT, Michael Noyes, and John Coyne, conference call with committee member Thom Hodgson on October 1, 2015.

Recommendation 4-4. BGCAPP should give attention to developing analysis tools such as statistical quality control prior to actual facility start-up.

REFERENCE