3
Evaluation of the WSLAT Feasibility Study
This chapter presents the results of the committee’s review of the WSLAT feasibility study1 that was specifically requested in the Statement of Work. The WSLAT feasibility study responds to the directive dated July 9, 2002, from the Director, Operation Test and Evaluation (DOT&E) to incorporate whole system testing with live biological warfare agents, both before and after the full-rate production decision for JBPDS.2 The committee’s consideration of the overall need for WSLAT, and a recommended alternative approach, are presented elsewhere in this report. This section is directed at the feasibility study of February 2004 and the requirement on which it is based, and is independent of the committee’s other recommendations.
The WSLAT feasibility study proposes a single design approach to satisfy the requirements of the DOT&E directive. It concludes with a plan and construction schedule to meet the technical and operational deadlines, with the clear proviso that certain decisions and funding should be made in a timely manner. The committee notes that the time for decision and funding is now several months overdue. As a result, the risks of not establishing WSLAT successfully and not
obtaining data and results to support the JBPDS decision schedule are already very high and becoming higher as the schedule compresses.
The committee is not optimistic that the proposed WSLAT approach will succeed. The design has major unresolved technical questions. The schedule is highly compressed and already behind. The committee also concluded that risk identification, analysis, and proposed management were particularly weak. The committee’s observations on the feasibility study will be presented in three primary groups. The first addresses the requirements analysis, the second addresses the scientific and engineering approach, and the third group of comments addresses regulatory and policy concerns.
In addition to these comments specifically about the WSLAT feasibility study, the committee has noted in the report that a credible testing program requires more than testing facilities. It also needs supporting methodology, materials (e.g., agents and simulants), and validated procedures. To be useful, the WSLAT should be operated according to an overall, integrated test and evaluation methodology. The live agents it will use must be validated and correlated with a suite of optimized and dependable simulants. Neither the methodology nor the suite of simulants is adequate, and WSLAT alone will not suffice.
SCIENTIFIC AND ENGINEERING CONCERNS ON THE PROPOSED WSLAT
The scientific and engineering concerns arise from the apparent lack of engineering design and analysis, even at a late stage of the schedule. The schedule itself includes many elements of high risk, and is already behind. The study has insufficient engineering detail, so the comments are more general in nature than they would have been had detailed design information been available for the committee. If a detailed engineering design for WSLAT is prepared, then the independent advisory panel recommended in this report could serve as a review body for the design.
IMPACT OF POLICIES AND REGULATIONS
In its evaluation of the regulatory aspects of WSLAT, the committee has no comment regarding DOD regulations and procedures, nor on DPG’s obligations to the State of Utah. The committee’s comments address only implications of compliance with the National Environmental Policy Act requirements.
The committee concludes that to be complete, the WSLAT feasibility study should address environmental requirements, and the potential for controversy and subsequent delay, in a rigorous fashion.
SUMMARY COMMENTS AND RECOMMENDATION
The committee is concerned that the WSLAT feasibility study is not mature and that the methodology, technology, time, and management attention available are not adequate to provide a reasonable expectation of success. The study writers themselves refer frequently to data that are not available (but essential), and the need to develop additional technology and methodology. The project is very high risk, and is already behind schedule. The difficulty the committee has experienced in obtaining information and data critical to its own study, cited in the Interim Report, is not a good indicator that the funds, attention, and management needed to execute this program will be made available.
Recommendation 3-1: An analysis of alternative WSLAT design approaches that would include comparisons of the various key performance requirements in each design approach should be undertaken. The analysis should include a thorough discussion and analysis of risk, mitigation methods, and development of a sound risk mitigation plan.
The committee considers these to be essential features before realistic funding and schedule decisions can be made.