3
Evaluation of the WSLAT Feasibility Study

This chapter presents the results of the committee’s review of the WSLAT feasibility study1 that was specifically requested in the Statement of Work. The WSLAT feasibility study responds to the directive dated July 9, 2002, from the Director, Operation Test and Evaluation (DOT&E) to incorporate whole system testing with live biological warfare agents, both before and after the full-rate production decision for JBPDS.2 The committee’s consideration of the overall need for WSLAT, and a recommended alternative approach, are presented elsewhere in this report. This section is directed at the feasibility study of February 2004 and the requirement on which it is based, and is independent of the committee’s other recommendations.

The WSLAT feasibility study proposes a single design approach to satisfy the requirements of the DOT&E directive. It concludes with a plan and construction schedule to meet the technical and operational deadlines, with the clear proviso that certain decisions and funding should be made in a timely manner. The committee notes that the time for decision and funding is now several months overdue. As a result, the risks of not establishing WSLAT successfully and not

1  

Feasibility Study for the Whole System Live Agent Testing Program, Battelle Memorial Institute, 505 King Avenue, Columbus, OH 43201. WDTC Document No. WDTC-FS-03-125, West Desert Test Center, U.S. Army Dugway Proving Ground, UT 84022-5000. February 2004. This document is referred to as the WSLAT feasibility study in this report.

2  

Memo, July 9, 2002, Office of the Secretary of Defense, Operational Test and Evaluation, Deputy Director, SUBJECT: Joint Biological Point Detection System (JBPDS) Test and Evaluation (T&E) Strategy.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 16
Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary 3 Evaluation of the WSLAT Feasibility Study This chapter presents the results of the committee’s review of the WSLAT feasibility study1 that was specifically requested in the Statement of Work. The WSLAT feasibility study responds to the directive dated July 9, 2002, from the Director, Operation Test and Evaluation (DOT&E) to incorporate whole system testing with live biological warfare agents, both before and after the full-rate production decision for JBPDS.2 The committee’s consideration of the overall need for WSLAT, and a recommended alternative approach, are presented elsewhere in this report. This section is directed at the feasibility study of February 2004 and the requirement on which it is based, and is independent of the committee’s other recommendations. The WSLAT feasibility study proposes a single design approach to satisfy the requirements of the DOT&E directive. It concludes with a plan and construction schedule to meet the technical and operational deadlines, with the clear proviso that certain decisions and funding should be made in a timely manner. The committee notes that the time for decision and funding is now several months overdue. As a result, the risks of not establishing WSLAT successfully and not 1   Feasibility Study for the Whole System Live Agent Testing Program, Battelle Memorial Institute, 505 King Avenue, Columbus, OH 43201. WDTC Document No. WDTC-FS-03-125, West Desert Test Center, U.S. Army Dugway Proving Ground, UT 84022-5000. February 2004. This document is referred to as the WSLAT feasibility study in this report. 2   Memo, July 9, 2002, Office of the Secretary of Defense, Operational Test and Evaluation, Deputy Director, SUBJECT: Joint Biological Point Detection System (JBPDS) Test and Evaluation (T&E) Strategy.

OCR for page 16
Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary obtaining data and results to support the JBPDS decision schedule are already very high and becoming higher as the schedule compresses. The committee is not optimistic that the proposed WSLAT approach will succeed. The design has major unresolved technical questions. The schedule is highly compressed and already behind. The committee also concluded that risk identification, analysis, and proposed management were particularly weak. The committee’s observations on the feasibility study will be presented in three primary groups. The first addresses the requirements analysis, the second addresses the scientific and engineering approach, and the third group of comments addresses regulatory and policy concerns. In addition to these comments specifically about the WSLAT feasibility study, the committee has noted in the report that a credible testing program requires more than testing facilities. It also needs supporting methodology, materials (e.g., agents and simulants), and validated procedures. To be useful, the WSLAT should be operated according to an overall, integrated test and evaluation methodology. The live agents it will use must be validated and correlated with a suite of optimized and dependable simulants. Neither the methodology nor the suite of simulants is adequate, and WSLAT alone will not suffice. SCIENTIFIC AND ENGINEERING CONCERNS ON THE PROPOSED WSLAT The scientific and engineering concerns arise from the apparent lack of engineering design and analysis, even at a late stage of the schedule. The schedule itself includes many elements of high risk, and is already behind. The study has insufficient engineering detail, so the comments are more general in nature than they would have been had detailed design information been available for the committee. If a detailed engineering design for WSLAT is prepared, then the independent advisory panel recommended in this report could serve as a review body for the design. IMPACT OF POLICIES AND REGULATIONS In its evaluation of the regulatory aspects of WSLAT, the committee has no comment regarding DOD regulations and procedures, nor on DPG’s obligations to the State of Utah. The committee’s comments address only implications of compliance with the National Environmental Policy Act requirements. The committee concludes that to be complete, the WSLAT feasibility study should address environmental requirements, and the potential for controversy and subsequent delay, in a rigorous fashion.

OCR for page 16
Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary SUMMARY COMMENTS AND RECOMMENDATION The committee is concerned that the WSLAT feasibility study is not mature and that the methodology, technology, time, and management attention available are not adequate to provide a reasonable expectation of success. The study writers themselves refer frequently to data that are not available (but essential), and the need to develop additional technology and methodology. The project is very high risk, and is already behind schedule. The difficulty the committee has experienced in obtaining information and data critical to its own study, cited in the Interim Report, is not a good indicator that the funds, attention, and management needed to execute this program will be made available. Recommendation 3-1: An analysis of alternative WSLAT design approaches that would include comparisons of the various key performance requirements in each design approach should be undertaken. The analysis should include a thorough discussion and analysis of risk, mitigation methods, and development of a sound risk mitigation plan. The committee considers these to be essential features before realistic funding and schedule decisions can be made.