National Academies Press: OpenBook
« Previous: 2 Overview and High-Level Recommendations
Suggested Citation:"3 Evaluation of the WSLAT Feasibility Study." National Research Council. 2005. Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary. Washington, DC: The National Academies Press. doi: 10.17226/11287.
×

3
Evaluation of the WSLAT Feasibility Study

This chapter presents the results of the committee’s review of the WSLAT feasibility study1 that was specifically requested in the Statement of Work. The WSLAT feasibility study responds to the directive dated July 9, 2002, from the Director, Operation Test and Evaluation (DOT&E) to incorporate whole system testing with live biological warfare agents, both before and after the full-rate production decision for JBPDS.2 The committee’s consideration of the overall need for WSLAT, and a recommended alternative approach, are presented elsewhere in this report. This section is directed at the feasibility study of February 2004 and the requirement on which it is based, and is independent of the committee’s other recommendations.

The WSLAT feasibility study proposes a single design approach to satisfy the requirements of the DOT&E directive. It concludes with a plan and construction schedule to meet the technical and operational deadlines, with the clear proviso that certain decisions and funding should be made in a timely manner. The committee notes that the time for decision and funding is now several months overdue. As a result, the risks of not establishing WSLAT successfully and not

1  

Feasibility Study for the Whole System Live Agent Testing Program, Battelle Memorial Institute, 505 King Avenue, Columbus, OH 43201. WDTC Document No. WDTC-FS-03-125, West Desert Test Center, U.S. Army Dugway Proving Ground, UT 84022-5000. February 2004. This document is referred to as the WSLAT feasibility study in this report.

2  

Memo, July 9, 2002, Office of the Secretary of Defense, Operational Test and Evaluation, Deputy Director, SUBJECT: Joint Biological Point Detection System (JBPDS) Test and Evaluation (T&E) Strategy.

Suggested Citation:"3 Evaluation of the WSLAT Feasibility Study." National Research Council. 2005. Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary. Washington, DC: The National Academies Press. doi: 10.17226/11287.
×

obtaining data and results to support the JBPDS decision schedule are already very high and becoming higher as the schedule compresses.

The committee is not optimistic that the proposed WSLAT approach will succeed. The design has major unresolved technical questions. The schedule is highly compressed and already behind. The committee also concluded that risk identification, analysis, and proposed management were particularly weak. The committee’s observations on the feasibility study will be presented in three primary groups. The first addresses the requirements analysis, the second addresses the scientific and engineering approach, and the third group of comments addresses regulatory and policy concerns.

In addition to these comments specifically about the WSLAT feasibility study, the committee has noted in the report that a credible testing program requires more than testing facilities. It also needs supporting methodology, materials (e.g., agents and simulants), and validated procedures. To be useful, the WSLAT should be operated according to an overall, integrated test and evaluation methodology. The live agents it will use must be validated and correlated with a suite of optimized and dependable simulants. Neither the methodology nor the suite of simulants is adequate, and WSLAT alone will not suffice.

SCIENTIFIC AND ENGINEERING CONCERNS ON THE PROPOSED WSLAT

The scientific and engineering concerns arise from the apparent lack of engineering design and analysis, even at a late stage of the schedule. The schedule itself includes many elements of high risk, and is already behind. The study has insufficient engineering detail, so the comments are more general in nature than they would have been had detailed design information been available for the committee. If a detailed engineering design for WSLAT is prepared, then the independent advisory panel recommended in this report could serve as a review body for the design.

IMPACT OF POLICIES AND REGULATIONS

In its evaluation of the regulatory aspects of WSLAT, the committee has no comment regarding DOD regulations and procedures, nor on DPG’s obligations to the State of Utah. The committee’s comments address only implications of compliance with the National Environmental Policy Act requirements.

The committee concludes that to be complete, the WSLAT feasibility study should address environmental requirements, and the potential for controversy and subsequent delay, in a rigorous fashion.

Suggested Citation:"3 Evaluation of the WSLAT Feasibility Study." National Research Council. 2005. Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary. Washington, DC: The National Academies Press. doi: 10.17226/11287.
×

SUMMARY COMMENTS AND RECOMMENDATION

The committee is concerned that the WSLAT feasibility study is not mature and that the methodology, technology, time, and management attention available are not adequate to provide a reasonable expectation of success. The study writers themselves refer frequently to data that are not available (but essential), and the need to develop additional technology and methodology. The project is very high risk, and is already behind schedule. The difficulty the committee has experienced in obtaining information and data critical to its own study, cited in the Interim Report, is not a good indicator that the funds, attention, and management needed to execute this program will be made available.

Recommendation 3-1: An analysis of alternative WSLAT design approaches that would include comparisons of the various key performance requirements in each design approach should be undertaken. The analysis should include a thorough discussion and analysis of risk, mitigation methods, and development of a sound risk mitigation plan.

The committee considers these to be essential features before realistic funding and schedule decisions can be made.

Suggested Citation:"3 Evaluation of the WSLAT Feasibility Study." National Research Council. 2005. Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary. Washington, DC: The National Academies Press. doi: 10.17226/11287.
×
Page 16
Suggested Citation:"3 Evaluation of the WSLAT Feasibility Study." National Research Council. 2005. Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary. Washington, DC: The National Academies Press. doi: 10.17226/11287.
×
Page 17
Suggested Citation:"3 Evaluation of the WSLAT Feasibility Study." National Research Council. 2005. Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary. Washington, DC: The National Academies Press. doi: 10.17226/11287.
×
Page 18
Next: 4 Recommended Evaluation Strategies »
Review of Testing and Evaluation Methodology for Biological Point Detectors: Abbreviated Summary Get This Book
×
Buy Paperback | $21.00 Buy Ebook | $16.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report examines the proposed testing methodology and facility that the Department of Defense (DOD) will use to test and evaluate the effectiveness of its detection system against biological warfare agents—an issue that impacts battlefield missions as well as homeland security missions. The report assesses a proposal to construct a whole system live agent testing facility at West Center Test Center, Dugway Proving Ground in Utah for testing the Joint Biological Point Detection System (JBPDS). Because of scientific and schedule-related risks, the report recommends an alternate approach that focuses test and evaluation efforts on leveraging existing data, improving simulated biological agents for use in testing, testing in conditions that more closely resemble the actual field conditions where the JBPDS would be deployed, and modeling for predicted performance against actual biological agents. The report concludes that an integrated testing and evaluation plan encompassing all of these factors will be needed.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!