single-stage developments. At the request of DoD, a committee of the National Academies planned and conducted a workshop to discuss the role of testing and evaluation in an evolutionary acquisition environment and to make appropriate conclusions and recommendations.

The specific questions addressed include: What are the appropriate roles and objectives for testing in an evolutionary environment? Can a systematic, disciplined process be developed for testing and evaluation in such a fluid and flexible environment? How can information from the earlier stages of the evolutionary acquisition process be used effectively in developing test designs for subsequent stages? Are there methodologies, either in the academic literature (statistics, operations research, management science, etc.) or best practices in industry that can be adapted for use in the evolutionary acquisition environment in DoD? Are there advantages to data archiving and documenting results from past stages of development? Is there adequate technical expertise within the acquisition community to fully exploit data gathered from previous stages and to effectively combine information from various sources for test design and analysis?

While discussing these questions, it became apparent that there are several broader, contextual issues that must also be addressed if the recommendations on test design are to be effective. Among these issues, the following were considered in the report: Is the meaning and intent of evolutionary acquisition sufficiently clear in DoD, or is there a need for clarity and consistency in the terminology and a need for enforcement of policies and procedures? Can the culture and organization of defense test and acquisition fully support the effective implementation of evolutionary acquisition? If not, what changes are needed in the DoD environment, the acquisition process, and incentives to ensure that the full benefits of testing in the evolutionary environment can be realized? Is the current level of cooperation among the program manager, contractors, and the developmental and operational testing communities adequate for supporting evolutionary acquisition?

While these broader issues are somewhat beyond the study’s original scope, the committee concluded that they must be discussed, even if only briefly. The committee draws some conclusions and makes some recommendations on these broader issues. However, the committee could not recommend how to address these problems fully due to the limited scope of this study.