the workshop and the experience and expertise of the panel’s members, we focused on the techniques that have been found to be useful in industrial system development and their applicability to the DOD environment, while acknowledging the differences in the two environments. To that end, we also considered the availability and access to data (especially test data), the availability of engineering and modeling expertise, and the organizational structure of defense acquisition.
Many, perhaps even most, of the industrial practices we discuss and recommend are or have been used in DOD, but they are not systematically followed. We do not offer new policy or procedural recommendations when (1) the techniques are already represented in DOD acquisition policies and procedures, (2) DOD has been trying to implement the desirable practices, or (3) the desirable practices have previously been recommended in other NRC reports or by other advisory bodies. In these cases we reiterate the benefits of and the need to fully adopt and follow the relevant policies, procedures, and practices. We do offer recommendations to determine if the defense acquisition community is moving in the wrong direction by reviewing policies, procedures, and practices that are new or have elements that are new.
Conclusion 1: It is critical that there is early and clear communications and collaboration with users about requirements. In particular, it is extremely beneficial to get users, developers, and testers to collaborate on initial estimates of feasibility and for users to then categorize their requirements into a list of “must haves” and a “wish list” with some prioritization that can be used to trade off at later stages of system development if necessary.
Although communication with users is common in defense acquisition, the emphasis at the workshop was on a continuous exchange with and involvement of users in the development of requirements. In addition, the industrial practice of asking customers to separate their needs into a list of “must haves” and a “wish list” forces customers to carefully examine a system’s needs and capabilities and any discrepancies between them and thus make decisions early in the development process. It is also important to use input from the test and evaluation community in the setting of initial requirements.
Conclusion 2: Changes to requirements that necessitate a substantial revision of a system’s architecture should be avoided as they