testing, must be sensitive to the wording of these requirements. According to the law, operational test and evaluation means: “(i) the field test, under realistic combat conditions, of any item of (or key component of) weapons, equipment, or munitions for the purpose of determining the effectiveness and suitability of the weapons, equipment, or munitions for use in combat by typical military users; and (ii) the evaluation of the results of such test” (10 U.S.C. § 138(a)(2)(B)).
Linking information collected in various stages of the acquisition process together has benefits besides assisting in the estimation of costs and performance. Models that are used to help estimate cost and performance can be evaluated and improved through the regular and systematic comparison of estimates based on models that use empirical observations later in the process.
Finally, it is worth mentioning that the present acquisition process often focuses rather early on an individual system. There are often advantages to developing two or three promising ideas in parallel. The advantages include matched-pair testing, resulting in better identification of superior systems, identification of subtle differences in various environments and against various threats, and better estimates of system cost. An obvious disadvantage is the extreme cost of developing more than one prototype. See Rich and Dews (1986) for further discussion of the pros and cons. However, in some situations even greater development costs will be outweighed by the above advantages. When a predecessor system exists, that system should serve as a relatively low-cost control (see the section above on experimental design).