. "Appendix E Role of Test and Evaluation in Department of Defense Acquisition." Test and Evaluation of Biological Standoff Detection Systems: Abbreviated Version. Washington, DC: The National Academies Press, 2008.
The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
Test and Evaluation of Biological Standoff Detection Systems: Abbreviated Version
attribute of a standoff detector is usability. Measures of detector usability might include weight, power use, mean time between failures, and difficulty in reading data.
Measures of effectiveness (MOEs) assess a system’s effectiveness in the accomplishment of a task. MOEs measure capabilities in terms of task accomplishment or system attributes. Tested capabilities should be related directly to operational capabilities in terms of engagement or battle outcome. MOE evaluation criteria (acceptability criteria) should be quantitative if possible. For example, measures of standoff detection include accuracy, false-alarm rates, response time, reliability, range of detection, and discrimination between target and threat.
Measures of performance (MOPs) gauge system or component capabilities or characteristics. MOPs are quantitative or qualitative measures of simulation capabilities and characteristics. They are based on capabilities and characteristics that are defined by the requirements of the intended application or that meet user-defined system performance requirements. Quantitative MOPs are used when it is difficult to assess an MOE directly or when quantitative criteria need to be established. Qualitative MOPs are categorical measures of performance that refer to the presence or absence of specified characteristics. Quantitative MOPs can frequently be related to a numerical scale. A MOP for a standoff detector would be how many simultaneous plumes can be detected, tracked, and analyzed. A qualitative MOP would be how much more rapidly a standoff detector allows a battlefield commander to decide to change a protective posture. Subjective measurement techniques are generally used to address qualitative MOPs.
Associated with each measure is a criterion that shows how well the measure needs to be addressed by the simulation if it is to be acceptable for the intended use. Those criteria are typically called acceptability criteria because they define a minimal level of performance, degree of effectiveness, level of success, or the like that the simulation needs to achieve to be acceptable to the user.