proceduralized. In these cases, detailed task analyses provide data that will permit at least a first-order approximation of the behavior of interest.

Sometimes small-scale analytical studies or field observations can provide detailed data suitable for filling in certain aspects of a model, such as the time to carry out a sequence of actions that includes positioning, aiming, and firing a rifle or targeting and launching a missile. Some of these aspects could readily be measured, whereas others could be approximated without the need for new data collection by using approaches based on prediction methods employed for time and motion studies in the domain of industrial engineering (Antis et al., 1973; Konz, 1995), Fitts' law (Fitts and Posner, 1967), or GOMS1 (John and Kieras, 1996; Card et al., 1983). These results could then be combined with estimates of perceptual and decision making times to yield reasonable estimates of human reaction times for incorporation into military simulations.

Inevitably, there will be some data and parameter requirements for which neither the literature nor modeling and analysis will be sufficient and for which it would be too expensive to conduct even an ad hoc study. In those cases, the developer should rely on expert judgment. However, in conducting this study, the panel found that expert judgment is often viewed as the primary source of the necessary data; we emphasize that it should be the alternative of last resort because of the biases and lack of clarity or precision associated with such judgments.

Much of the modeling of human cognition that will be necessary for use in human behavior representations—particularly those aspects of cognition involving higher-level planning, information seeking, and decision making—has not yet been done and will require new research and development. At the same time, these new efforts can build productively on many recent developments in the psychological and sociological sciences, some of which are discussed in the next chapter.

Verify, Validate, and Accredit the Model

Before a model can be used with confidence, it must be verified, validated, and accredited. Verification refers here to the process of checking for errors in the programming, validation to determining how well the model represents reality, and accreditation to official certification that a model or simulation is acceptable for specific purposes. According to Bennett (1995), because models and simulations are based on only partial representations of the real world and are modified as data describing real events become available, it is necessary to conduct verification and validation on an ongoing basis. As a result, it is not possible to ensure


GOMS (goals, operators, methods, and selection rules) is a relatively simple methodology for making quantitative estimates of the performance times for carrying out well-structured procedural tasks.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement