Defining the Decision Framework and the Value of Exposure Information in Military Deployments
Thomas E. McKone and Detlof von Winterfeldt1
A great deal of effort is being expended by the U.S. Department of Defense (DoD) to develop technologies for detecting chemical and biological agents and for tracking military personnel during deployments. The types and extent of exposure information needed depend largely on how the information will be used to take health-protective actions. As a hypothetical example, assume that a device could report the full spectrum of environmental concentrations for all known chemical and biological agents in real time. Although such a device would be an enormous technological breakthrough, it would not solve the problem of selecting the information to use, managing that information, and, most importantly, acting on the information. To be of value to the decision makers in the field whose judgments affect the success of a deployment and the short-term and long-term health risks to deployed forces, the output of such a device would have to be assembled, collapsed, organized, and summarized.
The first question that must be considered is the purpose of tracking, detection, and monitoring information. Clearly, the information could be used for many purposes, including planning military missions, improving decisions on the battlefield, protecting soldiers from exposure to harmful agents, and making better decisions about medical care during and
after deployment. To provide a better understanding of these purposes, a member of the advisory panel developed a taxonomy of situations that would benefit from improved tracking, detection, and monitoring of information. The taxonomy distinguishes between the use of information in the predeployment phase (e.g., for selecting protective equipment for use in a mission), during the deployment phase (e.g., for responding to immediate threats from harmful agents), and in the postdeployment phase (e.g., for reconstructing exposures and determining medical care).
The usefulness of tracking, detection, and monitoring information depends on its impact on decision making, the so-called ''value of information" (VOI) (e.g., Clemen, 1990; von Winterfeldt and Edwards, 1986). Information that only adds marginally to what is already known is not useful in the decision-making context. Too much information can be a nuisance and an obstacle to good decision making. VOI depends on two factors: (1) how well decisions can be made with readily available tracking, detection, and monitoring information; and (2) how much decisions can be improved by collecting information with new technologies. A formal definition of the VOI is the difference between the expected value of the decision with the information and the expected value of the decision without it (e.g., Clemen, 1990). The technologies that provide information have attributes, such as cost, size, and weight that must also be evaluated to assess the trade-offs among these attributes.
The three sections of this appendix are focused on the three topics of a taxonomy: (1) information needs, (2) VOI, and (3) attributes of new technologies.
Potential exposures to chemical and/or biological (CB) agents has been an important concern of military commanders throughout most of this century. Until recently, attention was focused almost excessively on lethal or incapacitating exposures, and little attention was paid to gathering information on low-level exposures to CB agents, toxic industrial chemicals (TICs), and other harmful agents. Since the emergence of the Gulf War, attention has been focused on correlations between reported symptoms and various types of exposures. DoD has initiated projects to track cumulative exposure information and collect health records for all military personnel as part of a comprehensive medical surveillance program.
In the predeployment stage, information about the nature of possible threats, locations of plants that could emit toxic agents, terrain, weather,
and prevailing wind directions can be used for planning a deployment and equipping troops. For example, the type of protective gear could be based on an assessment of the enemy's CB warfare capabilities. The location of chemical plants and the prevailing weather and wind directions are important factors in the timing of attacks on these plants, as well as for determining the preferred routes of ground forces. Information about local diseases, agricultural pesticides, and local air pollution would be useful for decisions about predeployment vaccinations and medical provisions.
In the deployment phase, especially on the battlefield, the most important information is whether or not a harmful agent is present and poses an immediate threat to troops. This information affects decisions about the use of protective gear, the evacluation and routing of troops to avoid harm, and short-term medical responses. The information must be reliable and available quickly but does not necessarily have to provide a detailed time-profile of individual exposures.
Joint Vision 2010 (JCS, 1996) describes how a commander includes information about potential CB agents in the overall mission strategy to ensure that troops have full dimension protection. Exposure information is also important to other concepts in Joint Vision 2010, including dominant maneuver, focused logistics, precision engagement, and information superiority. The commander, therefore, needs exposure information to protect troops from acute and severe health risks, to determine where to send troops and when to relocate them, to determine when and where to collect samples, and to decide when to call for the donning of protective gear. During deployment, especially on the battlefield, exposure information can be used to make decisions about evacuations or rerouting troops to avoid exposure, provisions of medical equipment and support, and giving an "all clear." Medical staff also make decisions based on exposure information, such as how to deploy medical resources and which treatments will be most effective.
In addition to reducing the risk of severe health consequences in the theater of deployment, comprehensive exposure assessments are now being used by DoD to monitor and improve the overall health status of personnel, which requires information on low-level exposures. Assessing the risk to troops during a past deployment, such as the Persian Gulf War, requires detailed information about the types of exposures, especially low-level exposures, the nature of the agent(s), and the time profile of
individual exposures. This information, which does not have to be available immediately, can be collected and stored for studies to reconstruct exposures after the deployment.
When troops return to garrison, those responsible for monitoring and maintaining their health status will also need exposure information to determine the appropriate level of health surveillance and medical support. They must also decide how to respond to questions about health complaints and the potential health hazards of deployment operations. Postdeployment medical personnel often need more information on combinations of low-level exposures than on peak concentrations. They need information on a much larger number of individuals and harmful agents than the field commander, whose concerns are survivability, troop performance, and fulfillment of the deployment mission. Long-term dose reconstruction, especially based on low-level exposures to a multitude of agents, requires detailed time-profiles of individual exposures. After deployment, exposure information can be used to determine retrospectively whether soldiers were harmed by exposures to CB and other harmful agents to make informed decisions on long-term medical care and compensation and to resolve legal claims.
As the previous discussion shows, a key distinction in the taxonomy is the phase of the deployment. Two other important distinctions are: (1) the time for health effects to appear, which ranges from immediate effects from acute exposures to long-term and delayed effects; and (2) the agent class of the substance to which troops are exposed, ranging from CB warfare agents to other agents available in the deployment environment, including TICs, endemic biological agents, and background chemical agents (e.g., high levels of naturally occurring metals, such as arsenic, lead, cadmium, etc.) All three distinctions are incorporated into the taxonomy of decisions and exposure information shown in Figure A-1. Each of the 12 cells in the diagram will lead to different decisions and requires different information.
UNCERTAINTY AND THE VALUE OF INFORMATION
The previous section focused on collecting information to improve decision making. This section is focused on measuring improvements in decisions. A first-order approximation of the VOI is the reduction of uncertainties. Figure A-2 shows schematically how information collected by tracking, detection, and monitoring devices influences uncertainties about health effects, a key factor in the decision to take (or not to take) protective action. Of course, these decisions are also influenced by other factors, such as mission objectives and degradation of troop performance as a result of donning protective gear.
Perfect information would completely describe the true state of each variable in each ellipse. In reality, uncertainties can be reduced but not eliminated. As they are reduced, decisions about health effects are likely to improve. However, two important caveats must be kept in mind. First, reducing, or even eliminating, uncertainty in any one of the variables influencing exposure and health effects may not eliminate uncertainty about health effects. Some uncertainties, such as the relationship between doses and responses, may be far more important than other uncertainties. Therefore, reducing uncertainties through tracking, detecting, and monitoring information may not substantially improve decision making.
Second, reducing uncertainty is a necessary, but not sufficient, condition for improving decisions. For example, if a mission always requires that soldiers wear protective gear in an area near a toxic facility (known through predeployment information), knowing more about the nature, concentration, and location of toxic agents in the area will not change this decision. In general, if a decision cannot be changed by the information, the information has no value for that decision.
Example 1. Potential Exposure and Protective Clothing
To determine the VOI, the decision without the information must first be evaluated. The concept of the VOI can be illustrated with a simple decision tree (Figure A-3), which provides a logical structure for evaluating a decision by laying out interim decisions (square nodes), subsequent events (round nodes), and consequences (triangular nodes).
In this example, prior evidence suggests that a harmful chemical agent has been released and is threatening 100 soldiers. It is assumed that the agent is not lethal but will incapacitate the soldiers for five days by causing severe intestinal problems. Based on prior evidence, the probability of the release of the agent is 20 percent. The commander must decide whether or not to order soldiers to don protective clothing.
This decision clearly depends on the consequences of the decision-event combinations. If the commander decides not to order the use of protective clothing and the harmful agent is present, 100 soldiers will be incapacitated for five days, resulting in a loss of 500 soldier days. If the commander decides to order his soldiers to use protective gear, their performance will be degraded, leading to an equivalent loss of soldier days. The equivalent loss is based on the assumption that soldiers would be required to wear protective gear for six hours (25 percent of a day) and that during that time they would only be 50 percent effective. For 100 soldiers,
this would lead to 12.5 "equivalent lost soldier days" (100 × 0.25 × 0.50). The consequences are assigned to the end nodes of the decision tree.
The decision tree is then analyzed by multiplying the consequences (lost soldier days) by their respective probabilities (Figure A-4). As a rule, the best decision minimizes lost soldier days. In this case, the decision to do nothing results in 100 lost soldier days, and the decision to use protective clothing results in only 12.5 lost soldier days. Thus, if there is a 20 percent chance of a real threat, using protective clothing is clearly the best decision.
Now assume that perfect information is available concerning the presence of a harmful agent. Figure A-5 compares a decision based on
perfect information with a decision based on prior evidence, but not perfect information. With perfect information, a commander would know for sure whether or not the agent was present. This scenario is analyzed in Figure A-6. Clearly, if the agent is present, the best decision is to order troops to put on protective gear. The expected lost soldier days with perfect information are now reduced to 2.5 (compared to 12.5 with imperfect information). Thus, perfect information "saves" 10 soldier days. In concrete terms, the commander should be willing to give up 10 of his soldiers for one day or one soldier for 10 days to obtain this perfect information.
Perfect information provides an upper bound to the VOI. Unfortunately, information is never perfect. Figure A-7 shows the situation in which an agent may or may not be present, and the device used to detect the agent is fallible. At any given time, the sample information may either "detect" an agent or "reject" it. Quotation marks indicate that detection can occur even though no agent is present and that rejection can occur even though an agent is present. The marginal probability of "detecting" the agent is q, of rejecting it is 1-q. The conditional probability that an agent is present, given that it is "detected," is r. The marginal probability that an agent is present, given that it is ''rejected," is s. With r < 1 and
s > 0, the detection device is imperfect. Therefore, the commander will still have to make a decision based on some uncertainty about the presence of the agent.
Figure A-7 is the standard textbook example of a decision made with imperfect or "sample" information (e.g., Clemen, 1990; von Winterfeldt and Edwards, 1986). In the current context, we can assume that the commander will order the use of protective clothing if the sample "detects" an agent and that he will do nothing if the sample information "rejects" the presence of an agent. This leads to the somewhat unorthodox but simpler decision tree shown in Figure A-8. Note that in this tree the presence of the agent is considered first and then whether or not the sample detects it. After obtaining the sample information (but without knowing for sure whether or not the agent is present), the commander makes a decision. If the sample information detects an agent, he orders the use of protective clothing; otherwise he does nothing.
Although this representation is unorthodox, it has exactly the same solution as the more conventional tree in Figure A-7, as long as the optimal decision is based solely on the sample information. The quality of the sample information can be characterized by four conditional probabilities:
h, the probability that an agent is present and the device detects it (a "hit")
m, the probability that an agent is present, but the device fails to detect it (a "miss")
f, the probability that an agent is not present, but the device detects it (a "false alarm")
c, the probability that an agent is not present, and the device fails to detect it (a "correct rejection")
Since h = 1 -m and f = 1 -c, detection devices can be characterized by h and f alone. Good devices maximize the probability of hits (h) and minimize the probability of false alarms (f). A device that delivers perfect information has a hit rate of 1 and a false alarm rate of 0. In this example, the hit rate is 0.98, and the false alarm rate is 0.10. Using these rates, Figure A-9 analyzes the decision tree with imperfect information. The decision to obtain imperfect information leads to 5.5 expected lost soldier days (three more than with perfect information), and the difference between imperfect information and no special information is reduced to seven lost soldier days. The reduction of the VOI reflects the hit and false alarm rates.
Example 2. Purchasing Detection Equipment
Figure A-10 shows a simple decision tree for deciding whether or not to purchase a new technology for detecting and monitoring harmful
agents on the battlefield. How valuable is the information provided with this new technology compared to the old technology? The VOI depends on the decision at hand, which has to be made in light of uncertainties about whether or not the agent is present.
To determine the VOI with the new technology, the expected value of the decision to use it must be compared to the expected value of making the decision with the old technology. With the new technology, a threat may occur at a given time with probability p. If the threat occurs, the new technology may detect it (a hit) or miss it with the respective probabilities labeled h(new) and 1-h(new). The new technology may also detect a threat even though it does not exist (false alarm), f(new), or correctly reject the existence of the threat with probabilities 1-f(new).
In the case of a hit, the commander will appropriately order protective action, which will save lives but reduce the effectiveness of the troops. In the case of a miss, no protective action will be taken, and lives will be lost. In the case of a false alarm, the commander will order the use of protective equipment, thus unnecessarily reducing troop effectiveness. In
the case of a correct rejection, no protective action will be taken, no lives will be lost, and the troops will remain fully effective.
The same sequence of decisions and events occurs in the lower part of the decision tree, starting with the decision to use the old technology. The only difference is in the probabilities h(old) and f(old). Presumably, the hit rate, h, is higher for the new technology, and the false alarm rate, f, is lower. These differences in hit and false alarm rates completely determine the differences in the expected value of the old and new technologies and thus determine the incrementally higher VOI of the new technology.
To conduct a formal VOI analysis for real monitoring, detection, and tracking technologies is, of course, impossible. These technologies could be used in many decisions and situations, and assessing the probabilities and consequences for all of them would be extremely difficult. However, a qualitative VOI analysis can be conducted based on the following questions:
What decisions are influenced by the information provided by the old and new technologies?
What are the stakes in the decision (i.e., the range of consequences)?
Would perfect information change the decision?
What are the hit and false alarm rates of the new technology compared to the rates with the old technologies?
For example, if perfect information would not change the decision, the value of an imperfect device will be zero. If perfect information would
change the decision but the range of consequences would be small, the value of an imperfect device would be small. If perfect information could change the decision and the range of consequences is large, the potential VOI for an imperfect device would be large depending on the hit and false alarm rates.
Evaluations with Multiple Attributes
In the preceding examples, the VOI was the only measure used to characterize the consequences of a decision. However, in actual decisions about the use of monitoring, detection, and tracking technologies, other attributes of the devices are also important. These include cost, size, and weight. In deployment situations, devices must be small, lightweight, and unobtrusive to minimize interference in a soldier's performance on the battlefield. Before evaluating these detection and monitoring devices, the attributes that distinguish between them should first be defined. The following attributes may be relevant:
VOI before deployment
VOI during deployment
VOI after deployment
maturity of the technology
size of the device
weight of the device
cost of the device
An evaluation matrix that lays out alternative devices against these attributes should then be filled out with numbers (e.g., for size and cost) and words (e.g., VOI may be characterized as "high" or "low"). This type of matrix is similar to a typical Consumer Report description of the advantages and disadvantages of consumer products.
This matrix may be sufficient to make choices among alternative monitoring, tracking, and detection devices. However, devices that perform well on some attribute (e.g., small size and weight) may not perform well on others (e.g., cost). In these cases, a multiattribute utility analysis can be used to quantify crucial trade-offs and evaluate the alternatives with a single number, their utility (Keeney and Raiffa, 1993). A multiattribute utility analysis can also be combined with a VOI analysis so that all consequences of the alternative devices could be counted at the end of the decision tree. A multiattribute utility analysis would be used to convert the vector of consequences into a single utility number, and the VOI calculation would be based on the utility numbers.
Before undertaking the development of new detection, tracking, or monitoring devices, they should be evaluated in terms of the information they will provide. Evaluations should be based on how the information will be used and the VOI. If the information, even if perfect, is not necessary to a decision, the device should have low priority for development funds. Criteria for evaluating the VOI are the effects and class of agents and the deployment stage of the operation. Other criteria are the maturity of the technology and the size, weight, and cost of the device. Using a matrix to compare alternative devices can help decision makers set priorities.
Clemen, R. 1990. Making Hard Decisions. New York: PWT Kent.
JCS (Joint Chiefs of Staff). 1996. Joint Vision 2010. Washington, D.C.: Joint Chiefs of Staff. Available on line at: http://www.dtic.mil/jv2010/jvpub.htm
Keeney, R.L., and Raiffa, H. 1993. Decisions with Multiple Objectives. New York: Cambridge University Press.
von Winterfeldt, D., and W. Edwards. 1986. Decision Analysis and Behavioral Research. New York: Cambridge University Press.