K
Behavioral-Surveillance Techniques and Technologies

The primary question in behavioral science as applied to the use of behavioral technologies in the antiterrorism effort is, How can detection of particular behaviors and the attendant biological activity be used to indicate current and future acts of terrorism?

K.1
THE RATIONALE FOR BEHAVIORAL SURVEILLANCE

Some behavioral methods attempt to detect terrorist activity directly (for example, through surveillance at bridges, docks, and weapon sites). However, the focus in this appendix is on behavioral methods that are more indirect. Such methods are used to try to detect patterns of behavior that are thought to be precursors or correlates of wrongdoing (such as deception and expression of hostile emotions) or that are anomalous in particular situations (for example, identifying a person who fidgets much more and has much more facial reddening than others in a security line).

Many behavioral-detection methods monitor biological systems (such as cardiac activity, facial expressions, and voice tone) and use physiological information to draw inferences about internal psychological states (for example, “on the basis of this pattern of physiological activity, this person is likely to be engaged in deception”). In most situations, the easiest and most accurate way to determine past, current, and future behavior might be to ask the person what he or she has been doing, is doing, and plans



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 250
K Behavioral-Surveillance Techniques and Technologies The primary question in behavioral science as applied to the use of behavioral technologies in the antiterrorism effort is, How can detection of particular behaviors and the attendant biological activity be used to indicate current and future acts of terrorism? K.1 THE RATIONALE FOR BEHAVIORAL SURVEILLANCE Some behavioral methods attempt to detect terrorist activity directly (for example, through surveillance at bridges, docks, and weapon sites). However, the focus in this appendix is on behavioral methods that are more indirect. Such methods are used to try to detect patterns of behav- ior that are thought to be precursors or correlates of wrongdoing (such as deception and expression of hostile emotions) or that are anomalous in particular situations (for example, identifying a person who fidgets much more and has much more facial reddening than others in a security line). Many behavioral-detection methods monitor biological systems (such as cardiac activity, facial expressions, and voice tone) and use physiologi- cal information to draw inferences about internal psychological states (for example, “on the basis of this pattern of physiological activity, this person is likely to be engaged in deception”). In most situations, the easiest and most accurate way to determine past, current, and future behavior might be to ask the person what he or she has been doing, is doing, and plans 0

OCR for page 250
 APPENDIX K to do. However, the terrorist’s desire to avoid detection and the “cat and mouse” game that is played by terrorists and their pursuers make such a verbal mode of information-gathering highly unreliable. Because verbal reports can be manipulated and controlled so easily, we might turn to biological systems that are less susceptible to voluntary control or that provide detectable signs when they are being manipulated. Once we move to the biological level, however, we have abandoned direct observation of terrorist behavior and moved into the realm of inference of likely behavior from more primitive and less specific sources. Biobe- havioral methods can be powerful and useful, but they are intrinsically subject to three limitations: • Many-to-one. Any given pattern of physiological activity can result from or correlate with a number of quite different psychological or physi- cal states. • Probabilistic. Any detected sign or pattern conveys some likelihood of the behavior, intent, or attitude of interest but not an absolute certainty. • Errors. In addition to the highly desirable true positives and true negatives that are produced, there will be the troublesome false posi- tives (an innocent person is thought to be guilty) and false negatives (a guilty person is thought to be innocent). Depending on the robustness of the biobehavioral techniques involved, it may be possible in the face of countermeasures for a subject to induce false negatives by manipulating his or her behavior. In addition, even if deception or the presence of an emotion can be accurately and reliably detected, information about the reason for deception, a given emotion, or a given behavior is not available from the measurements taken. A person exhibiting nervousness may be excited about meeting someone at the airport or about being late. A person lying about his or her travel plans may be concealing an extramarital affair. A person fidgeting may be experiencing back pain. None of those persons would be the targets of counterterrorist efforts, nor should they be—and the possibility that their true motivations and intents may be revealed has definite privacy implications. K.2 MAJOR BEHAVIORAL-DETECTION METHODS Most behavioral methods are based on monitoring the activity of neural systems that are thought to be difficult to control voluntarily or that reveal measurable signs when they are being controlled.

OCR for page 250
 PROTECTING INDIVIDUAL PRIVACY IN THE STRUGGLE AGAINST TERRORISTS K.2.1 Facial Expression Facial muscles are involved in the expression and communication of emotional states. They can be activated both voluntarily and involun- tarily,1 so there is ample opportunity for a person to interfere with the expression of emotion in ways that serve personal goals. There is strong scientific evidence that different configurations of facial-muscle contrac- tions are associated with what are often called basic emotions.2 Those emotions include anger, contempt, disgust, fear, happiness, surprise, and sadness. There is also evidence that other emotions can be identified on the basis of patterns of movement in facial and bodily muscles (for example, embarrassment3) and that distinctions can be made between genuine felt happiness and feigned unfelt happiness according to whether a smile (produced by the zygomatic major muscles) is accompanied by the contraction of the muscles (orbicularis oculi) that circle the eyes. 4 Facial-muscle activity can be measured accurately by careful examina- tion of the changes in appearance that are produced as the muscles cause facial skin to be moved.5 Trained coders working with video recordings can analyze facial expressions reliably, but it is extremely time-consum- ing (it can take hours to analyze a few minutes of video fully). Greatly simplified methods that focus only on the key muscle actions involved in a few emotions of interest and that are appropriate for real-time screen- ing are being developed and tested. Some basic efforts to develop auto- mated computer systems for analyzing facial expressions have also been undertaken,6 but the problems inherent in adapting them for real-world, naturalistic applications are enormous.7 1 W.E. Rinn, “The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expressions,” Psychological Bulletin 95(1):52- 77, 1984. 2 P. Ekman, “An argument for basic emotions,” Cognition and Emotion 6(3-4):169-200, 1992. 3 D. Keltner, “Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame,” Journal of Personality and Social Psychology 68(3):441-454, 1995. 4 P. Ekman and W.V. Friesen, “Felt, false and miserable smiles,” Journal of Nonerbal Behaior 6(4):238-252, 1982. 5 P. Ekman and W.V. Friesen, Facial Action Coding System, Consulting Psychologists Press, Palo Alto, Calif., 1978. 6 J.F. Zlochower, A.J. J. Lien, and T. Kanade, “Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding,” Psychophysiology 36(1):35- 43, 1999. 7 For example, according to a German field test of facial recognition conducted in 2007, an accuracy of 60 percent was possible under optimal conditions, 30 percent on average (depending on light and other factors). See Bundeskriminalamt (BKA), Face Recognition as a Tool for Finding Criminals: Picture-man-hunt, Final report, BKA, Wiesbaden, Germany, Febru- ary 2007. Available in German at http://www.cytrap.eu/files/EU-IST/2007/pdf/2007-07- FaceRecognitionField-Test-BKA-Germany.pdf.

OCR for page 250
 APPENDIX K There are several other ways to measure facial-muscle activity. The electrical activity of the facial muscles themselves can be measured (with electromyelography [EMG]). That requires the application of many elec- trodes to the face, each placed to maximize sensitivity to the action of particular muscles and minimize sensitivity to the action of other muscles. Because of the overlapping anatomy of facial muscles, their varied sizes, and their high density in some areas (such as around the mouth), the EMG method may be better suited to simple detection of emotional valence (positive or negative) and intensity than to the detection of specific emo- tions. Another indirect method of assessing facial-muscle activity is to measure the “heat signature” of the face associated with changes in blood flow to different facial regions.8 That information can be read remotely by using infrared cameras; however, the spatial and temporal resolutions are problematic. Even if a method emerged that allowed facial-muscle activity to be measured reliably, comprehensively, economically, and unobtrusively, there would be the issue of its utility in a counterterrorism effort. Neces- sary (but not sufficient) conditions for utility would include: • The availability of tools that can determine the specific emotion that is being signaled if and when emotional facial expression is displayed. • The superiority of a facial-expression–based emotion-predic- tion system to a system based on any other biological or physiological markers. • The detectability of indicators that a person is attempting to con- ceal his or her true emotional state or to shut down facial expression entirely, such as Small, fleeting microexpressions of the emotion being felt. Tell-tale facial signs of attempted control (such as tightening of some mouth muscles). Signs that particular emotions are being simulated. Characteristic increases in cardiovascular activity mediated by the sympathetic branch of the autonomic nervous system.9 Because no specific facial sign is associated with committing or plan- 8 See, for example D.A. Pollina and A. Ryan, The Relationship Between Facial Skin Surface Temperature Reactiity and Traditional Polygraph Measures Used in the Psychophysiological Detec - tion of Deception: A Preliminary Inestigation, U.S. Department of Defense Polygraph Institute, Ft. Jackson, S.C., 2002. 9 J.J. Gross and R.W. Levenson, “Emotional suppression: Physiology, self-report, and ex- pressive behavior,” Journal of Personality and Social Psychology 64(6):970-986, 1993; J.J. Gross and R.W. Levenson, “Hiding feelings: The acute effects of inhibiting negative and positive emotion,” Journal of Abnormal Psychology 106(1):95-103, 1997.

OCR for page 250
 PROTECTING INDIVIDUAL PRIVACY IN THE STRUGGLE AGAINST TERRORISTS ning a terrorist act, using facial measurement in a counterterrorism effort will have to be based on some combination of the detection of facial expressions thought to indicate malevolent intent (such as signs of anger, contempt, or feigned happiness in some situations), the detection of facial expressions thought to indicate deception,10 and the detection of facial expressions that are anomalous compared with those of other people in the same situation. Results of research on the connection between facial expression and emotional state suggest correlations between the two. However, the sug- gestive findings have generally not been subject to rigorous, controlled tests of accuracy in a variety of settings that might characterize real-world application contexts. K.2.2 Vocalization In addition to the linguistic information carried by the human voice, a wealth of paralinguistic information is carried in pitch, timbre, tempo, and the like and is thought to be related to a person’s emotional state. 11 Those paralinguistic qualities of speech can be difficult to control voluntarily, so they are potentially useful for detecting underlying emotional states and deception. In the emotion realm, much of the promise of mapping paralinguistic qualities of vocalization onto specific emotions has yet to be realized, and the history of using paralinguistic markers in the deception realm is not very encouraging. At one time, a great deal of attention was given to the detection of deception by quantifying microtremors in the voice (“voice stress analyzers”), but this approach has failed to withstand scientific scrutiny.12 Why has more progress not been made in using paralinguistic quali- ties of speech to detect emotions and deception? There are several pos- sible reasons. The relationships between paralinguistic qualities of speech and psychological states are much weaker than originally thought. The field has not yet identified the right characteristics to measure. And siz- 10 P. Ekman and M. O’Sullivan, “From flawed self-assessment to blatant whoppers: The utility of voluntary and involuntary behavior in detecting deception,” Behaioral Sciences and the Law. Special Issue: Malingering 24(5):673-686, 2006. 11 K.R. Scherer, Vocal Measurement of Emotion, Academic Press, Inc., San Diego, Calif., 1989. 12 See, for example, National Research Council, The Polygraph and Lie Detection, The Nation- al Academies Press, Washington, D.C., 2003; Mitchell S. Sommers, “Evaluating voice-based measures for detecting deception,” The Journal of Credibility Assessment and Witness Psychology 7(2):99-107, 2006, available at http://truth.boisestate.edu/jcaawp/2006_No_2/2006_99-107. pdf; J. Masip, E. Garrido, and C. Herrero, “The detection of deception using voice stress analyzers: A critical review,” Estudios de PsicologÃa 25(1):13-30, 2004.

OCR for page 250
 APPENDIX K able individual differences in speech need to be accounted for before interindividual consistencies will emerge. In the interim, new approaches that do not rely on paralinguistic vocalizations in isolation but rather combine them with other indicators of deception and emotion (such as facial expressions and physiological indicators) may prove useful. Is is ironic that it is fairly simple to obtain high-quality, noninvasive samples of vocalizations in real-world contexts. Moreover, cost-effective, accurate instrumentation for analyzing the acoustic properties of speech is readily available. Thus, the tools are already in place; it is just the science that is lagging. K.2.3 Other Muscle Activity Technology is readily available for quantifying the extent of overall motor activity (sometimes called gross motor activity or general somatic activity). It can be done with accelerometers attached to a person (some are built into watch-like casings) or with pressure-sensitive devices (such as piezoelectric transducers) placed under standing and sitting areas. The latter can be used to track motion in multiple dimensions and thus enable characterization of patterns of pacing, fidgeting, and moving. Although clearly not specifically related to any particular emotional or psychologi- cal state, high degrees of motor activity may be noteworthy when they are anomalous in comparison with usual levels of agitation and tension. K.2.4 Autonomic Nervous System The autonomic nervous system (ANS) controls the activity of the major organs, including the heart, blood vessels, kidneys, pancreas, lungs, stomach, and sweat glands. Decades of methodological development in medicine and psychophysiology have produced ways to measure a wide array of autonomic functions reliably and noninvasively. Some of the measures are direct (such as using the electrical activity of the heart muscle to determine heart rate), and some are indirect (such as estimating vascular constriction by using the reflection of infrared light to determine the amount of blood pooling in peripheral sites or using impedance meth- ods to measure the contractile force of the left ventricle as it pumps blood from the heart to the rest of the body). Additional work has been directed toward developing methods of ambulatory monitoring that enable track- ing of ANS activity in freely moving people. Remote sensing of autonomic function is still in its infancy, but some progress has been made in using variation in surface temperature to indicate patterns of blood flow. Measures of ANS activity are essentially measures of arousal and reflect the relative activation and deactivation of various organ systems to

OCR for page 250
 PROTECTING INDIVIDUAL PRIVACY IN THE STRUGGLE AGAINST TERRORISTS provide the optimal milieu to support current body activity (such as sleep, digestion, aggression, and thinking). Debate has raged over the decades as to whether specific patterns of autonomic activity are associated with particular psychological states, including emotions. For emotion, the issue is whether the optimal bodily milieu for anger (ANS support for fight- ing) is different from that for disgust (ANS support for withdrawal and expulsion of harmful substances). Evidence in support of that kind of autonomic specificity for at least some of the basic emotions is drawn from experimentation, metaphors found in language (such as association of heat and pressure with anger or of coolness with fear), and observable signs of autonomic activity (such as crying during sadness but not during fear or gagging during disgust but not during angers). There are a number of reviews of these issues and the associated scientific evidence. 13 Over the years, patterns of ANS activity have been mapped onto several nonemotionl states. Among the more durable of them have been the distinction between stimulus intake (elevated skin conductance plus heart rate deceleration) and stimulus rejection (elevated skin conductance plus heart rate acceleration),14 and the more recent distinction between the cardiovascular responses to threat (moderate increases in cardiac contractility, no change or decrease in cardiac output, and no change or increase in total peripheral resistance) and to challenge (increase in cardiac contractility, increase in cardiac output, and decrease in total peripheral resistance).15 Regardless of the putative pattern, using the existence of any particu- lar pattern of ANS activity by itself to infer psychological or emotional states is fraught with danger. The ANS is the slave to many masters, and any ANS pattern may reflect any of a host of nonpsychological and psy- chological states. The other way in which ANS monitoring has been used extensively is to detect deception. The use of autonomic measurement in lie-detection technology has a long history in law enforcement, security screening, and personnel selection. Despite its history (which continues), most of the major scientific investigations of the validity of the polygraph have raised serious reservations. For example, an independent review of the 13 R.W. Levenson, “Autonomic specificity and emotion,” pp. 212-224 in Handbook of Affec- tie Sciences, R.J. Davidson, K.R. Scherer, and H.H. Goldsmith, eds., Oxford University Press, New York, N.Y., 2003. 14 J.I. Lacey, J. Kagan, B.C. Lacey, and H.A. Moss, “The visceral level: Situational determi- nants and behavioral correlates of autonomic response patterns,” pp. 161-196 in Expression of the Emotions in Man, P.H. Knapp, ed., International University Press, New York, N.Y., 1963. 15 J. Tomaka, J. Blascovich, R.M. Kelsey, and C.L. Leitten, “Subjective, physiological, and behavioral effects of threat and challenge appraisal,” Journal of Personality and Social Psychol- ogy 65(2):248-260, 1993.

OCR for page 250
 APPENDIX K use of the polygraph commissioned by the Office of Technology Assess- ment concluded that16 there is at present only limited scientific evidence for establishing the validity of polygraph testing. Even where the evidence seems to indi- cate that polygraph testing detects deceptive subjects better than chance (when using the control question technique in specific-incident criminal investigations), significant error rates are possible, and examiner and examinee differences and the use of countermeasures may further affect validity. (p. 96) In 2003, a review by the National Research Council was similarly critical,17 concluding that the polygraph has a better than chance but far less than perfect performance in detecting specific incidents of deception but that it is not acceptable for use in general screening and is highly vul- nerable to countermeasures. In considering the use of the polygraph in antiterrorism efforts, it is important to weigh its possible utility in “guilty knowledge” situations (for example, the person being interrogated is denying knowing something that he or she knows) against the likelihood that the person will be trained in using countermeasures. Empirically, “guilty knowledge” studies indicate that the polygraph confers at best a minimal advantage in identifying such situations and suggest that guilty parties may not need to take countermeasures at all to evade detection by a polygraph. K.2.5 Central Nervous System The brain is clearly the source of motivated behavior—both good and evil. Thus, measuring brain activity is appealing if the goal is to detect intentions, motives, planned behaviors, allegiances, and a host of other mental states related to terrorism and terrorist acts. The electrical activity of the brain can be measured directly with electroencephalography (EEG) and indirectly with such methods as magnetoencephalography (MEG, which detects changes in magnetic fields produced by the brain’s electri- cal activity), positron-emission tomography (PET, which uses radioactive markers to track blood flow into the brain areas that are most active), and functional magnetic resonance imaging (fMRI, which uses strong mag- netic fields to detect changes in the magnetic properties of blood flowing 16 Office of Technology Assessment, Scientific Validity of Polygraph Testing: A Research Reiew and Ealuation, U.S. Congress, Office of Technology Assessment, Washington, D.C., 1983. 17 National Research Council, The Polygraph and Lie Detection, The National Academies Press, Washington, D.C., 2003.

OCR for page 250
 PROTECTING INDIVIDUAL PRIVACY IN THE STRUGGLE AGAINST TERRORISTS through the brain that occur when active brain areas use the oxygen car- ried by red blood cells). The electrical activity of the brain can be monitored using these tech- nologies while an individual is undertaking fairly complex behavioral activities and can sometimes be linked to particular discrete stimulus events. An overarching goal of the research using these methods has been to understand how and where in the brain such basic mental activities as error detection, conflict monitoring, emotion activation, and behavioral regulation occur. In most brain research, the focus has been more on spe- cific cognitive processes than on specific emotions. Some patterns of brain activity can be used to predict when a person is experiencing emotion but not the particular emotion. In addition to emotional activation, some pat- terns indicate attempts at emotion regulation and control.18 Each of the existing measures of brain activity has advantages and disadvantages in temporal resolution, spatial resolution, invasiveness, susceptibility to movement artifact, methodological requirements, and expense. Much of the current excitement in the field is focused on fMRI. Viewed from the perspective of counterterrorism, fMRI presents numer- ous challenges: subjects must be supine in a tube for a long period (typi- cally 15 minutes to 2 hours), temporal resolution is low, and the method is highly vulnerable to movement artifacts (movements greater than 3 mm can result in unusable images). Although the committee heard testimony about detection of deception with fMRI, the paucity of research support- ing it and the considerable constraints associated with it make it difficult to imagine its having any immediate antiterrorism utility. K.3 ASSESSING BEHAVIORAL-SURVEILLANCE TECHNIQUES Proponents and advocates (especially vendors) often seek to dem- onstrate the validity of a particular approach to behavioral surveillance or deception detection by presenting evidence that it discriminates accu- rately between truthfulness and deception in a particular sample of exam- inees. Although such evidence would be necessary to accept claims of validity, it is far from sufficient. The 2003 National Research Council report on the polygraph and lie detection19 provided a set of questions that guide the collection of cred- ible evidence to support claims of validity of any proposed technique for deception detection and a set of characteristics of high-quality studies 18 K.N. Ochsner and J.J. Gross, “The cognitive control of emotion,” Trends in Cognitie Sci- ences 9(5):242-249, 2005. 19 National Research Council, 2003, op. cit.

OCR for page 250
 APPENDIX K that address issues of accuracy. Those questions and characteristics are presented in Box K.1. K.4 BEHAVIORAL AND DATA MINING METHODS: SIMILARITIES AND DIFFERENCES Behavioral and data mining methods have many similarities and some key differences. Perhaps most important, they face many of the same challenges and can both be evaluated in the overall framework presented in this report. These are some characteristics that are common to the two methods: • Probabilistic. Data mining and behavioral surveillance seek patterns that are likely to be associated with terrorist acts. Successful methods will need to have high rates of true positives and true negatives and low rates of false positives and false negatives. Because of the low base rate of terrorism in most contexts (for example, in airport security lines), both methods will detect many acts of malfeasance that are not directly related to terrorism (for example, acts by people who have committed or are plan- ning other crimes). The value and cost of these “true positives of another sort” must be considered in evaluating any applications of the methods. • Remote and secret monitoring. Data mining and some kinds of behav- ioral surveillance allow information to be collected and analyzed without direct interaction with those being monitored. • Countermeasures. Data mining and behavioral surveillance are vul- nerable to countermeasures and disinformation. • Gateways to human judgment. Data mining and behavioral surveil- lance may best be viewed as ways to identify situations that require fol- low-up investigation by skilled interviewers, analysts, and scientists. • Priacy. Data mining and behavioral surveillance raise serious con- cerns for the protection of individual liberties and privacy. • Need for prior empirical demonstration. Data mining and behavioral surveillance should deployed operationally on a wide scale only after their utility has been empirically demonstrated in the laboratory and on a limited scale in operational contexts. • Need for continuing ealuation. The use of data mining and of behav- ioral surveillance should be accompanied by a continuing process of evaluation to establish utility, accuracy and error rates, and violation of individual privacy. The following are some of the important differences: • Collection ersus analysis. Techniques for detecting deception require

OCR for page 250
0 PROTECTING INDIVIDUAL PRIVACY IN THE STRUGGLE AGAINST TERRORISTS BOX K.1 Questions for Assessing Validity and Characteristics of Accurate Studies Questions for Assessing Validity • Does the technique have a plausible theoretical rationale, that is, a proposed psychological, physiological, or brain mechanism that is consistent with current physi- ological, neurobiological, and psychological knowledge? • Does the psychological state being tested for (deception or recognition) reliably cause identifiable behavioral, physiological, or brain changes in individuals, and are these changes measured by the proposed technique? • By what mechanisms are the states associated with deception linked to the phenomena the technique measures? • Are optimal procedures being used to measure the particular states claimed to be associated with deception? • By what mechanisms might a truthful response produce a false positive result with this technique? What do practitioners of the technique do to counteract or correct for such mechanisms? Is this response to the possibility of false positives reasonable considering the mechanisms involved? • By what means could a deceptive response produce a false negative result? That is, what is the potential for effective countermeasures? What do practitioners of the technique do to counteract or correct for such phenomena? Is this response to the possibility of false negatives and effective countermeasures reasonable considering the mechanisms involved? • Are the mechanisms purported to link deception to behavioral, physiological, or brain states and those states to the test results universal for all people who might be examined, or do they operate differently in different kinds of people or in different situa- tions? Is it possible that measured responses do not always have the same meaning or that a test that works for some kinds of examinees or situations will fail with others? • How do the social context and the social interactions that constitute the examina- tion procedure affect the reliability and validity of the recordings that are obtained? • Are there plausible alternative theoretical rationales regarding the underlying mechanisms that make competing empirical predictions about how the technique per- forms? What is the weight of evidence for competing theoretical rationales? Research Methods for Demonstrating Accuracy • Randomized experimentation. In analog studies, this means that examinees are randomly assigned to be truthful or deceptive. It is also useful to have studies in which examinees are allowed to decide whether to engage in the target behavior. Such studies gain a degree of realism for what they lose in experimental control. • Manipulation checks. If a technique is claimed to measure arousal, for example, there should be independent evidence that experimental manipulations actually create different levels of arousal in the different groups.

OCR for page 250
 APPENDIX K • Blind administration and blind evaluation of the technique, Whoever administers and scores tests based on the technique must do so in the absence of any information on whether the examinee is truthful or deceptive. • Adequate sample sizes. Most of the studies examined [in the National Research Council 2003 polygraph report] were based on relatively small sample sizes that were sometimes adequate to allow for the detection of statistically significant differences but were insufficient for accurate assessment of accuracy. Changing the results of only a few cases might dramatically affect the implications of these studies. • Appropriate comparison conditions and experimental controls. These conditions and controls will vary with the technique. A suggestion of what may be involved is the idea in polygraph research of comparing a polygraph examination with a bogus poly- graph examination, with neither the examiner nor the examinee knowing that the test output might be bogus. • Cross-validation of any exploratory data analytic solution on independent data. Any standardized or computerized scoring system for measurements from a technique cannot be seriously considered as providing accurate detection unless it has been shown to perform well on samples of examinees different from those on whom it was developed. • Examinees masked to experimental hypotheses if not to experimental condi- tion. It is important to sort out precisely what effect is being measured. For example, the results of a countermeasures study would be more convincing if examinees were instructed to expect that the examiner is looking for the use of countermeasures, among other things, rather than being instructed explicitly that this is a study of whether coun- termeasures work and can be detected. • Standardization. An experiment should have sufficient standardization to allow reliable replication by others and should analyze the results from all examinees. It is important to use a technique in the same way on all the examinees, which means: clear reporting of how the technique was administered; sharply limiting the examiner’s discre- tion in administering the technique and interpreting its results; and using the technique on all examinees, not only the ones whose responses are easy to classify. If some examinees are dropped from the analysis, the reasons should be stated explicitly. This is a difficult test for a procedure to pass, but it is appropriate for policy purposes. • Analysis of sensitivity and specificity or their equivalents. Data should be re- ported in a way that makes it possible to calculate both the sensitivity and specificity of the technique, preferably at multiple thresholds for diagnostic decision making or in a way that allows comparisons of the test results with the criterion on other than binary scales. SOURCE: National Research Council, The Polygraph and Lie Detection, The National Academies Press, Washington, D.C., 2003, pp. 222-224.

OCR for page 250
 PROTECTING INDIVIDUAL PRIVACY IN THE STRUGGLE AGAINST TERRORISTS the collection of physiological and biological data, whereas data-mining is a technique for analyzing already-collected data. • Degree of intrusieness. Traditional jurisprudence and ethics gener- ally regard a person’s body as worthy of a higher degree of protection than his or her information, residences, or possessions. Thus, techniques that require the collection of physiological and biological data (especially data relevant to one’s thoughts) are arguably more intrusive than collec- tion schemes directed at different kinds of personal data.