Application of Cognitive Performance Assessment Technology to Military Nutrition Research
Mary Z. Mays1
Military ration developers are charged with the seemingly impossible task of creating operational rations that are highly palatable, nutrient-dense, shelf-stable, lightweight, fully prepared meals. The rations must be sufficient to fuel maximum intellectual and physical performance in order to sustain combat effectiveness adequately. Thus, a substantial portion of military nutrition research literature is devoted to cognitive performance assessment.
Tests of cognitive performance attempt to quantify the measurable end points of complex intellectual behaviors. Logically, if performance were flawed on one or more of the surrogate markers, it would be an indication that more complex intellectual tasks would be done incompletely, imprecisely, or ineffectively. In a military setting, the intellectual behaviors of interest cover a
wide spectrum, including tasks such as driving a tank, planning a covert reconnaissance operation, conducting medical triage, and handling high explosives.
The history of U.S. Army research on military nutrition can be traced easily from World War II to the present (Askew, 1994; Hirsch and Kramer, 1993; Johnson and Sauberlich, 1982; Lieberman and Shukitt-Hale, 1996; Mays, 1995; Meiselman and Kramer, 1994). Interdisciplinary teams of researchers tested rations, measuring the spectrum of biochemical, physical, psychological, and social parameters on soldiers of all ranks while they were engaged in military field operations (Crowdy et al., 1982; Hirsch et al., 1984; Johnson and Sauberlich, 1982; Shippee et al., 1994; USACDEC/USARIEM, 1986). Such settings limited the type and manner of cognitive performance assessments that were feasible. Assessments typically were made by one of three methods: observation of intact behavior, pencil and paper tests, or automated versions of pencil and paper tests. Occasionally, researchers included tests of manual dexterity and eye-hand coordination in a battery of cognitive performance tests.
In the mid-1980s, military nutrition researchers recognized the need to determine the boundary conditions within which nutritional deficits could be shown to influence cognitive performance. A report by the Committee on Military Nutrition Research (CMNR) clearly articulated the value of cognitive performance assessment in military nutrition research (NRC, 1986). It carefully reviewed methods of assessing cognitive performance by characterizing the breadth and depth of the methods available at that time for investigating the effects of nutritional deficits on memory, reasoning, decision making, and attention. The report provided a detailed discussion of specific test batteries in use in neuropsychology, neurotoxicology, psychopharmacology, and neurobiology.
The potential of the test batteries was never realized. First, the exponential growth of personal computer technology in the late 1980s rendered most of the available test batteries obsolete before they could be disseminated widely and tested for effectiveness in military nutrition research settings. New assessment technology, using computer games or performance tracking devices built into actual or simulated military vehicles and aircraft, made work on refining and tailoring batteries of cognitive performance tests seem dull and unnecessary. Second, the human and financial resources needed to lay the groundwork for a comprehensive study of the effects of nutritional deficits on cognitive performance were simply not available. Identifying those tests that were (1) psychometrically valid surrogates for cognitive components of critical military tasks, (2) appropriately reliable under adverse field conditions, and (3) extraordinarily sensitive to subtle nutritional deficiencies was well beyond the scope of the military nutrition research mission. The emphasis of the CMNR report (NRC, 1986) on the importance of using an empirically validated multivariate approach in the study of complex intellectual behavior prevented a simple choice of one method or test battery over another and left military
nutrition researchers without a clearly attainable goal (Mays, 1995). Furthermore, scientific and technological breakthroughs in other disciplines were changing the focus of military nutrition research (Askew, 1994). Investigations of three different problems, food deprivation, underconsumption, and nutritional neuroscience, are typical of the role of cognitive performance assessment in military nutrition research during the decade since the CMNR report.
TYPICAL MILITARY APPLICATIONS
Behavioral scientists who collaborate with military nutrition researchers have accepted the tremendously difficult task of assessing the cognitive performance of individuals who are (1) normal, not sick; (2) highly trained and motivated, but required to perform under duress; and (3) experiencing an energy deficit, but not nutritionally deficient (Mays, 1995; Meiselman and Kramer, 1994). Under the vast majority of circumstances, the small performance decrements that can be expected to occur under such conditions would be considered inconsequential. However, combat effectiveness requires sustained vigilance, precise reasoning, and prompt decision making under stress. Subtle deficits in these intellectual behaviors can degrade military performance dramatically, resulting in significant morbidity and mortality (Belenky et al., 1994).
Several different concerns motivate military studies of food deprivation. For example, soldiers on covert missions must carry all their supplies on their backs. The operational requirement forces them to make trade-offs between ammunition, radio batteries, food, and water. Researchers must explicitly define the minimal caloric and nutrient requirements of individuals under such conditions. Similarly, the ability of survival rations to sustain intellectual and physical performance under extreme conditions must be well understood. A series of military nutrition studies in the 1960s and 1970s documented the malaise, memory lapses, and inability to concentrate that characterize semistarvation (Consolazio et al., 1967, 1968; Johnson and Sauberlich, 1982; Johnson et al., 1971). There was a renewed interest in this work in the 1990s due to concerns over the rigors of training for Special Operations soldiers (Moore et al., 1992; Shippee et al., 1994). Field research of this type requires assessment technologies that are field hardened, minimally invasive, and amenable to group administration.
Military studies of underconsumption are essential to the development of improved military operational rations. Regardless of how nutritious rations may be, they will not sustain intellectual and physical performance if they are not eaten. Research has documented unequivocally the phenomenon of underconsumption of military rations (Baker-Fulco, 1995; Hirsch, 1995). Unfortunately, researchers have not tested adequately the significance of this underconsumption for cognitive performance (Mays, 1995). In this setting, cognitive performance tests must have well-established norms, very small errors in measurement, high reliability across dynamic environmental conditions, and meticulous sensitivity to nutritional manipulations in particular.
Research in the field of nutritional neuroscience over the last decade has had far-reaching implications for the engineering of military rations (Askew, 1994). It is conceivable that rations could facilitate cognitive performance by increasing or inhibiting the synthesis or destruction of neurotransmitters and receptor proteins (Wurtman, 1994). Tests of performance-enhancing ration components require yet another set of methods for assessing changes in cognitive performance. In many cases, researchers will have to test the potential of performance-enhancing ration components in controlled dose-response studies in the laboratory. They will need to borrow methods from the fields of psychopharmacology and neurotoxicology, which sample behavior over time in order to establish the time course of effects. Simulation and game technology are particularly well suited to this type of investigation.
In summary, the objective of behavioral scientists should not be to design an assessment battery that meets all the needs of military nutrition researchers. It should be to document carefully the level of sensitivity, fidelity, flexibility, field expediency, and normative data associated with the method used in a given investigation. Documenting why a method was chosen and how trade-offs among critical attributes of the method were balanced against the objectives of the investigation is an essential part of evaluating the reliability and relevance of the data. Researchers will need to judge the ability of emerging technologies to provide significant improvements in such attributes.
TWENTY-FIRST CENTURY TECHNOLOGY
The basic methods that behavioral scientists use to measure cognitive performance are not likely to change in the near future (Chouinard and Braun, 1993; Gamberale et al., 1990; Gottschalk, 1994; Iregen and Letz, 1992; Kane, 1991; Kane and Kay, 1992; Matarazzo, 1992; Reitan and Wolfson, 1994;
Retzlaff and Gibertini, 1994; Retzlaff et al., 1992; Turnage et al., 1992; White and Proctor, 1992; White et al., 1994). A thorough search of the recent civilian nutrition literature yielded some interesting data on the influence of dieting and diets on the cognitive performance of adults (Brownell and Rodin, 1994; Green et al., 1994; Heatherton et al., 1993; Riggs et al., 1996; Wing et al., 1995). Considerably more data exist on the influence of restricted diets on the cognitive performance of children and the elderly (Gold, 1995; Lopez et al., 1993; Pollitt, 1995; Rosenberg and Miller, 1992; White and Wolraich, 1995). However, none of the nutritional studies suggested that innovative cognitive performance assessment technologies might emerge in the near future. In contrast, there is little doubt that personal computer hardware and software technology will mature significantly in the early twenty-first century. Emerging technologies in automation have the potential to improve substantially the precision, timeliness, and relevance of cognitive performance assessment.
One of the most productive changes in technology for behavioral scientists in military nutrition research will be the miniaturization of the personal computer's power supply and improvements in battery technology. Increases in durability, safety, and biodegradability and decreases in weight and cost are all feasible (Pen Computing, 1996a). These improvements will increase the field expediency of a number of methods and make true automation of traditional pencil and paper tests worthwhile. If the technological advances in reducing the weight, volume, and cost of power supplies were large, it would permit soldiers to carry the assessment device with them in the field. A handheld device that included a miniature cellular modem and global positioning device would allow researchers to download data from remote sites (Pen Computing, 1996a). The ability to collect data remotely would substantially reduce the invasiveness of cognitive performance assessment and improve the ability to take repeated samples of behavior over time. Having an individual soldier carry a performance assessment device with him might increase the fidelity of assessment. The assessment device could become the analog of the pilot's ''black box," which records key elements of a pilot's performance in a standard format (Maitland and Mandel, 1994; Weinstein, 1995).
An area in which miniaturization will not be useful is the hardware interface between operator and computer. Anyone who has used a calculator-sized microcomputer, such as the Newton™ or the Wizard™, knows the difficulty of doing meaningful work with a tiny keyboard. However, the touch-screen and pen-based graphics interfaces of such products are quite useful (Pen Computing,
1996b). Significant increases in the speed and durability, decreases in the power and light requirements, and improvements in the resolution of the display will enable researchers to design a "natural" interface for a cognitive performance test on an appropriately sized assessment device (Sollenberger and Milgram, 1993; Trautman et al., 1995). A natural interface is one that takes advantage of the techniques humans traditionally use to interact with the world around them, such as three-dimensional imagery, spoken language, and gestures. Thus, a natural interface has two important benefits. It does not require special training, and it increases the speed of input-output processes (both for the soldier and the computer). Additionally, improvements in interfaces will permit researchers to use identical assessment techniques in the laboratory and the field, vastly increasing the relevance and usefulness of both types of investigations (Frohlich et al., 1995; Kenyon and Afenya, 1995; Massimino and Sheridan, 1994; Walker et al., 1993).
For many years, the scientists and engineers tasked with developing improved weapons platforms have been perfecting artificial intelligence software to serve as the operator's "associate" (Haas, 1995). The associate's job is to analyze information and provide it to the operator in ways that sustain optimum performance. Similarly, several personal computer software developers, including both Apple® and Microsoft®, have developed prototype operating systems that function as "intelligent agents," mimicking the attributes of "butlers" or well-trained "pets'' who anticipate the needs of the user and fetch appropriate tools and information (Azar, 1995). The wide acceptance of the World Wide Web has spawned a large industry aimed at producing an "intelligent agent" to roam the Internet's Information Highway and bring back information tailored to the individual computer user's interests (Hawn, 1996; Mobilis, 1996a, b). Future associate programs based on psychological principles will be more powerful than today's clever marketing gimmicks (Doane et al., 1992; Lee and Moray, 1992; Roth et al., 1992; Stanney and Salvendy, 1995). Serious associates will permit the developers of computer-based cognitive performance tests to tailor the tasks and the manner of presentation to individual research subjects and specific research settings (Adelman et al., 1993; DeLucia and Warren, 1994; Ferri, 1995; Kirlik, 1993; Mitta and Packebush, 1995; White and Procter, 1992). Similarly, low-cost, high-speed, high-fidelity handwriting and voice recognition technologies in a handheld computer will permit behavioral science associates to conduct realistic "one-on-one interviews" with subjects in a natural language format (Gottschalk, 1994; Harrington et al., 1995; Moore, 1995; Wilpon, 1995; Wolpaw and McFarland, 1994).
TWENTY-FIRST CENTURY INCENTIVES
Military nutrition researchers are not alone in their desire to improve cognitive performance assessment technology. Vendor shows at meetings of the American Psychological Association make it clear that the development and publication of tests of cognitive performance are a highly competitive industry. Moreover, current changes in the marketplace will stimulate the growth of this industry over the next decade. One of the consequences of health care reform will be an intense interest in precisely quantifying the efficacy of treatment of neuropsychological illness and injury. A consequence of the increasing concern over occupational safety and environmental pollution will be an increased emphasis on developing and using cognitive performance tests that are extraordinarily sensitive to the neuropsychological effects of chronic sleep disturbances and the neurotoxic effects of occupational exposure to hazardous materials. A consequence of educational reform, corporate "right-sizing," and the competitive search for productive diversity in the work force will be improved methods of intelligence and achievement testing. In short, cognitive performance assessment will underlie many measurements of therapeutic efficacy and most decisions concerning job-related selection, training, promotion, and retention in the early twenty-first century.
AUTHOR'S CONCLUSIONS AND RECOMMENDATIONS
The value of cognitive performance assessment in military nutrition research has been clearly established. Military nutrition research continues to have unique needs for cost-effective, field-expedient, and automated versions of cognitive performance tests with well-established military norms and proven validity. These needs will require continued refinement of assessment devices in the twenty-first century. Predictable commercial advances in personal computer software and hardware technologies will have a significant positive impact on the devices used to conduct cognitive performance assessments. Obvious incentives exist for commercial development of emerging assessment technologies by health care corporations, occupational safety and environmental protection regulatory agencies, educational and psychological test publishers, and corporate human resource management consultants. Four conclusions can be derived from the analysis presented. First, significantly more cost-effective, practical, and relevant methods of cognitive performance assessment will be available in the next decade. Second, off-the-shelf technologies suitable for research in the field will be available for use by qualified behavioral scientists. Third, the Department of Defense can ensure that its particular needs are met in a timely fashion, if it systematically supports the development of emerging technologies by commercial firms. Finally, military nutrition researchers will continue to have unique needs for handheld assessment devices that researchers
in the private sector are not likely to develop. Thus, the following recommendations seem appropriate:
Leverage emerging technologies in the computer industry.
Support the development of assessment technologies by commercial firms.
Develop handheld assessment devices compatible with military field operations.
Adelman, L., M.S. Cohen, T.A. Bresnick, J.O. Chinnis, and K.B. Laskey 1993 Real-time expert systems interfaces, cognitive processes, and task performance: An empirical assessment. Hum. Factors 35(2):243–261.
Askew, E.W. 1994 Nutritional enhancement of soldier performance at the U.S. Army Research Institute of Environmental Medicine, 1985–1992. Pp. 65–75 in Food Components to Enhance Performance, An Evaluation of Potential Performance-Enhancing Food Components for Operational Rations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Azar, B. 1995 Meet "Bob": Psychology's answer to easy computing. APA Monit. 26(4):1, 9.
Baker-Fulco, C.J. 1995 Overview of dietary intakes during military exercises. Pp. 121–149 in Not Eating Enough, Overcoming Underconsumption of Military Operational Rations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Belenky, G., D.M. Penetar, D. Thorne, K. Popp, J. Leu, M. Thomas, H. Sing, T. Balkin, N. Wesenten, and D. Redmond 1994 The effects of sleep deprivation on performance during continuous combat operations. Pp. 127–135 in Food Components to Enhance Performance, An Evaluation of Potential Performance-Enhancing Food Components for Operational Rations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Brownell, K.D., and J. Rodin 1994 The dieting maelstrom. Is it possible and advisable to lose weight? Am. Psychol. 49(9):781–791.
Chouinard, M.J., and C.M. Braun 1993 A meta-analysis of the relative sensitivity of neuropsychological screening tests. J. Clin. Exp. Neuropsychol. 15(4):591–607.
Consolazio, C.F., L.O. Matoush, H.L. Johnson, R.A. Nelson, and H.J. Krzywicki 1967 Metabolic aspects of acute starvation in normal humans (10 days). Am. J. Clin. Nutr. 20:672–683.
Consolazio, C.F., L.O. Matoush, H.L. Johnson, H.J. Krzywicki, G.J. Isaac, and N.F. Witt 1968 Metabolic aspects of calorie restriction: Hypohydration effects on body weight and blood parameters. Am. J. Clin. Nutr. 21:793–802.
Crowdy, J.P., C.F. Consolazio, A.L. Forbes, M.F. Haisman, and D.E. Worsley 1982 The metabolic effects of a restricted food intake on men working in a tropical environment. Hum. Nutr. Appl. Nutr. 36A:325–344.
DeLucia, P.R., and R. Warren 1994 Pictorial and motion-based depth information during active control of self-motion: Size-arrival effects on collision avoidance. J. Exp. Psychol. Hum. Percept. Perform. 20(4):783–798.
Doane, S.M., D.S. McNamara, W. Kintsch, P.G. Polson, and D.M. Clawson 1992 Prompt comprehension in UNIX command production. Mem. Cognit. 20(4):327–343.
Ferri, F. 1995 The medical folder as an active tool in defining the clinical decision-making process. Med. Inf. (Lond.) 20(2):97–112.
Frohlich, B., G. Grunst, W. Kruger, and G. Wesche 1995 The responsive workbench: A virtual working environment for physicians. Comput. Biol. Med. 25(2):301–308.
Gamberale, F., A. Iregen, and A. Kjellberg 1990 Computerized performance test in neurotoxicology. Pp. 359–395 in Behavioral Measures of Neurotoxicity, Report of a Symposium, R.W. Russell, P.E. Flattau, and A.M. Pope, eds. U.S. National Committee for the International Union of Psychological Science, Commission on Behavioral and Social Sciences and Education, National Research Council. Washington, D.C.: National Academy Press.
Gold, P.E. 1995 Role of glucose in regulating the brain and cognition. Am. J. Clin. Nutr. 61(4 suppl.):987S–995S.
Gottschalk, L.A. 1994 The development, validation, and applications of a computerized measurement of cognitive impairment from the content analysis of verbal behavior. J. Clin. Psychol. 50(3):349–361.
Green, M.W., P.J. Rogers, N.A. Elliman, and S.J. Gatenby 1994 Impairment of cognitive performance associated with dieting and high levels of dietary restraint. Physiol. Behav. 55(3):447–452.
Haas, M.W. 1995 Virtually augmented interfaces for tactical aircraft. Biol. Psychol. 40:229–238.
Harrington, M.E., R. W. Daniel, and P.J. Kyberd 1995 A measurement system for the recognition of arm gestures using accelerometers. Eng. Med. 209(2):129–134.
Hawn, M. 1996 Seek and ye shall find: Searching the Web. Macworld May:123–127.
Heatherton, T.F., J. Polivy, C.P. Herman, and R.F. Baumeister 1993 Self-awareness, task failure, and disinhibition: How attentional focus affects eating. J. Pers. 61(1):49–61.
Hirsch, E.S. 1995 Short-and long-term studies of food intake in military exercises: An overview. Pp. 151–173 in Not Eating Enough, Overcoming Underconsumption of Military Operational Rations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Hirsch, E.S., and F.M. Kramer 1993 Situational influences on food intake. Pp. 215–243 in Nutritional Needs in Hot Environments, Applications for Military Personnel in Field Operations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Hirsch, E.S., H.L. Meiselman, R.D. Popper, G. Smits, B. Jezior, I. Lichton, N. Wenkham, J. Burt, M. Fox, S. McNutt, M.N. Thiele, and O. Dirige 1984 The effects of prolonged feeding of Meal, Ready-to-Eat (MRE) operational rations. Technical report TR85/035. Natick, Mass.: U.S. Army Natick Research, Development and Engineering Center.
Iregen, A., and R. Letz 1992 Computerized testing in neurobehavioral toxicology. Appl. Psychol. Int. Rev. 41(3):247–255.
Johnson, H.L., and H.E. Sauberlich 1982 Prolonged use of operational rations: A review of the literature. Unnumbered technical report. Presidio of San Francisco, Calif.: Letterman Army Institute of Research.
Johnson, H.L., C.F. Consolazio, H.J. Krzywicki, G.J. Isaac, and N.F. Witt 1971 Metabolic aspects of calorie restriction: Nutrient balance with 500-kilocalorie intakes. Am. J. Clin. Nutr. 24:913–923.
Kane, R.L. 1991 Standardized and flexible batteries in neuropsychology: An assessment update. Neuropsychol. Rev. 2(4):281–339.
Kane, R.L., and G.G. Kay 1992 Computerized assessment in neuropsychology: A review of tests and test batteries. Neuropsychol. Rev. 3(1):1–117.
Kenyon, R.V., and M.B. Afenya 1995 Training in virtual and real environments. Ann. Biomed. Eng. 23(4):445–455.
Kirlik, A. 1993 Modeling strategic behavior in human-automation interaction: Why an "aid" can (and should) go unused. Hum. Factors 35(2):221–242.
Lee, J., and N. Moray 1992 Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10):1243–1270.
Lopez, I., I. deAndraca, C.G. Perales, E. Heresi, M. Castillo, and M. Colombo 1993 Breakfast omission and cognitive performance of normal, wasted and stunted schoolchildren. Eur. J. Clin. Nutr. 47(8):533–542.
Lieberman, H.R., and B. Shukitt-Hale 1996 Food components and other treatments that may enhance mental performance at high altitudes and in the cold. Pp. 453–465 in Nutritional Needs in Cold and in High-Altitude Environments, Applications for Military Personnel in Field Operations, B.M. Marriott and S.J. Carlson, eds. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Maitland, M.E., and A.R. Mandel 1994 A client-computer interface for questionnaire data. Arch. Phys. Med. Rehabil. 75(6):639–642.
Massimino, M.J., and T.B. Sheridan 1994 Teleoperator performance with varying force and visual feedback. Hum. Factors 36(1):145–157.
Matarazzo, J.P. 1992 Psychological testing and assessment in the 21st century. Am. Psychol. 47(8):1007–1018.
Mays, M.Z. 1995 Impact of underconsumption on cognitive performance. Pp. 285–302 in Not Eating Enough, Overcoming Underconsumption of Military Operational Rations , B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine, Washington, D.C.: National Academy Press.
Meiselman, H.L., and F.M. Kramer 1994 The role of context in behavioral effects of foods. Pp. 137–157 in Food Components to Enhance Performance, An Evaluation of Potential Performance-Enhancing Food Components for Operational Rations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
Mitta, D.A., and S.J. Packebush 1995 Improving interface quality: An investigation of human-computer interaction task learning. Ergonomics 38(7):1307–1325.
Mobilis 1996a Interview with General Magic's Jim White (Part I). Mobilis 2(3) URL http://www.volksware.com/mobilis/march.96/interv1.htm.
1996b Interview with General Magic's Jim White (Part II). Mobilis 2(4) URL http://www.volksware.com/mobilis/april.96/interv1.htm.
Moore, R.C. 1995 Integration of speech with natural language understanding. proc. Natl. Acad. Sci. USA 92(22):9983–9988.
Moore, R.J., K.E. Friedl, T.R. Kramer, L.E. Martinez-Lopez, R.W. Hoyt, R.E. Tulley, J.P. DeLany, E.W. Askew, and J.A. Vogel 1992 Changes in soldier nutritional status and immune function during the Ranger training course. Technical Report No. T13-92. Natick, Mass.: U.S. Army Research Institute of Environmental Medicine.
NRC (National Research Council) 1986 Cognitive Testing Methodology, Proceedings of a Workshop Held on June 11–12, 1984. Committee on Military Nutrition Research, Food and Nutrition Board, Commission on Life Sciences. Washington, D.C.: National Academy Press.
Pen Computing 1996a Pen computers in transportation. Pen Computing 3(10):18–33.
1996b Pen computers in healthcare. Pen Computing 3(9):18–35.
Pollitt, E. 1995 Does breakfast make a difference in school? J. Am. Diet. Assoc. 95(10):1134–1139.
Reitan, R.M., and D. Wolfson 1994 A selective and critical review of neuropsychological deficits and the frontal lobes. Neuropsychol. Rev. 4(3):161–198.
Retzlaff, P.D., and M. Gibertini 1994 Neuropsychometric issues and problems. Pp. 185–209 in Clinician's Guide to Neuropsychological Assessment, R.D. Vanderploeg, ed. Hillsdale, N.J.: Lawrence Erlbaum.
Retzlaff, P., M. Butler, and R. Vanderploeg 1992 Neuropsychological battery choice and theoretical orientation: A multivariate analysis. J. Clin. Psychol. 48(5):666–672.
Riggs, K.M., A. Spiro, K. Tucker, and D. Rush 1996 Relationship of vitamin B12, vitamin B6, folate, and homocysteine to cognitive performance in the Normative Aging Study. Am. J. Clin. Nutr. 63(3):306–314.
Rosenberg, I.H., and J.W. Miller 1992 Nutritional factors in physical and cognitive functions of elderly people. Am. J. Clin. Nutr. 55(6 suppl.):1237S–1243S.
Roth, E.M., D.D. Woods, and H.E. Pople 1992 Cognitive simulation as a tool for cognitive task analysis. Ergonomics 35(10):1163–1198.
Shippee, R., K. Friedl, T. Kramer, M. Mays, K. Popp, E.W. Askew, B. Fairbrother, R. Hoyt, J. Vogel, L. Marchitelli, P. Frykman, L. Martinez-Lopez, E. Bernton, M. Kramer, R. Tulley, J. Rood, J. Delaney, D. Jezior, J. Arsenault, B. Nindl, R. Galloway, and D. Hoover 1994 Nutritional and immunological assessment of Ranger students with increased caloric intake. Technical report T95-5. Natick, Mass: U.S. Army Research Institute of Environmental Medicine.
Sollenberger, R.L., and P. Milgram 1993 Effects of stereoscopic and rotational displays in a three-dimensional path-tracing task. Hum. Factors 35(3):483–499.
Stanney, K.M., and G. Salvendy 1995 Information visualization: Assisting low spatial individuals with information access tasks through the use of visual mediators. Ergonomics 38(6):1184–1198.
Trautman, E., M.A. Trautman, and P. Moskal 1995 Preferred viewing distances for handheld and structurally fixed displays. Ergonomics 38(7):1385–1394.
Turnage, J.J., R.S. Kennedy, M.G. Smith, D.R. Baltzley, and N.E. Lane 1992 Development of microcomputer-based mental acuity tests. Ergonomics 35(10):1271–1295.
USACDEC/USARIEM (U.S. Army Combat Developments Experimentation Center and U.S. Army Research Institute of Environmental Medicine) 1986 Combat Field Feeding System-Force Development Test and Experimentation (CFFS-FDTE). Technical Report CDEC-TR-85-006A. Vol. 1, Basic Report; vol. 2, Appendix A; vol. 3, Appendixes B through L. Fort Ord, Calif.: U.S. Army Combat Developments Experimentation Center.
Walker, N., D.E. Meyer, and J.B. Smelcer 1993 Spatial and temporal characteristics of rapid cursor-positioning movements with electromechanical mice in human-computer interaction. Hum. Factors 35(3):431–458.
Weinstein, C.J. 1995 Military and government applications of human-machine communication by voice. Proc. Natl. Acad. Sci. USA 92(22):10011–10016.
White, J.W., and M. Wolraich 1995 Effect of sugar on behavior and mental performance. Am. J. Clin. Nutr. 62(1 suppl.):242S–249S.
White, R.F., and S.P. Proctor 1992 Research and clinical criteria for the development of neurobehavioral test batteries. J. Occup. Med. 34(2)140–148.
White, R.F., F. Gerr, R.F. Cohen, R. Green, M.D. Lezak, J. Lyberger, J. Mack, E. Silbergeld, J. Valciukas, and W. Chappell 1994 Criteria for progressive modification of neurobehavioral batteries. Neurotoxicol. Teratol. 16(5):511–524.
Wilpon, J.G. 1995 Voice-processing technologies—their application in telecommunications. Proc. Natl. Acad. Sci. USA 92(22):9991–9998.
Wing, R.R., J.A. Vasquez, and C.M. Ryan 1995 Cognitive effects of ketogenic weight-reducing diets. Int. J. Obes. Relat. Metab. Disord. 19(11):811–816.
Wolpaw, J.R., and D.J. McFarland 1994 Multichannel EEG-based brain-computer communication. Electroencephalogr. Clin. Neurophysiol. 90(6):444–449.
Wurtman, R.J. 1994 Effects of nutrients on neurotransmitter release. Pp. 239–261 in Food Components to Enhance Performance, An Evaluation of Potential Performance-Enhancing Food Components for Operational Rations, B.M. Marriott, ed. A report of the Committee on Military Nutrition Research, Food and Nutrition Board, Institute of Medicine. Washington, D.C.: National Academy Press.
ROBERT NESHEIM: Thank you very much. I think you were concluding all the way through there, Mary.
MARY MAYS: You asked me for my opinion.
ROBERT NESHEIM: And you have never been reluctant, as I understand. Any comments or questions for Mary?
HARRIS LIEBERMAN: Thanks for that stimulating talk, as always. I have concerns about whether we are really going to be able to use commercial kinds of products most of the time for our cognitive assessment. In general, it seems to me that when we have taken things that are widely used, they get a little bit sloppy and do not exactly answer the question that we are trying to answer.
Maybe there are some large groups of people working on tests, who think about things more from the practical point of view than the theoretical, the way most performance tests and most cognitive tests have been developed up until now.
MARY MAYS: That was the interesting part of doing some homework for this presentation. I was very impressed in interviewing people—their comments were not for attribution, so I do not want to tell you the corporations—but I interviewed these people specifically about that. [I asked them] "Is there an emerging technology or is this simply an improvement on old business?"
I think that you will be really impressed, especially with this notion of managed care. There is a tremendous fight right now among psychologists who
do not want to be managed by physicians [and HMOs], but who need to be a part of that HMO so that they can provide psychological services. And a part of that whole controversy and that debate is the use of diagnostic tests, but these are really performance tests.
What they are trying to diagnose is your ability to cope in the normal world, in the normal environment, and I think what we are going to see come out of this, again as a result of hardware developments, not necessarily good work by psychologists, is a field-hardened, operator-safe kind of assessment battery.
It will not be the traditional pattern, it will not be the kind of thing that the Department of Defense has always discussed. The tests will be very simple things that will be aimed at component behaviors, testing memory, testing attention, testing reasoning, testing analysis in very quick fashion. They will be, just as in a medical model they would be, surrogate markers of something else.
The point is, the commercial firms have the wherewithal and the incentive to insist that those markers are good markers. Several people at this workshop have mentioned the fact that CD4 [a cell-surface antigen on T-helper cells] is not a good marker any more, that we are starting to reject that as a marker of HIV staging and so forth. There probably will be some errors and it may be that the military researchers will discover them; for example, "This is not a very good marker of anything, I would never use this commercial test."
I do not think we should underestimate the industry's ability to put millions of dollars toward this because they expect to make billions of dollars from it.