Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
The Current Status of Test Development in Neurobehavioral Toxicology Ann M. Williamson There has been a proliferation in the number of tests used in neurobehavioral toxicology in recent years and an increase in the number of groups producing test batteries. Nevertheless, the area still remains a difficult one despite the increased interest in it, and many questions still remain unanswered. Neurobehavioral methods have been used to examine an increas- ing number of toxicants, but the impact of the findings from such studies has differed considerably across countries. Evidence from neurobehavioral testing has been highly influential in lowering acceptable health standards for lead and solvent exposure, for example, in Scandinavian countries, but it has had little effect in many other countries (e.g., Australia and Britain). Political forces no doubt play a large part in these differences, but the fact remains that evidence from neurobehavioral tests is simply not convincing to many decision makers in the latter countries. The reasons for this are the problems encountered in neurobehavioral testing, namely, problems of selection of controls; accounting for confounding variables such as alcohol, drug use, education, age, and socioeconomic status; selection of sensitive and comprehensive tests; and quantifying exposure. It is apparent that these problems are often regarded as sufficient evidence to reject an overwhelming weight of evidence that would otherwise be seen as convincing. For occupational lead exposure, for example, a review of the lit- erature since 1980 demonstrates that of the 14 or so papers published 56
CURRENT STATUS OF TEST DEVELOPMENT 57 on neurobehavioral effects, all but 2 show statistically significant im- pairments in some tests in lead workers compared to controls. In all studies, lead exposure levels were within the subclinical range (i.e., below 3.8 ~mol/L). Despite this, critics of the field tend to "throw the baby out with the bathwater" and concentrate their attention on the undeniable flaws in most of the studies without looking at the literature as a whole. It therefore becomes essential to ensure that neurobehavioral testing for toxic effects is as rigorous as possible if it is to have a significant impact on decision makers. RESEARCH DESIGN AND TEST SELECTION Although the major factor of interest here is the choice of tests, it is not really possible to divorce questions of experimental design from those relating to test selection. A number of workers have reviewed the study design problem for neurobehavioral toxicology (Gamberale, 1985; Valciukas and Lilis, 1980~. Virtually all studies in this area are cross sectional, in which exposed workers are compared to nonexposed workers on their performance of a battery of neurobehavioral tests. Test selection is a problem in this type of design because many tests are sensitive to extraneous differences between exposed and nonexposed workers such that they contribute to, and potentially confound, the finding of differences in test performance. For example, in a study by Parkinson et al. (1986), when the effects of age, education, and income were removed statis- tically, significant differences between lead-exposed workers and controls on some neurobehavioral tests disappeared. Problems of confounding in this design are usually dealt with by matching exposed and control groups or by statistical means. Appro- priately chosen tests however can produce the same effect through the selection of tests that are not vulnerable to the effects of confounding factors such as age, education, or ethnic background, at least for working populations. This has not been done in any study to date. It is common though for researchers to investigate the effect of putative confounding variables prior to using various statistical techniques to minimize confounding (Hogstedt et al., 1983; Valciukas and Lilis, 1982~. For the few prospective cohort studies done in this area, the prob- lem of test selection lies in selecting tests that are not susceptible to the effects of practice. Because workers are tested on more than one occasion, it is essential that test-retest results are not muddied by the fact that workers will improve from one test session to another simply because they have seen the test before. The results of the single
58 ANN M. WILLIAMSON prospective study performed on occupational lead exposure (Mantere et al., 1984), for example, were less convincing because of strong training effects on the performance of a number of tests. REASONS FOR TEST SELECTION i In most studies a set or battery of psychological or behavioral tests s used. Typically, the tests are chosen for one or more of the four following reasons: 1. The test is a well-standardized psychological test for which the distribution of scores in the population is already established, e.g., subtests from the Wechsler Adult Intelligence Scale (WAIS; Wechsler, 1955~. 2. The tests will measure some aspects of functioning that are known to be influenced by the toxic~substance based on clinical evi- dence; e.g., knowing that inorganic mercury exposure produces un- intentional hand tremor would result in a tremor test or some other motor test being included in the test battery. 3. Each test corresponds roughly to a particular psychological function, e.g., including the Santa Ana test for motor functions, the Digit Sym- bol test for attention, and Digit Span test for memory functions. 4. Tests may be chosen on some theoretical grounds such that each test is juxtaposed against another in some logical manner (Williamson et al., 1982~. In some studies however, no rationale is provided for the choice of tests in a particular battery. Various test batteries are currently employed for a range of neurotoxic substances, and there are a number of reviews that describe them (Anger, 1984, 1986; Gullion and Eckerman, 1986; Johnson and Anger, 1983~. Some of these batteries, such as that devised at the Finnish Institute of Occupational Health (Hanninen and Lindstrom, 1979), have been used in multiple studies of a range of toxic hazards. Con- sequently, some estimate can be made about their sensitivity. Many batteries, however, are simply put together for a single purpose, and if no rationale is provided for test selection, the reliability, validity, and sensitivity are largely unknown, particularly if no sound rationale exists from previous work. RECENT APPROACHES TO TEST BATTERY DESIGN There have been three new approaches to test battery design in the last few years, each of which focuses on a different aspect of battery design.
CURRENT STATUS OF TEST DEVELOPMENT Neurobehavioral Core Test Battery 59 The Neurobehavioral Core Test Battery (NCTB) battery was devel- oped at a meeting of a group of experts from a range of disciplines as a joint initiative of the World Health Organization (WHO) and the National Institute for Occupational Safety and Health (NIOSH). The aim of the meeting was to devise a test battery that could be used to screen for neurotoxic effects with particular reference to its use in developing countries. Although the rationale for test selection was not much different from that used in many other studies, this initiative constitutes a significant leap forward because it is an attempt to set up norms for each test that are applicable to comparisons between and within cultural boundaries. It is argued that by applying the test battery in a range of countries it should be possible to estimate the influence of cultural differences on test performance. Moreover, on a more practical level, such cross-cultural comparison should allow for better interpretations between studies performed in different countries. The Computerized Battery Approach The computerized battery approach capitalizes on the recent boom in personal computer development by designing a battery that is administered by computer. Two advantages of computerized testing are that test administration is standardized and reproducible, requiring minimal involvement by the test administrator, and that data handling and scoring are made easier so that the results can be reported imme- diately. Two main research groups have developed computer-administered test batteries. Probably the most well known is the battery devel- oped by Baker and Letz at Harvard (Baker et al., 1985~. After some standardization procedures, the final test battery includes three memory tests from the Wechsler Memory Scale (WMS) or Wechsler Adult Intelligence Scale (WAIS); two tests classified as measuring verbal concept formation both from the WAIS; four visuomotor tests two of which are from the WAIS; and a mood scale. Three of these tests are from the NCTB. This battery has been subjected to some validation (Baker et al., 1983), although the description of the investigation of test reliability is not clear (Fidler et al., 1987~. Some estimates can be made as to the sensitivity of this battery because it has been used to show impairments in both lead-exposed (Baker et al., 1984) and solvent-exposed work- ers (Fidler et al., 1987~. The second well-known computer-assisted battery was developed
60 ANN M. WILLIAMSON at the University of North Carolina by Foree et al. (1984). This bat- tery (also known as the Microtox battery) was based on the work of Carroll (1980), who proposed a theory of 10 factors to represent the range of cognitive abilities. Carroll holds that all test performance is partitionable into smaller building blocks which he called elementary cognitive tasks (ECTs). The battery consists of three sensory tests, two psychomotor tests, two attention tests, eight memory tests, and one test that is classified as "other." Investigations of validity and reliability were performed by Carroll in the development of the theory. The sensitivity of the battery has been evaluated to some extent in studies of the effects of carbon monoxide and alcohol (Force et al., 1984~. Mode! or Theory-Based Tests and Test Batteries A few test batteries have some theory of psychological function as their basis. The aim of this approach is to facilitate interpretation of results. Smith and Langolf (1981), for example, take the view that tests selected for a battery should be ability-specific and have some underlying theoretical structure. This, they argue, allows interpretation to be made in terms of the processing stages or systems that produce performance on a test; furthermore, scores can be given for individual stages in processing. The argument advanced by Smith and Langolf appears to rest mostly on their use of the Sternberg memory scanning test (Sternberg 1966, 1975), which has a well-developed theoretical basis. There has been considerable debate, however, on the adequacy of Sternberg's theory to account for all aspects of performance on this test. Gullion and Eckerman (1986) state that this debate is sufficient to make unwarranted any inferences about the intactness of underlying cognitive processes on the bases of performance on the memory scanning task. Rather than choosing particular tests that have well-developed theoretical structures, two research groups have employed theory- based test batteries. The first is the Microtox test battery described above. In this battery, Carroll's elementary cognitive task theory (Carroll, 1980), is used to guide test selection. The second is a battery devised by Williamson and colleagues (Williamson and Teo, 1986; Williamson et al., 1982, 1987), in which the choice of tests is based on information- processing theory (Wickens, 1984, 1987~. This battery includes a sen- sory test, three psychomotor tests, a sustained attention test, and four memory tests: a sensory store memory test, two short-term or working memory tests, and a long-term memory test. Validation has
CURRENT STATUS OF TEST DEVELOPMENT 61 been carried out on this test battery, and some limited studies of reliability have been done, but these results have not been published. The sensitivity of this battery has also been evaluated in detecting effects of inorganic mercury (Williamson et al., 1982), inorganic lead (Williamson and Teo, 1986), and prolonged exposure to the underwater environment (Williamson et al., 1987~. Model or theory-based test batteries have the advantage of provid- ing a comprehensive coverage of neurobehavioral functions that is often missing from other approaches to battery design. Screening test batteries in particular should be designed on this basis. For screening purposes, battery design should proceed as if nothing is known about the neurobehavioral effects of the toxin in question because clinical symptoms may be very misleading. For example, a commonly reported symptom in toxically exposed workers is fatigue (Fidler et al., 1987; Hanrunen et al., 1979; Valciukas et al., 1979~. However, a researcher would have great difficulty selecting an appropriate test to investigate this symptom further unless a holistic or theoretical approach was taken in designing the test battery. Clinically manifested fatigue can have mental or physical origins so which should be tested for? In addition, if, for example, the fatigue is due to problems in maintaining mental effort, it would be impossible to determine which of the functional areas or stages of processing is responsible. Unless all possibilities are tested no clear conclusions can be made regarding the action of the supposed toxin. Another difficulty with designing a battery without some global structure is that interpretation of test results can be made only for single tests. For example, a typical study of the effects of lead expo- sure in which the battery included tests of a range of neurobehavioral functions may have found impairments in reaction time, learning, and memory functions in lead workers, compared to nonexposed controls, but no apparent impairment of other functions. In this case the- con- clusion would be made that lead exposure affects motor, learning, and memory functions. If, however, the tests could be related on the basis of a cohesive theory, the interpretation might be very different. For example, in the study of lead exposure by Williamson and Tea (1986) the clustering of performance impairments seen in lead work- ers compared to controls suggested that sensory motor, learning, sensory store, and short-term but not long-term memory functions were affected. By using the information-processing principles on which the tests were selected, however, because vision was involved in the perfor- mance of each test and vision was impaired, it is just as likely that lead is affecting only the sensory function measured (in this case,
62 ANN M. WILLIAMSON vision) and that problems relating to the adequacy of stimulus input would explain the impairments in the other functions. This possibil- ity is being pursued. INFLUENCE ON TESTING SELECTION OF DIFFERENT OBJECTIVES FOR TESTING The position taken above regarding the utility of theory-based test batteries is also relevant to the question of the most important objec- tives for neurobehavioral testing. It is commonly agreed that there are two levels of testing (Hanninen, 1981~. The primary level focuses on screening for neurobehavioral insult by particular substances; the secondary level, on determination of the functional and, if possible, the neurological sites and mechanisms of the toxic action. Test selection is affected by the level or reason for testing. For screening batteries it is argued that tests should be quick and easy to administer and should concentrate on the known effects of the toxin, whereas for the second level of testing it is maintained that tests can be more complex and time-consuming. Although this dichotomy of levels has real merit on practical grounds, there has been a tendency to focus too much on the "getting the job done" approach (Gullion and Eckerman, 1986) at the expense of "un- derstanding the phenomenon." There has been a tendency in doing so to use tests that are fast and expedient rather than comprehensive and informative about the toxic effect on the system. This does not necessarily mean understanding the phenomenon but, rather, being aware of the breadth of the problem. A test battery designed around an information-processing model, for example, will provide a proper screening tool that reduces the possibility of Type 1 errors occurring simply because the affected function was not tested adequately or at all. As Wickens (1987) states about information-processing theory, From the perspective of human factors, the importance of the distinction between processing stages results because knowing that a particular environ- mental stressor, chemical tox~cant or system characteristic influences one processing stage and not another has important implications for system re- design or reconfiguration. For example, knowing that a given stressor influ- ences response processes and not encoding should lead the designer to focus on the improvement In control, rawer Man the display interface. In the same way, neurobehavioral toxicology needs to be able to distinguish effects on function by appropriate choice of tests at the screening level of testing. Questionnaires are simply not adequate for initial screening. Rather, screening should first involve a very
CURRENT STATUS OF TEST DEVELOPMENT 63 careful analysis of the functional effects of the putative toxin. Once this is done and the boundaries of the toxic effect have been established, a second battery can be developed which could consist of a screening questionnaire for subjective symptoms and a short, economical objective test battery (Hanninen, 1981~. EXPECTED DIRECTIONS IN TEST DEVELOPMENT The development of tests in this area will almost certainly turn to concentrating much more on establishing valid, reliable, and sensi- tive test batteries. What tests are employed in these batteries will depend on the reason for using neurobehavioral tests in the first place. The main reason for using neurobehavioral tests is either to identify whether the substance is toxic at a particular level of exposure or to determine the nature of the effect on the nervous system, or both. If the rationale for testing is the former, the types of tests needed are the same as those currently in use. As discussed above, development in this area should be toward simple tests that can be broken down into basic functional elements and have a place in a comprehensive, theoretical framework in order to improve interpretation of the toxicity question. If the latter reason constitutes the rationale for testing, development would be in the direction of finding appropriate tests that are analogues of underlying electrophysiological and biochemical processes. This type of development, however, has considerably further to go than the first and will be discussed again in the next section. A third reason for testing that is likely to emerge is to demonstrate the need to respond to the toxicity problem. As discussed in the beginning of this chapter, a major problem in this area is establishing, to the satisfaction of the wider community, that impairments in neurobehavioral functioning of the type typically shown do constitute both a health effect and a possible compromise to safety (e.g., the slowed reaction time that occurs in solvent-exposed workers may cause more accidents). If this is the aim of the investigation, the direction of test development would be toward tests that mirror "real- life" functioning in much the same way as tests were developed to investigate the effects of alcohol on driving (e.g., Moskowitz, 1973~. A likely offshoot of improvements in the validity, reliability, and sensitivity of test batteries, as well as the significant efforts currently being made toward setting up norms for neurobehavioral tests (e.g., the WHO/NIOSH test program), is that tests will become much more useful as clinical tools for diagnosing individual responses to toxic substances. This development has important implications for preven-
64 ANN M. WILLIAMSON lion of toxic effects if early or "subclinical" effects can be detected in particular workers. Last, an important area for investigation in test development is in designing tests that will examine the strength of adaptive capacities to overcome the effects of toxic insult to the nervous system. This is an extremely important question because there are strong possibilities that "behavior's global nature also may allow compensatory mechanisms to thwart the early detection of an irreversible pathological process" (Weiss, 1983~. This was demonstrated in a study by Albers et al. (1987) in which a previously mercury-exposed cohort who had shown no ill-effects related to their exposure during their working lives were compared after retirement with a group of matched controls. The previously exposed group showed statistically significant impairments in psychomotor per- formance that were also related to the extent of their lifetime exposure. These findings were interpreted in terms of changes with age in the capacity of the nervous system to adapt or compensate for deficiences · Or Injury. One possible facet of functioning that could provide a window on such concealed compensatory changes involve the strategies that in- dividuals use in tackling tasks or solving problems. For example, in a study of professional (abalone) divers (Williamson et al., 1987), analysis of the results of the memory scanning task showed that divers responded to stimuli faster than matched controls but made significantly more errors in doing so. The divers were clearly sacrificing accuracy for speed. It was not surprising that these divers showed risk-taking tendencies, given the nature of their work. This tendency obviously had implications for the results of other tests in the battery, and for any testing that might be carried out in the future, which would not have been revealed if a less informative test had been used. Tests need to be developed that will focus on the way that the problem is solved, not just on the speed (although this measure may reflect a slowing due to the adoption of less familiar or less efficient strategies) or the number of errors made. The approach of using tests that can be broken down into elementary cognitive tasks (Carroll, 1980) would provide a good beginning for this new direction. REMAINING BARRIERS TO TEST DEVELOPMENT Most of the remaining barriers to test development are due to the state of knowledge in neurobiology and neuropsychology. To proceed much further with test development, more must be known, for example, about the biological correlates of existing tests, and tests need to be devised for which this is known. In addition, psychological theories
CURRENT STATUS OF TEST DEVELOPMENT 65 or models of behavior also need to be refined further in terms of tests that measure fundamental functions. Gullion and Eckerman (1986) argue very strongly that the present status of psychological theory is too weak to be useful in neurobehavioral toxicology. They maintain that the theories have not been tested adequately for each type of validity or for reliability and, furthermore, that such testing needs to be carried out in the field, on populations similar to those that are likely to be exposed to toxic hazards, not just on healthy college students. It should be noted, however, that Gullion and Eckerman base their argument on criticism of a theory that attempts to describe only one aspect of psychological functioning, namely, short-term memory. Focusing only on individual tests could in itself be seen as a barrier to the development of useful tests for neurobehavioral toxicology. It is important that efforts in theory-based test development concentrate on holistic theories of behavior or information processing, not just on particular aspects. Another potential barrier to the effective development of tests is the widespread use of computer-aided testing. There is little doubt that computers are extremely useful in making the task of data collection much quicker and easier, and they can increase the scope of the data collected (Fidler et al., 1987; Gullion and Eckerman, 1986~. Their use can present problems, however, in that because of the ease of admin- istration, untrained and inexperienced testers can be used. Test administration involves much more than simply showing the subject what to do. A great deal of insight can be gained about subjects' performance by watching how they perform. This would be of particular importance in the early stages- of screening for the effect of a toxic hazard and can easily be dispensed with if the computer is doing all the work. In addition, although standardizing test administration is important, the conditions under which testing is carried out are also important. This point can be overlooked if the tester is not intimately involved in the test process and again, particularly, if the tester is untrained. Problems can also be encountered due to the subject's lack of familiarity with computers. This is especially likely to be a problem for the type of workers who will be exposed to toxic hazards. Unless this is closely monitored, test results may well be confounded by extraneous factors due to the computer itself. There is a significant possibility that the development of tests will be forced to take a back seat in favor of research focusing on the effects of the many toxic substances that have not yet been studied. This may well be the major barrier to test development. To policymakers
66 ANN M. WILLIAMSON and the providers of research grants, test development may look like "contemplation of the navel." Neurobehavioral toxicology must make test development at least as important as the analysis of toxic effects if it is to make further progress. ~ RECOMMENDATIONS FOR FURTHER RESEARCH OR DEVELOPMENT One of the major problems in neurobehavioral toxicology is de- sign~ng studies which eliminate, or at least minimize, the effects of extraneous variables that confound any demonstration of impairment in exposed compared to nonexposect workers. The use of prospec- tive or cohort designs, which is the most satisfactory solution to this problem, has implications for the development of test batteries. Further research is needed to devise tests that are resistant to learning or practice effects. Research is also needed in the development of tests that can detect the effect of adaptive changes which may camouflage functional im- pairment. From the point of view of prevention, this particular area hoicis real promise for the future. If this approach becomes success- fu] it would then be possible to conceive of early detection criteria for long-term and even delayed effects. This would! also provide a much more comprehensive picture of the breadth of the effects of a particular toxic hazard. Finally, research should concentrate on development of tests that can be used for screening of individual workers. The initiative taken by the WHO/NIOSH where the NCTB is being applied worldwide is a step in this direction. It should, however, now be followed up by careful standardization in terms of a full investigation of the validity (particularly construct and concurrent validity) and reliability of the tests and the test battery. Once this is done, neurobehavioral toxicology will have available a set of well-defined tests that have established norms. Not only will testing of individuals be a realistic possibility, but the effectiveness of group testing will also be much improved. This is particularly important in attempting to overcome the vexing problem of comparison of results from different laboratories and different parts of the world. REFERENCES Albers, J., D. Escheverria, P. Donofrio, L. Fine, E. Kallenbach, G. Langolf, and R. Wolfe. 1987. Persistent adverse health effects of occupational mercury exposure. Paper presented at International Congress on Occupational Health, Sydney, Australia.
CURRENT STATUS OF TEST DEVELOPMENT 67 Anger, W. K. 1984. Neurobehavioural testing of chemicals: Impact on recommended standards. Neurobehavioural Toxicology and Teratology 6:147-153. Anger, W. K. 1986. Workplace exposures. In Neurobehavioural Toxicolo~v Z. Ar~nau ed. Baltimore: Johns Hopkins Press. Baker, E. L., R. G. Feldman, and R. F. White. 1983. Monitoring neurotoxins in indus- - try development of a Neurobehavioural test battery. Journal of Occupational Medi- cine 25:125-130. Baker, E. L., R. G. Feldman, and R. F. White. 1984. Occupational lead neurotoxicity a behavioural and electrophysiological evaluation. Study design and year one results. British Journal of Industrial Medicine 41:352-361. Baker, E. L., R. Letz, and A. Fidler. 1985. A computer administrated Neurobehavioural evaluation system for occupational and environmental epidemiology. Journal of Occupational Medicine 27:206-212. Carroll, J.B. 1980. Individual Difference Relations in Psychometric and Experimental Cognitive Tasks. L. L. Thurstone Laboratory Report No. 163. Chapel Hill, N.C.: University of North Carolina. Fidler, A., E. L. Baker, and R. Letz. 1987. Neurobehavioural effects of occupational exposure to organic solvents among construction painters. British Journal of Indus- trial Medicine 44:292-308. Foree, D., D. Eckerman, and S. L. Elliott. 1984. MTS: An adaptable microprocessor- based testing system. Behavioural Research Methods, Instrumentation Computing 16:223-229. Gamberale, F. 1985. Use of behavioural performance tests in the assessment of solvent toxicity. Scandinavian Journal of Work Environment and Health ll(Suppl 1):65-74. Gullion, C.M., and D. Eckerman. 1986. Field testing for Neurobehavioural toxicology: Methods and methodological issues. In Neurobehavioural Toxicology, Z. Annau, ed. Baltimore: Johns Hopkins Press. Hanninen, H. 1981. Behavioural methods in the assessment of impairments in central nervous function. In Biological Monitoring and Surveillance of Workers Exposed to Chemicals, A. Alto, V. Riihimaki, and H. Vainio, eds. Washington: Hemisphere Publishing Corporation. Hanninen, H., and K. Lindstrom. 1979. Behavioural Test Battery for Toxicopsychological Studies: Used at the Institute of Occupational Health in Helsinki, second revised edition. Helsinki: Institute of Occupational Health. Hanninen, H., P. Mantere, and S. Hernberg. 1979. Subjective symptoms in low-level exposure to lead. Neurotoxicology 1:333-347. Hanninen, H. 1981. Behavioural methods in the assessment of impairments in central nervous function. In Alto, A et al. (Eds.), Biological Monitoring and Surveillance of Workers Exposed to Chemicals, Washington: Hemisphere. Hogstedt, C., M. Hane, and A. Agrell. 1983. Neuropsychological test results and symptoms among workers with well-defined long-term exposure to lead. British Journal of Industrial Medicine 40:99-105. Johnson, B. L., and W. K. Anger. 1983. Behavioural toxicology. In Environmental and Occupational Medicine, W. R. Rom, ed. Boston: Little, Brown. Mantere, P., H. Hanninen, and S. Hernberg. 1984. A prospective follow-up study on psychological effects in workers exposed to low levels of lead. Scandinavian Journal of Work Environment and Health 10:43-50. Moskowitz, H. 1973. Laboratory studies of the effects of alcohol on some variables related to driving. Journal of Safety Research 5:185-199. Parkinson, D. K., C. Ryan, E. J. Bromet, and M. M. Connell. 1986. A psychiatric epidemiologic study of occupational lead exposure. American Journal of Epidemi- ology 123:261-269. vet ~ .
68 ANN M. WILLIAMSON Smith, P. J., and G. D. Langolf. 1981. The use of Sternberg's memory-scanning para- digm in assessing effects of chemical exposure. Human Factors 23:701-708. Sternberg, S. 1966. High speed scanning in memory. Science 153:652-654. Sternberg, S. 1975. Memory scanning: New findings and current controversies. Quarterly Journal of Experimental Psychology 27:1-32. Valciukas, J. A., and R. Lilis. 1980. Psychometric techniques in environmental re- search. Environmental Research 21:275-297. Valciukas, J. A., and R. Lilis. 1982. A composite index of lead effects. International Archives of Occupational and Environmental Health 51:1-14. Valciukas, J. A., R. Lilis, and H. A. Anderson. 1979. The neurotoxicity of polybromi- nated biphenyls: Results of a medical field survey. Annals of the New York Acad- emy of Sciences 320:337-367. Wechsler, D. 1955. Wechsler Adult Intelligence Scale. New York: Psychological Corporation. Weiss, B. 1983. Behavioural toxicology and environmental health science: Opportu- nity and challenge for psychology. American Psychologist 91:1174-1186. Wickens, C. D. 1984. Engineering Psychology and Human Performance. Columbus: Charles Merrill. Wickens, C. D. 1987. Information processing, decision-making and cognition. In G. Salvendy, ed. Handbook of Human Factors. New York: John Wiley & Sons. Williamson, A. M., and R. K. C. Tea. 1986. Neurobehavioural effects of occupational exposure to lead. British Journal of Industrial Medicine 43:370380. Williamson, A.M., R.K.C. Tea, and J. W. Sanderson. 1982. Occupational mercury exposure and its consequences for behaviour. International Archives of Occupa- tional and Environmental Health 50:273-289. Williamson, A. M., B. Clarke, and C. E. Edmonds. 1987. The neurobehavioural effects of professional abalone diving. British Journal of Industrial Medicine 44:459-466.
