Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
3 Research on Instruction Whatever is known about the acquisition of reasoning skills in mathematics and science, such knowledge needs to be translated into classroom instruction. Teachers and curriculum are key to the amount and quality of time spent on instruction; testing assesses the outcomes of instruction. This chapter discusses research on each of these three elements. RESEARCH ON TEAC B RS To use time given to ~ nstruction effectively, teachers must be competent and willing to exert sufficient effort. (See Levin, 1980, for a cogent discussion of teacher inputs to educational productivity.) Teacher competence involves adequate cognitive mastery of the subject matter to be taught and, in the case of science especially, proficiency in handling experimental materials that can lead students to form new concepts from observation and evidence. For example, Arons (1981) argues that even the best curricula will be ineffective unless teachers are trained to deal with various modes of abstract logical reasoning, for example, the logic of arithmetic involved in ratios and division, the logic of control of variables, dealing with propositional statements, recognizing gaps in available information, making inferences and pre- dictions from mental models, doing hypothetico-deductive reasoning, and the like. In fact, the processes and problems involved in educating teachers to acquire these capacities are not very different from those involved for any other learners. But, Arons (1983) also argues, subject-matter courses taken by prospective teachers-- usually the standard courses offered by science 15
16 departments--often cover too much content at too rapid a pace and seldom pay explicit heed to developing reasoning capacities. Hence, prospective and practicing teachers often lack a genuine understanding of concepts and lines of reasoning that characterize the subject(s) they are teaching and, having missed effective training themselves, are unable to cultivate and enhance the basic abstract reasoning capacity of their students. The shortcomings in his or her college courses may be one of the reasons a teacher's background in science (as well as preparation in professional education) shows relatively low correlations with student outcomes (Drava and Anderson, 1983). However, there may be other pos- sible explanations for these low correlations, including lack of significant variations in teacher training, low correspondence between subject matter taught and the content of tests (Freeman et al., 1~83), and such other factors as teacher motivation and energy level. Whatever its effect, little is known about the subject matter preparation of the 2.37 million teachers in the current pool (Raizen and Jones, 1985), much less about their competence for teaching science and mathematics. Two ongoing surveys, one by the National Center for Education Statistics and one by Research Triangle Institute, will provide some relevant information, but it will be limited in scope. Much general information on teachers is also being collected in connection with the National Assessment of Educational Progress; assessments in mathematics and science and concomitant teacher surveys are scheduled for 1986. Meanwhile, in the absence of sufficient knowledge about the nature of teacher prepara- tion programs, assessment of teacher quality has been based on reviewing SAT scores of college freshmen planning to be teachers (Weaver, 1979; Schlechty and Vance, 1983) and increasingly on the scores of newly entering teachers on the National Teacher Examination or on state-constructed teacher tests. Over the last five years, states have made various policy changes intended to increase the quality of teachers: 32 states have changed teacher certification requirements; 28 states have changed teacher education curricula, and 20 states have raised entrance require- ments for teacher education programs (Goertz et al., 1984). Other policies that have been proposed include salary increases and structural changes in compensation for teachers, requiring liberal arts majors for all teachers and possibly a five-year rather than a four-year
17 degree program (Scannell and Guenther, 1981; Boyer, 1983; National Commission for Excellence in Teacher Education, 1984), and ensuring teacher competence through a nation- ally recognized licensing examination (Albert Shanker, as reported in the Chronicle of Higher Education, 1985a). With respect to increasing the quality of science and mathematics teaching specifically, monies have been appropriated by Congress and by a number of states to provide loans and scholarships for students preparing to enter these fields (U.S. Department of Education, 1984), and a program for developing models for teacher education has been established by the National Science Foundation. Few of these policy changes are based on research evidence that relates the proposed interventions to observed responses on the part of individuals who might enter teaching (Murnane, 1985) or to the acquisition of knowledge and skills deemed necessary for science and mathematics teaching. Indeed, there are indications the t some of the new policies may prove ineffective or have some undesirable consequences. For example, Summers and Wolfe (1977) found a statistically significant negative correlation between teachers' scores on the National Teachers Exam and their students' test score gains. Increases in credentialing requirements may rob local districts of the flexibility to hire individuals who exhibit the capacities for teaching mathematics and science but lack the credentials; abolishing traditional credentials, as New Jersey has done and other states are considering, may have the perverse effect of setting lower rather than higher standards (Chronicle of Hither Education, 1985b). Moreover, simply raising requirements to enter teacher education programs is likely to reduce the socioeconomic and racial and ethnic diversity of the nation's teaching force at a time when schools will be educating a larger nether of minority students (Goertz et al., 1984). Systems of compensation such as merit pay that require evaluating teacher performance are hampered by the difficulties of developing and implementing such evaluations (Wise et al., 1984) and, perhaps for that reason, historically have had a short life span (Educational Research Service, 1979). The current experimentation with incentives, teacher education programs, and credentialing sharpens the need to understand better (a) who gets access to teacher preparation programs under various conditions, (b) the content of these programs, and (c) the regulation of access to teaching positions. These factors are poorly
18 understood even for the current pool of teachers. The unplanned variations resulting from current policy changes provide a rich opportunity for assessing the effects of various alternatives. Therefore, the committee recommends the development of a national data base on teacher preparation and qualifica- tions sufficiently detailed and appropriately stratified to reflect conditions in different types of school dis- tri~cts and for varying student populations. We recommend a research program to develop improved understanding of: (1) the response to various monetary incentives designed to attract able individuals to mathematics and science teaching and keep them in these fields; (2) how to improve the subject-matter education of both pre- and inservice teachers, including optimal volume and pace of subject- matter coverage in different sciences and experiences that develop and enhance abstract reasoning capacity; and (3) the effects of alternative requirements for entering and being certified in the profession, particularly with respect to developing an adequate pool of teachers competent to teach mathematics and science. Effective use of instructional time involves not only the teacher's capacity, but also the teacher's effort (Levin, 1980). Direct measures of the quantity of teacher effort in the classroom (e.g., the amount of time spent by the teacher on direct instruction or active teaching) and indirect measures (e.g., the amount of time students are on-task n or engaged in learning) show positive correlations with student performance (Brophy and Ever tson, 1976; Good and Grouws, 1977; Fisher et al., 1980). Particular aspects of active teaching have also been investigated as to their effectiveness--for example. strategies for giving information (Rosenshine, 1968; Armento, 1977; Smith and Sanders, 1981) and reacting to their responses (Clark et al., 1979; Evertson et al., 1980) and for assigning and checking homework (Good and Gronws, 1977; Walberg and Rasher, 1985). Bowever, attempts to assess teacher behavior have been limited to specif ic instructional settings (Gage, 1978), and no consistent pattern of success across subject areas or specific groups of students has emerged (Brophy and Evertson, 1976; Medley, 1979). m e exception is work on the assignment of and feedback on homework--apparently an effective way of extending learning time through teacher effort. Teacher effort is not solely a consequence of indi- vidual attributes; it is also influenced by the organ) -
19 Rational characteristics of schools: the degree of autonomy allowed teachers in their own classrooms and their contribution to school practices and policies (Levin, 1980; Lightfoot, 1983), opportunities for professional interaction and encouragement of innovation (Grant, 1981; Little, 1981; LipSitz, no date), explicit and tacit reward structures and sanctions (Lortie, 1975; Sykes, 1983), and the general values and attitudes of teachers, for example, consensus on academic goals and norms for behavior (Brookover et al., 1979; Rutter, 1979) Unfortunately, assessing the effects of such factors on teacher effort is even more difficult than measuring teacher capacity or teacher behavior in the classroom. Nevertheless, it is important to conduct research on how societal pressures, school organization, and educational policies affect the effort teachers are able and willing to invest in instructing their students. RESEARCE ON CURRICULA AND CURRICULAR MATERIALS Advanced scholarship in a subject is based on theories and concepts that serve to make a domain accessible to subject-matter experts. However, a theory for the expert may not be good pedagogical theory for the novice. As already noted, recent work in cognitive psychology has described how acquired knowledge is organized and repre- sented, and how cognitive models can facilitate reasoning and thinking as students use and test these models to solve problems and revise what they already know (Estes et al., 1982; Rumelhart and Norman, 1981). Such research has had little influence, however, on the rigidly hier- archical conception of science and mathematics that under- girds most classroom instruction. Nevertheless, effec- tive teachers use their experience of how students learn to shape the subject matter they present. This craft knowledge provides a second source for developing peda- gogical theory for teaching science and mathematics to students at different levels of competence and education. Still a third source is the current experimentation with computer systems for intelligent tutoring, based on models of how successful students perform various cognitive tasks tSleeman and Brown, 1982; Anderson et al., 1985). Based on work from these sources, the committee recommends research directed toward effective instruc- tional strategies based on explorations of: (1) the
20 design of pedagogical theories that students can test, evaluate, and modify; (2) the techniques of ingenious teachers who are able to devise such temporary models or pedagogical theories; and (3) the design of intelligent computer-assisted instruction that incorporates interroga- tion and exploration. In addition to these general research issues on cur- riculum, there are specific questions surrounding the subject matter of technology. Unlike the more traditional domains of science and mathematics, technology and com- puter science do not have well-established curricula. Many schools are now introducing "computer literacy. courses. Often, such courses focus on teaching program- ming in a particular computer language. In other instances,-computer literacy courses deal with the capabilities and functioning of computers, either with or without hands-on experience, and may include topics on the effects of computers on the workplace and society. At more advanced levels, science and mathematics courses may devote some class time to illustrating changes in these fields that have come about because of the avail- ability of poweful computational tools. - In the committee's view, there is insufficient knowledge about the age and grade levels at which the computer and programming should be introduced and about the effects of alternative curricula in computer literacy. Systematic attention must also be given to how the knowledge structures and the processes of the sciences and mathematics have changed as a result of readily available computation and what these changes imply about the school curriculum. For example, the advent of calculators made traditional drill in using logarithm tables superfluous. Similar issues need to be explored regarding the advent of more powerful computers for all the science and mathematics courses taught in school. The committee recommends research targeted at providing characterizations of the cognitive skills and knowledge needed for understanding of and successful performance in technological systems; based on such characterizations, development of usable school curricula in computer literacy; and investigating the effects of computers on the knowledge structure of mathematics and various sciences and the changes implied for the school curriculum. Recent research with preschool children suggests that changing the context of the learning task, or ~recon- textualiz ins, n can help students acquire some basic
21 cognitive skills (holding information in memory, building structural representations for later use, comparing perspectives) essential to achievement in science. Istomina (1975) compared the performance of Preschool children on a test-like version of a free recall task and the same task embedded in a role-playing game of being Activa- tion of the four- to five-year-olds' still-crude memoriz- ing operations was greatly facilitated by the play situation. _ sent to a make-believe store for a list of items. Similarly, Margaret Donaldson (1978) and her students addressed the presumed inability of children younger than 10 to 11 years of age to take account of another person's visual point of view (Piaget and Inhelder, 1975). They demonstrated that perspective-taking ability is present in very young children in the right circumstances. Donaldson arranged for the problem to involve toy children hiding from a toy policeman. Only by taking the policeman's point of view could the child subjects know where the toy children should hide. Four- to f ive-year- olds succeeded at this problem even when they had to coordinate the points of view of two policemen, whose view of the scene was different from their own. As a final example, decades of research on delayed responses," in which an object is hidden in one of several boxes and children are required to search for it several seconds or a few minutes later, has shown children to be deficient in their ability to keep the location of objects in mind. DeLoache and Brown (1979) repeated this experiment with two- to three-year-olds their homes. Instead of hiding a piece of candy, children favorite toys were hidden under a piece of furniture. Under these conditions, children would remember the location of the hidden object for 24 hours, the longest intervals tested. This research suggests that a fundamental way of changing how much time is needed for a particular task is to change the context of the task as Presented to and understood by the learner. In The cognitive task was more successfully completed when it was embedded in some larger activity involving familiar scripts and human intentions. These examples from research on young children are not intended to suggest that recontextualization for older learners must always strive for simplification or that it should only involve making materials more familiar and obviously utilitarian. Knowledge is unavailable on how
22 the insights gained from the work with young children can be applied at higher levels of the curriculum in the areas of science, mathematics, and technology. However, there is evidence that science curricula combining activity-based instruction with appropriate text materials are more effective than traditional curricula in teaching higher-order skills (Shymansky et al., 1983; Boldzkom and Lutz, 19841. On the basis of such research, it has been argued that programs and school curricula In science and mathematics should stress utility and practical applications rather than heavy reliance on theory (Harms and Yager, 1981). Hands-on, laboratory, and activity-oriented are accurate descriptions of most programs identified in recent surveys as exemplary, especially at the elementary level (Penick, 1983). Bowever, activity-based teaching is notoriously difficult to carry out and appears at times to be in conflict with the high level of control of the teacher over classroom activities advocated by some proponents of research results on effective teaching (Starlings, 1975; Hunter, 1984; Brophy and Good, 1985). Moreover, a broad conclusion rejecting more abstract curricular forms is clearly premature. For example, a trademark of the SEED program (Johntz, no date) is demonstrating the success of minority students in performing highly theoretical mathe- matical manipulations with little focus on applications or ties to anything concrete. At least in the hands of an extremely competent and knowledgeable instructor-- usually a scientist or mathematician in the case of the SEED model theoretical training works. Because of the great importance of curricular orienta- tion and context to learning, particularly to the learning of mathematics, science, and technology, the committee urges special emphasis on this research area. Priorities include research on how important tasks can be embedded in contexts that reduce the time needed for learning; under what circumstances and in what ways activity systems using physical objects and "real. events [whether hands- on experience, models based on systematic laws, or story lines that mirror common experiences) can be used to enhance learning; and what makes theory-oriented instruc- tion work, especially with individuals from some minority groups and women generally said to require a more prag- matic, utilitarian approach. Curricula depend on and are built around educational materials. Textbooks and, to an increasing degree, educational computer software are central factors in
23 determining what is learned on what time schedule (Stake and Easley, 1978). The content of textbooks is influenced by the authors' sense of appropriate learning goals, the publishers' perception of the demands of the education market, and state and local district priorities and pro- cedures for textbook approval and selection. There is some information on choice of textbooks, but it dates back to a 1977 survey (Weiss, 1978). This survey will be repeated in 1985, but even though it may yield informa- tion on what texts are used, there is no systematic effort under way to analyze the content of these texts. During the development of reform curricula in the 1960s, much attention was paid to the balance between emphasis on facts and emphasis on concepts and learning how to learn. Students using the reform curricula did appear to make greater gains than their counterparts on reasoning and problem-solving skills as well as on general achievement measures (Shymansky et al., 1983). There is no equivalent current information on textbook content (Walker, 1981), although analysis of the struc- ture and language of science textbooks has documented that the learning of special or technical vocabulary. i.e., rote memorization, is a central feature of these texts (Yager, 1983). The increasing use of standardized tests to assess student achievement assumes that a curriculum covers the material on the test. Based on a detailed analysis of fourth-grade mathematics texts and tests, Freeman et al. (1983) found (p. 511) "the proportion of topics covered on a standardized test that received more than cursory treatment in a textbook was never more than 50%.~ Limited as it is, such evidence indicates possible inaccuracies in general assumptions about the curricular content of educational materials in current use. Further evidence on disparities between the assumed (.intended~) and actual (nimplemented.) curriculum comes from several large-scale studies on student achievement. For example, the studies conducted by the International Association for the Evaluation of Educational Achievement (IEA) have attempted to relate the items on student achievement tests to the opportunity students had to learn the material through asking their teachers whether pertinent instruction had been provided. The opportunity to learn the material turned out to be highly correlated with student test scores (Husen, 1967; wolf, 1977; Crosswhite et al., 1985). The National Assessment of Educational Progress (1985) also includes measures of the implemented
24 curriculum, such as asking teachers of students taking the mathematics tests what topics are included in mathematics instruction in grades 6-8. Differences in implemented curricula presented to different sorts of students affect their opportunity to learn (Alexander and McDill, 1976; Entwisle and Hayduk, 1982; Barr and Dreeben, 19831. Thus, the effects of ability grouping and tracking on learning are realized not only through differences in instructional strategies and peer influences but also through differences in the curriculum to which different groups of students are exposed (Rosenbaum, 1976; Cicourel and Kitsuse, 1983; Hallinan and Sorensen, 19841. The committee recommends a concerted research effort on how educational curricula and materials are created, their content, and how they are used, specifically, on (1) whether and how the treatment of substantive content in current textbooks and software supports the learning of reasoning, thinking, and problem-solving skills as well as lower-order recall and memorization tasks; (2) the exploration of new content areas within various fields and at various grade levels that might be pro- ductive additions to promoting higher-order skills; (3) the abilities, skills, and perspectives of those who write textbooks and software (for example, to what extent do they understand the importance of curricular context, as discussed above) and the means for attracting better prepared individuals to those fields; (4) the development of consensus on appropriate subgoals, content, and sequencing by grade level to facilitate greater emphasis on higher-order skills; (5) the effects of state approval processes on content issues; and (6) further studies on the relation between what is tested and what is included in textbooks and software and between the intended and the implemented curriculum. RESEARCE ON TESTING The testing of cognitive achievement and aptitude plays a powerful role in American schools. Tests are used to group and track pupils, resulting in the differ- entiation of pupil experiences. Tests are used to diagnose current knowledge and skill prior to instruc- tion. Tests are used to assess mastery of instructional objectives. Tests are used to evaluate teaching and instruction. There is a widespread consensus among
25 cognitive scientists that many of these procedures are inadequate, particularly in the assessment of higher- order thinking skills (Frederiksen, 1984). Assessing reasoning ability is not easy. Tradi- tionally, it has best been done by open-ended tests requiring problem solving and free-form answers (e.g., essay and problem tests). Such tests are difficult to administer and to grade, particularly for large numbers of individuals. There is a long and productive line of research within the field of psychometrics on the prac- tical measurement of a variety of intellectual skills. There is a new and promising line of research that links traditional psychometrics to the growing understanding of reasoning skills described above (Hunt et al., 1973; Glaser, 1981 ; Sternberg , 1977, 1984 ) . Inexpensive, powerful computers provide a new possibility for more effective interactive testing. Using current microcom- puters to test students could be more accurate and less time-consuming for those taking tests as well as less labor-intensive for those administering tests (Weiss, 1983), but further research is required to substantiate that possibility. Tests also play a role in the learning process itself. They tell students what in the curriculum is impor tent and shape the teaching and learning process (Frederiksen, 1984). If, for example, testing is confined to memorizable end results, students will concentrate on these end results, ignoring the more sophisticated levels of understanding and reasoning to which teachers and text materials may be rendering lip service. Teachers and school administrators also use tests as a guide to cur- riculum emphasis, especially when student performance on given tests is used as a measure of teacher and school performance. The committee recommends a program of research on testing, including: (1) the development of practical tests that reliably assess reasoning ability, perhaps using interactive testing made possible by microcompu- ters; (2) improving the testing of mathematics and science achievement to reflect important instructional goals and objectives; and (3) techniques for educating teachers to become better writers of test questions, particularly of questions that test for the higher-order intellectual skills and levels of learning.
