Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
LA UREN B. RESNICK 15 to the meaning of the symbols (Peterson et al., 1984; Resnick, 1987~. Strong math learners also engage in more task analysis (Dweck, in press); that is, they figure out alternative strategies for attacking problems and generating solvable subproblems. These sense-making and knowledge-extending activities parallel those that are so well documented for high levels of reading skill. They are also activities generally viewed as characteristic of high levels of mathematics think- ing and problem solving. Thus, we again see a convergence between the processes identified by cognitive research and those associated with traditional elite mathematics education. GENERAL REASONING: IMPROVING INTELLIGENCE Mathematics and reading are not unique in the extent to which high-level performance depends on processes of monitoring one's understanding, imposing meaning and structure, and raising quest tions about presented material. Much the same story can be told about all the subject matter in the school curriculum and about all but the most routine job performances. Recent research in science problem solving, for example, shows that experts do not respond to problems as they are presented-writing equations for every relation- ship described and then using routine procedures for manipulating equations. Instead, they reinterpret the problems, recasting them in terms of general scientific principles until the solutions become almost self-evident (Larkin et al., 1980~. Expert writers treat the process of composing an essay as a complex task of shaping a com- munication that will appeal to and convince an intended audience rather than as a simple task of writing down everything they know about a topic (Bereiter and Scardamalia, 1982; Flower and Hayes, 1980~. In the social sciences, trained thinkers call upon a wide range of knowledge relevant to a topic to construct proposals for action and to build justifications for those proposals that conform to many of the classical principles of rhetoricad argumentation (doss et al., 1983~. Skilled technicians repairing equipment do not just proceed through routine checklists; instead, they construct Mental modems of complex systems and use these to reason about possible causes of observed breakdowns and potential repairs (e.g., de Kleer and Brown, 1980~. In all of these cases, certain kinds of higher order thinking recur: experts elaborate and reconstruct problems into new forms; they Took for consistencies and inconsistencies in proposed solutions; they
16 ED UCATION AND LEARNING TO THINK pursue implications of initial ideas and make modifications rather than seeking quick solutions and sticking with initial ideas; they reason by analogy to other, similar situations. These similarities, long noted in discussions of intelligence (see Journal of Educational Psychology, 1921; Simon, 1976; Sternberg and Detterman, 1979) and problem solving (Tuma and Reif, 1980), lead naturally to the question of whether there might not be some general thinking skills that would produce improved ability to learn across many traditional curriculum areas. If such skills exist and if we can find effective ways to teach them, we can imagine an important increase in educational efficiency, for it would seem a relatively narrow instructional effort might produce wide learning results. The search for general learning skills is not a new one. both educators and psychologists have long sought to identify and to char- acterize such skills, the former because of the educational efficiency such skills could help them realize, the latter in search of unifying characteristics of human thought. Psychological research gives us reason to believe in the reality of general skills for learning as well as reason to maintain a degree of skepticism. In the next section we will review recent efforts to teach higher order skills. These efforts provide the newest body of evidence on the question of whether such skills are teachable. Before proceeding to that review, however, we should first consider what the body of past research would suggest. Past Research Psychometric research provides the best-established evidence for the existence of cognitive skills that play a role in diverse kinds of learning. When two or more cognitive abilities are tested, there is almost always a positive correlation between the measures. People who do well on one ability test are, on the average, likely to do wed on the others. Virtually the only conditions under which such a correlation is not found are those in which tests have been specif- ically designed not to correlate. For example, investigators have built tests of creativity explicitly designed to be psychometrically independent of IQ. Tests that correlate positively are presumed to share underlying processes. The fact that most intelligence tests do correlate strongly and that a general factor can always be identified through statistical methods such as factor analysis suggests that all tests have some processes in common. These common processes are, presumably, general abilities.
LA UREN B. RESNICK 17 Of course, such findings only raise new questions. How do we characterize these common processes? Is there reason to think they are teachable? When cognitive scientists do information-processing analyses of complex skills, they find the same kinds of basic problem- solving processes used in task after task (Simon, 1976~. For example, one of the earliest uses of computers to explore processes of human reasoning resulted in the construction of a program that solved sym- bolic logic problems. This program was called the General Problem Solver (GPS) in the belief that its processes would play a role in s~olv- ing many kinds of problems, not just those of symbolic logic. This has turned out to be partly true. Although GPS itself can solve only a limited range of problems, the kinds of processes used by GPS appear over and over again in simulations of human performance of complex tasks. Processes such as means-ends analysis (comparing one's final goal with results that would be produced by procedures currently available), subgoal formation (forming a new goal that is easier to solve and that Is en route to the final goal), generate-and-test rou- tines (generating actions and testing them against constraints), and other general problem-solving routines are used in tasks as varied as inventing buggy arithmetic routines, planning compositions, con- structing geometry proofs, and troubleshooting electronic devices. The reason that a single artificial intelligence program cannot solve a wide variety of problems is not that the fundamental processes it applies are widely different across domains, but rather that the pro- gram must apply these processes to very specific, organized bodies of knowledge. Each simulation must build in the relevant knowI- edge, and so it becomes specific to its knowledge base (see Dehn and Schank, 1982~. Other processes that appear repeatedly in analyses of complex task performance play a kind of ~executive" or self-regulatory role in thinking. People use these processes to keep track of their own understanding, to initiate review or rehearsal activities when needed, and to deliberately organize their attention and other resources in order to learn something. These activities have been shown to be characteristic of effective learners, good readers and writers, and strong problem solvers. The same processes are relatively absent in younger or less intelligent individuals. These skills are sometimes called ~metacognitive skills" (see Brown et al., 1983) because they operate on an individual's own cognitive processes. They have been suggested frequently as processes that could be taught and that would enhance learning and thinking in a wide range of specific situations.
18 EDUCATION AND LEARNING TO THINK The problem-solving skills identified in cognitive simulation re- search and the metacognitive skills identified in developmental psy- chology research have both been proposed as candidates for teaching. The hope is held out that if we can improve specific skills through some form of direct teaching, then people's ability to perform various kinds of learning, thinking, and problem-solving tasks in which such skills have been observed will also improve. On the other hand, the very body of research that has helped to identify the candidate 'Cgeneral" skills also provides reason for questioning their educational importance. Cognitive research yields repeated demonstrations that specific content area knowledge plays a central role in reasoning, thinking, and learning of all kinds. ~ have already alluded to several examples of the importance of specific knowledge. Specific knowledge about a text's topic affects processes of language comprehension, for example. Skilled science problem solvers rely On their knowledge of scientific principles to recast prom lems into more elegant and easily solvable forms. Political scientists' argumentation becomes degraded when they know little about the particular problem or the particular part of the world under discus- sion (doss et al., 1983~. Even on the tasks used to assess general intelligence or scholastic aptitude, recent analyses have made it clear that much depends on specific knowledge: of vocabulary, of particular number relationships, of possible transformations of visual displays, and the like (cf. Glaser, 1984~. General skills such as breaking down a problem into simpler problems or checking to see whether one has captured the main idea of a passage may be impossible to apply if one does not have a store of knowledge about similar problems-or know enough about the topic to be able to recognize its central ideas. Of course, to appreciate the dependence of general skills application on specific knowledge is not to deny that such general skills exist. Yet such an understanding raises questions about the wisdom of at- tempting to develop thinking skills outside the context of specific knowledge domains. It suggests that a more promising route may be to teach thinking skills within specific disciplines and perhaps hope for some transfer to other disciplines as relevant knowledge is acquired. On first consideration the hope for transfer of thinking abilities across disciplines seems misplaced. A long history of research exists on transfer among school subjects. Over the decades, educators have espoused a recurring belief that certain school subject matters "discipline the minds and therefore should be taught not so much for
LA UREN B. RESNICK 19 ., ... their inherent value as for their efficacy in facilitating other learning. Latin was defended for many years in these terms; mathematics and logic are often so defended today. Most recently, computer programming has been proposed as a way to develop general problem- solving and reasoning abilities (e.g., Papert, 1980~. The view that we can expect strong transfer from learning in one area to improvements across the board has never been well supporter] empirically. At the turn of the century, Thornclike and Woodworth (1901) studied transfer among school subjects and found that it was more efficient to study the subject of interest (English vocabulary, for example) than to study some other subject (e.g., Latin) that Prepared one's mind. Subsequent reviews of research on transfer of school subject matter generally have reconfirmed Thorndike ant] Woodworth's finding. Nevertheless, the history of transfer research need not be to- tally discouraging; most of this research does not directly address the questions of most concern to those whose goal is the improve ment of general thinking and learning abilities. First, the subject matter teaching in these studies has rarely been aimed at develop ing transferable skill and knowledge. We thus do not know wha leverage there might be in instruction explicitly aimed at producing general skills in the context of a particular discipline. Second, eval- uations of learning outcomes have rested mainly on what knowledge was acquired in the transfer discipline, rather than on whether skills for acquiring knowledge in that discipline have been enhanced. The issue of transferability of thinking and learning skills, then, is still open. Current Programs for Teaching Higher Order Skills Recently, a variety of courses and programs claiming to teach reasoning and problem-solving abilities have emerged (see Nickerson et al., 1985; Segal et al., 1985~. These represent the newest wave of optimism concerning the teachability of general higher order cogni- tive skills. Some programs focus on problem solving and reasoning in particular disciplines. But most are aimed at enhancing general skills or at using a combination of both approaches. Recent programs thus offer an opportunity to update the empirical record concerning the effects of various kinds of training in thinking and reasoning skills. In the course of this study, nominations have been sought of programs aimed at teaching various aspects of higher order thinking.
20 EDUCATION AND LEARNING TO THINK A large number of programs and reports have been examined. They are discussed here in several broad categories. Problem Solving m the Disc~nes Faculty members in a number of disciplines have developed courses or course-adjuncts designed to improve the problem-solving ability of students in their disciplines. These are generally college- leve! programs aimed at the full range of students in the discipline. The majority of such courses have been developed in the physical sci- ences (e.g., Reif and St. John, 1979), engineering (e.g., Fuller, 1978; Rubinstein, 1980; Woods, 1983; Woods et al., 1984), and mathemat- ics (e.g., Schoenfeld, 1982, 1983, 1985~. Wales axld Stager (1977; see also Wales and Nardi, 1985) have proposed a general strategy, which they call "Guided Design, for teaching problem solving and decision making within the context of a variety of subject matters. Guided Design courses have been offered in high schools as well as colleges and in the humanities and social sciences as well as in the physical . · ~ sciences and engineering. Problem-solving courses and programs vary considerably in scope and in style, ranging from individual courses or laboratory programs to a multicourse sequence spread over several years of college. The reported programs are probably representative of similar programs being used on many campuses that have not been formally described. Some of the programs are highly structured, with printed materials, special lab notebooks, standard exercises, and regular evaluations. Others are essentially suggestions to instructors, including guidance in how to conduct classroom discussions to favor the development of problem-solving skills. Central to all prograrrLs is extensive practice in solving problems or in designing and carrying out experiments. Supportive help is of- fered, and problem complexity gradually increases. Some programs also teach students to use particular heuristic strategies including special forms of problem representation. For example, Fuller's chem- ical engineering course requires students to prepare special graphical representations (Polya maps) that show a problem's structure. Reif's laboratory requires lab reports in which students organize hierarchi- cally the important aspects of an experiment. Various forms of social interaction are used, both to make visible normally covert aspects of the problem-solving process and to in- crease students' self-conscious monitoring and management of their
LA UREN B. RESNICK 21 thought processes. These include having the instructor think aloud while solving problems set by students, having students work in pairs or larger teams, and having students justify solutions to one another and evaluate each other's solutions. Particular attention is often paid to the uncertainties of problem solving and to the process of making and correcting rather than avoiding or denying errors. Formal evaluation of problem-solving programs is rare. The most extensive quantitative evaluation data are presented by Wales (1979) for freshmen for the first six years of the Guided Design program in engineering at West Virginia University. Wales found definite rises in both freshman and four-year grade point averages (GPAs) even after controlling for grade inflation that occurred during the study period. Before the introduction of Guided Design, engineering students' average freshman GPAs were below the university aver- age; after Guided Design, their GPAs were well above the average. Students who had participated in the Guided Design program as freshmen also had higher four-year GPAs than (transfer) students who had not participated. During the same period, entering stu- dents' ACT (American College Testing Program Assessment) scores remained roughly constant. The percentage of students completing the four-year course also increases; thus, the grade increase cannot be attributed to a more selective university policy. Other Guided Design users have reported similar results. Other problem-solving programs have not reported this kind of extensive quantitative data, but several document favorable student evaluations of their programs and describe examples of improved problem solving displayed by individual students (e.g., Fuller, 1975; Reif and St. John, 1979; Woods et al., 1984~. In general, most program authors cited here can point to long-term use of their courses on their campuses, attesting to both faculty and student enthusiasm. Further, because these programs are designed, by and large, to teach skills that are directly desired in their disciplines, the question of transfer is not as relevant as for some of the other programs to be discussed. Nevertheless, more attention to evaluation issues-and especially the use of more informative measures than overall gracle averages would strengthen the case for these types of courses. General Problem-Solving Skills Another group of programs aims to teach general problem- solving abilities that will be applicable in many different settings.
22 EDUCATION AND LEARNING TO THINK The CoRT Thinking Program (de Bono, 1976, 1985) and the Pro- ductive Thinking Program (Covington, 1985, in press) represent two visible and useful examples of this kind of program. CoRT grows out of a tradition of training executives and design- ers to increase fluency and creativity in practical problem solving (see de Bono, 1970~. A version of the program suitable for schooichil- dren recently has been produced and commercially marketed. It is probably the most widely used thinking skills program, having been translated into several languages and officially aclopted for school use in several countries. CoRT focuses on mastering a set of ~attention- directing" tools that, when applied, lead one to consider multiple sides of an issue, to consider consequences, to select objectives and weigh factors involved in a situation, to generate and evaluate evi- dence, and the like. Lessons are as content-free a" possible- that is, they use farn~liar situations and very short presentations to establish contexts in which the tools can be used. A great premium is placed on quick use of taught strategies and on the number and variety of ideas generated. De Bono refers to this as perceptual rather than log- ical thinking and is more concerned with effective "real-life" thinking than with improving school performance. The Productive Thinking Program was designed specifically for upper elementary schoolchildren. It, too, teaches a variety of strate- gies for planning, managing, and monitoring one's own thinking. Although stated in quite different language and embedded in more complex (though still nonacademic) problem settings, the strate- gies taught appear similar in intent to those of the CoRT program. Both programs seem to teach versions of the planning and metacog- nitive strategies that have been identified in information-processing research on problem solving (cf. Poison and Jeffries, 1985) along with the kind of fluency in idea generation associated with certain defini- tions of creativity. Covington's theory and program also emphasize motivation and self-concept, helping students to think of themselves as problem solvers and to resist immobilization caused by fear of failure. The Productive Thinking Program has been evaluated quite extensively over a number of years (see Covington, in press, for the most recent reports). There ~ evidence that students in the program become good at generating ideas and questions and increase their use of the planning strategies in the kinds of problem situations on which training is given. Furthermore, trained students' advantages last for some months. Most important, students seem to apply the
LA UREN B. RESNICK 23 program's planning strategies (e.g., analyzing the task, outlining an action plan) to school tasks such as preparing a report or exhibit. However, the latter assessments consisted of self-reports; therefore, we do not know if students actually apply these skills in practice. The CoRT program has been evaluated less often than its wide- spread adoption might suggest. Nickerson et al. (1985, pp. 217- 220; see also Edwards et al., 1984) summarize several studies; these show that students taking the course tend to become substantially more fluent in producing ideas, may make some progress toward higher levels of abstraction, and may take a more balanced view of problems. Changes also often occur in students' conceptions of them- seIves as learners. However, these findings come from performances on problems very similar to those used in the CoRT training. The only assessments of transfer to practical or school problem solving come from students who report using the strategies in their everyday lives. Thus, judgments of CoRT's educational value must depend on the importance one attaches to the strategies directly taught and to ideational fluency as such. We clo not have empirical evidence of the kind of effects these have on school learning or on success in practical problem solving, although many people fee} that the CoRT program has helped them or their children in both. Reading and Study Strategies Perhaps the largest set of training approaches and programs is clirected at teaching strategies for reading and studying from texts (e.g., Dansereau, 1985; Jones et al., 1985; Jones et al., 1984; Paris et al., 1984; Weinstein and Underwood, 1985~. Programs for enhancing reading and studying skills have been developed for virtually every educational level from elementary school to the university. Some authors stress the study skill aspect of their programs; others em- phasize the reading skill aspect. In fact, however, it is often difficult to distinguish between the two. Programs anti research studies use different labels to describe a common set of strategies including skim- ming, using context to figure out words ant] meaning, self-testing to check one's understanding, and generating summaries as one reads. The strategies taught in these programs are all based on cognitive research in reading; they involve various kinds of elaborations the reader can make on the basis of the text. The strategies taught are those that have been observed in expert readers and in strong stu- dents but that are often found to be lacking in weaker readers. They
24 EDUCATION AND LEARNING TO THINK are also strategies that accord well with theories of reading expertise and with cognitive science models of the reading process. Some techniques are reminiscent of older study skill techniques. These include special forms of notetaling intended to highlight rela- tions among different parts of the text's content and to help readers organize their knowledge (Dansereau, 1985; Jones et al., 1985~. Tr some cases, the study skills and reading strategies are embedded in fairly extensive programs that also help students plan their time, manage study activities, control anxiety and mood, and apply delis orate learning strategies in typical academic study situations (e.g., Dansereau, 1985; Weinstein and Underwood, 1985~. Considerable effort has gone into quantitative evaluations of these strategy training programs. Evaluation results reveal the the- oretical and practical complexities of these research efforts. Paris and his colleagues, for example, have studied carefully the ejects of training elementary schoolchildren in strategies such as skimming, using context to figure out unfamiliar words, and taking notes (Paris and Jacobs, 1984; Paris et al., 1984~. In a series of studies, they have shown that students became more aware of comprehension strategies and report using them more often. On the other hand, the effect of these improvements on general reading skill is slight when measured by traditional comprehension measures, which typically require an- swering questions about short passages. The trained children do excel in tasks that evoke deliberate attention to the structure and meaning of the text, such as detecting errors and filling in missing words. Because good performance on such tasks is known to corre- late well with reading comprehension, one might expect transfer to the more commonly used passage comprehension measures. Deter- mining why such transfer does not occur or what additional training features might produce transfer is likely to occupy investigators in the field for some time. Weinstein has reported that her college-level study skills course has positive effects on reading performance, using a general reading test. She also documents Towered test anxiety and improvements in student-reported study habits. Dansereau h" shown similar results, using more direct study measures in which students were given an hour to study 3,00~word passages and were tested a week later; these tests included essay questions as well as more standard test items. As with other programs, some evaluation problems did exist. In both program evaluations it was difficult to establish optimal control groups. Furthermore, the effect of a total study skills program, rather
LAURENB. RESNICK 25 than the effect of a particular study strategy or teaching method, was under scrutiny. However, Dansereau has also conducted a number of separate studies of particular component strategies. This mixture of global evaluation with detailed analyses of the effects of specific component strategies, pursued in a cumulative fashion and extended so that long-term effects and transfer can be evaluated, is precisely what we need to establish which elements of complex programs are important to their overaI! ejects. Self-Monitor~ng SkiBe Direct strategy training may be only partially helpful in in- creasing performance because many individuals primarily lack good judgment regarding when strategies should be applied. Extensive re- search supports this prediction. For example, research with retarded individuals shows that it is relatively easy to improve memory task performance by simply instructing people to rehearse or to engage in verbal elaboration and other mnemonic activities. Typically, the im- provement comes almost immediately, suggesting that the strategies are, in some sense, already known. However, in these studies there was almost complete lack of transfer, even to tasks that were only slightly modified. This meant that retarded individuals' difficulty was in not knowing when memory strategies were called for rather than in being unable to use the strategies. Recent training stud- ies that focused on appropriate application of strategies have shown more promising results (see Brown et al., 1983, for a review of this research). Overuse of deliberate strategies can also be maladaptive. Read- ing would be neither pleasurable nor efficient if one continuously slid the kinds of deliberate processing taught in the study skill exper- iments just described. These strategies are useful when automatic processing breaks down, but they can be very intrusive and disrupt tive when applied unnecessarily. The more skilled the reader, the more likely he or she will know when to apply the strategies and when to avoid them. Weak readers tend to apply strategies indis- criminately, thus disrupting comprehension, or tend to drop them entirely when there is no longer a teacher present to insist on their use and demonstration. Because of these observations, some investigators have suggested that readers-particularly weak readers- might profit more from developing self-monitoring skills than from practicing specific text
26 EDUCATION AND LEARNING TO THINK interpretation strategies. PaTincsar and Brown's (1984) work rem resents the most striking advance in this direction. Working wit middIe-schoot children who had extremely weak reading comprehen- sion skills, they introduced a process of Reciprocal teaching" in which children worked cooperatively to develop an interpretation of a text. To facilitate interpretation, children took turns posing ques- tions about and summarizing the texts. Sometimes they also made predictions about what would be said in a following section of text or asked for clarification. The teacher modeled these processes for the children in think-aloud form. Other group members commented on the quality of questions or summaries and tried to help improve them. There was no practice in answering questions or in any particular strategies for using context, analyzing words, or the like. Reciprocal teaching sessions were conducted daily for several weeks. During this training period the children's skill at answering questions about passages that they read privately also began to rise. They maintained improved reading test performance even after an eight-week period without reciprocal teaching sessions. Furthermore, scores on science and social studies comprehension tests, given in the classroom rather than in the special reciprocal teaching laboratory, also rose significantly. Comparisons with groups of children who en- gaged in intensive reading practice without the reciprocal teaching support establish the importance of reciprocal teaching in producing these results. These lever of retention and transfer are rare in educa- tional intervention studies. More important, accumulating evidence demonstrates that variants of reciprocal teaching can be effectively carried out by regular classroom teachers as part of their normal instruction. Other studies focusing on self-monitoring and meaning construc- tion skills have also shown promising although not an dramatic results as Palincsar and Brown's (e.g., Bereiter and Bird, 1985; Collins et al., 1981; Day, 1980~. In all of these studies, learning proceeded in a social setting in which tutor and students shared responsibil- ity for text interpretation. The tutor modeled certain interpretive processes; these were then taken over by students. There was some attention to building students' awareness of their own level of under- standing as well. Schoenfeld (1985) has used a similar approach in teaching mathematics problem solving. The findings on reciprocal teaching and its cousins point to a promising educational intervention. However, they also highlight how little we know about exactly how such training produces its
LA UREN B. RESNICK 27 effects. How can instruction focused on overt, self-conscious strate- gies that may not be actual components of skilled performance im- prove normally automatic processes? Some cognitive scientists be- lieve that question asking and summarizing become automated in the course of learning ant] are present in skilled reading in abbreviated, fast, and therefore largely invisible form. Others suggest that these abilities are not actively invoked during the course of automatic comprehension although they may well be used during studying and when smooth comprehension breaks down. In that case, the monitoring strategies taught and children's subsequent skilled read- ing performance would be only indirectly related. Perhaps prac tice in deliberate, rn~ndful, or ~intentional" reading activates certain powerful knowledge structures that can be applied in subsequent reading. Perhaps practice mitigates emotional difficulties associated with years of perceiving oneself as a poor reader. At present, many explanations seem possible, but the actual learning mechanisms have not been identified. Research has located a "psychological spacer in which eclucationally powerful effects seem to occur, but it has not yet adequately explained what happens in the space to produce the effects. Until we can provide a more substantial theoretical expla- nation, we can probably expect mixed results from both laboratory and classroom experiments aimed at training self-monitoring skills and strategies because it will be difficult to determine in advance the essential components of a training approach. Con~onents of Intelligence A number of programs aim to improve general intelligence through special training. Among the best known of these are Whim- bey and Lochhead's (1982, 1984) program for high school and college students, Feuerstein's Instrumental Enrichment Program (Feuerstein et al., 1985), the Venezuela Project Intelligences program (Bolt Beranek and Newman, 1983), and Sternberg's (1986) program for developing practical intelligence. The program of actually defining intelligence is addressed only indirectly by most of these program developers. Their programs pros vice practice and feedback on the kinds of tasks that usually appear in intelligence and aptitude tests. These include vocabulary-building activities, exercises involving synonyms and antonyms, analogies, spatial reasoning items, and certain kinds of logic tasks of a more or
28 EDUCATION AND LEARNING TO THINK less puzzlelike nature. By including such tasks, the program devel- opers implicitly accept the validity of established tests as indicators of intelligence. However, the history of the field (e.g., Journal of Ed- ucational Psychology, 1921; Sternberg and Detterman, 1979) shows that psychologists have never arrived at a fully satisfactory definition of intelligence. Recognizing this limitation, two of the programs extend their reach substantially beyond the usual testlike tasks. Sternberg's pro- gram aims to teach problem-solving techniques drawn from cognitive research, strategies for memorizing and reading, various practical skills (e.g., interviewing and clinical reasoning), and methods for overcoming emotional blocks. The program text, intended for high school or college courses, assumes that students' performance will improve when they receive information about psychological theo- ries. In this sense, it can be seen as the most recent in a series of self-improvement courses designed by psychologists to reflect cog ~ _ ... . .. · ~ ~ 1 t~ ~ ~ [I I__ 1~Q1. nitive research on th1nkmg and problem solving (CI. Hayes, loot; WickeIgren, 1974~. The Venezuela Project Intelligence course also includes tasks that go beyond intelligence test types of exercises. These include lessons on the structure of language and the analysis of arguments that are similar to material taught in the informal logic and critical thinking programs discussed in the next section of this essay. Other lessons cover the use of graphic, tabular, and simula- tion representations. A range of problem-solving, decision-making, and design activities, similar to those included in programs on prow lem solving in the disciplines, is also included. By contrast, several programs marketed under the titles of Critical thinking," "reason- ing,~ or Thinking skilled are actually composed mainly of testlike exercises. Two intelligence training programs, Whimbey and Lochhead's and Feuerstein's, particularly stress social mediation in learning cog- nitive skill. Whimbey and Lochhead suggest that their exercises be used in a Pair problem-solving" process in which students alter- nate the roles of problem-solver (thinking aloud) and listener-critic. The intent, as in some of the mathematics and engineering problem- solving programs described earlier, is to make the problem-solving process overt and to give students practice in analyzing problems and working through errors rather than avoiding them. Feuerstein's Instrumental Enrichment Program is intended for functionally back- ward students. It tries to provide, in condensed form, the kind of
LA UREN B. RESNICK 29 help in explicitly analyzing tasks, formulating strategies, and e~ralu- ating outcomes that is provided incidentally in normal development through interaction with parents and other caretakers. In Instru- mental Enrichment training, student-teacher interaction, together with specially structured group discussions following the completion of indiviclual exercises, plays this mediating role. Adequate program evaluation is sparse, except in the case of the Venezuela program. That program has been subjected to a fairly extensive evaluation involving experimental and control classes in the seventh grade (students aged 11-17) in barrio (impoverished urban district) schools. Evaluation demonstrated a clear effect of the course on a verbal A measure and on several general ability tests, including reacting. Experimental students performed better than control students on several measures of the skills directly taught in the course. In addition, a smaller student sample also took special oral and written posttests assessing qualitative aspects of thinking such as appropriateness of a design, clarity of expression, and use of supporting reasons. Here, too, the experimental group outperformed the control. The special posttest in the Venezuela evaluation is important because it examines transfer of the skills taught to educationally and practically relevant tasks. Researchers must establish this kind of transfer whenever teaching focuses on activities that are valued because of their association with socially valued competence, rather than valued for their own worth. This is clearly the case for A tests. These tests are used in evaluation studies because the tests are quite good at predicting school performance. But students trained to do well on the tests themselves will not necessarily clo better in school. IQ tests probably correlate with school performance mainly because doing well on both the A tasks and school tasks depends on learning abilities and strategies not directly observed in either. Therefore, specialized, targeted training on {Q-like tasks may not generalize. Direct assessment of transfer is needecI. Unfortunately, apart from the promising but limited evidence from the Venezuela program, such assessments have not been made. Performance on particular types of items or on A tests as a whole has been shown to improve with training (e.g., Feuerstein et al., 1985; Sternberg, 1986~. However, evidence that improved test scores predict improved performance on problem solving or learning tasks closer to those of school or great lifer is rare (see Lochhead, 1985, for a perspicacious discussion of the difficulties of evaluations that include this kind of transfer criterion).
30 EDUCATION AND LEARNING TO THINK Informal I`ogic and Critical Thinking The final approach to the teaching of higher order skills to be considered here emerges from a philosophical rather than a psycho- Togical tradition. In the put several years philosophers at a number of universities have turned their attention to problems of teaching general reasoning and argumentation skills. Their work is rooted in ancient traditions of rhetoric and In recent work on the logic of argumentation (see, e.g., Toulmin et al., 1979~. The current focus on the analysis of extended discourse on complex topics, usually social issues, represents a new thrust within philosophy, offering an alter- native to the traditions of mathematical logic and formal proof. The new approaches maintain the normative stance of philosophy; they prescribe acceptable forms of thinking based on standards of logic. This contrasts with psychologists' efforts to discover and then to teach students the actual processes used by good thinkers. Philoso- phers promote an approach designed to discipline thinking and to guard against the propensities of humans to accept fallacious ar- guments and draw inappropriate conclusions. Indeed, the scholarly heart of the informal logic movement is the analysis of fallacies com- mon in undisciplined reasoning. Most efforts to teach informal logic have focused on college-level courses. Although organized Programs at this level are uncom- mon, certain textbooks that are frequently used for informal logic courses provide a reasonable sense of the field (see Johnson, 1981, and Johnson and Blair, 1980, for reviews and analyses of several of these texts). The books typically contain examples of texts for analysis and often present techniques for displaying the relationships among various segments of an argument. In most cases, the texts emphasize identification of particular reasoning fallacies and include technical vocabulary for describing argument structures and their associated fallacies. In addition to philosophers, a small number of people from other disciplines are linked to the informal logic mover meet. For example, rhetoric has become a major element in many English departments; in these programs, courses in writing and com- position often concentrate on principles of argument construction (see Lazere, 1982, for one such approach). Some social scientists (e.g., Browne and Keeley, 1981; Hursh et al., 1983) have developed courses and textbooks in critical thinking that share the concerns of the informal logic movement, although not always the particular analytic vocabulary.
LA UREN B. RESNICK 31 Extensive attention to informal logic at the elementary and sec- ondary school levels is quite recent. It has been spurred by the recent press for critical thinking in the schools and by the inclusion of critical thinking components in some states' competency testing programs (e.g., California, Connecticut, and New Jersey). The only fully developed and extensively assessed program for precollege stu- dents is Matthew Lipman's Philosophy for Children. Philosophy for Children's basic teaching method is extensive discussion organized around issues raised in the course of storylike texts. These texts pose traditional philosophical problems problems of meaning, truth, am thetics, reality and imagination, ethics, and the like. In this context, a variety of informal logic skills all focused on logical relations as expressed in ordinary language-are expected to be developed. The oldest and most widely used text, Harry Stottlemeier's Discovery (Lipman, 1974/1982), is aimed at fifth- and sixth-gracie students. Texts exist for younger and older students as well. This brief consideration cannot do justice to the variety of prac- tice and range of opinion in the critical thinking and informal logic movement. For example, some programs focus largely on identifying and correctly labeling reasoning fallacies; others concentrate more or developing skills of argumentation in extended discourse, without extensive formal analysis. An important debate in the field exactly parallels psychologists' discussions of whether general cognitive skills or specific knowledge is most central to intellectual competence. Most informal logic philosophers believe that general reasoning capacity can be shaped and that it transcends specific knowledge domains (e.g., Ennis, 1980, 1985~. In an even stronger claim, Paul (1982, in press) argues that we should seek to develop in students a broadly rational personality rather than any set of technical reasoning skills. This view usually, but not always, supports calls for independent critical thinking courses. However, a competing view, most strongly stated by McPeck (1981), argues that no general reasoning skill is possible and that all instruction in thinking should be situated in particular disciplines. Despite their parallel concerns, psychologists studying the teachability of cognitive skills and philosophers promot- ing critical thinking instruction have communicated very little with one another. That is beginning to change, with each group express ingmoreinterestin the other's work (e.g.,Norris, 1985; Perkins, 1982), and more mutual influence is probable in the future. The college-level courses discussed here have enjoyed little or no formal assessment apart from regular course examinations. There is
32 EDUCATION AND LEARNING TO THINIf an implicit claim that the kinds of analysis taught in informal logic courses can and should permeate performance throughout the uni- versity curriculum, although this has not been tested empirically. As in the case of science, math, and engineering problem-solving courses, then, judgments of the educational importance of university-level in- formal logic courses must depend for the moment on the extent to which the forms of argument analysis taught are judged to be valu- able aspects of learning in their own right. Several evaluations of Philosophy for Children, most of which were conducted by evalua- tors not directly connected with program development or implemen- tation, provide evidence that the program when well implemented and given adequate time in the instructional calendar can produce rather general gains on tests, including improvement on reading com- prehension and A scores (Lipman, 1985~. This program, then, more than most, has been subjected to evaluations on a transfer criterion and has fared quite well. Problems of Assessment: Some General Comments Before summarizing the evidence on the teachability of general thinking skills, it is important to reflect on the question of what constitutes appropriate evaluation of programs designed to teach problem-solvir~g and reasoning skills. The most common evaluation reported for the programs we have considered is mastery performance (Arbitman-S~riith et al., 1984), that is, performance on exercises sim- ilar to those included in the program itself. In other words, evaluation provides evidence that students who have used a program learn to clo the things the program teaches. This is a necessary first evaluation step, a minimal test that the program in question is worthwhile. Although necessary, such evidence is rarely sufficient to establish the program's educational value. If the program teaches skills that are in themselves considered valuable, then clear evidence that stu- dents learn and maintain those skills Is adequate. But if a program is meant to teach skins that facilitate other learning but are not valued in themselves, then more is needed than merely tests of the performances directly taught. In these cases, assessments of trans- fer beyond the course or program itself must be included. Various measures of such transfer can be used, including standardized test scores, subsequent grade point averages, measures of course reten- tion, or advanced program placement. What matters is that the ultimate measures assess socially valued performances.
LAUREN B. RESNICK 33 There are strong theoretical and practical reasons for this. Even when two measures have been correlated repeatedly for example, Scholastic Aptitude Test (SAT) scores and college grades-nothing guarantees that the correlation will still exist if conditions leading to high scores in either measure are changed. Under normal learning conditions it is safe and practical to treat SAT scores as an indicator of probable college grades. But if special, targeted training produces an increase in SAT scores, one cannot safely assume that college grades will also go up. The correlation was establisher} under partic- ular learning conditions; if those conditions change, the correlation must be reestablished by verifying empirically that the program pro- ducing increased SAT scores also produces increased college grades. The same is true for metacognitive skills associated with reading. We know that students who perform well on standardized reacting tests usually exhibit more metacognitive behaviors such as elaborat- ing on what the text says, summarizing as they react, and raising questions. But this does not necessarily mean that if we teach stu- dents to elaborate, to summarize, and to ask questions, their reading test scores will go up. Useful evaluations of higher order skill train- ing programs require that the educational outcomes of interest be directly assessed. We cannot afford to rely on evidence that cer tain performances traditionally associates! with strong educational outcomes have improved. On this criterion, even reading tests, probably the most fre quently used measure in the studies reviewed, are somewhat prom lematic. These tests examine abilities that are themselves valued. They are thus better for evaluation purposes than intelligence tests. However, many of the higher order training programs aspire to types and levels of cognitive functioning to which standardized reading tests are not likely to be adequately sensitive. How, for example, should we assess whether skills of argument analysis have permeated students' study of the social sciences or their reading of the ciaily newspapers? How can we determine whether the problem-solving skills taught in a high school or freshman college course have altered performance in science courses or on-thejob creativity? A crude (and not infrequently used) indicator of academic improvement is course grades. But even gracles are only indirect indicators of changed cog- nitive abilities. They do not reveal the quality of thinking, and they offer no indications of transfer beyond purely academic settings. Clearly, a most important challenge facing the movement for increasing higher order skill learning in the school is the development
