2
Research and the Improvement of Education

Few Americans would question the proposition that research has been a potent force for improved medical care in the twentieth century. Few would deny that research has played an equally powerful role in the emergence of modern agriculture. When it comes to education, however, research enjoys no such flattering reputation. Whether or not the judgment is justified, research in education is more likely to be dismissed as trivial or irrelevant than it is to be considered a fundamental ingredient in understanding how children learn and in improving how they are taught.

One example of this low regard is the very small portion of federal research and development funding that goes to education: slightly more than $350 million of $64.1 billion in fiscal 1991—one half of 1 percent. In comparison, the federal government spends 3 times as much on research and development activities for agriculture, 21 times as much for space research and development, and 30 times as much for research related to health. If relative funding levels are any indication, Congress is clearly not convinced that federal support of research can benefit public education in the way it has benefited the nation's health and agriculture. Members of Congress are not alone in their general low regard for research as an integral part of a robust system of education: teachers commonly indicate that they do not use research and do not see its connection to what they do on a daily basis in the classroom (Louis et al., 1984).

There are many reasons for the undistinguished reputation of research in education, only some of which are well founded. Part of the cause can be



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement 2 Research and the Improvement of Education Few Americans would question the proposition that research has been a potent force for improved medical care in the twentieth century. Few would deny that research has played an equally powerful role in the emergence of modern agriculture. When it comes to education, however, research enjoys no such flattering reputation. Whether or not the judgment is justified, research in education is more likely to be dismissed as trivial or irrelevant than it is to be considered a fundamental ingredient in understanding how children learn and in improving how they are taught. One example of this low regard is the very small portion of federal research and development funding that goes to education: slightly more than $350 million of $64.1 billion in fiscal 1991—one half of 1 percent. In comparison, the federal government spends 3 times as much on research and development activities for agriculture, 21 times as much for space research and development, and 30 times as much for research related to health. If relative funding levels are any indication, Congress is clearly not convinced that federal support of research can benefit public education in the way it has benefited the nation's health and agriculture. Members of Congress are not alone in their general low regard for research as an integral part of a robust system of education: teachers commonly indicate that they do not use research and do not see its connection to what they do on a daily basis in the classroom (Louis et al., 1984). There are many reasons for the undistinguished reputation of research in education, only some of which are well founded. Part of the cause can be

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement found in the practical orientation of teacher education. Schools of education generally do not prepare the nation's future teachers to value disciplined inquiry or even, at a more mundane level, to keep track of relevant research. Once on the job, the conditions of work do not encourage school teachers to study the research literature. No matter how enlightening research may be, it cannot contribute to improvements in education if it is not understood, used intelligently, and refined in the context of local experience. This situation is aggravated by the national tendency to want quick solutions to problems—even if they have been generations in the making. Much of the public discussion of education research has a distinctly utilitarian cast: it assumes that researchers conduct studies, their findings are translated into products or programs for use in the schools, and education is improved. This view is at once too narrow and too grandiose. It implies that the only valuable research is research that can be directly translated into classroom practice, a view that gives short shrift to much research. And it encourages unrealistic expectations about what research can—or should be able to—accomplish. The effects of research on educational practice are seldom straightforward and quick. As in other fields, there are few definitive studies, but rather a gradual accretion of knowledge drawn from overlapping studies in many fields of study, conducted over a long period of time, punctuated by an occasional breakthrough. In physics and chemistry, as well as social and behavioral science, decades of basic research provide the seed bed for new approaches and methods. Improvement in education will occur only if all participants—parents, students, teachers, the public, and policy makers—are willing to make strong intellectual commitments to work together using new insights, approaches, and techniques to improve education. The undistinguished reputation of education research is also partly attributable to some of the work. There has been some methodologically weak research, trivial studies, an infatuation with jargon, and a tendency toward fads with a consequent fragmentation of effort. The committee, however, does not share the widespread negative judgments about the contributions of research to the reform of education. Our review of research-based programs to improve teaching, strengthen curricula, restructure institutions of learning, and assess and monitor the progress in U.S. schools has convinced us not only that research can improve education, but also that it has been demonstrably useful. This chapter provides a brief introduction to a few of the contributions from research and development for education. Some of the work has been funded by the Office of Educational Research and Improvement, some has been funded by the National Institute of Child Health and Human Development and the National Science Foundation, and some by other federal agencies. We first discuss just one stream of basic research, cognitive science,

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement and how it has informed understanding of the teaching and learning of mathematics and reading. We next introduce seven innovative curricula and teaching approaches, several of which are based partly on the findings from cognitive science, and then two school restructuring processes, based on other research from the social sciences. The third section describes some of the major efforts to monitor the status of American schools and teachers and the achievement of students. Finally, we discuss some of the ways in which Congress and congressional agencies have used education research. There is much research and development that we have not covered, such as work on the social and cultural contexts of school and learning, school finance, the economics of education, administrative and organizational studies, classroom observational studies, curriculum analysis, and studies of postsecondary education. Our exclusion of these lines of work is no reflection on their importance but rather a result of the committee's limited time and our decision to cover fewer topics in greater depth. For a broader introduction to the field, see the Encyclopedia of Educational Research (Alkin, 1992). BASIC RESEARCH IN COGNITIVE SCIENCE Research has enriched knowledge of learning and teaching in many ways. One of these is knowledge about early development of thinking, reading, and mathematics skills. A number of the basic theories of the development of cognitive processes presented in this section informed the design of programs discussed below. In some cases, the findings of cognitive researchers have reinforced traditional practices used to assist children in acquiring reading and mathematics skills. For instance, the practice of reading with a child and discussing the story has been shown to build cognitive skills of summarizing, clarifying, predicting, and questioning. But just as often, cognitive researchers working in areas such as artificial intelligence and expert systems have suggested new approaches to teaching. For many years the principles espoused by B.F. Skinner dominated human experimental psychology. His approach was based on determining the relationships between observable stimuli and observable responses, with little consideration for what went on in between. Since the late 1960s, however, the emphasis has shifted to the study of cognitive processes, modeling what the mind knows and how it knows—an approach that is more compatible with providing guidance for teaching and learning. According to Resnick (1987a:7): The process of making explicit the abilities formerly left to the intuitions of gifted learners and teachers is precisely what we need to establish a scientific foundation for the new agenda of extending thinking and reasoning abilities to all segments of the population.

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement Cognitive scientists—including researchers in psychology, computer science, linguistics, neuroscience, and anthropology—have differentiated and expanded understanding of how thought and knowledge develop and interact. The notion of schemata, first discussed by Bartlett in 1932, has reemerged as a principal concept. A schema is a mental framework for acquiring and organizing new knowledge and skill and interpreting new experience; it also contains both the elements of knowledge and the rules for relating the elements. The development of expertise involves more than the acquisition of new knowledge, it involves the remodeling of one's prior perspective. According to cognitive theorists, individuals have several schemata, each of which may result in a different interpretation of an event. Thinking skills are sets of strategies for analysis and self-regulation that build on prior knowledge and experience and generate increasingly complex frameworks for understanding (Chipman et al., 1985; Glaser, 1984; Resnick, 1989). Some aspects of thinking are common across domains; others are quite specialized and domain specific (Benton and Kiewra, 1987). And thinking is influenced by social support, shared experience, and role models (Brown and Palincsar, 1989; Rogoff and Lave, 1984). In the past it was believed that young children were essentially empty vessels to be filled with knowledge, and when faced with unfamiliar problems, their errors were the result of random guessing. Work in cognitive science has since shown that many errors made by children in the first grade of school are based on the consistent application of incorrect rules (Brown and VanLehn, 1982; Fisher and Bullock, 1984). With this new understanding of learning processes, cognitive scientists began to explore and categorize faulty rules, looking for the principles underlying the errors made in different learning tasks. The results of this work have provided the ability to identify the cause of children's errors and to design instructional strategies to eliminate them. One approach to supporting cognitive development in young children is guided intervention, a collaborative process based on shared experiences and understanding (Vygotsky, 1978). In this approach, children develop thinking and subject-related skills through guided, social contact with adults. The adult models a behavior that is slightly beyond the child's current capabilities, coaches the child in the behavior, and guides him or her in reflecting on the new experience for purposes of mastering the behavior. In this way, the child acquires not only the new skill but also the adult's understanding of the skill. Another important line of research in cognitive science is modeling the knowledge structures and judgments of experts and novices and then comparing the two as a way to understand the nature of expertise and the training needed to turn novices into experts. For example, Chi et al. (1981) have examined the differences in the knowledge structures and problem

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement approaches of expert and novice physicists to better understand how the acquisition of knowledge and rules affects problem-solving strategies. The schemata and algorithms used by experts can be studied by using such methods as cognitive task analysis or think-aloud protocols (Newell and Simon, 1972). According to Glaser et al. (1991), models representing stages in the progression from novice to expert skill would be useful in guiding the learning process. The principles of cognitive science have provided important guidance to the developers of many promising programs on curriculum design and teaching approaches. Two examples are Cognitively Guided Instruction and Reciprocal Teaching (described below). Mathematics Education Recent research on mathematics learning shows that preschool children develop mathematical concepts that they apply to a variety of practical situations. Not surprisingly, many of their concepts and algorithms are incorrect. In a careful study of the processes used by children in multidigit subtraction, Brown and VanLehn (1983) found that many errors were systematic and consistent and could be traced to erroneous variations in procedures known as "bugs". For example, when the top digit is smaller than the bottom digit, children often subtract the top digit from the bottom digit instead of borrowing. At the time of their article, Brown and VanLehn had found 88 primitive bugs and 300 combined bugs based on children's flawed hypotheses. Many children experience difficulty in learnings mathematics in school because they and their teachers do not understand the relationship between the rules and algorithms taught in school and the children's own, independently developed mathematical intuitions. Understanding the rules followed by children as they make errors can be useful in diagnosing specific learning problems and in developing effective instructional strategies. For instance, researchers at the Learning Research and Development Center are currently working on a reasoning-based mathematics program designed to help children build on their intuitions, showing them how to correct and extend them. At the Wisconsin Center for Educational Research, Fennema et al. (1989) developed a taxonomy of word problems in addition and subtraction. The taxonomy helps teachers identify the mathematical concepts a student understands and those that must be mastered to solve given problems correctly. Reading Several lines of research have contributed to understanding how to teach beginning reading. One central stream is the decades of work on alphabetic

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement coding, phoneme awareness, and word recognition. Adams (1990), a researcher at the OERI's Center for the Study of Reading, provides a detailed analysis of this work. She describes the reading system, based on four processors: the orthographic processor perceives the sequence of letters in the text; the phonological processor maps the letters onto their spoken equivalents; the meaning processor generates meaning from words; and the context processor constructs an on going understanding of the text. Experimental research on eye movements and fixations of skilled readers provides important insights into how each of these processors is used and how they interact as a reader moves from print to meaning. Phoneme awareness has been shown to be a prerequisite for mapping alphabetic symbols to sound, and alphabetic mapping is believed to be necessary for learning to identify words. Chall (1983) found that the two best predictors of early reading achievement are letter knowledge and the ability to discriminate phonemes. Findings reported by Adams (1990) and Vellutino (1991) suggest that strategies based on the direct teaching of letter-sound combinations to facilitate the generative use of sounds for word decoding are necessary but not sufficient conditions for learning to read: meaning-based strategies are also important for comprehension. Skilled reading requires mastering the basic processes of letter and word recognition to the point that they are automatic. When reading is not fluent, comprehension has been found to be deficient perhaps because ''less than optimum facility in word identification drains off cognitive resources that would normally be diverted to comprehension processes, thereby impeding these processes'' (Vellutino, 1991:438). In 1972 a small group of distinguished social scientists was assembled by OERI's predecessor, the National Institute of Education (NIE), to examine written and oral communication from the standpoint of information processing theory. The group concluded (Miller, 1973): "NIE should actively support efforts to understand the cognitive processes involved in acquiring basic reading skills and the cognitive processes involved in comprehending linguistic messages." More than 100 researchers were subsequently involved in a consensus building process to plan an appropriate research program. In response to the ensuing request for field-initiated proposals, 100 were received and approximately 25 were funded. Funding was also provided to the Center for the Study of Reading and the Center for The Study of Writing. This work and that supported by other federal agencies substantially expanded understanding of cognitive aspects of learning to read. In a major summary of the work 10 years later, Anderson et al. (1985) indicate that constructing meaning from written text is central to the reading process and that this involves selecting and using knowledge about people, places, and things, as well as developing the skills of summarizing, clarifying, and

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement predicting. Skilled readers use prior knowledge about the topic and about the syntax of language to fill in gaps, integrate different pieces of information, and infer meaning. Consequently, skilled readers are strategic: They assess their prior knowledge on a topic; adjust their approach on the basis of the complexity of the text, their familiarity with the topic, and their purpose for reading; and monitor their comprehension. It has long been known that parents reading to young children is useful for intellectual, social, and emotional development. In addition, however, research has shown that the intellectual stimulation is enhanced if the parent engages the child in discussions of the stories. Asking children to recall facts, provide descriptions, and reflect on the experiences in the stories introduces them to reading as a constructive and strategic process (Anderson et al., 1985; Dole et al., 1991). Another new research finding involves writing. It used to be thought that young children could not learn to write meaningful text until they mastered basic reading skills. However, when researchers tried to teach writing and reading simultaneously to first and second graders, they were not only successful, but they also serendipitously found that writing instruction accelerated the acquisition of reading skills (Graves, 1983). Research on reading and writing has contributed to the development of innovative programs, such as Reciprocal Teaching, Reading Recovery and Success For All, and has informed parents, teachers, and policy makers through a series of widely distributed publications. A popular summary of the research work, Becoming a Nation of Readers (Anderson et al., 1985) was published by the Center for the Study of Reading and sold 250,000 copies. In addition, OERI published nearly a million copies of three companion booklets—for parents, for teachers, and for principals. CURRICULUM DEVELOPMENT AND IMPROVED TEACHING APPROACHES The committee's informal search for examples of programs designed to improve curriculum was not comprehensive—programs suggested by committee members and a review of Educational Programs That Work (National Dissemination Study Group, 1990) produced a list of 30 candidates. From those we selected seven that appeared to have at least moderately credible evaluation data, that had evidence of at least moderate impact, and that illuminated a variety of approaches. We did not assess whether the examples are the most effective or efficient programs available for a given purpose. The programs selected for discussion are striking in their variety, but all provide evidence that the translation of research to improved teaching and learning is a lengthy and expensive process that often requires numer-

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement ous iterations of research, development, and refinement. Two of the programs are still in the early stages of research, but all are the product of 4 or more years of work, and several have been in various stages of development or dissemination for more than 10 years. In-service teacher education and development is part of each program. For most of the programs, initial training is for less than 1 week, but Cognitively Guided Instruction requires a 4-week seminar, and Reading Recovery requires a full year of training. Federal funding for research and development portions of the programs ranged from $330,000 to $8 million. Many of the innovative programs were also supported by state and private funds and by the in-kind contributions of school districts that participated in their development and demonstration. The seven programs reviewed in this section present examples of promising programs in four areas: higher order thinking skills, reading, mathematics, and generic instructional approaches for use in any subject area. The first program, Project IMPACT, integrates instruction in thinking skills into the curriculum as part of each subject-matter course. The next two programs are designed to enhance reading instruction, and they draw on research in both reading theory and cognitive science. One of the programs, Reading Recovery, is described in detail because it offers an example of a fully developed program for the poorest readers in the first grade that has been demonstrated to have a continuing positive effect over at least the next two grades. The other reading program, Reciprocal Teaching, demonstrates Vygotsky's model of guided intervention and provides a direct test of the theory of the centrality of comprehension monitoring in strategic reading. Reciprocal Teaching also offers an example of an extensive basic research program designed to study metacognition that has also begun to benefit educational practice. The two mathematics programs are the Comprehensive School Mathematics Program (CSMP) and Cognitively Guided Instruction. CSMP offers an interesting example of a field-initiated effort based on new concepts of instruction that was subsequently developed by a laboratory of NIE. The idea of including thinking skills in the instruction of mathematics is now becoming more widespread. Cognitively Guided Instruction provides an example of how teachers can learn to diagnose a child's level of comprehension and create instruction that builds directly on that level. The program draws on research examining the mathematical concepts of preschool children conducted at the University of Wisconsin and the Learning Research and Development Center. The final two examples, computer-assisted instruction (CAI) and cooperative learning, are illustrative of successful, widely disseminated approaches that are used for instruction in all subject areas. Both approaches have a history of at least 25 years of research, development, refinement, and appli-

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement cation, and both have been studied extensively by several researchers. These long-term efforts illustrate the usefulness of research in identifying and confirming innovations that can make a positive difference in education. Student Team Learning, a cooperative learning program created by researchers at the Johns Hopkins Center for Social Organization of the Schools, is an example of a program based on prior research in social and organizational psychology and modified by the developers' own subsequent research and evaluations. All of the programs described in the section used research designs in which the performance of students in an experimental group was compared with national norms or with the performance of students in a control group. However, the committee, was generally disappointed in the quality of the evaluations available from these programs, which exhibited several problems. Part of the gains reported for some of the evaluations may be due to a statistical artifact known as regression toward the mean, a problem that occurs when subjects for a remedial treatment become candidates for selection by exhibiting lower than average achievement scores. The most commonly used criterion for program success was a statistically significant increase in performance on a standardized achievement test. Although these tests are useful, they often provide a relatively narrow indication of student achievement. In addition, few of the programs have collected follow-up data to determine whether initial effects were maintained after students left the program. In many cases, the evaluation data are only from carefully managed demonstrations and not from subsequent dissemination sites except for computer-assisted instruction and cooperative learning, where full adoption of the program is not assured. Moreover, except for computer assisted instruction and cooperative learning, all of the evidence for effectiveness has been provided by program developers. Lastly, the reports of evaluations were often less thorough than we would have liked. Despite these limitations, the programs have been subject to two or more evaluations, often using different approaches, and the results were similar. This increased our confidence in the findings. Nevertheless, because of the important limitations in the evaluations, most of the programs and approaches described should be considered promising, rather than of proven effectiveness. Project IMPACT Program Description Project IMPACT was designed to improve the critical thinking of students by incorporating instruction in thinking skills into the regular course work of reading, mathematics, and science. Development of this field-

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement initiated program began in 1979 with remedial students in grades 6–9; it has since been expanded to include all grades from kindergarten through high school. The premise of the program is that all students with intellectual potential at or near the normal range will benefit from instruction in higher level thinking skills. The IMPACT program provides curriculum materials and training. The first section of the materials is designed to assist curriculum planners in analyzing existing courses for embedded instruction on thinking skills: classifying, seeing cause-and-effect relationships, making generalizations, forming predictions, and making assumptions. The second section of the program materials is designed to help teachers fill in identified gaps in thinking skills instruction. These materials contain lesson plans for classifying and categorizing information, ordering and setting priorities for information, formulating effective questions, and various reasoning exercises (Winocur, 1987). Training is available at two levels. At the first level, teachers participate in a 3-day in-service workshop in which they are provided with strategies for introducing and guiding the process of critical thinking within the context of various subject areas. Special instruction is provided in effective use of Project IMPACT curriculum materials. The second level of training is designed to prepare graduates of the first level of training to act as trainers and disseminators in their local school districts. Testing and Evaluation The original evaluation study was conducted in 1983 and involved four school districts. All student subjects were in the seventh, eighth, or ninth grades and were enrolled in remedial reading or mathematics classes. The students selected to participate in both the experimental and control groups were those who failed to pass their district proficiency test and scored at or below the 37th percentile in reading comprehension on the Comprehensive Test of Basic Skills. The 426 students in a remedial IMPACT classes were taught by IMPACT-trained teachers; the 352 students in remedial control classes were taught by regular teachers. Standardized tests of basic skills and the Cornell Critical Thinking Test were used as measures of students' gains. Pretests were administered in September and post-tests in February; all testing was conducted by classroom teachers. The results show that students in the Project IMPACT classes significantly outperformed students in control classes in thinking skills, mathematics, and reading at all three grade levels. According to the analysis, the magnitude of the gains for IMPACT students was 1 standard deviation on both the Comprehensive Test of Basic Skills and the Cornell Critical Thinking Test, compared with an average gain of 0.1 standard deviations for students in control classes (Winocur, 1983).

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement Results obtained in a later study by the same investigator showed that 83 students in seventh grade classes taught with IMPACT significantly improved their reading comprehension between the pretest and the post-test (as measured by the Comprehensive Test of Basic Skills): the mean performance of these students increased from the 40th percentile to the 51st percentile of the norm group, which included a nationally representative student (Winocur, 1987). Project IMPACT offers an approach that can be embedded into most curricula and used as part of most teaching methods. As a result, the program appears to have widespread utility. Since 1983 the National Diffusion Network has helped disseminate this program. According to IMPACT staff, there have been approximately 6,500 adoptions across all grades. Reading Recovery Program Description Reading Recovery is an early intervention program designed to assist the lowest 20 percent of readers in the first grade, as determined by a special battery of diagnostic tests developed by Marie Clay (1985). The goal of this field-initiated program is to teach these children reading skills that are comparable to those of the average students in their class. Current data show that 86 percent of the children in Reading Recovery have successfully met that goal within 12 to 20 weeks. The Reading Recovery program is designed to be supplementary to regular reading instruction in the classroom. A child in the program leaves the classroom to work one-to-one with a specially trained teacher for 30 minutes every day. Within a lesson, reading and writing activities axe integrated, based on the idea that development in one skill area supports advancement in the other. Program materials include several hundred small books graded at 20 levels of difficulty. The easier levels of these books contain illustrations on every page and are written in predictable language patterns compatible with children's ability to understand. The approach is to encourage children to read by building on the knowledge and skills they already have. In the early stages of the program, a teacher works very closely with a child, examining each word and each sentence in a small story for recognition, pronunciation, and meaning. Throughout the story reading, the child and the teacher discuss what the characters might do next. As the child progresses through the lessons, he or she is supported by the teacher in the development of strategies that good readers appear to acquire naturally, such as summarizing, clarifying, and predicting. Teachers stress meaning cues and comprehension in both reading and writing activities. Children are considered ready to leave the program when they have

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement Throughout the program's development and implementation, the focus was on involving the parents, encouraging participation among all interested parties, and working to create an understanding of children's emotional, social, and academic needs. The mental health team worked with children and taught their teachers how to respond appropriately to disruptive or antisocial behavior. A special program, the discovery room, included a variety of tools, toys, and material for students to use individually, in pairs, or in groups, and was staffed by an understanding and accepting teacher who helped the children work through their fears, anxieties, and anger. Many of the children in these schools were not emotionally or socially prepared for school: they came from insecure family environments that did not encourage cognitive or emotional growth. As a result, it was necessary for the school to provide an environment for this growth. In addition, to create more stability for the children, they were assigned to the same teacher for 2 years. Administrators, parents, and teachers worked together on all aspects of the schools' programs. Curriculum changes and the introduction of special innovations, such as the discovery room, were contingent on the approval of parents. According to Comer (1980, 1988), parental participation was critical to program success. The first year of the project was marked by dissension and lack of parental support, but by the second and third years there was growing cooperation and participation. In the second year, teachers developed a set of guidelines for parents to use when observing a class: what to look for, what sorts of questions to ask, etc. Testing and Evaluation Data reported by Comer (1988) show marked improvement for mathematics and reading scores of fourth graders attending the first two New Haven Schools; no performance data have been provided on students in other grades. In 1969 the fourth grade students in these schools were functioning slightly below a third grade level; 10 years later they were performing at grade level; and by 1984 they were scoring 2 years above grade level. Moreover, school attendance, at all grade levels, improved to second highest in the city, and student conflict was reported as minimal. The program was further evaluated by developers in a 1987 study using a randomly selected sample of 306 African American students in grades 3–5. Of the total sample, 176 students were attending seven School Development Program schools around the country, 91 were attending four control schools (comparable schools not using the program), and 39 were in three special schools. The results show significant gains in reading scores, as measured by classroom grades, for students in program schools but not for students in control schools; there were no significant gains in mathematics for either

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement group. Children in program schools were significantly more positive toward their classroom environment after the program was introduced, and absenteeism declined significantly (Comer et al., 1989). In another study, students in ten predominantly African American schools in Prince Georges County (Maryland) using Comer's program showed significantly higher percentile gains on the California Achievement Test than those reported for the district as a whole (Comer, 1988). Between 1985 and 1987 third grade students in Comer's program schools gained 18 percentile points in mathematics, 17 in language and 9 in reading; throughout the district, students gained 10 percentile points in mathematics, 8 in language, and 5 in reading. At the fourth grade level, students in the program schools gained more than 20 percentile points in mathematics, 12 in language, and 7 in reading, compared with district-wide gains of 11, 7 and 4 percentile points respectively. Prince Georges County is the fifteenth largest in the country and has 105,000 students, 62 percent of whom are African American. Outcomes Driven Development Model Program Description Development of the Outcomes Driven Development Model (ODDM) began in 1971 in response to dissatisfaction with student performance of administrators and teachers in New York's Johnson City Central School District. ODDM is a procedural model involving the direct participation of teachers, administrators, and boards of education in restructuring all aspects of a school to achieve a specified set of outcomes. ODDM provides a set of procedures for aligning all facets of a school or district with a goal of excellence in student achievement and for guiding teachers and administrators in using the best research available for these purposes—research on instructional practices, curriculum design, school climate, change theory, and school management. One of the first steps involves changing the belief systems of teachers and administrators concerning student capabilities. ODDM creates conditions in which all teachers and administrators can participate in decisions that influence the direction of the organization. Restructuring also involves changes in the administrative supports, the classroom supports, and the community supports: ODDM provides a blueprint for making these changes. All adopters are required to follow an eight-phase training plan over a 2-year period, starting with the development of a leadership team and ending with diffusion; the training is provided by Johnson City Central School District staff and other certified trainers. All adopters are also required to carry out the full ODDM process. Since the first work at Johnson City, the program has been expanded to serve high schools.

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement Testing and Evaluation For the developers, the original goal was to have at least 75 percent of the students achieve scores of 6 months above grade level in reading and mathematics by the end of the eighth grade. In 1976 the percentage of eighth grade students at or above this level was 44 percent for reading and 53 percent for mathematics; 7 years later the percentages were 75 percent for reading and 80 percent for mathematics. In Utah, five districts have used ODDM for 3 or more years; four of them have data showing dramatic improvements, using pretest and posttest performance on the Comprehensive Test of Basic Skills. One district showed average increases of two and three grade levels by fifth and sixth grade students; a second showed math score increases of 1.5, 2.0, and 3.0 grade levels by students in the third, fourth, and fifth grades, respectively; a third raised the average reading percentile approximately 5 points and the average math percentile by 10–20 points; and the fourth raised reading and language arts percentiles by approximately 10 points and mathematics percentiles by 15–20 points. A particularly encouraging aspect of these findings is that the effects appear to be cumulative—probably because of the systematic redirection of instruction throughout all the grade levels in a school or district. ODDM has been actively disseminated through the National Diffusion Network since 1985. Future plans for the project involve additional evaluation and continuing dissemination. MONITORING THE STATE OF PUBLIC EDUCATION One important function of education research is to inform policy makers about the course of education. In this regard, the federal government, principally through the National Center on Educational Statistics (NCES), has long played a major role by large-scale collection of data on the condition of education in this country. NCES produces data on the demographic, financial, physical, and performance characteristics of U.S. school systems. In this section we discuss some of the most widely used databases and reports prepared by NCES, including statistical compilations describing current conditions, longitudinal studies, national assessments of student performance, and international comparisons. National Statistics The principal descriptive databases currently prepared by NCES include the Common Core of Data, the Schools and Staffing Survey, the Integrated Postsecondary Education Data System, the National Postsecond-

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement ary Student Aid Survey, and the Survey of earned Doctorates Awarded in the United States. The Common Core of Data (CCD) on state education agencies, local education agencies, and public schools is collected annually. It includes enrollments by grade, the racial and ethnic composition of the enrollments, the percentage of students eligible for the free lunch program, number of handicapped students, number of graduates, number of teachers and other staff, teacher salaries, and an array of other financial data. The Schools and Staffing Survey (SASS) collects more detailed information on school staff and workplace conditions from a sample of public and private schools. The Integrated Postsecondary Education Data System (IPEDS), which superseded the Higher Education General Information Survey (HEGIS), collects data on types of programs offered; tuition; full- and part-time enrollment; racial and ethnic characteristics of students; age distributions of students; degrees completed by race, ethnicity and gender; full-time equivalent staffing; rank, tenure, and salaries of staff; and various other financial data. The National Postsecondary Student Aid Study (NPSAS) provides the most comprehensive nationwide data on how students and their families pay for postsecondary education. The Survey of Earned Doctorates Awarded in the United States has been conducted each year since the 1920s. Drawing on these databases, NCES regularly publishes three major compendia of statistics: Condition of Education, Digest of Educational Statistics, and Projections of Education Statistics. NCES also publishes numerous annual and biennial reports from the individual databases, and special reports on specific topics. The reported statistics are used by Congress, federal agencies, state and local education agencies, professional associations of educators, researchers, businesses that sell goods and services to schools, the media, and officials in other nations. They are used to project future enrollments at each grade, assess the supply and demand of teachers, profile the teacher work force, portray school climate and working conditions, describe various characteristics of the educational programs, compare public and private schools, examine educational attainments, and monitor expenditures. Longitudinal Studies Longitudinal studies provide important information about the changes in the behavior and performance of a cohort of students over time. NCES has supported three particularly notable large-scale longitudinal studies of students: the 1972 National Longitudinal Study, the 1980 High School and Beyond Study, and the 1988 National Educational Longitudinal Study. These studies have been ''valuable in basic scientific and policy research, and they have occasionally been useful in monitoring trends in educational transitions'' (Hauser, 1991:3).

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement The 1972 National Longitudinal Study (NLS-72) was designed to describe the transition of students from high school to college and then into the work force. The sample was composed of 16,683 high school seniors around the country. The data that were collected included demographic and school characteristics, courses taken, academic achievement, and current status of and future plans for both education and work. Follow-up data were collected in 1973, 1974, 1976, 1979, and 1986 to determine changes in education, work, and marital status. NLS-72 has provided the basis for hundreds of studies. One example is Student Progress in College: NLS-72 Postsecondary Education Transcript Study (Knepper, 1989), which examined patterns of student progress through the postsecondary education system. Although the majority of 1972 high school seniors went on to postsecondary education, most did not follow the traditional pattern of completing college within 4 years. Students who attended 4-year schools entered sooner after high school graduation than those who began other forms of postsecondary education. Generally, men took longer than women to complete each level of education. Transfer students generally took an average of 8 months more to complete a B.A. degree than nontransfer students. High School and Beyond was designed to provide additional data on the school-to-work transition, particularly in light of concerns raised by increases in school dropout rates and decreases in academic achievement. In 1980 a sample of 28,000 sophomores and 30,000 seniors was drawn from a highly stratified national probability sample of 1,100 secondary schools. The longitudinal design included follow-up surveys of large subsets of the two cohorts in 1982, 1984, and 1986, with another follow-up of the sophomore cohort scheduled for 1992. The High School and Beyond study expanded the data collection categories used in NLS-72 to include parents' attitudes and financial planning, teachers' observations, and students' scores on a variety of achievement tests. The findings from the first follow-up study of sophomores provided important information concerning school dropout problems and the factors that influence student aspirations during the last 2 years of high school (National Center for Education Statistics, 1991c). Comparisons between the results of NLS-72 and High School and Beyond provide insight into the effects of social and cultural changes on the attitudes and performance of high school students. The 1988 National Educational Longitudinal Study (NELS-88) was designed to examine the major factors contributing to student achievement, persistence in school, and participation in postsecondary education. The study will follow a cohort of 25,000 eighth graders for a decade, and it will collect more data on family, school, and classroom characteristics than NLS-72 and High School and Beyond. Tests of reading, mathematics, science, and social science skills were administered in 1988 and 1990 and will be administered again in 1992. Although most of the questions being addressed by NELS-88 cannot be answered for

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement several years, the first round of data provides some important baseline information concerning the eighth grade students in the sample: 44 percent of African Americans, 39 percent of Hispanics, and 36 percent of Native Americans have two or more background characteristics that put them at risk for school failure. The risk factors are single-parent family, family income less than $15,000 annually, child home alone more than 3 hours per day, parents without a high school diploma, a sibling who has dropped out of school, and limited English proficiency. These data also reveal that eighth graders spend an average of 21.4 hours per week watching television, 5.6 hours doing homework, and 2 hours doing outside reading. National Assessment of Educational Progress Development of the National Assessment of Educational Progress (NAEP) was initiated with federal funds in the late 1960s to measure the achievement of elementary and secondary school students in science, mathematics, reading, writing, citizenship, history, geography, social studies, art, music, and literature. NAEP is now conducted in even-numbered years using samples of students in the fourth, eighth, and twelfth grades. Currently, reading and mathematics are assessed every 2 years, writing and science every 4 years, and the other subjects on an irregular basis. NAEP is both a product and a source of education research and development. The matrix sampling of test items, the analysis of potential items for bias, and the scale scores used in reporting findings are the result of complex psychometric work accomplished over the past three decades by psychologists, statisticians, and education researchers. Without this work, NAEP would have been more time-consuming for students to take, more expensive to administer, and less accurate as a measure of achievement. NAEP has revealed much about students' knowledge and how their knowledge has changed over time. In 1990, only 74 percent of twelfth graders knew how many hours equal 150 minutes; only 49 percent knew what percent 7 is of 175; and only 15 percent knew how much a $1,000 deposit would be worth after 6 months if it earned interest of 1 percent per month on the initial amount (National Assessment Governing Board, 1991). Trend data indicate that reading and mathematics skills have increased only slightly over the past two decades while science achievement declined and then rose to about the 1970 levels (National Center for Education Statistics, 1991d). One of the lesser known trends is that the reading and mathematics scores of African American and Hispanic students have been rising faster than the national average. The first decade of NAEP data partly provoked the 1980s school reform efforts (National Research Council, 1986); the second decade of data has prompted policy makers to rethink their reform

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement strategies because the effects of those reforms on NAEP scores have been less than they had hoped. Despite its prominence and use, NAEP is the target of controversy and misunderstanding. There is debate over which knowledge and skills should be assessed and how they should be measured. Some observers are concerned that students do not try to do their best because the scores have no personal consequences. And many people do not realize that changes in scores, or lack thereof, reflect changes in the student populations and social conditions as well as changes in school instruction. If the quality of instruction in schools improved, there would be no change in the scores if, for instance, the level of student motivation and family support declined. Conversely, increases in scores are not necessarily evidence that school instruction has improved. International Studies International studies are helpful in interpreting the educational achievement of U.S. students. NCES has been a major contributor of resources and data in coordinating comparisons among countries. For example, NCES has contributed to the 1982 Second International Mathematics Study, the 1984 International Education Association (IEA) Writing Study, the 1985 Second International Science Study, the 1988 International Assessment of Education Progress, and the 1991 IEA Reading Literacy Study, and it is currently funding the planning of performance assessments for the upcoming Third International Science Study (TIMSS). Although each of the studies has some limitations, they have repeatedly found that American students' academic skills lag behind those of many other nations. For instance, the Second International Math Study found that U.S. eighth graders were slightly above average in arithmetic calculation, but well below average in problem solving. The results of international studies of mathematics and science achievement were cited prominently in A Nation At Risk (National Commission on Excellence in Education, 1983), a publication that helped precipitate the school reform efforts of the 1980s. These results also inspired one of the National Goals for Education: "By the year 2000 U.S. students will be first in the world in science and mathematics achievement." Several of the above studies, particularly the more recent ones, have collected some data on characteristics of the schooling provided in each country. These data permit analysis of the relationships between school achievement and such factors as curricula, amount of time spent on school work, teacher development, classroom size, parental involvement, and other factors. Other international studies focus intensively on comparisons of

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement curriculum and teaching practices in different countries. For example, it has been found that American mathematics textbooks often develop ideas by progressing slowly through a hierarchy of learning tasks, while textbooks used in several Asian countries immerse students in more demanding problems from the beginning (Fuson et al., 1988). The recently developed and widely hailed standards for mathematics teaching in this country endorse the latter practice (National Council of Teachers of Mathematics, 1989). CONGRESSIONAL USE OF EDUCATION RESEARCH There are four agencies that serve as conduits for education research and development information to Congress. They are the Congressional Research Service (CRS), the General Accounting Office (GAO), the Congressional Budget Office (CBO), and the Office of Technology Assessment (OTA). The committee interviewed 17 of the education specialists in the first three agencies and reviewed the OTA publications that deal with educational matters. Of the specialists, 14 indicated that they often use data or reports from NCES; 13 often use the bibliographic and document retrieval system of OERI's Educational Resources Information Center (ERIC). Only 3 of the specialists said they often use the studies of the OERI laboratories and centers (see Chapter 3) or consult with their staff, but another 9 do so occasionally. Several of the education specialists also mentioned using the results of education research and development from the National Science Foundation (NSF), the Department of Health and Human Services, and the National Institute of Mental Health. All four congressional agencies have used reports from NCES's High School and Beyond study, and one staffer described it as a "treasure trove." Several staff members mentioned that the federal government is usually the only source of longitudinal studies, and several reported using NAEP data and reports. Staff at one congressional agency found that the work they were planning was already under way at OERI centers and laboratories. And when staff at a congressional agency were working on testing issues they used data from OERI's Center on Assessment and Evaluation and "consulted regularly" with its director. Another agency staffer, with a Ph.D. in economics, described the work of the former Center on Education and Employment as ''impressive high quality research.'' An examination of OTA's reports covering education reveal use of a broad range of OERI work, including NAEP, High School and Beyond, the National Longitudinal Survey, the Digest of Education Statistics , the Condition of Education, NCES bulletins, and reports from the laboratories and centers. NSF's work is also frequently cited.

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement Several staff in the congressional agencies noted a marked improvement in NCES's services since 1987. Suggestions for additional change included more timely collection and delivery of data, release of confidential data with appropriate safeguards, more longitudinal studies with broad scope, consultation between NCES and congressional agencies when planning data collection efforts, and easy access to the ongoing work of the centers and laboratories. One example of an education research study that directly influenced policy is the National Assessment of Vocational Education, completed in 1989 (Wirt et al., 1989). According to the counsel for the House Education and Labor Committee, Congress was much influenced by the study when drafting the Perkins Vocational and Applied Technology Act. This legislation focused federal support for vocational education on districts that have the highest proportion of poor families, that integrate academic and vocational instruction, and that operate effective programs and produce the desired results. The legislation also encouraged an easing of state regulatory burdens and the transfer of more authority over these programs to the local level (Jennings, 1991). Why did this study have a major impact on Congress? At least four factors seem to have contributed: the study was mandated by Congress to help inform its reauthorization of the Perkins Act; the researchers consulted with Congress on the design of the study; interim findings were released as they became available; and the researchers responded with additional analyses that were requested by Congress. In short, the researchers worked closely with Congress to make the study meet its needs. CONCLUSIONS The programs discussed in this chapter are examples of how research has (1) expanded fundamental understanding of child development, learning, and teaching; (2) pointed the way to the discovery of effective elements of curriculum, instruction, and school organization; (3) provided a basis for evaluating worthwhile innovations; and (4) monitored the progress of reform efforts, assessing whether the reforms have been implemented as planned and are having the intended effects. Findings from basic research have significantly broadened knowledge concerning the underlying processes of acquiring, organizing, storing, and retrieving information. Moreover, researchers studying early cognitive development have provided important insights into preschool children's understanding of mathematical and language concepts, thus giving developers of educational programs important guidelines for enhancing instruction. Two of the programs—Reciprocal Teaching and Cognitively Guided Instruction—were designed as basic research on theories developed by cognitive scien-

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement tists, but they have also led to some practical guides for classroom instruction. Several of the other programs also drew heavily on one or more of the concepts developed by cognitive scientists. The focus in this chapter has been on the contributions of cognitive science to learning and teaching, but many other disciplines have made important contributions to the process of education: some of this research is at a detailed physiological level; other research concerns the functioning of groups and organizations. Our review shows that the effects of research on educational practice are for the most part indirect and slow. The programs examined in this chapter range in development time from 4 to more than 20 years and are based on decades of research work. Some of the programs are heavily grounded in basic research; all of them have made use of principles drawn from basic research. The three most highly developed programs—Reading Recovery, Comprehensive School Mathematics Program, and Student Team Learning—have had the advantages of continuous support and the work of dedicated researchers since the initial stages of research, and all have been under development for longer than 20 years. One important shortcoming of most of these innovative programs is the lack of follow-up assessment to determine if the gains measured at the end of the treatment are maintained over time. Another shortcoming is the lack of evaluations at dissemination sites where the programs are no longer under the direct control of the developers. Once a program is disseminated, evaluation activities are usually reduced or discontinued because of logistical difficulties and the lack of funding. A third shortcoming of most of the programs is the limited number of adoptions—without adoptions, programs cannot have impact. Both the laboratories and centers that are supported by OERI and individual scholars have been responsible for producing promising research and innovative programs. The Learning Research and Development Center and the Wisconsin Center for Educational Research have been leaders in research on cognitive science and its contributions to education; the Center for the Study of Reading has been a leader in reading research (and produced A Nation of Readers); the Central Midwestern Regional Laboratory designed the Comprehensive School Mathematics Program, and the Midcontinent Regional Educational Laboratory disseminated it; and the Johns Hopkins' centers have developed and expanded Student Team Learning. Field-initiated research and development has also been a rich source of ideas and has provided a wide range of promising programs for schools. Most of the innovative programs we reviewed were built on a research base accumulated largely from the field-initiated work of individual social and behavioral scientists. In addition, many of the innovative programs were developed by individuals working on their own initiative. For example, the use of computers in the schools was introduced and tested through numer-

OCR for page 19
Research and Education Reform: Roles for the Office of Educational Research and Improvement ous field-initiated efforts; Reading Recovery is based on the research of Marie Clay, who then proceeded to develop this powerful intervention; IMPACT was a field-initiated effort aimed at integrating instruction in thinking and reasoning into content courses; and the developers of both the School Development Program and the Outcomes Driven Development Model combined prior basic research and their own practical experience to create comprehensive approaches to school restructuring. Research has also informed policy makers and the public about the status of the U.S. education system, it problems, and the progress towards reaching the nation's goals in education. The major activities of the National Center for Educational Statistics and the data it collects and reports are invaluable to congressional agency staff and to social science researchers in general. Our review demonstrates that successful education research and development requires a sustained investment of time and money for research, development, and dissemination. We conclude that no one mechanism for the support of research should dominate federal grant-making policy. A vigorous program for support of field-initiated research is as important as the support of laboratories, centers, and other such institutions. Furthermore, no one discipline should be given priority. Advances in education have been built on research in the cognitive sciences, psychology, sociology, anthropology, organizational behavior, and clinical work in and outside of classrooms.