Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
lp per 1 em Pa~llletship · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ THE CHALLENGE A powerful consensus has emerged in the United States about the importance of improving student learning, particu- larly for children in elementary and secondary schools. This consensus has led to significant federal and state investment in education. To make that investment productive, however, will require a commensurate effort to investigate systematically how to improve teaching and learning. A proliferation of content and accountability standards has not been accompanied by com- panion efforts to ascertain whether and how those standards can be reached for the highly diverse population of students to whom they apply. It is perplexing that so high a national prior- ity has to date generated so little sustained, systematic attention to the very complex problems of teaching and learning in the classroom, and has fostered so little fruitful collaboration among researchers, education practitioners, and policy makers. The current state of affairs cannot, in our view, be attributed simply to a lack of attention by researchers to problems that bear on student learning or to a shortage of intellectual para- digms that might profitably be applied. There are powerful examples of both. Nor can it be attributed to a lack of intriguing experiments in the communities of educational practice that provide fodder for research. Efforts at innovation and reform abound. Rather, the existing pockets of opportunity created by research and experimentation have been largely untapped. The committee's judgment is that there are two explanations for this: first, historically there have been few incentives for change in education, and second, there are no institutions within which collaborative efforts to improve student learning are facilitated. T H E N E E D F O R A N E W P A R T N E R S H I P 9
The links between research and practice, between scientists and educators, are tenuous and fragile. A change in incentives is on the horizon. State and federal policies focused on establishing standards and measuring stu- dent performance place new demands on schools to improve student learning. In response, policy makers and school officials are beginning to actively seek the sort of research and develop- ment that would help schools improve performance outcomes. But the structures that would respond effectively to that de- mand are largely absent. The question therefore is "How do we construct a focused program of research and development that informs and improves day-to-day educational practice?" In our view, the answer lies not in more of the same but in a fundamentally different mode! for education research and de- velopment. The centerpiece of this new mode! is collaboration between practitioners and researchers, enabled and supported by a new organizational infrastructure. We propose the Strate- gic Education Research Partnership (SERP) as that new model. At the core of the SERP vision are teams of leading scientists and practitioners working together on a coherent, highly fo- cused program of research and development that is tightly coupled and interactive with practice. Building on behavioral and social science research, disciplinary research, and the sys- tematic study of effective practice, their efforts can produce, over time, a powerful body of usable knowledge. But this will happen only if SERP, through its organization and program, develops and nurtures the capacity for the work that is envisioned. A critical element of the SERP plan, then, is to cultivate practitioners who have the knowledge and training needed to work effectively with research teams, helping to de- velop, test, and use research-based materials and methods; re- searchers who focus their work on the problems of educational practice as they develop and test hypotheses in collaboration with classroom teachers; and developers who have learned how to work with practitioners and researchers to incorporate robust findings into usable, carefully tested instructional methods, pro- grams and tools, organizational environments, and professional development programs. To achieve this end, the SERP organi- zation must foster the development of the shared language, mutual regard, and working atmosphere required for effective partnering. 10 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
Equally important, the collaboration we envision extends beyond the SERP research activities. As the research accumu- lates, the same intense collaborative effort must go into synthe- sis, sense-making, and communication, so that SERP can be- come a place where teachers and school administrators, policy officials, schools of education, researchers, and other partici- pants in the education enterprise can go for high-quaTity, care- fully evaluated research-based information; a place where inter- ested school districts and researchers can link up around the SERP agenda; a place where new members of the field of teach- ing and its related research disciplines can join a program of research and development productively channeled to improv- ing teaching and learning (we will propose fellowship and in- ternship programs); and a place where many kinds of funders of education research and development can become part of an ongoing collaborative effort to improve student outcomes. U N T A P P E D R E S O U R C E S Would a new research and development infrastructure im- prove educational outcomes? The committee's optimism about the potential to do so is rooted in three sets of untapped re- sources that could make this new kind of research enterprise highly productive. First, there are tantalizing opportunities to leverage intellec- tual developments in allied fields and disciplines in pursuit of improved educational outcomes. The outpouring of scientific discovery on the mind and brain, on the processes of thinking and learning, on the neural underpinnings of learning and cog- nition, and on the development of intellectual competencies provide a rich context for innovation in education (National Research Council, 1999, 2000~. The enthusiastic response to the National Research Council report, How People Learn: Brain, Mind, Experience, and School, suggests a growing interest in that re- search base and its relevance for teaching and learning. Second, remarkable "natural experiments" in educational practice are occurring every day. Yet for the most part these have not been the object of systematic, rigorous, sustained study and evaluation by researchers. These natural experiments range from the work of individual teachers who have consistent records of supporting high student achievement, to schools and school districts that undertake reform efforts, to new forms of organi- T H E N E E D F O R A N E W P A R T N E R S H I P 1 1
cation like charter schools. But while prescriptions for educa- tional reform are numerous, there are relatively few careful research efforts designed to evaluate how and why differences in the form and content of education affect student learning. Third, and perhaps most distressing, there are numerous examples of promising educational innovations that have been validated through rigorous research but that have had rela- tively little impact on educational practice. The Number Worlds curriculum, for example, was designed to build on years of · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ BOX 1.1 Primary School Mathematics From an early age, children begin to develop an informal understanding of quantity and number. Careful research conducted by developmental and cognitive psychologists has mapped the progression of children's conceptual understanding of number through the preschool years. just as healthy children who live in language- rich environments will develop the ability to speak according to a fairly typical trajectory From single sound utterances to grammatically correct explanations of why a parent should not turn out the light and leave at bedtime), children follow a fairly typical trajectory from differentiating more from less, to possessing the facility to add and subtract accurately with small numbers. just as a child's environment influences language development, it influences the acquisition of number concepts. For many children whose early years are characterized by disadvantage, there is a substantial lag in the development of the number concepts that are prerequisite to first grade mathematics. Between the ages of 4 and 6 most children develop what Case and Sandieson ( 1987) refer to as the "central conceptual structure" for whole number mathematics: 1. The ability to verbally count using number words. This ability is initially developed as a sequencing of words (one, two, three. . .) without an understanding of the specific meaning attached to the words. Quantity is still understood nonnumerically as more or less, big or small. 2. The ability to count with one to one correspondence. When this ability develops, children are able to point at objects as they count, mapping the counting words onto the objects so that each is tagged once and only once. This ability is initially developed as a sensorimotor activity, with an understanding of quantity still absent. Children who can successfully count four objects and five objects cannot answer the question, "Which is more, four or five?" 3. The ability to recognize quantity as set size. With development of this ability, children do understand that "three" refers to a set with three members. Initially this understanding is concrete, and children will often use their fingers as indicators of set membership. 4. The ability to "mentally simulate" the sensorimotor counting. When this ability is in place, children can carry out counting tasks as though they were operating with a mental number line. They understand that movement from one set size to the next involves the addition or subtraction of one unit. Children from middle and higher socioeconomic backgrounds generally come to school with the central conceptual structure in place, whereas many children from disadvantaged backgrounds do not. When first grade math instruction assumes that knowledge, these children are less likely to succeed. A curriculum called Number Worlds deliberately puts the central conceptual structure for whole numbers in place in kindergarten (Griffin and Case, 1997~. Developed, tested, and refined with classroom teachers and children, the program consists primarily of 78 games that provide children with ample opportunity for hands-on, inquiry-based learning. Number is represented in a variety of forms on dice, with chips, as spaces on a board, as written numerals. An important component of the program is the Number Knowledge Test, which allows teachers to quickly assess each individual student's current level of understanding and then choose individual or class activities that will solidify fragile knowledge and take students the next step. 12 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
careful research on children's understanding of whole number (see Box 1.1~. The experimental results suggest that disadvan- taged children who begin school as much as two years behind their peers in number knowledge can be brought up to and in some cases surpass the level of those peers in the early el- ementary years. Other innovations in instruction (e.g., Recipro- cal Teaching, Thinker Tools), in professional development (e.g., Cognitively Guided Instruction) and in aspects of school orga- nization (e.g., reduced class size, small schools) have demon- · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ The Number Worlds program has been tested with disadvantaged populations in numerous controlled trials in both the United States and Canada with positive results. One longitudinal study charted the progress of three groups of children attending school in an urban community in Massachusetts for three years: from the beginning of kindergarten to the end of second grade. Children in both the Number Worlds treatment group (n = 54) and in the control group (n = 48) were from schools in low-income, high-risk communities where about 79 percent of children were eligible for free or reduced-price lunch. A third normative group (n = 78) was drawn from a magnet school in the urban center that had attracted a large number of majority students. The student body was predominantly middle income, with 37 percent eligible for free or reduced-price lunch. The Number Knowledge Test scores on the vertical axis can be mapped against developmental expectations for children at various ages (at age 6, the expected developmental score is 1.0, at 7 it is 1.5, and at 8 it is 2.0~. As Figure l. l shows, the normative group began kindergarten with substantially higher scores on the Number 2.5 2.0 - 1.5 - 1.0 - 0.5 - 0.0 - - ~? '-he ,. , ,. , . , ,. i'. /,,...~. ,.. ,. . ... Pre K Post K Post 1 Post2 (5 4 yrs) (6.0 yrs) (7.0 yrs) (8.0 yrs) Treatment - Normative Control FIGURE l. l Mean developmental level scores on Number Knowledge Test at four time periods (SOURCE: Case et al., I 999~. T H E N E E D F O R A N E W P A R T N E R S H I P Knowledge Test than children in the treatment and the control groups. The gap indicated a developmental lag that exceeded one year and was closer to two years for many children in the treatment group. By the end of the kindergarten year, however, the Number Worlds children narrowed the gap with the normative group to a small fraction of its initial size. By the end of the second grade, the treatment children actually outperformed the magnet school group. In contrast, the initial gap between the control group children and the normative group did not narrow over time. The control group children did make steady progress over the three years; however, they were never able to catch up. 13
strable strengths. Some of these have been based on careful research into learning and teaching (Palincsar, 1986; Palincsar and HerrenkohI, 2002; White and Frederiksen, 1998) and have benefited from being implemented in situations where practi- tioners could modify their design and enhance their utility. Many of these innovations have been demonstrated in research studies to be effective in improving learning substantially, even to weaken the customary link between performance and such factors as family income, race, and other demographic factors. Despite these successes, the innovations have not penetrated deeply into mainstream practice. THE NEED FOR AN R&D INFRASTRUCTURE To effectively mine untapped resources for purposes of im- proving education will require, in the committee's view, the development of a research and development infrastructure to that has several key functions. Recruiting Disciplinary Scholarship In many sectors of the economy, what is of interest to aca- demic researchers is not the same and may hardly intersect- with what is of interest to practitioners in that sector. Theories and paradigms of researchers may be relevant to, but not ap- plied to, problems of practice. This problem is not unique to education. Indeed, the inspi- ration for the SERP idea at the National Research Council was the Strategic Highway Research Program, launched in the 1980s to focus research efforts on the problems of highway construc- tion. At that time, policy makers charged with the construction of a highway system and construction companies charged with building those highways did not have the knowledge base to support the construction of longer lasting roads that were cus- tomized to local environmental conditions. They were uninter- ested in funding research, however, because what researchers produced was, in their view, of little practical importance. The National Academy of Sciences proposed, and the U.S. Congress funded, a 10-year program of research focused on the problems of practice. The collaborative efforts of those from the worlds of For further discussion of these and other programs, see the companion 14 report, Learning and Instruction: A SERP Research Agenda. STRATEGIC EDUCATION RESEARCH PARTNERSHIP
research, policy, and construction through the Strategic High- way Research Program were generally viewed as highly pro- ductive (National Research Council, 1999~. In other sectors as well, research and development infra- structures were designed to tackle this same issue: to bring together scientific resources and the problems of practice. Land grant colleges and agricultural experiment stations are prime examples, as are teaching hospitals in medicine. Although medi- cine and agriculture differ in many important respects from education,2 the historical importance in those sectors of devel- oping opportunities and settings to carry out a systematic pro- gram of research on practice is instructive.3 Access to school settings is particularly important for a pro- gram of research and development focused on educational prac- tice because most researchers lack intimate knowledge of K-12 practice. The typical researcher's thinking about teaching, learn- ing, or organizational change is quite different from and much more abstract than that of the typical teacher or administrator. Implementation, for example, tends to be seen as an issue of second-order importance. Practitioners, in contrast, spend much of their time focused on the moment, and they lack the re- sources or training to evaluate their own practice systematically or to share what they have learned about implementation in organizational context with colleagues. Grounding research in the problems and needs of practice will require productive rela- tionships that are neither linear nor unidirectional; instead, re- search and practice must interact in meaningful and progres- sively more sophisticated ways. Making educational practice the focus of serious research attention will also require access to data that are at a level of specificity that allows for an understanding of the effects of characteristics of teachers, students, instructional programs, and classroom environments on learning outcomes. These data must be collected longitudinally if the Tong-term impact of policies, practices, and interventions is to be understood. Longitudinal data can also play a central role in theory development, since the observation of patterns in data can stimulate hypothesis development and testing (see Box 1.2~. 2Including market conditions. 3The committee is indebted to leading agricultural economists Vernon W. Ruttan of the University of Minnesota and Robert E. Evans of Yale University for sharing their deep knowledge of this issue. T H E N E E D F O R A N E W P A R T N E R S H I P 15
BOX 1.2 The Amazing Miss A A 1978 study of student achievement at the "Ray School," an elementary school located in one of the poorest areas of a large northeastern city, uncovered a startling fact: IQ can be affected by teaching. The research, conducted by Eigil Pedersen, focused on people who were pupils at the school over a 25-year period, probably beginning in the 1 930s. The school had a reputation of being the most difficult among the 80 elementary schools in the district; its students consistently had the lowest mean scores on IQ tests. Only 50 percent of those who graduated from the elementary school completed tenth grade, and only 10 percent of that group completed high school. Professor Pedersen attended the Ray School from the age of 4, and he returned there to teach fifth grade as a young teacher. Even as life took him elsewhere, he kept track of some of those who had been in his classes, becoming increasingly disheartened at how few completed high school, or even tenth grade. He ultimately came upon the idea of interviewing graduates of the Ray School to see if this might help him devise more effective teaching methods. In order to try to locate them, Pedersen reviewed the permanent record cards of pupils at the school who would now be adults. In doing so, he noticed an oddity: many pupils exhibited large differences in IQ scores between the third grade and sixth grade administrations of the tests, which went against contemporary notions of the stability of IQ. This apparent anomaly led Pedersen and his colleagues to undertake several studies of the effects of school characteristics on IQ change, making what appears to be wonderfully clever use of the permanent records to gain estimates of such phenomena as self-esteem, self-fulfilling prophesies, and reinforcement by teachers, always with IQ score as the primary dependent variable. At some point along the way, Pedersen got curious about whether achievement in first grade is an indicator of later IQ score. Records showed that there were three first grade teachers who were at the school during the whole period under study, as well as many others who stayed for only a few years. He found that, of the long-time teachers, Miss A had taught a high proportion of the pupils who showed an increase in IQ score between third and sixth grades. Miss B had taught a high proportion of the girls whose scores increased, and Miss C had taught a high proportion of the students whose IQ scores had decreased between third and sixth grades. The students were otherwise undifferentiated by economic or other background characteristics. Pedersen and his colleagues managed to locate 60 people in their early 30s who had attended the Ray School as children. They were interviewed in depth according to a carefully devised protocol, which yielded, among other things, a measure of "adult status," a distillation of six factors including occupation, type of housing occupied, and education. Adult status scores were then tabulated against all of the many variables in the study. "[O]ne simple tabulation was so stunning it caused them to stop and look at their data differently.... It was the cross tabulation of 'adult status' with first-grade teacher" (Fallon, 2001~. The mean adult status score of those who were Miss C's pupils was 4.3 in a distribution that ranged from 1.0 to 9.0. For Miss B's students it was 4.8. And for Miss A's it was 7.0. Considered on a simple scale of high, medium, and low, the adult status scores again showed dramatically differing results: for all teachers together, only 29 percent achieved high status as adults; almost 40 percent were classified as low status. None of Miss A's former students were classified as low status, and 64 percent of her students achieved high adult status although they were indistinguishable from the others in terms of background characteristics. Fallon, speaking with and for Pedersen, emphasizes two important aspects of the study. The first was the finding that teacher quality makes a difference (against the then conventional wisdom that gains in student achievement would come from changes in poverty, socioeconomic status, and the like rather than from within the school). The second was that only a rigorous and creative use of multiple research methods enabled the authors to reach their important conclusion. Being able to link the achievement of individual students over time with specific teachers, curricula, and schools enabled this research to address the kinds of questions that get to the heart of the matter. SOURCE: Distilled from Daniel Fallon, "The Amazing Miss A and Why We Should Care About Her," Homecom- ing Speech, University of South Carolina, October 19, 2001. 16 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
Nurturing Research Over an Acloquate Time Span Solid research that has implications for teaching and learn- ing exists (National Research Council, 2000~. Often, however, that knowledge is not elaborated at a level of detail that is useful for practice and then incorporated into carefully tested pro- grams and tools (directed both at student learning and at teacher learning) that allow it to infuse the larger system. A research and development infrastructure could provide an environment that nurtures promising work through the various stages needed for classroom relevance and reliability. Sometimes research stalls at the identification of important principles of learning and teaching that are not made specific enough for practice. For example, a now substantial body of research points to the misconceptions that students harbor in physics. In making sense of everyday experience, people de- velop understandings, or informal models, of how the world works that shape everyday ideas about scientific relationships. These ideas usually contain partial truths but are not scientifi- cally correct. For example, as we move closer to a heat source, temperature rises. Students often assume that the higher tem- peratures of the summer must mean that the earth is closer to the sun, and classroom lessons that explain the seasons in terms of the angle of the earth's axis with respect to the sun quite often fad! to change that conception (Schneps and Sadler, 1987~. In studies of students of all ages, everyday models of the physical world prove to be highly resistant to change (DiSessa, 1982; National Research Council, 2000, 2003b; Vosniadou and Brewer, 1989~. The difficulty of changing everyday conceptions is a phe- nomenon that extends into every area of the curriculum (for examples in history and mathematics, see National Research Council, 2003a). Knowing about this principle is critical to effec- tive teaching yet it is of little help to teachers unless the work has been done to reveal the kinds of conceptions student t~vpi- cally harbor regarding the topic a teacher is about to tackle, as well as the teaching strategies needed for supporting concep- tual change in students. A program of research focused on making the most important findings from research usable in the classroom would pursue such an agenda. The stalling point is not always at the level of general prin- ciples, however. Many promising curricular or pedagogical in- T H E N E E D F O R A N E W P A R T N E R S H I P 17
novations do not penetrate the system because individual re- searchers are left with the challenge of raising the funds and carrying the work forward with very little support. Reciprocal Teaching, for example, is a procedure for addressing a very pervasive problem of K-12 education students' independent comprehension of text. The program showed impressive re- sults, but it is not widely used. The research was not carried through to a point at which it was sufficiently embedded in curricular programs so as to be readily accessible to a large number of teachers (see Chapter 2~. Number Worlds does have a well-articulated and extensive curriculum but using that cur- riculum effectively requires that teachers understand the teach- ing and learning of whole number differently. A research and development program focused on teacher knowledge and learn- ing designed to prepare teachers to use it well has not yet been developed. The expertise and interest required for the many different tasks of research, development, evaluation, communi- cation, and professional development are not likely to emerge from individuals working alone or even with a few colleagues. But an appropriate organizational infrastructure can knit to- gether communities with these different strengths so that efforts can be carried through all the necessary stages for usability in the classroom. There are other examples in which theories about learning and pedagogy are incorporated into curricula and classroom programs and tools, but research to rigorously test if, when, and for whom the programs and tools are effective is often lacking. Several National Science Foundation- (NSF) supported science and math curricula fall into this category (Education Develop- ment Center, Inc., 2001~. Adequate evaluation is taken to mean not just analysis of learning outcomes, but also evidence about a program's feasibility in the classroom and its accessibility to the average teacher. It also requires attention to the perspectives of teachers trying to use the program with a wide variety of stu- dents in a variety of school settings, as well as the distribution of results across the population of students (not just evidence about average achievement outcomes). An infrastructure that supports a major research and devel- opment program is not, of course, required in order for an evaluation to be done. What a sustained R&D program can bring to bear, however, is the capacity to assess the theoretical underpinnings of a program in order to isolate those that show 18 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
particular promise for improving understanding of important issues of teaching and learning. And in proposing evaluation design and instrumentation, it can create the opportunity to learn not only if something works, but why and how. This more thoroughgoing evaluation is more likely to be the product of a research program focused on generating a deeper knowledge base regarding how students learn and teachers teach than it is to be the result of a one-shot effort to determine whether a program "works." In a program of research and development, outcomes of evaluations are a milestone but not a finish line. Both successes and failures provide clues about the mechanisms at work and the effects of context that can support new under- standings and hypothesis development. Indeed, in order to un- derstand when results can be generalized and when they can- not, knowledge of the mechanisms and of the role of context is critical. This point applies not only to programs, but to changes in the organization of schooling, such as reducing class size. The failure of the positive results of the Tennessee STAR experiment to be reproduced in California points to the critical role that sustained attention to context and causal mechanisms can play. It also points to the value and importance of detailed study of what actually is going on in the classroom. In some cases, contextual factors contributing to success or failure may be complex. But they can also be quite straightfor- ward, as a study of the implementation of the Everyday Math curriculum adopted by the Pittsburgh schools suggests (Briars and Resnick, 2000~. The study documented that Everyday Math improved student outcomes impressively, but only in some schools. More detailed analysis of results showed that the larg- est performance gains occurred in what the authors call "strong implementation" schools, where principals and teachers em- braced the new pedagogical approach. In many schools, organi- national commitment and support appear not to have been present and central office staff were often unwilling to confront those who were not fully implementing the new and not yet locally proven program. In these schools, performance effects were negligible (see Box 1.3~. These findings suggest the impor- tance of designing research to illuminate why innovations work. Finally, replication of research findings is a canon of good science. But without a research infrastructure to ensure that it is a priority, it can be easily overlooked. For researchers establish- T H E N E E D F O R A N E W P A R T N E R S H I P 19
BOX 1.3 Everyday Mathematics In 1992, the Pittsburgh Public School District adopted a plan to align standards, tests, curricula, instructional materials, and professional development. Beginning in 1993, a new elementary school mathematics curriculum, Everyday Mathematics, was introduced (University of Chicago School Mathematics Project, 1995~. This is a closely scripted curriculum for grades K-5 that is informed by research on early cognitive development. The program is directly mapped to the National Council of Teachers of Mathematics standards (1989) and closely aligned to the Pittsburgh Core Curriculum Framework for math (National Council of Teachers of Mathematics, 1989, 2000~. Implementation began with the children entering kindergarten in 1993- 1994; by 1997- 1998, they were fourth graders. In the 1996- 1997 school year, the Pittsburgh Reform in Mathematics Education project (PRIME) began to offer extensive professional development for teachers and administrators. PRIME was designed explicitly to develop teachers' capacity to implement Everyday Mathematics; it provides in-class support including demonstra- tion lessons, joint planning, and coaching, as well as after-school and summer workshops. Standards-based tests that reflected the three core curricular goals (skills, concepts, and problem solving) were given to fourth graders beginning in 1996. Briars and Resnick examined fourth grade mathematics achievement scores citywide over the three years from 1996 to 1998.* The 1998 fourth graders were the first cohort to have experienced the Everyday Math program throughout their elementary schooling. By 1998, performance improved dramatically. Between 1996 and 1998, the percentage of students who met the district achievement standard in the Skills category increased from 30 to 52 percent. Seen from another vantage point, the good news is that the percentage of students at the very lowest of the five achievement categories on the skills subtest fell from 23 percent in 1966 to only 7 percent in 1998. There were substantial, if more modest, gains in the percentage of students meeting the standard in concepts and problem solving as well, although starting from a much lower 1996 baseline (8 percent).** But there is a larger point to this story. If one looks at the data school by school rather than citywide, the degree of administrative and teacher support for the new system emerges as critical to the performance gains evident from 1996 to 1998. As Figure 1.2 illustrates, the schools with strong implementation showed dramatic improvement in all three areas, while in the schools with weak implementation there was little change from year to year on any of the measured achievement dimensions. *The norm-referenced tests previously used in the district were also administered. We focus here on the New Standards assessment results. **This reflects the concentration of the traditional math curriculum in the U.S. on computational skills; teachers and students alike needed to learn much more about mathematical concepts and problem solving. 90 - 80 - 70 ~ 60 - 50 - 40 ~ 30 - 20 ~ 10 ~ O - Strong implementation Weak implementation II 1 1 Skills Concepts Problem solving FIGURE 1.2 SOURCE: Briars and Resnick (2000:27~. 20 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
ing a reputation, it has little cachet. For funders looking for promising new ideas, it can seem an unexciting addition to a portfolio. Yet it is critical to knowledge accumulation. A stable program of research focused on consolidating knowledge can make this a priority in a way that is unlikely to happen other- wise. Replication, of course, can provide new insight when it is conducted in natural settings as much of the envisioned work would be. Innovations that work in some settings, with some students, might well turn out to be ineffective with other stu- dents or teachers. Rather than taking inconsistent outcomes as evidence that the innovation should be abandoned as a failure, such findings should themselves be the target of analysis. Be- cause there is no one best curriculum for all students, or all teachers, analyzing the conditions under which various innova- tions do and do not improve outcomes should be a part of the process of evaluation and of preparing teachers to employ new teaching practices. Aciciressing the Complexity of the System Excellent curricula and instructional materials are impor- tant, but educational practice is not embodied solely in the tools and protocols of the trade. Rather, instructional tools, teacher knowledge, and the organization of the school are interdepen- dent. They serve as three legs of the stool supporting student learning. While each is analytically independent of the other two, the effectiveness of any one in supporting student achieve- ment depends on the strength of the other two. This lesson has been demonstrated repeatedly in education reform efforts. There- fore, focusing research attention on one leg of the stool without simultaneously attending to the other two is a strate~v that holds little promise for success. tJJ A compelling example is provided by the implementation in the Pittsburgh schools of an algebra curriculum developed by researchers at Carnegie Mellon University. The positive results in experimental trials led the researchers to expect a far more significant impact of their curriculum than the test results showed. Upon investigation, the researchers discovered that the number of hours devoted to mathematics teaching for the students using the program was far below the time require- ments for mastery. Without reorganizing the daily schedule of the students in the program, the gains that the curriculum could T H E N E E D F O R A N E W P A R T N E R S H I P 21
produce were marginal (Anderson, 1983). This example sharply illustrates the importance of understanding what is required to make a program work, the difficulties in identifying "replica- tions," and the need for detailed observation of actual practice in order to come to understand what can make a program successful. Making Knowlecige Usable There has been relatively little systematic accumulation, syn- thesis, and sense-making in the education research enterprise that practitioners and policy makers can turn to for help with decision making. Instead, there is a huge volume of uncon- nected and undigested material available, and no authoritative source of carefully screened and vetted research knowledge. As a consequence, even the most promising research-based cur- ricula, the most effective programs, and the most important insights into human learning are often little known and have little effect on U.S. schools. The U.S. Department of Education has recently funded the What Works Clearinghouse to provide an authoritative evalua- tion of educational interventions. If support for the activity and initial efforts to establish high-qualit~v standards are sustained, it will make a valuable contribution to decision making. A SERP infrastructure can substantially enhance the quality of an effort like that of the What Works Clearinghouse in several respects. First, answers regarding what works depend critically on an understanding of the outcomes to be achieved. In reading com- prehension, for example, standardized measures test recall of text detail. There is research to suggest, however, that recall is not the same as comprehending the meaning of text for pur- poses of problem solving, and it does not guarantee that the information recalled will actually be integrated with the stu- dents' existing knowledge. In fact, students who do best at recall may not be the same students who do best at deeper comprehension (Mannes and Kintsch, 1987; National Research Council, 2003b). The answer to the question regarding what works will therefore depend critically on the outcome measures used. An R&D infrastructure like SERP can pursue research on the nature of reading comprehension and its assessment that will support better answers to questions of what works (Na- tional Research Council, 2003b). 22 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
Second, the optimal usability of findings requires more than ready access to the knowledge base. Educators often hold con- ceptions of teaching and learning that are at odds with the scientific principles underlying well-developed instructional in- novations (Palincsar et al., 1989~. Teachers need opportunities to learn about the underlying conceptions in order to use the inno- vation well, and the more innovative the practice, the more challenging will be the teacher learning involved. A program of research on teacher learning coupled with an effort to commu- nicate with teachers effectively about what works is likely to enhance the prospect that programs with demonstrated success will be used successfully when they are taken to scale. Finally, teachers and administrators need opportunities to understand innovative practices in a way that permits them to estimate the fit with their schools and the institutional supports that are required for successful implementation. They need to know not only that a program works, but also for whom it works, under what conditions it works, and why it works. While the Clearinghouse will ask these questions, the research base to answer them is not currently available. A research pro- gram that can systematically pursue these questions for promis- ing programs would substantially strengthen the effectiveness of the effort to provide schools with a knowledge base concern- ing successful programs and practices. · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ SERF CAPABILITIES The challenges of effectively linking research and practice identified here help to define a set of capabilities that a SERP enterprise must have if it is to succeed in that endeavor. What would have to be in place for the Number Worlds curriculum, Reciprocal Teaching, or class-size reduction to be more fully developed and made widely useful across settings? What struc- ture would make the resources residing in the disciplines of cognitive science, psychology, sociology, and economics more available to the improvement of educational practice? What would be necessary for the nation to cull knowledge from the naturally occurring variations in educational practice that could be broadly useful to policy makers? What kind of entity would enable the wisdom and knowledge of highly effective practition- T H E N E E D F O R A N E W P A R T N E R S H I P 23
ers and administrators to be systematized, reviewed, verified, accumulated, and made public? In order to maximize the value of promising education innova- tions, SERP would have to carry out a coordinated set of re- search and development activities designed to identify candi- date pedagogical, curricular, systemic, or organizational innovations that are worthy of study. It then would need to examine these systematically with careful attention to the con- ditions under which they have their effects, the particular popu- lation of students who are likely to benefit from them, and the teacher learning and organizational supports that are required for their effective adoption and implementation. In order to benef~tfrom advances in the relevant disciplines, SERP would need to create incentives for researchers to work on problems that have a likelihood of informing educational prac- tice. Furthermore, it would need to create incentives to keep researchers and practitioners involved through the often lengthy process of development, implementation, and adaptation so as to ensure continued learning from and about innovative prac- tices. In addition, SERP would need to provide the arrange- ments under which mutually informative conversations between researchers and practitioners can occur, so as to ensure that researchers' questions are genuinely informed by the most burn- ing issues of practice. In order to capitalize on and learnfrom naturally occurring varia- tions in practice, SERP would need a process for surfacing and vetting both problems and practices that are worthy of sus- tained study for example, what teachers need to know to teach algebra II to all of their students, or to ensure subject-matter learning for secondary students with limited literacy skills; or the strategies that would inspire engagement in a rigorous cur- riculum. One approach might be to describe current approaches to solving these problems, in particular those used by highly effective practitioners, and then to organize systematic field trials of alternate approaches that seem promising. Naturally occurring variation in practices, for example, charter schools, new approaches to school finance, teacher learning communi- ties, and comprehension strategy instruction, could similarly provide a source of organized learning if subjected to system- atic comparison and analysis. An organization with these capabilities would be a resource for those responsible for today's classrooms, as well as for those ~ . 24 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
in charge of Tong-term reform efforts. It would by its design blur the traditional distinction between basic and applied research, helping practitioners to think more like researchers and re- searchers to appreciate the challenges of practice. To accom- plish its mission, it would need a process for selecting among the many possible research activities, for prioritizing commit- ments, and for consulting with a full array of stakeholders. It would need to create research environments in which student learning, teaching, and school organization can be studied si- multaneously. And it must develop a capacity to reach those communities that can make use of the fruits of its efforts. H o w S E R P R E ~ A T E S T O O T H E R E F F O R T S As the first report of the SERP initiative made clear (see National Research Council, 1999:17-20), the Strategic Education Research Partnership we propose, although different from other research and reform efforts, is emphatically not a replacement for them. For the SERP idea to come to life, education leaders will have to see its potential for leveraging existing investments by the federal government, state governments, school systems, and private-sector organizations. The U.S. Department of Edu- cation, the National Science Foundation, and the National Insti- tute of Child Health and Human Development have important ongoing programs supporting research and education reform. Likewise, major private foundations have long been working to improve education. In our view, a sustained and focused strate- gic research and development program like SERP can strengthen and leverage these efforts, helping them to realize their goals and bringing greater coherence and staying power to the whole. In the sectors in which research has had a substantial impact on practice (and vice versa), it is well understood that research and development is an ongoing process, an unfolding of knowI- edge and understanding, marked now and then by important breakthroughs, producing new products along the way, revis- ing and backtracking as effects are better understood. Close interaction between research and practice is important both to enable proposals developed in research to be tested and ad- justed so that they can work in practice, and to enable research- ers to understand the problems, puzzles, and constraints of the operational environment. The research investment takes years, even decades, to reach fruition. The history of investment in T H E N E E D F O R A N E W P A R T N E R S H I P 25
education research, with some notable exceptions, is one marked by impatience and abandonment of research endeavors in a relatively short time frame. The pressure for immediate results is not surprising when education research takes place largely within the confines of a political institution. Voters whose children are in the public schools want those schools to improve their performance now. A 10- or 15-year time horizon is, from a parent's point of view, entirely useless. Because education is perceived as so central to the future opportunities of today's children, a focus on what can produce results in the short run is demanded of those in a new administration. And it is entirely understandable for each new administration to put its mark on education improvement by emphasizing what it will do differently what it sees as poten- tial paths to improvement. What is needed is a program of research that proceeds from an understanding of how students learn, to research on the design and testing of instructional tools and programs, to repli- cation in a range of contexts, to a study of teacher knowledge requirements, and finally, to an examination of the organiza- tional requirements that support instruction and teacher learn- ing. Without such sustained attention, research-based knowI- edge cannot accumulate and grow more sophisticated, and the potential contribution of research to practice cannot be realized. A successful SERP research enterprise working at some re- move from the political arena could serve as a significant asset to the existing education and education research agencies. Like the core holdings in an investment portfolio, SERP would repre- sent long-term positions, accumulating steadily over time, which the nation holds for years. SERP would anchor the "managed portfolios" of political leaders or private philanthropies. What- ever the goals for educational improvement of a particular po- litical administration, the availability of a solid research founda- tion would facilitate policy making. Whatever the particular education reform interests of foundations be it urban schools or small schools or improving the prospects of minority and disadvantaged youth the accumulating SERP data on learn- ing, instruction, and schools as organizations would help them shape their action programs; partnering with SERP would help improve their investments. If SERP had existed when Congress mandated the State- wide Systemic Initiative in 1990, for example, the National Sci- 26 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
ence Foundation could have turned to SERP for an integrated knowledge base on the barriers to change in complex organiza- tions, or effective organizational and environmental supports for academic learning. If a future administration were to decide to pursue a reform agenda like Reading First, with its legislative mandate that state grant recipients implement research-based reading programs, SERP could provide the states with critically needed informa- tion about the characteristics of high-quality curricula and with examples of curricula that had been tested in various school settings. SERP could also produce research-based information on the teacher knowledge and teacher training associated with the effective use of a given curriculum. To take another example, the Gates Foundation is presently devoting major resources to a small-school experiment in the state of Washington. To understand under what conditions size makes a difference or how to take optimal advantage of small school size, it might engage SERP in a systematic exploration of the contributions of size, curriculum, teacher quaTity/profes- sional development, and other factors critical to student perfor- mance. Over the long term, this should significantly increase the payoff of its experiment in education. If a strong SERP existed, some of the heat could be removed from the education debates that presently tend to cripple progress in education. A SERP enterprise one step removed from politics could take on questions about theories of learning or the efficacy of different instructional approaches (basic skills versus inquiry-based, phonics versus whole language) and sub- ject such questions to systematic study. These questions are answerable. While policy positions will not and should not- be determined by research outcomes alone, they should be in- formed by scientific research on learning, and by data on the relative effectiveness of various programs for various kinds of children and the conditions that maximize effectiveness. In sum, SERP will offer action-oriented program links to the best research knowledge. It will offer funding agencies and foundations productive ways to work at the intersection of research and practice. It will offer those who worry about train- ing future leaders of the research and education establishments an opportunity to create new careers that grow out of the inter- action of research and practice, while providing for productive collaboration with the other parts of the education research T H E N E E D F O R A N E W P A R T N E R S H I P 27
enterprise. But most importantly, it will offer teachers, adminis- trators, and state policy makers a steadily improving knowI- edge base to support them in their critical mission of educating the nation's children. In the chapters that follow, we offer the committee's pro- posal for the design of an organization with the capabilities to carry out the broad SERP mission, a vision of how such an organization would attract the participation and funding needed for success, and illustrative examples of the kind of work the enterprise might undertake. But before we turn to that discus- sion, we address an even more fundamental question: would an infrastructure like the proposed SERP make a significant differ- ence in teaching and learning? Although proof cannot be pro- vided a priori, we devote Chapter 2 to examples that make a powerful case that such an infrastructure would considerably enhance the productivity of the education system. 28 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
