Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
A SERP Networks: 10 J hat an r l IT J Ill ~ U 1l l flea loo? n Chapter 3 we sketched a picture of each of the SERP organizational Darts. In this chanter we turn to anima- . ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ tion of the 5~;l~l' core: the research and development networks. What incentives would bring researchers, practitioners, and school systems to collaborate in the networks? And once they joined the efforts, what type of work would they do? Finally, we consider the incentives for those engaged in the delivery of education to make use of what SERP would offer. · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ CREATING NETWORK PARTNERSHIPS She nature of the collaboration contemplated for SERP is different in qualit~v and scope from any currently in place in education. She committee has found the teaching hospital a useful metaphor for the envisioned collaborative relationships: the functions of practice and research would in good part be located in the same site (SERP field sites), professional prepara- tion of both practitioners and researchers would be merged with high-level research activity, and links with university re- search departments would be strong. · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WHO WOULD COME? Where are many examples of existing partnerships between researchers and practitioners, including the cases described in Chapter 2. When scaled against the size of the K-12 education S E R P N E T W O R K S 79
system, however, the number is dwarfed by the need. Even more importantly, these partnerships generally require Herculean efforts on the part of individuals and are very diffi- cult to sustain when those individuals move on. How can SERP create a set of incentives for participation by both researchers and practitioners that makes collaboration more commonplace and that facilitates the maintenance of those collaborations in a sustainable fashion? B R ~ N G ~ N G R E S E A R C H E R S T O S E R P To create an organization with the capacity to attract high- quaTit~v researchers, we need to know what motivates the deci- sions of researchers' regarding the work they pursue. The committee's hypothesis is that researchers decisions can be roughly characterized as balancing five considerations, although the weight given to each varies tremendously across individu- als. The five are reputation, career opportunity, intellectual stimulation, income, and the ability to make civic contributions through their work to the public good. While an individual researcher may value one of these highly and another very little, we think each of the five is important to some researchers. The more participation in SERP can advance or at least not jeopar- dize these goals, the more successful the institution will be at attracting outstanding researchers. Reputation and career opportunity are closely linked. Both are promoted when researchers produce high-quaTity, publishable work. SERP would promote productivity and quaTit~v in re- search in its program in several ways, among them: · By building a high-quaTit~v, well-maintained, longitudinal database. High-quaTity, longitudinal datasets are the workhorses of empirical research. They allow tests of hypotheses regarding causal relationships that cross-sectional data cannot support. And they allow correlations over time to be observed that sup- port new hypotheses and theory building. They also fuel a great number of publications. The National Education Longitudinal Study (NELS), conducted by the National Center for Education Statistics, is a case in point. A large body of research has been 80 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
spawned in the years since the first follow-up in 1990 to the initial data collection in 1988. In order to carry out the program of research envisioned for SERP, longitudinal data collection that allows for empirical in- vestigation of the long-term effects of curricular interventions, instructional strategies, organizational environment, and policy choices will be required. In this sense, the needs of the research program overlap directly with the professional needs of re- searchers. To meet both needs will require that data collection be a high priority and be adequately funded by the institution from the start. · By supporting the capacity for rigorous research design. The overall quality of research produced through SERP will determine whether affiliation with the institution enhances or detracts from a researcher's reputation. Providing institutional standards and support to engage in high-quaTity work can there- fore enhance both the quality of the individual's work and the draw of the institution in the community of researchers. SERP would support quality by creating and maintaining high stan- dards for research design through the scientific review board and peer review processes described in Chapter 3. Quality research is likely to enhance one's reputation more if it is noticed. By building a coherent program of research, we would expect individual contributions to carry more signifi- cance because they are part of a body of work that together supports a set of understandings and provides a foundation for decision making. In this sense, the more effective the SERP effort to steer the research and development program in productive directions, the greater will be the draw of the institution. Many researchers see their greatest career opportunities in university positions. For this reason, affiliation with SERP must be compatible with university commitments. The proposed SERP structure, with most of the research and development being carried out in dispersed field sites where researchers and practi- tioners partner, allows for a simultaneous commitment to a university and to SERP research. In addition, SERP would need to work actively with universities to obtain permissions for S E R P N E T W O R K S 81
researchers who come to SERP headquarters to obtain multiyear, fixed-term leaves of absence. . We would expect intellectual stimulation to be a natural by- product of SERP. We would argue that stimulation is likely to be highest in two situations: when people from the same field come together to push the boundaries of understanding in that field, and when people from different fields work together on a problem, allowing for multiple paradigms and models to gener- ate new understandings and ideas. SERP would expand the opportunities for both types of stimulation substantially. The research agendas of the networks would keep research teams in close contact and collaboration in order to advance the knowI- edge base. And SERP would create a venue for a type of inter- disciplinary work on education issues that has been relatively rare because education departments in universities have gener- ally not been successful at attracting scholars from other disci- plines. But perhaps the most powerful lever SERP would wield to increase the intellectual options for researchers would come from the opportunities that SERP is specifically designed to create and nurture: allowing for disciplinary researchers to work closely with research-oriented practitioners. Negotiating the re- lationships with expert practitioners that lead to real collabora- tive research is not easy, and yet anyone who has benefited from such a relationship recognizes that it is highly productive and enormously stimulating. Researchers who value making a civic contribution through their work may be thwarted by the logistical difficulty of doing so. The lack of an infrastructure that links research and practice makes it difficult for education researchers to contribute to the education system. Education Week recently published a story about a Harvard education researcher, Richard Murnane, who decided to spend a sabbatical year offering his help to the local school district. While Murnane's effort eventually resulted in an ongoing connection with the schools through which he is help- ing train practitioners to effectively use data about their stu- dents, getting there was not easy. Neither Murnane nor the schools knew at the outset what, if anything, he could do to be helpful. His determination kept him in a situation that was at first quite awkward. By facilitating links between researchers and schools, SERP would allow for those interested in making a civic contribution to do so without having to leap such a high barrier to search for a place in which to be useful individually. 82 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
Attracting researchers to problems of education like all other research problems will be greatly facilitated if there are financial rewards for doing so. There are clear precedents in military, space, and medical research. The relatively meager funding of education research and the instability of the federal commitment of funds for education research over the years have not created a strong pull on intellectual resources to the problems of education. If SERP operates with a relatively stable, sizable budget as described in Chapter 3, there will be a finan- cial incentive for researchers to turn their attention to the prob- lems of educational practice that could become quite powerful as the commitments of states grow. ATTRACTING PRACTITIONERS TO SERP . What would motivate teachers to participate in collabora- tive research? We hypothesize rather different incentives in the case of teachers. Because the rewards of the profession are quite different, what is likely to be required in order to compete for a teacher's time commitment will be different as well. While the financial rewards of teaching are not as great as those in re- search, what teaching does provide is an opportunity to engage with students in a meaningful, sometimes life-shaping, way. It gives one an opportunity to open new worlds, and new oppor- tunities, to students. SERP, we believe, will be attractive to teachers if the SERP work allows them to do their jobs more successfully and to influence the lives of their students more positively. SERP will provide an opportunity for teachers in the sense that the knowledge from research will be brought to bear in context and on the problems of practice. We envision SERP as providing professional development opportunities for teachers and summer stipends to attend SERP seminars and workshops. Teaching can also be attractive for the job stability it pro- vides and for the predictability of the schedule. But precisely those characteristics can pose a challenge to attracting teachers to SERP. The rigidity of a teacher's schedule does not typically allow for other commitments during a school year, nor for job security or seniority in the case of an extended absence. For SERP to attract teachers, institutional arrangements will need to be made with schools to provide for either a teacher's extended leave without penalty or a reduced workload to allow for par- ticipation in a research project during a school year. The nego- S E R P N E T W O R K S 83
tiation of these arrangements through the office of research services should create opportunities for teachers that are now lacking. As with researchers, teachers will vary tremendously in their drives for intellectual stimulation and to make a contribu- tion to their profession writ large. Teachers who seek venues for these purposes currently have few options. Teachers and ad- ministrators lead activit~v-driven lives; they are rewarded for working long hours and being endlessly "available." They are not rewarded for reflection on their practice. Moreover, much of their work is done in isolation from others in the profession. As several of the cases in Chapter 2 suggest, practitioners who want to empirically test teaching approaches or interventions are often left to find their own path to the world of research through unmarked territory. Certainly not every teacher is in- terested in working with research teams. But for those who are, SERP will provide a venue that does not now exist. Finally, much of the SERP work would be designed to learn from teaching practice, so the knowledge that teachers bring will be actively employed and valued. For teachers who do look to research, it can be quite frustrating that what is available often does not address the complexity of the problems they face. The opportunity to influence the research agenda is likely to be an important draw for this group. ATTRACTING FIELD SITES Why would schools, school districts, or states agree to serve as field sites for SERP research? The primary answer, in our view, is that education leaders at all three levels have a desire- whether motivated by civic contribution, reputation, career op- portunity, or accountability measures to do a better job of educating students. If SERP is successful in its early years at linking research and practice to improve student learning out- comes, we think that attracting new field sites will be facilitated by the expectation of continued success. It may be more of a challenge, however, in the start-up period before proof of con- cept has been demonstrated. For this reason, early efforts to establish field sites should target schools and districts that have demonstrated interests in engaging in research and development efforts. School districts in Boston, Chicago, Pittsburgh, and Providence, for example, 84 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
currently work with researchers. While drawing on exciting opportunities is not likely to produce a representative sample of schools or students, it should provide the foundation for suc- cess early on that will create the possibility of a representative sample of field sites further down the road. The benefits of the SERP research will be available to all schools in all states that join the SERP compact. It therefore ~ . stands to reason that the costs of the field site research should be borne primarily by the organization rather than the participat- ing schools. To encourage field site participation and an effi- cient distribution of costs, we propose that SERP funds be used to buy out teacher time and cover other new costs imposed by the research. When expenditures substitute for those made rou- tinely by a participating site (e.g., an investment in data collec- tion), costs should be shared so that the school is neither taxed nor subsidized by involvement with SERP. · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WHAT WOULD THEY DO? If SERP is successful at attracting researchers, practitioners, and field sites, what will they do when they join a research and development network? The answer to that question will be determined by those who are given leadership responsibility in the new organization. Below we provide our vision of what that leadership might do. The committee is proposing the inauguration of three net- works during the start-up years, although others may eventu- ally be added. We recommend that a learning and instruction network be initiated at the very start because the relative matu- rity of research in the cognitive and developmental sciences holds promise for real improvement in how teachers are pre- pared and what they do in the classroom. A second network on schools as organizations would, we think, be a critical compan- ion to the learning and instruction network because the organi- zational environment of the school creates the conditions and incentives required to support and sustain instructional change. We anticipate that as work progresses during the start-up years, each network would develop five to eight related strands of work that speak to its hub question, employing the full panoply of research methods (experimental design, longitudinal studies, S E R P N E T W O R K S 85
observational techniques, formative and summative evaluations, etc.) in an iterative process that produces ever more refined knowledge. We propose a third network to house research and develop- ment on issues that profoundly influence both instruction and organization management: education policy. This network would embrace research on issues like accountability testing, class/school size, education finance, school choice, desegrega- tion, and other policy issues that create the incentives and envi- ronment to which teachers and schools respond. How quickly a third network is put in place will depend in large part on the funds available to launch the SERP enterprise. AN ILLUSTRATIVE AGENDA FOR A SERP NETWORK ON LEARNING AND I NSTRUCTION To make the case for the value of a SERP research and development program more concretely, an expert pane! of prac- titioners and researchers was convened by the National Re- search Council to complement the work of our Committee. Its task was to design an illustrative agenda for one network that would simulate, in a sense, the role of the agenda-setting advi- sory board and the leadership of a learning and instruction network. We selected this network from among those proposed because the National Academies have in recent years produced important syntheses of the research literature on human learn- ing (National Research Council, 1999) and on assessment of learning (National Research Council, 2001), as well as disci- pline-specific syntheses in reading and in mathematics (Na- tional Research Council, 199S, 2001~. These and other explora- tions of the knowledge base on learning and instruction (National Institute on Child Health and Human Development, 2000; RAND, 2002a, 2002b) provided a rich foundation on which our effort could build. This focus is not intended to suggest preeminence of the learning and instruction network. In our judgment, the organi- zational and implementation issues associated with educational change are also absolutely central to SERP's mission. Efforts to improve learning and instruction, the motivation and engage- 86 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
ment of students and teachers, and the quality of curricula, assessments, and instructional materials cannot succeed unless they are attentive to the variety of organizational and institu- tional contexts in which instruction occurs. Indeed, in our view, it is in combination with the work of other networks that the research and development on learning and instruction will have maximum likelihood of influencing practice. GENERATING A CONCEPTUAL FRAMEWORK Because the intended focus of the SERP program of research and development is educational practice, the conceptual frame- work proposed by the learning and instruction panel begins not with the questions posed by any research discipline, but with the questions that define teaching practice. The point of present- ing this conceptual framework here, before we get to the specif- ics of the agenda proposed for the area of early mathematics, is to demonstrate how broadly one needs to be able to think about educational research and improvement how many different perspectives and sources of knowledge are needed to think productively about even a single, specific, focused research prob- lem. SERP's competitive advantage is precisely that it can bring together the array of researchers and practitioners who can keep these many different perspectives in play simultaneously. We can view classroom instruction as organized around a set of core questions that apply no matter what the subject: · What do we want students to know or be able to do? · What are the typical understandings and precon- ceptions students hold on this topic at the outset? · What is the expected progression of understanding and skill mastery, and what are the predictable points of difficulty or hurdles that must be overcome? · What instructional interventions can move students along a path from their initial understandings and skills to the desired outcome (curricula, instructional activi- ties, etch? · What general and discipline-specific norms and practices best comprise and support student learning? · And finally, how can the individual student's progress be monitored and the student be engaged in S E R P N E T W O R K S 87
the instructional activities that she or he needs to take the next step toward increased understanding and skill? Whether explicitly or implicitly, by design or by default, teachers answer the above questions in the course of their teach- ing practice. In doing so, teachers (like professionals in other fields) draw on standards of practice, professional preparation, background knowledge, tradition, and personal inclinations and intuitions. A program of research and development can im- prove the answers to those questions by providing a solid knowI- edge base to support both teaching practice and professional preparation, as well as by expanding the instructional and as- sessment tools available to teachers. The questions that define educational practice can be in- formed by several very different fields of research and knowI- edge: 88 · What students should know or be able to do in an area is informed (but not fully determined) by disciplinary ex- pertise. It requires an understanding of the core con- cepts around which the disciplinary knowledge is orga- nized, characteristic methods of reasoning and problem solving, and language and patterns of discourse. What to teach becomes not only a matter of the information and skills considered important but also of helping the student to build the conceptual framework that trans- forms or helps to organize information into understand- ngs. · Knowledge of common student conceptions of a topic and the expected progression of student thinking requires careful research on the typical trajectory of understand- ing. In part this research attempts to identify the nature and limits of children's changing cognitive abilities with age and instruction. And in part it attempts to uncover common understandings that can either support learn- ing (the ability to halve or double relatively easily in mathematics) or undermine it (the belief that heat and temperature are the same thing). Research findings dem- onstrate the remarkable resilience of students' everyday understandings even after instruction to the contrary (and often to the great surprise of teachers). This high- STRATEGIC EDUCATION RESEARCH PARTNERSHIP
lights the importance of a carefully designed research program to inform and support practice. Research of this sort is often done by cognitive scientists and educa- tion researchers, although the knowledge may emerge from the experience of expert teachers and the observa- tion of exemplary practice. · Educative experiences intended to move students along a learning path constitute the core of what we consider to be "instruction." These experiences are ulti- mately created by the teacher, but teachers usually draw on materials generated by curriculum developers or (less often) researchers. Instructional programs involve as- sumptions about the contributors to skill development, knowledge acquisition, and conceptual change that should themselves be a research agenda, and the effec- tiveness of the instructional approach is a matter for empirical testing. · General and discipline-specific norms and practices that support student learning. Learning takes place in ciass- rooms that are themselves communities. Every commu- nit~v is distinguished by norms for work and interac- tions, ranging from when and how people collaborate to how they speak with one another. Some of those norms are general rooted in the understanding of schools in a democratic society; others are specific what it means to do mathematics differs from what it means to do literary analysis or chemistry or history. How individual norms contribute to or undermine student learning, and how this differs by community, are empirical questions that draw on sociological and psychological understandings, as well as on a rigorous evaluation of classroom prac- tices. · Assessing the current level of an individual student's understanding is itself an interdisciplinary undertaking because it requires an understanding of both what con- stitutes learning and how to measure it. To be useful in the learning process, the assessment must be tied to instructional responses. Assessment, then, requires that the knowledge relevant to each of the above questions be incorporated into the design and testing of specific instruments. S E R P N E T W O R K S 89
The knowledge bases described above must be incorporated into teacher education opportunities, instructional programs, curriculum materials, and other tools that facilitate the work of the teacher. The learning and instruction network must there- fore be concerned both with shoring up the knowledge base on each of the questions above and on incorporating that knowI- edge into education programs, tools, and teaching protocols. Practice is not embodied solely in the tools and protocols of the trade, however. Rather, these work in tandem with both the knowledge and skill of the practitioner. TEAC H E R KNOWLE DG E AN D LEARN ~ NG Teacher knowledge and skill matters a great deal in student learning. Ferguson (1991) analyzed data from 900 Texas school districts and found that teacher licensing exam scores, masters degrees, and experience accounted for over 40 percent of the variance in students' reading and mathematics achievement scores after controlling for socioeconomic status. Other studies suggest a similarly powerful effect (Ferguson and Ladd, 1996; Strauss and Sawyer, 1986~. Yet despite its importance, the re- search base on teacher learning is relatively undeveloped; the grasp of the content of the teacher knowledge that produces achievement is shallow. The questions above that we argue define teaching practice apply just as aptly to teacher learning as to student learning. For teachers, however, we have a good start on the first question: "What do we want the teacher to know and be able to do?" The answer is defined by the questions that frame teaching practice. We want teachers to understand the learning process of the student well enough to assess and guide it; the content well enough to select appropriate instructional materials, guide the pace and direction of instruction, and flexibly respond to stu- dent questions and thoughts; the curriculum materials well enough to use them as a means to an end rather than as the end itself; the norms and practices that constitute effective practice well enough to create a supportive learning environment in the classroom; and assessments well enough to interpret the out- comes and respond appropriately. What is not well defined are the forms of knowledge a teacher must master in order to reach that end and what levels of mastery are needed. What mathematics must a teacher know, 90 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
and what pedagogical knowledge is required, to make and implement appropriate decisions about the best instructional steps to develop student thinking about rational number, for example? Although these questions are central to effective prac- tice, little research has been done to provide answers. Moreover, learning is as complex an undertaking when the teacher is the target as it is when the student is the target. Teachers' conceptions of learning and instruction, of student thinking and age-specific capabilities, and of the subject matter often diverge considerably from research-based findings (Na- tional Research Council, 2000; Palincsar and Herrenkohl, 2002; Palincsar et al., 1989~. These existing conceptions must be un- derstood and engaged. And experiences that bring about con- ceptual change for the teacher must be designed and effectively deployed for learning to occur. More complex still than creating conceptual change is the relationship between a teacher's knowledge and instructional practice. Practice requires knowing what elements of the knowl- edge base are relevant in a specific situation and what character- istics define the situation itself. This is what is meant by "conditionalized knowledge." A comparison to medicine is illu- minating. Understanding well how the human body functions and malfunctions does not ensure that a medical student con- fronted with a patient will know which avenues to explore in response to a description of symptoms, or which features of the patient are particularly worthy of note. A medical student who has finished course work is not permitted to practice alone without first having extensive experience in observing the knowl- edge used in the context of practice. Much that teachers need to know cannot be learned apart from practice, just as learning to ride a bike requires experience with the thing that is being learned (Polyanyi, 1967~. This raises several questions for inquiry: Under what conditions can teach- ers best learn while engaged in practice? What knowledge and skill must teachers acquire at the beginning of their careers? What knowledge and skill is best acquired once they enter the profession? What organizational, material, and human resources are necessary to support and sustain teacher learning over time? The conceptual framework adopted by the learning and instruction panel, then, is defined by the questions that teachers must answer to effectively educate their students, and the ques- tions teacher-educators must answer to effectively prepare teach- S E R P N E T W O R K S 91
ers for practice. The framework can be applied no matter what the subject matter to be taught. A STRATEGIC AGENDA In the Tong run, providing research-based knowledge to support answers to the above questions for every subject taught in schools is a desirable end just as one expects the treatment of any ailment by a physician to be based on research-based knowledge (amplified by craft knowledge). Yet the reality of the limited resources devoted to education research and the exist- ing capacity to conduct that research suggest the need for focus on a limited set of subjects in order to ensure that work can be carried through all stages necessary for usability. As a knowI- edge base is consolidated in some areas, attention can be de- voted to new subject areas. The pane! chose three areas for focus: mathematics, science, and reading. The rationale for its choices, as well as the full agenda in each of the domains, appears in a companion report, Learning and Instruction: A SERF Research Agenda (National Re- search Council, 2003b). To develop a strategic agenda, the com- mittee sought to identify subject areas in each domain that fall into two categories: (1) areas in which considerable progress has already been made in answering some of the important ques- tions of instruction. Additional work that builds on that success can be expected to contribute to improvements in practice in the relatively near term. Work on whole number, early reading, and physics falls into this category. (2) subjects characterized by fundamental gaps in the knowledge required to inform instruc- tion. Algebra, elementary and middle school science, and read- ing comprehension fall into this category. While the first group takes advantage of existing opportunities, the second begins with pressing problems of practice. In each area, the panel considered the relative, and in some cases unique, advantage of the SERP infrastructure for support- ing research and development. In this chapter we discuss just one of the focal areas from the full panel report as an example: elementary mathematics. ELEMENTARY MATHEMATICS ~ . Investment in recent decades by federal agencies and pri- vate foundations has produced a wealth of knowledge on the 92 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
development of mathematical understanding, as well as numer- ous curricula that incorporate that knowledge. As a result, el- ementary mathematics is ripe for productive investments in making that knowledge usable and used widely by schools. The committee had the benefit of drawing on a recent synthesis of research on elementary mathematics (National Research Coun- cil, 2001) and on the work of a RAND study group that pro- duced a mathematics research agenda (RAND, 2002b). What Do We Want Chilciren to Know or Be Able to Do ? A recent consensus report at the National Research Council (2001) shows that U.S. students fare poorly in international comparisons of mathematics achievement. They show weak understanding of basic mathematical concepts, and although they can perform straightforward computational procedures, they are notably weak in applying mathematical skills to solve even simple problems. These results have generally been attrib- uted to the shallow and diffuse treatment of topics in elemen- tary mathematics relative to that in other countries, as well as an instructional emphasis on repeated practice with paper and pencil skills in arithmetic (National Research Council, 2001~. This report also provides an argument for what elementary schoolchildren should know and be able to do in mathematics that draws on a solid research base in cognitive psychology and mathematics education. The consensus includes mastery of pro- cedures but places far more emphasis on understanding when and how to apply those procedures. The latter is rooted in a deeper understanding of mathematical concepts, and a facility with mathematical reasoning. The report identifies five inter- twining strands that constitute mathematical proficiency (Na- tional Research Council, 2001~: · Conceptual understanding comprehension of math- ematical concepts, operations, and relations; · Proceduralfluency skill in carrying out procedures flexibly, accurately, efficiently, and appropriately; · Strategic competence ability to formulate, represent, and solve mathematical problems; · Adaptive reasoning capacity for logical thought, re- flection, explanation, and justification; S E R P N E T W O R K S 93
· Productive disposition habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one's own efficacy. The instructional issues, then, involve the means to achiev ing this more ambitious goal of mathematical proficiency. Progression of Unclerstancling Research has uncovered an awareness of number in infants shortly after birth. The ability to represent number and the development of informal strategies to solve number problems develops in the child over time. Many studies have explored how preschoolers and young schoolchildren understand basic number concepts and begin operating with number informally well before formal instruction begins. Children's understanding progresses from a global notion of a little or a Tot to the ability to perform mental calculations with specific quantities (Griffin and Case, 1997~. Initially, the quantities children can work with are small, and their methods are intuitive and concrete. In the early elementary grades, they proceed to methods that are more general (less problem depen- dent) and more abstract. Children display this progression from concrete to abstract in operations first with single-digit num- bers, then with multidigit numbers. Importantly for instruction, the extent and the pace of development depend on experiences that support and extend the emerging abilities. Researchers have identified two issues in early mathematics learning that pose considerable challenges for instruction: 1. Differences in individual experiences result in some children primarily those from economically disad- vantaged backgrounds entering kindergarten as much as two years behind their peers in the develop- ment of number concepts (Griffin and Case, 1997; see Chapter 1~. Children's informal mathematical reasoning and emer- gent strategy development can serve as a powerful foundation for mathematics instruction. However, in- struction that does not explore, build on, or connect with children's informal reasoning processes and ap- proaches can have undesirable consequences. Children 94 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
can learn to use more formal algorithms, but are likely to apply them rigidly and sometimes inappropriately (see Box 4.1~. Mathematical proficiency is lost because procedural fluency is divorced from the mastery of concepts and mathematical reasoning that give the procedures power. Curriculum Development Past investments in research and development have pro- duced curricular interventions to address each of the two prob- lems raised above. A well-developed and promising research base on the Number Worlds curriculum suggests that well- planned activities designed to put each step required in master- ing the concept of quantity securely in place can allow disad- vantaged students to catch up to their more advantaged peers right at the start of formal schooling. The curriculum has a · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ BOX 4.1 Buggy Algorithms When students attempt to apply conventional algorithms without conceptually grasping why and how the algorithm works, "bugs" are sometimes introduced. For example, teachers have long wrestled with the difficulties that second and third graders frequently have with multidigit subtraction in problems, such as 51 -14 A common error is 51 -14 43 The subtraction procedure above is a classic case: Children subtract "up" when subtracting "down" tried first is not possible. Here, students would try to subtract 4 from I and, seeing that they could not do this, would subtract I from 4 instead. These "buggy algorithms" are often both resilient and persistent. Consider how reasonable the above procedure is: in addition problems which look similar, children can add up or down and get a correct result either way: 51 +14 65 Bugs often remain undetected when teachers do not see the highly regular pattern in students' errors, responding to them more as though they were random miscalculations. S E R P N E T W O R K S 95
companion assessment tool (the Number Knowledge Test) to help the teacher monitor and guide instruction. If results in controlled trials (see Chapter 1) could be attained in schools across the country that serve disadvantaged populations, this would represent a major success with respect to narrowing the achievement gap a long-standing national goal that has proven difficult to realize. With respect to the second concern, research done in the 1990s investigated the effects on student achievement of in- struction that builds on informal understandings and empha- sizes mathematical concepts and reasoning. Cobb et al.'s Prob- lem Centered Mathematics project (Wood and Sellers, 1997) and Cognitively Guided Instruction (CGI) in problem solving and conceptual understanding (Carpenter et al., 1996) both reported positive effects. With support from the National Science Foun- dation (NSF), several full-scare elementary mathematics cur- ricula with embedded assessments have been developed, di- rected at supporting deeper conceptual understanding of mathematics concepts and building on children's informal knowledge of mathematics to provide a more flexible founda- tion for supporting problem solving. Three curricula developed separately take somewhat different approaches to achieving those goals: the Everyday Mathematics curriculum, the Investi- gations in Number, Data and Space curriculum, and the Math Trailblazers curriculum (Education Development Center, Inc., 2001~. While theories of learning help to identify problems these curricula have been designed to address, the curricula them- selves involve theories of instruction that must be tested. Do their efforts to provide more contextual learning opportunities that link students' informal thinking to mathematical problem solving produce students with stronger mathematics skills over- all, or are there trade-offs among the component skills? Do they perform as well for students who excel in mathematics as for students who struggle? All three curricula show positive gains in student achieve- ment in implementation studies in which the developers collect data on program effects. While such findings are encouraging, they must be viewed with a critical eye, both because those providing the assessment have a vested interest in the outcome and because the methodology employed does not allow for 96 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
direct attribution of the results to the program. Third-party evaluations using comparison groups have been done, but none of these has involved random assignment (the condition that maximizes confidence in attributing results to the intervention). Nor do these studies measure either fidelity of implementation of the reform curriculum for the experimental group or the specific program features of the alternative used with the con- tro! group (see, for example, Fuson et al., 2000~. From the perspective of practice, these are important omis- sions. To make informed curriculum decisions, teachers and school administrators need to know what type of implementa- tion of a specific curriculum produces what results, compared with what alternatives. Ideally, the reform curricula would be compared with traditional curricula that are highly rated or widely used (or both) in order to advance the knowledge base for practice. Yet to provide the information that is most useful to practice is a major undertaking. These questions are answer- able, but research carefully designed to provide the answers will take a substantial, Tong-term investment. A s s e s s m e n ~ The curricula described above have embedded assessments that allow teachers to track student learning. As previously mentioned, a key feature of the Number Worlds curriculum is the Number Knowledge Test that allows teachers to closely link instructional activities for children to the assessment results. How well other curricula link assessment and instruction is an issue worthy of investigation. A separate issue is the assessment over time of the five strands that constitute mathematical proficiency. The last de- cade has seen the emergence of a spate of new tests and mea- sures. No consensus has emerged, however, on critical mea- surest While there are some standard and widely used assessment tools to appraise young children's emergent reading and language skills and competence, no such tools are used on any comparable basis in primary mathematics. This type of assessment will be required to evaluate the effectiveness of a particular curriculum and to make compari- sons across curricula. For the most part, we lack sophisticated methods for tracking student learning over time or for examin- S E R P N E T W O R K S 97
ing the contribution of any particular instructional interven- tions, whether large or small, on students' learning. A research project that focused on mathematics teaching and learning might begin by developing such tools. Teacher Knowlecige Little is known about what it might take for teachers to use particular instructional approaches effectively, a necessary ele- ment of taking any particular approach to scale. The challenges can be substantial. The curricula mentioned above introduce major changes in approach to teaching mathematics, and effec- tive implementation will require that teachers change their view of mathematics teaching and learning dramatically. In Every- day Mathematics, for example, teachers are expected to intro- duce topics that will be revisited later in the curriculum. Com- plete mastery is not expected with the first introduction. This has created some confusion for teachers, who are often unclear about when mastery is sufficient to move on to the next topic (Fuson et al., 2000~. All of the curricula encourage building on students' own strategies for problem solving and supporting engagement through dialogue about the benefits of alternative strategies. The change required on the part of the teacher to relinquish control of the answer in favor of a dialogue among students, where it has been studied, has proven difficult to master (Palincsar et al., 1989~. The risks of change must be considered as well: if a teacher does turn control of the discus- sion over to students but is not prepared to guide that discus- sion productively, precious little learning may go on. Critical to the effectiveness of efforts to implement such curricula on a large scale, then, is that there be adequate teacher preparation and ongoing support for an entirely different approach to teach- ing. This is clearly an important area for further study. One clue regarding teacher knowledge requirements can be found in research pursued for the most part separately from the work on student learning and the design of curriculum ap- proaches, tools, and materials discussed above. Investigations of teachers' knowledge reveal that although teachers can, for the most part, "do" the mathematics themselves, they often are unable to explain why procedures work, distinguish different interpretations of particular operations, or use a model to closely map the meaning of a concept or a procedure. For example, 98 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
teachers may be able to use concrete materials to verify that the answer to the subtraction problem in Box 4.1 is 37 and not 43. They can operate in the world of base ten blocks to solve 51 - 14 but may not be able to use base ten blocks to demonstrate the meaning of each step of the conventional (or other) algorithm. Similarly, teachers may be able to compute using familiar standard algorithms but not be able to recognize, interpret, or evaluate the mathematical quality of an alternative algorithm. They may not be able to ascertain whether a nonconventional method generalizes or to compare the relative merits and disad- vantages of different algorithms (for example, their transpar- ency, efficiency, compactness, or the extent to which they are either error-prone or likely to avert a calculation error). Over and over, evidence reveals that knowing mathematics for one- self (i.e., to function as a mathematically competent adult) is insufficient knowledge for teaching the subject. In the domain of early number, studies suggest that most teachers' own knowI- edge is solid, but that their understanding of conceptual foun- dations is uneven. Following this work, some materials for use in teachers' professional development have been developed.2 Modules and other curriculum materials contain focused work aimed at help- ing teachers learn the sort of mathematical knowledge of whole numbers and operations that is needed for teaching. As with the curricula developed for students' learning discussed above, de- velopers of teacher learning materials provide some evidence of teachers' learning of mathematics for teaching, but they have studied less the role of this learning in the teacher's subsequent instructional practice and effectiveness. Still less is known about what teacher developers them- selves need to know to support teachers' learning and how their professional learning might be supported. The demand for skilled leaders who can teach teachers is growing, but the field, though highly remunerative, is unregulated and all too often typified by inadequately trained instructors and badly designed ~ Base ten blocks are a common material used to model place value con- cepts and operations that rely centrally on place value. The materials consist of a unit cube, a ten-stick built of 10 cubes, a flat square built of 100 cubes or 10 ten-sticks, and a block composed of 1,000 cubes, or 10 flats, or 100 ten-sticks. 2 See, for example, work by Schifter and her colleagues at Education Devel- opment Center, Developing Mathematics Instruction (Schifter et al., 1999~. S E R P N E T W O R K S 99
delivery methods. Scaling up materials that can support teach- ers' learning of mathematics for teaching will require worrying about the knowledge requirements of those who will guide and support the teachers. The SERP Agenda Given the current state of practice and knowledge about learning and teaching of early number, then, what might a SERP program of research and development seek to do? How might it build on what currently exists and begin to extend and fill gaps in what is known and done, with the ultimate goal of more reliably and productively building evidence-based instruc- tional practice? In other words, how could work be planned and carried out that would extend what is known and take that to scale in U.S. schools? The proposed agenda is comprised of three major initia- tives. The first focuses on developing assessments to measure student knowledge, a second evaluates promising curricula and the effects of their particular design features on student out- comes, and a third focuses on the teacher knowledge require- ments to comfortably and effectively use curricula that are built on research-based findings regarding student learning. Initiative I: Developing Early Mathematics Assessments. Qual- ity assessments depend on three things: (1) clarity about the competencies that the assessment should measure; (2) tasks and observations that effectively capture those competencies; and (3) appropriate qualitative and quantitative techniques to give interpretive power to the test results. Clarity about the compe- tencies to be measured requires a theoretical understanding (that is empirically supported) of mathematics learning. Unlike many other areas of the curriculum, early mathematics has the theoretical and conceptual models, as well as supporting em- pirical data, on which to build quality assessments. Substantial work has already been done to specify critical concepts and skills within this domain, providing assessment developers with resources on which to draw in drafting the elements of a mea- surement strategy. Even with a strong foundation on which to build in early mathematics, much work remains in designing and testing as- sessment items to ensure that inferences can be accurately drawn too STRATEGIC EDUCATION RESEARCH PARTNERSHIP
about student knowledge and competencies. And this work must be carefully crafted for the specific purpose and use of the assessment, for example, formative assessment for use in the classroom to assist learning; summative assessment for use at the classroom, school, or district level to determine student attainment levels; or assessment for purposes of program evalu- ation. Formative assessments are essentially diagnostic. They can, for example, provide feedback to the teacher on a student's mastery of a particular skill or concept or on whether individual students need more time and practice before moving on to new material. Summative assessments are also used in the class- room, but they come at the end of a unit. They give a teacher feedback on how well the students have mastered and brought together the set of concepts and skills taught in the unit. These may be helpful to the teacher in redesigning instruction for the next year, providing valuable data on students' strengths and weaknesses that can inform instruction at the next level. School- or district-level assessments have more general policy purposes, most commonly to determine attainment levels for groups of students in order to evaluate the effectiveness of an instruc- tional program; to monitor attainment by racial, ethnic, or dis- ability category; and in some cases to hold schools accountable for the performance of their students. Currently the different types of assessment are loosely con- nected at best. Tensions are introduced when strong instruc- tional programs and accountability assessments are at odds. Better aligning assessments and tying all assessments firmly to the theoretical and empirical knowledge base are widely regarded as critical to improving learning outcomes. The con- struction of such a system represents a major research, develop- ment, and implementation agenda that would require the kind of stability, longevity, and support that SERP intends as its hallmark. The above work should be pursued as a collaborative effort involving teachers, content area specialists, cognitive scientists, and psychometricians. The effort could use as a departure point well-established standards in mathematics (e.g., National Coun- ci! of Teachers of Mathematics), standards-based curricular re- sources, and rigorous research on content learning to identify and define what students should know in early mathematics, S E R P N E T W O R K S ]0]
how they might be expected to show what they know, and how to appropriately interpret student performance. In the case of formative assessment, this extends to an understanding of the implications of what the evidence suggests for subsequent in- struction. In the case of summative assessment, this means un- derstanding the implications of student performance for mas- tery of core concepts and principles and the growth of this mastery over time. While there are several possible approaches to developing such a system of student assessments in early mathematics, one obvious place to begin is with a review of the assessment mate- rials in existing widely used and exemplary curricular pro- grams for formative and summative assessments, commercial testing programs, and state and national tests for policy making and accountability. These can be reviewed in light of cognitive theories of mathematical understanding, including empirical data regarding the validity of specific assessments. Research needs to focus on evidence of the effectiveness of specific assess- ments for capturing the range of student knowledge and profi- ciency for particular mathematical constructs and operations. A related line of inquiry would focus on issues of assessment scoring and reliability, particularly ease of scoring, consistency of scoring within and across individuals, and consistency of interpretation of the results relative to the underlying cognitive constructs. The development of assessments in early mathematics should be closely tied to complementary initiatives in the areas of teacher knowledge and curriculum effectiveness. Thus a strand of research focused on implementation issues should address the set of questions critical to successful use of quality assessments: · What teacher knowledge is necessary to support effective use of assessments in their instructional prac- tice? These include teacher understanding of the assess- meets and their purpose, as well as practical consider- ations of the time to administer, score, and interpret results. · What forms of technology support are needed to assist teachers in the administration, scoring, and inter- pretation of a range of standards-based and theory-based assessments? 102 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
· How and to what extent does the process of imple- menting curriculum-based and standards-based assess- ments lead to changes in teachers' instructional prac- tices, and how do these changes affect student learning outcomes? This investigation should focus both on changes in the near term and the stability of changes in the Tong term. High-quality evidence that permits practitioners, research- ers, and policy makers to ask and answer comparative ques- tions will be critical to making the SERP research and develop- ment usable in practice. Initiative II: Teacher Knowledge. To take advantage of existing investments in research and development in elementary math- ematics will require further work regarding teacher learning and knowledge requirements and the supports that allow teach- ers to use these curricula comfortably and effectively. This re- search should begin with a clear articulation of the principles and assumptions about student learning that the curriculum incorporates, comparing these to carefully solicited understand- ings of teachers. Learning experiences should be designed to address the points of divergence and tested for their power to change teacher conceptions. Further research should test the effectiveness of different components of professional development on both teacher learn- ing and the learning of their students. The relative benefits of teacher guides, videotaped cases, and opportunities to pose questions and receive support should be tested, as well as the timing effect (before instruction begins, during instruction, etc.) for different teacher learning opportunities. Initiative III: Curriculum Evaluation. The identification (and further development) of a set of approaches to the teaching of number and operations that vary on distinct and theoretically important dimensions would permit careful comparisons of how particular instructional regimes impact students' learning. Programs and approaches already developed, such as Number Worlds, Cognitively Guided Instruction, the three NSF- supported curricula mentioned above, and well-regarded and widely used traditional curricula would form the initial core, S E R P N E T W O R K S '03
but analysis would permit such a core set to be complemented with other theoretically and practically important alternatives. Many of the evaluations of the curricula set out to answer the question, "Does the curriculum improve student achieve- ment?" While this is an important question and of particular interest to those who market a curriculum the questions of importance for Tong-term improvements in practice are why, for whom, and compared with what? Number Worlds shows very promising results for disadvantaged children; Everyday Mathematics does as well. How, and for whom, do those out- comes differ? Are there trade-offs in the competencies children gain from each? Does the context in which they work best differ? Each of the three NSF elementary mathematics curricula takes a somewhat different approach to instruction. How are those differences reflected in outcomes for students? Does one better address the needs of Tow- or high-achieving students? What are their respective organizational and implementation requirements? Are there lessons in the outcomes that could be used to improve any of the curricula or to combine features not now found in a single curriculum? An analysis of existing candidate materials could illuminate important differences, and strategic selections could be made. The implementation, adaptation, and use of these different ap- proaches could be followed over time, attending to instructional practice, students' opportunities to learn, and implementation issues. In addition, based on what is known about teachers' knowledge of whole number and operations for teaching, as well as about teacher learning, systematic variations could be designed to support the implementation of these different in- structional approaches. For example, in one set of schools, a teacher specialist model might be deployed, and, in others, teachers might engage in a closely focused study of practice (instruction, student learning, mathematical tasks), co-planning and analyzing lessons across the year. In still others, teachers might be provided with both incentives to spend time planning and adequate teacher guides. The work could be conducted in carefully controlled, Tongi- tudinal studies carried out in SERP field sites. A SERP organiza- tion like that described in Chapter 3 would be well positioned to carry out such work. Because it would have mutually beneficial relationships established with a number of field sites and data collection efforts in those sites already under way, taking on a ]04 STRATEGIC EDUCATION RESEARCH PARTNERSHIP
controlled experimental study of alternative curricula would be a far less daunting task than it would be for researchers working independently. Moreover, the concern for undertaking research that is maximally useful to educational practice and the ability to design and conduct or oversee the conduct of that re- search will be combined in a single organization. This is a situa- tion that does not now exist. The research initiatives described above provide a glimpse through a single window of a large-scale SERP research and development enterprise. The companion report provides a more extensive agenda, but even that is limited to the learning and instruction network. Perhaps the greatest benefit of the pro- posed SERP organization is that programs of research on schools as organizations and on education policy will be developed alongside that of learning and instruction. Yet, even within the confines of the early mathematics agenda considered here, dis- tinguishing features of SERP are apparent. This includes an effort to define a program of research that focuses on the prob- lems of practice, strategically building on strengths in the exist- ing knowledge base and shoring up its weakness. · ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WOULD SERP CHANGE PRACTICE? Even the highest quality SERP research and development will make a difference only if it is used in practice. What incen- tives will teachers, schools, districts, and states have to make use of the SERP work? AS we mentioned in the opening chapter, the current cli- mate in which schools are being held accountable for student performance creates some motivation to search for the means to improvement. While this may lubricate the wheels of change, large-scare accountability will not be a sufficient motivator. It is, by its nature, low-level accountability it generally attends to gross measures of skill performance. The SERP program, in contrast, is targeted at improving learning for understanding. This more difficult change in instruction will require more pow- erful motivators. These, in the committee's view, are (a) solid evidence that change will bring clear benefits in student perfor- mance and (b) support for implementing the change in real classroom environments. S E R P N E T W O R K S '05
Regarding the first, we have proposed an investment at the outset in measures of program outcome for all of the SERP work. As the case of the Cognitive Tutor Algebra I in Oklahoma suggests (see Chapter 2), the resistance to making a change often lies in the doubt that the new program will in fact be better, as well as the risk that it may be worse. Careful efforts to document student gains will, we think, serve as a significant inducement for teachers and schools to change their practices. In the proposed research and development agenda, the effort to measure impact permeates every strand of research. Further- more, the organizational design assigns to those responsible for quality assurance the task of measuring the impact of SERP research and development. In doing their job, an additional source of pressure will be placed on the research and develop- ment program to define clearly the expected program outcomes, so that impact data can be compiled as programs are imple- mented. These data will help inform school districts and states about the potential improvements associated with change. To- gether we expect these efforts to provide a powerful motivation for change. But motivation is not itself enough, either for the higher- level change sought by SERP or the Tower-level change encour- aged through accountability standards. Motivated teachers still must have the support to change their practice. The idea of attending seriously to what it takes to use research-based prac- tices at the school level runs throughout the envisioned SERP program. Carrying out much of the work in a range of class- room settings will allow the problems of classroom use to them- selves be a subject of study and an issue for development and program design. Indeed, in the Oklahoma case, the combination of evidence of success and support from the school district and the program developer to change curriculum resulted in all teachers preferring the new program, making continued con- trolled experimentation difficult. It is the unique combination of attention to carefully measured outcomes and attention to the requirements for classroom use that leads the committee to be convinced that SERP can make the kind of change possible that has been so difficult to achieve in the past. 106 STRATEGIC EDUCATION RESEARCH PARTNERSHIP