Click for next page ( 80


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 79
A SERP Networks: 10 J hat an r l IT J Ill ~ U 1l l flea loo? n Chapter 3 we sketched a picture of each of the SERP organizational Darts. In this chanter we turn to anima- . ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~ tion of the 5~;l~l' core: the research and development networks. What incentives would bring researchers, practitioners, and school systems to collaborate in the networks? And once they joined the efforts, what type of work would they do? Finally, we consider the incentives for those engaged in the delivery of education to make use of what SERP would offer. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ CREATING NETWORK PARTNERSHIPS She nature of the collaboration contemplated for SERP is different in qualit~v and scope from any currently in place in education. She committee has found the teaching hospital a useful metaphor for the envisioned collaborative relationships: the functions of practice and research would in good part be located in the same site (SERP field sites), professional prepara- tion of both practitioners and researchers would be merged with high-level research activity, and links with university re- search departments would be strong. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WHO WOULD COME? Where are many examples of existing partnerships between researchers and practitioners, including the cases described in Chapter 2. When scaled against the size of the K-12 education S E R P N E T W O R K S 79

OCR for page 79
system, however, the number is dwarfed by the need. Even more importantly, these partnerships generally require Herculean efforts on the part of individuals and are very diffi- cult to sustain when those individuals move on. How can SERP create a set of incentives for participation by both researchers and practitioners that makes collaboration more commonplace and that facilitates the maintenance of those collaborations in a sustainable fashion? B R ~ N G ~ N G R E S E A R C H E R S T O S E R P To create an organization with the capacity to attract high- quaTit~v researchers, we need to know what motivates the deci- sions of researchers' regarding the work they pursue. The committee's hypothesis is that researchers decisions can be roughly characterized as balancing five considerations, although the weight given to each varies tremendously across individu- als. The five are reputation, career opportunity, intellectual stimulation, income, and the ability to make civic contributions through their work to the public good. While an individual researcher may value one of these highly and another very little, we think each of the five is important to some researchers. The more participation in SERP can advance or at least not jeopar- dize these goals, the more successful the institution will be at attracting outstanding researchers. Reputation and career opportunity are closely linked. Both are promoted when researchers produce high-quaTity, publishable work. SERP would promote productivity and quaTit~v in re- search in its program in several ways, among them: By building a high-quaTit~v, well-maintained, longitudinal database. High-quaTity, longitudinal datasets are the workhorses of empirical research. They allow tests of hypotheses regarding causal relationships that cross-sectional data cannot support. And they allow correlations over time to be observed that sup- port new hypotheses and theory building. They also fuel a great number of publications. The National Education Longitudinal Study (NELS), conducted by the National Center for Education Statistics, is a case in point. A large body of research has been 80 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
spawned in the years since the first follow-up in 1990 to the initial data collection in 1988. In order to carry out the program of research envisioned for SERP, longitudinal data collection that allows for empirical in- vestigation of the long-term effects of curricular interventions, instructional strategies, organizational environment, and policy choices will be required. In this sense, the needs of the research program overlap directly with the professional needs of re- searchers. To meet both needs will require that data collection be a high priority and be adequately funded by the institution from the start. By supporting the capacity for rigorous research design. The overall quality of research produced through SERP will determine whether affiliation with the institution enhances or detracts from a researcher's reputation. Providing institutional standards and support to engage in high-quaTity work can there- fore enhance both the quality of the individual's work and the draw of the institution in the community of researchers. SERP would support quality by creating and maintaining high stan- dards for research design through the scientific review board and peer review processes described in Chapter 3. Quality research is likely to enhance one's reputation more if it is noticed. By building a coherent program of research, we would expect individual contributions to carry more signifi- cance because they are part of a body of work that together supports a set of understandings and provides a foundation for decision making. In this sense, the more effective the SERP effort to steer the research and development program in productive directions, the greater will be the draw of the institution. Many researchers see their greatest career opportunities in university positions. For this reason, affiliation with SERP must be compatible with university commitments. The proposed SERP structure, with most of the research and development being carried out in dispersed field sites where researchers and practi- tioners partner, allows for a simultaneous commitment to a university and to SERP research. In addition, SERP would need to work actively with universities to obtain permissions for S E R P N E T W O R K S 81

OCR for page 79
researchers who come to SERP headquarters to obtain multiyear, fixed-term leaves of absence. . We would expect intellectual stimulation to be a natural by- product of SERP. We would argue that stimulation is likely to be highest in two situations: when people from the same field come together to push the boundaries of understanding in that field, and when people from different fields work together on a problem, allowing for multiple paradigms and models to gener- ate new understandings and ideas. SERP would expand the opportunities for both types of stimulation substantially. The research agendas of the networks would keep research teams in close contact and collaboration in order to advance the knowI- edge base. And SERP would create a venue for a type of inter- disciplinary work on education issues that has been relatively rare because education departments in universities have gener- ally not been successful at attracting scholars from other disci- plines. But perhaps the most powerful lever SERP would wield to increase the intellectual options for researchers would come from the opportunities that SERP is specifically designed to create and nurture: allowing for disciplinary researchers to work closely with research-oriented practitioners. Negotiating the re- lationships with expert practitioners that lead to real collabora- tive research is not easy, and yet anyone who has benefited from such a relationship recognizes that it is highly productive and enormously stimulating. Researchers who value making a civic contribution through their work may be thwarted by the logistical difficulty of doing so. The lack of an infrastructure that links research and practice makes it difficult for education researchers to contribute to the education system. Education Week recently published a story about a Harvard education researcher, Richard Murnane, who decided to spend a sabbatical year offering his help to the local school district. While Murnane's effort eventually resulted in an ongoing connection with the schools through which he is help- ing train practitioners to effectively use data about their stu- dents, getting there was not easy. Neither Murnane nor the schools knew at the outset what, if anything, he could do to be helpful. His determination kept him in a situation that was at first quite awkward. By facilitating links between researchers and schools, SERP would allow for those interested in making a civic contribution to do so without having to leap such a high barrier to search for a place in which to be useful individually. 82 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
Attracting researchers to problems of education like all other research problems will be greatly facilitated if there are financial rewards for doing so. There are clear precedents in military, space, and medical research. The relatively meager funding of education research and the instability of the federal commitment of funds for education research over the years have not created a strong pull on intellectual resources to the problems of education. If SERP operates with a relatively stable, sizable budget as described in Chapter 3, there will be a finan- cial incentive for researchers to turn their attention to the prob- lems of educational practice that could become quite powerful as the commitments of states grow. ATTRACTING PRACTITIONERS TO SERP . What would motivate teachers to participate in collabora- tive research? We hypothesize rather different incentives in the case of teachers. Because the rewards of the profession are quite different, what is likely to be required in order to compete for a teacher's time commitment will be different as well. While the financial rewards of teaching are not as great as those in re- search, what teaching does provide is an opportunity to engage with students in a meaningful, sometimes life-shaping, way. It gives one an opportunity to open new worlds, and new oppor- tunities, to students. SERP, we believe, will be attractive to teachers if the SERP work allows them to do their jobs more successfully and to influence the lives of their students more positively. SERP will provide an opportunity for teachers in the sense that the knowledge from research will be brought to bear in context and on the problems of practice. We envision SERP as providing professional development opportunities for teachers and summer stipends to attend SERP seminars and workshops. Teaching can also be attractive for the job stability it pro- vides and for the predictability of the schedule. But precisely those characteristics can pose a challenge to attracting teachers to SERP. The rigidity of a teacher's schedule does not typically allow for other commitments during a school year, nor for job security or seniority in the case of an extended absence. For SERP to attract teachers, institutional arrangements will need to be made with schools to provide for either a teacher's extended leave without penalty or a reduced workload to allow for par- ticipation in a research project during a school year. The nego- S E R P N E T W O R K S 83

OCR for page 79
tiation of these arrangements through the office of research services should create opportunities for teachers that are now lacking. As with researchers, teachers will vary tremendously in their drives for intellectual stimulation and to make a contribu- tion to their profession writ large. Teachers who seek venues for these purposes currently have few options. Teachers and ad- ministrators lead activit~v-driven lives; they are rewarded for working long hours and being endlessly "available." They are not rewarded for reflection on their practice. Moreover, much of their work is done in isolation from others in the profession. As several of the cases in Chapter 2 suggest, practitioners who want to empirically test teaching approaches or interventions are often left to find their own path to the world of research through unmarked territory. Certainly not every teacher is in- terested in working with research teams. But for those who are, SERP will provide a venue that does not now exist. Finally, much of the SERP work would be designed to learn from teaching practice, so the knowledge that teachers bring will be actively employed and valued. For teachers who do look to research, it can be quite frustrating that what is available often does not address the complexity of the problems they face. The opportunity to influence the research agenda is likely to be an important draw for this group. ATTRACTING FIELD SITES Why would schools, school districts, or states agree to serve as field sites for SERP research? The primary answer, in our view, is that education leaders at all three levels have a desire- whether motivated by civic contribution, reputation, career op- portunity, or accountability measures to do a better job of educating students. If SERP is successful in its early years at linking research and practice to improve student learning out- comes, we think that attracting new field sites will be facilitated by the expectation of continued success. It may be more of a challenge, however, in the start-up period before proof of con- cept has been demonstrated. For this reason, early efforts to establish field sites should target schools and districts that have demonstrated interests in engaging in research and development efforts. School districts in Boston, Chicago, Pittsburgh, and Providence, for example, 84 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
currently work with researchers. While drawing on exciting opportunities is not likely to produce a representative sample of schools or students, it should provide the foundation for suc- cess early on that will create the possibility of a representative sample of field sites further down the road. The benefits of the SERP research will be available to all schools in all states that join the SERP compact. It therefore ~ . stands to reason that the costs of the field site research should be borne primarily by the organization rather than the participat- ing schools. To encourage field site participation and an effi- cient distribution of costs, we propose that SERP funds be used to buy out teacher time and cover other new costs imposed by the research. When expenditures substitute for those made rou- tinely by a participating site (e.g., an investment in data collec- tion), costs should be shared so that the school is neither taxed nor subsidized by involvement with SERP. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WHAT WOULD THEY DO? If SERP is successful at attracting researchers, practitioners, and field sites, what will they do when they join a research and development network? The answer to that question will be determined by those who are given leadership responsibility in the new organization. Below we provide our vision of what that leadership might do. The committee is proposing the inauguration of three net- works during the start-up years, although others may eventu- ally be added. We recommend that a learning and instruction network be initiated at the very start because the relative matu- rity of research in the cognitive and developmental sciences holds promise for real improvement in how teachers are pre- pared and what they do in the classroom. A second network on schools as organizations would, we think, be a critical compan- ion to the learning and instruction network because the organi- zational environment of the school creates the conditions and incentives required to support and sustain instructional change. We anticipate that as work progresses during the start-up years, each network would develop five to eight related strands of work that speak to its hub question, employing the full panoply of research methods (experimental design, longitudinal studies, S E R P N E T W O R K S 85

OCR for page 79
observational techniques, formative and summative evaluations, etc.) in an iterative process that produces ever more refined knowledge. We propose a third network to house research and develop- ment on issues that profoundly influence both instruction and organization management: education policy. This network would embrace research on issues like accountability testing, class/school size, education finance, school choice, desegrega- tion, and other policy issues that create the incentives and envi- ronment to which teachers and schools respond. How quickly a third network is put in place will depend in large part on the funds available to launch the SERP enterprise. AN ILLUSTRATIVE AGENDA FOR A SERP NETWORK ON LEARNING AND I NSTRUCTION To make the case for the value of a SERP research and development program more concretely, an expert pane! of prac- titioners and researchers was convened by the National Re- search Council to complement the work of our Committee. Its task was to design an illustrative agenda for one network that would simulate, in a sense, the role of the agenda-setting advi- sory board and the leadership of a learning and instruction network. We selected this network from among those proposed because the National Academies have in recent years produced important syntheses of the research literature on human learn- ing (National Research Council, 1999) and on assessment of learning (National Research Council, 2001), as well as disci- pline-specific syntheses in reading and in mathematics (Na- tional Research Council, 199S, 2001~. These and other explora- tions of the knowledge base on learning and instruction (National Institute on Child Health and Human Development, 2000; RAND, 2002a, 2002b) provided a rich foundation on which our effort could build. This focus is not intended to suggest preeminence of the learning and instruction network. In our judgment, the organi- zational and implementation issues associated with educational change are also absolutely central to SERP's mission. Efforts to improve learning and instruction, the motivation and engage- 86 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
ment of students and teachers, and the quality of curricula, assessments, and instructional materials cannot succeed unless they are attentive to the variety of organizational and institu- tional contexts in which instruction occurs. Indeed, in our view, it is in combination with the work of other networks that the research and development on learning and instruction will have maximum likelihood of influencing practice. GENERATING A CONCEPTUAL FRAMEWORK Because the intended focus of the SERP program of research and development is educational practice, the conceptual frame- work proposed by the learning and instruction panel begins not with the questions posed by any research discipline, but with the questions that define teaching practice. The point of present- ing this conceptual framework here, before we get to the specif- ics of the agenda proposed for the area of early mathematics, is to demonstrate how broadly one needs to be able to think about educational research and improvement how many different perspectives and sources of knowledge are needed to think productively about even a single, specific, focused research prob- lem. SERP's competitive advantage is precisely that it can bring together the array of researchers and practitioners who can keep these many different perspectives in play simultaneously. We can view classroom instruction as organized around a set of core questions that apply no matter what the subject: What do we want students to know or be able to do? What are the typical understandings and precon- ceptions students hold on this topic at the outset? What is the expected progression of understanding and skill mastery, and what are the predictable points of difficulty or hurdles that must be overcome? What instructional interventions can move students along a path from their initial understandings and skills to the desired outcome (curricula, instructional activi- ties, etch? What general and discipline-specific norms and practices best comprise and support student learning? And finally, how can the individual student's progress be monitored and the student be engaged in S E R P N E T W O R K S 87

OCR for page 79
the instructional activities that she or he needs to take the next step toward increased understanding and skill? Whether explicitly or implicitly, by design or by default, teachers answer the above questions in the course of their teach- ing practice. In doing so, teachers (like professionals in other fields) draw on standards of practice, professional preparation, background knowledge, tradition, and personal inclinations and intuitions. A program of research and development can im- prove the answers to those questions by providing a solid knowI- edge base to support both teaching practice and professional preparation, as well as by expanding the instructional and as- sessment tools available to teachers. The questions that define educational practice can be in- formed by several very different fields of research and knowI- edge: 88 What students should know or be able to do in an area is informed (but not fully determined) by disciplinary ex- pertise. It requires an understanding of the core con- cepts around which the disciplinary knowledge is orga- nized, characteristic methods of reasoning and problem solving, and language and patterns of discourse. What to teach becomes not only a matter of the information and skills considered important but also of helping the student to build the conceptual framework that trans- forms or helps to organize information into understand- ngs. Knowledge of common student conceptions of a topic and the expected progression of student thinking requires careful research on the typical trajectory of understand- ing. In part this research attempts to identify the nature and limits of children's changing cognitive abilities with age and instruction. And in part it attempts to uncover common understandings that can either support learn- ing (the ability to halve or double relatively easily in mathematics) or undermine it (the belief that heat and temperature are the same thing). Research findings dem- onstrate the remarkable resilience of students' everyday understandings even after instruction to the contrary (and often to the great surprise of teachers). This high- STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
lights the importance of a carefully designed research program to inform and support practice. Research of this sort is often done by cognitive scientists and educa- tion researchers, although the knowledge may emerge from the experience of expert teachers and the observa- tion of exemplary practice. Educative experiences intended to move students along a learning path constitute the core of what we consider to be "instruction." These experiences are ulti- mately created by the teacher, but teachers usually draw on materials generated by curriculum developers or (less often) researchers. Instructional programs involve as- sumptions about the contributors to skill development, knowledge acquisition, and conceptual change that should themselves be a research agenda, and the effec- tiveness of the instructional approach is a matter for empirical testing. General and discipline-specific norms and practices that support student learning. Learning takes place in ciass- rooms that are themselves communities. Every commu- nit~v is distinguished by norms for work and interac- tions, ranging from when and how people collaborate to how they speak with one another. Some of those norms are general rooted in the understanding of schools in a democratic society; others are specific what it means to do mathematics differs from what it means to do literary analysis or chemistry or history. How individual norms contribute to or undermine student learning, and how this differs by community, are empirical questions that draw on sociological and psychological understandings, as well as on a rigorous evaluation of classroom prac- tices. Assessing the current level of an individual student's understanding is itself an interdisciplinary undertaking because it requires an understanding of both what con- stitutes learning and how to measure it. To be useful in the learning process, the assessment must be tied to instructional responses. Assessment, then, requires that the knowledge relevant to each of the above questions be incorporated into the design and testing of specific instruments. S E R P N E T W O R K S 89

OCR for page 79
companion assessment tool (the Number Knowledge Test) to help the teacher monitor and guide instruction. If results in controlled trials (see Chapter 1) could be attained in schools across the country that serve disadvantaged populations, this would represent a major success with respect to narrowing the achievement gap a long-standing national goal that has proven difficult to realize. With respect to the second concern, research done in the 1990s investigated the effects on student achievement of in- struction that builds on informal understandings and empha- sizes mathematical concepts and reasoning. Cobb et al.'s Prob- lem Centered Mathematics project (Wood and Sellers, 1997) and Cognitively Guided Instruction (CGI) in problem solving and conceptual understanding (Carpenter et al., 1996) both reported positive effects. With support from the National Science Foun- dation (NSF), several full-scare elementary mathematics cur- ricula with embedded assessments have been developed, di- rected at supporting deeper conceptual understanding of mathematics concepts and building on children's informal knowledge of mathematics to provide a more flexible founda- tion for supporting problem solving. Three curricula developed separately take somewhat different approaches to achieving those goals: the Everyday Mathematics curriculum, the Investi- gations in Number, Data and Space curriculum, and the Math Trailblazers curriculum (Education Development Center, Inc., 2001~. While theories of learning help to identify problems these curricula have been designed to address, the curricula them- selves involve theories of instruction that must be tested. Do their efforts to provide more contextual learning opportunities that link students' informal thinking to mathematical problem solving produce students with stronger mathematics skills over- all, or are there trade-offs among the component skills? Do they perform as well for students who excel in mathematics as for students who struggle? All three curricula show positive gains in student achieve- ment in implementation studies in which the developers collect data on program effects. While such findings are encouraging, they must be viewed with a critical eye, both because those providing the assessment have a vested interest in the outcome and because the methodology employed does not allow for 96 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
direct attribution of the results to the program. Third-party evaluations using comparison groups have been done, but none of these has involved random assignment (the condition that maximizes confidence in attributing results to the intervention). Nor do these studies measure either fidelity of implementation of the reform curriculum for the experimental group or the specific program features of the alternative used with the con- tro! group (see, for example, Fuson et al., 2000~. From the perspective of practice, these are important omis- sions. To make informed curriculum decisions, teachers and school administrators need to know what type of implementa- tion of a specific curriculum produces what results, compared with what alternatives. Ideally, the reform curricula would be compared with traditional curricula that are highly rated or widely used (or both) in order to advance the knowledge base for practice. Yet to provide the information that is most useful to practice is a major undertaking. These questions are answer- able, but research carefully designed to provide the answers will take a substantial, Tong-term investment. A s s e s s m e n ~ The curricula described above have embedded assessments that allow teachers to track student learning. As previously mentioned, a key feature of the Number Worlds curriculum is the Number Knowledge Test that allows teachers to closely link instructional activities for children to the assessment results. How well other curricula link assessment and instruction is an issue worthy of investigation. A separate issue is the assessment over time of the five strands that constitute mathematical proficiency. The last de- cade has seen the emergence of a spate of new tests and mea- sures. No consensus has emerged, however, on critical mea- surest While there are some standard and widely used assessment tools to appraise young children's emergent reading and language skills and competence, no such tools are used on any comparable basis in primary mathematics. This type of assessment will be required to evaluate the effectiveness of a particular curriculum and to make compari- sons across curricula. For the most part, we lack sophisticated methods for tracking student learning over time or for examin- S E R P N E T W O R K S 97

OCR for page 79
ing the contribution of any particular instructional interven- tions, whether large or small, on students' learning. A research project that focused on mathematics teaching and learning might begin by developing such tools. Teacher Knowlecige Little is known about what it might take for teachers to use particular instructional approaches effectively, a necessary ele- ment of taking any particular approach to scale. The challenges can be substantial. The curricula mentioned above introduce major changes in approach to teaching mathematics, and effec- tive implementation will require that teachers change their view of mathematics teaching and learning dramatically. In Every- day Mathematics, for example, teachers are expected to intro- duce topics that will be revisited later in the curriculum. Com- plete mastery is not expected with the first introduction. This has created some confusion for teachers, who are often unclear about when mastery is sufficient to move on to the next topic (Fuson et al., 2000~. All of the curricula encourage building on students' own strategies for problem solving and supporting engagement through dialogue about the benefits of alternative strategies. The change required on the part of the teacher to relinquish control of the answer in favor of a dialogue among students, where it has been studied, has proven difficult to master (Palincsar et al., 1989~. The risks of change must be considered as well: if a teacher does turn control of the discus- sion over to students but is not prepared to guide that discus- sion productively, precious little learning may go on. Critical to the effectiveness of efforts to implement such curricula on a large scale, then, is that there be adequate teacher preparation and ongoing support for an entirely different approach to teach- ing. This is clearly an important area for further study. One clue regarding teacher knowledge requirements can be found in research pursued for the most part separately from the work on student learning and the design of curriculum ap- proaches, tools, and materials discussed above. Investigations of teachers' knowledge reveal that although teachers can, for the most part, "do" the mathematics themselves, they often are unable to explain why procedures work, distinguish different interpretations of particular operations, or use a model to closely map the meaning of a concept or a procedure. For example, 98 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
teachers may be able to use concrete materials to verify that the answer to the subtraction problem in Box 4.1 is 37 and not 43. They can operate in the world of base ten blocks to solve 51 - 14 but may not be able to use base ten blocks to demonstrate the meaning of each step of the conventional (or other) algorithm. Similarly, teachers may be able to compute using familiar standard algorithms but not be able to recognize, interpret, or evaluate the mathematical quality of an alternative algorithm. They may not be able to ascertain whether a nonconventional method generalizes or to compare the relative merits and disad- vantages of different algorithms (for example, their transpar- ency, efficiency, compactness, or the extent to which they are either error-prone or likely to avert a calculation error). Over and over, evidence reveals that knowing mathematics for one- self (i.e., to function as a mathematically competent adult) is insufficient knowledge for teaching the subject. In the domain of early number, studies suggest that most teachers' own knowI- edge is solid, but that their understanding of conceptual foun- dations is uneven. Following this work, some materials for use in teachers' professional development have been developed.2 Modules and other curriculum materials contain focused work aimed at help- ing teachers learn the sort of mathematical knowledge of whole numbers and operations that is needed for teaching. As with the curricula developed for students' learning discussed above, de- velopers of teacher learning materials provide some evidence of teachers' learning of mathematics for teaching, but they have studied less the role of this learning in the teacher's subsequent instructional practice and effectiveness. Still less is known about what teacher developers them- selves need to know to support teachers' learning and how their professional learning might be supported. The demand for skilled leaders who can teach teachers is growing, but the field, though highly remunerative, is unregulated and all too often typified by inadequately trained instructors and badly designed ~ Base ten blocks are a common material used to model place value con- cepts and operations that rely centrally on place value. The materials consist of a unit cube, a ten-stick built of 10 cubes, a flat square built of 100 cubes or 10 ten-sticks, and a block composed of 1,000 cubes, or 10 flats, or 100 ten-sticks. 2 See, for example, work by Schifter and her colleagues at Education Devel- opment Center, Developing Mathematics Instruction (Schifter et al., 1999~. S E R P N E T W O R K S 99

OCR for page 79
delivery methods. Scaling up materials that can support teach- ers' learning of mathematics for teaching will require worrying about the knowledge requirements of those who will guide and support the teachers. The SERP Agenda Given the current state of practice and knowledge about learning and teaching of early number, then, what might a SERP program of research and development seek to do? How might it build on what currently exists and begin to extend and fill gaps in what is known and done, with the ultimate goal of more reliably and productively building evidence-based instruc- tional practice? In other words, how could work be planned and carried out that would extend what is known and take that to scale in U.S. schools? The proposed agenda is comprised of three major initia- tives. The first focuses on developing assessments to measure student knowledge, a second evaluates promising curricula and the effects of their particular design features on student out- comes, and a third focuses on the teacher knowledge require- ments to comfortably and effectively use curricula that are built on research-based findings regarding student learning. Initiative I: Developing Early Mathematics Assessments. Qual- ity assessments depend on three things: (1) clarity about the competencies that the assessment should measure; (2) tasks and observations that effectively capture those competencies; and (3) appropriate qualitative and quantitative techniques to give interpretive power to the test results. Clarity about the compe- tencies to be measured requires a theoretical understanding (that is empirically supported) of mathematics learning. Unlike many other areas of the curriculum, early mathematics has the theoretical and conceptual models, as well as supporting em- pirical data, on which to build quality assessments. Substantial work has already been done to specify critical concepts and skills within this domain, providing assessment developers with resources on which to draw in drafting the elements of a mea- surement strategy. Even with a strong foundation on which to build in early mathematics, much work remains in designing and testing as- sessment items to ensure that inferences can be accurately drawn too STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
about student knowledge and competencies. And this work must be carefully crafted for the specific purpose and use of the assessment, for example, formative assessment for use in the classroom to assist learning; summative assessment for use at the classroom, school, or district level to determine student attainment levels; or assessment for purposes of program evalu- ation. Formative assessments are essentially diagnostic. They can, for example, provide feedback to the teacher on a student's mastery of a particular skill or concept or on whether individual students need more time and practice before moving on to new material. Summative assessments are also used in the class- room, but they come at the end of a unit. They give a teacher feedback on how well the students have mastered and brought together the set of concepts and skills taught in the unit. These may be helpful to the teacher in redesigning instruction for the next year, providing valuable data on students' strengths and weaknesses that can inform instruction at the next level. School- or district-level assessments have more general policy purposes, most commonly to determine attainment levels for groups of students in order to evaluate the effectiveness of an instruc- tional program; to monitor attainment by racial, ethnic, or dis- ability category; and in some cases to hold schools accountable for the performance of their students. Currently the different types of assessment are loosely con- nected at best. Tensions are introduced when strong instruc- tional programs and accountability assessments are at odds. Better aligning assessments and tying all assessments firmly to the theoretical and empirical knowledge base are widely regarded as critical to improving learning outcomes. The con- struction of such a system represents a major research, develop- ment, and implementation agenda that would require the kind of stability, longevity, and support that SERP intends as its hallmark. The above work should be pursued as a collaborative effort involving teachers, content area specialists, cognitive scientists, and psychometricians. The effort could use as a departure point well-established standards in mathematics (e.g., National Coun- ci! of Teachers of Mathematics), standards-based curricular re- sources, and rigorous research on content learning to identify and define what students should know in early mathematics, S E R P N E T W O R K S ]0]

OCR for page 79
how they might be expected to show what they know, and how to appropriately interpret student performance. In the case of formative assessment, this extends to an understanding of the implications of what the evidence suggests for subsequent in- struction. In the case of summative assessment, this means un- derstanding the implications of student performance for mas- tery of core concepts and principles and the growth of this mastery over time. While there are several possible approaches to developing such a system of student assessments in early mathematics, one obvious place to begin is with a review of the assessment mate- rials in existing widely used and exemplary curricular pro- grams for formative and summative assessments, commercial testing programs, and state and national tests for policy making and accountability. These can be reviewed in light of cognitive theories of mathematical understanding, including empirical data regarding the validity of specific assessments. Research needs to focus on evidence of the effectiveness of specific assess- ments for capturing the range of student knowledge and profi- ciency for particular mathematical constructs and operations. A related line of inquiry would focus on issues of assessment scoring and reliability, particularly ease of scoring, consistency of scoring within and across individuals, and consistency of interpretation of the results relative to the underlying cognitive constructs. The development of assessments in early mathematics should be closely tied to complementary initiatives in the areas of teacher knowledge and curriculum effectiveness. Thus a strand of research focused on implementation issues should address the set of questions critical to successful use of quality assessments: What teacher knowledge is necessary to support effective use of assessments in their instructional prac- tice? These include teacher understanding of the assess- meets and their purpose, as well as practical consider- ations of the time to administer, score, and interpret results. What forms of technology support are needed to assist teachers in the administration, scoring, and inter- pretation of a range of standards-based and theory-based assessments? 102 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
How and to what extent does the process of imple- menting curriculum-based and standards-based assess- ments lead to changes in teachers' instructional prac- tices, and how do these changes affect student learning outcomes? This investigation should focus both on changes in the near term and the stability of changes in the Tong term. High-quality evidence that permits practitioners, research- ers, and policy makers to ask and answer comparative ques- tions will be critical to making the SERP research and develop- ment usable in practice. Initiative II: Teacher Knowledge. To take advantage of existing investments in research and development in elementary math- ematics will require further work regarding teacher learning and knowledge requirements and the supports that allow teach- ers to use these curricula comfortably and effectively. This re- search should begin with a clear articulation of the principles and assumptions about student learning that the curriculum incorporates, comparing these to carefully solicited understand- ings of teachers. Learning experiences should be designed to address the points of divergence and tested for their power to change teacher conceptions. Further research should test the effectiveness of different components of professional development on both teacher learn- ing and the learning of their students. The relative benefits of teacher guides, videotaped cases, and opportunities to pose questions and receive support should be tested, as well as the timing effect (before instruction begins, during instruction, etc.) for different teacher learning opportunities. Initiative III: Curriculum Evaluation. The identification (and further development) of a set of approaches to the teaching of number and operations that vary on distinct and theoretically important dimensions would permit careful comparisons of how particular instructional regimes impact students' learning. Programs and approaches already developed, such as Number Worlds, Cognitively Guided Instruction, the three NSF- supported curricula mentioned above, and well-regarded and widely used traditional curricula would form the initial core, S E R P N E T W O R K S '03

OCR for page 79
but analysis would permit such a core set to be complemented with other theoretically and practically important alternatives. Many of the evaluations of the curricula set out to answer the question, "Does the curriculum improve student achieve- ment?" While this is an important question and of particular interest to those who market a curriculum the questions of importance for Tong-term improvements in practice are why, for whom, and compared with what? Number Worlds shows very promising results for disadvantaged children; Everyday Mathematics does as well. How, and for whom, do those out- comes differ? Are there trade-offs in the competencies children gain from each? Does the context in which they work best differ? Each of the three NSF elementary mathematics curricula takes a somewhat different approach to instruction. How are those differences reflected in outcomes for students? Does one better address the needs of Tow- or high-achieving students? What are their respective organizational and implementation requirements? Are there lessons in the outcomes that could be used to improve any of the curricula or to combine features not now found in a single curriculum? An analysis of existing candidate materials could illuminate important differences, and strategic selections could be made. The implementation, adaptation, and use of these different ap- proaches could be followed over time, attending to instructional practice, students' opportunities to learn, and implementation issues. In addition, based on what is known about teachers' knowledge of whole number and operations for teaching, as well as about teacher learning, systematic variations could be designed to support the implementation of these different in- structional approaches. For example, in one set of schools, a teacher specialist model might be deployed, and, in others, teachers might engage in a closely focused study of practice (instruction, student learning, mathematical tasks), co-planning and analyzing lessons across the year. In still others, teachers might be provided with both incentives to spend time planning and adequate teacher guides. The work could be conducted in carefully controlled, Tongi- tudinal studies carried out in SERP field sites. A SERP organiza- tion like that described in Chapter 3 would be well positioned to carry out such work. Because it would have mutually beneficial relationships established with a number of field sites and data collection efforts in those sites already under way, taking on a ]04 STRATEGIC EDUCATION RESEARCH PARTNERSHIP

OCR for page 79
controlled experimental study of alternative curricula would be a far less daunting task than it would be for researchers working independently. Moreover, the concern for undertaking research that is maximally useful to educational practice and the ability to design and conduct or oversee the conduct of that re- search will be combined in a single organization. This is a situa- tion that does not now exist. The research initiatives described above provide a glimpse through a single window of a large-scale SERP research and development enterprise. The companion report provides a more extensive agenda, but even that is limited to the learning and instruction network. Perhaps the greatest benefit of the pro- posed SERP organization is that programs of research on schools as organizations and on education policy will be developed alongside that of learning and instruction. Yet, even within the confines of the early mathematics agenda considered here, dis- tinguishing features of SERP are apparent. This includes an effort to define a program of research that focuses on the prob- lems of practice, strategically building on strengths in the exist- ing knowledge base and shoring up its weakness. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ WOULD SERP CHANGE PRACTICE? Even the highest quality SERP research and development will make a difference only if it is used in practice. What incen- tives will teachers, schools, districts, and states have to make use of the SERP work? AS we mentioned in the opening chapter, the current cli- mate in which schools are being held accountable for student performance creates some motivation to search for the means to improvement. While this may lubricate the wheels of change, large-scare accountability will not be a sufficient motivator. It is, by its nature, low-level accountability it generally attends to gross measures of skill performance. The SERP program, in contrast, is targeted at improving learning for understanding. This more difficult change in instruction will require more pow- erful motivators. These, in the committee's view, are (a) solid evidence that change will bring clear benefits in student perfor- mance and (b) support for implementing the change in real classroom environments. S E R P N E T W O R K S '05

OCR for page 79
Regarding the first, we have proposed an investment at the outset in measures of program outcome for all of the SERP work. As the case of the Cognitive Tutor Algebra I in Oklahoma suggests (see Chapter 2), the resistance to making a change often lies in the doubt that the new program will in fact be better, as well as the risk that it may be worse. Careful efforts to document student gains will, we think, serve as a significant inducement for teachers and schools to change their practices. In the proposed research and development agenda, the effort to measure impact permeates every strand of research. Further- more, the organizational design assigns to those responsible for quality assurance the task of measuring the impact of SERP research and development. In doing their job, an additional source of pressure will be placed on the research and develop- ment program to define clearly the expected program outcomes, so that impact data can be compiled as programs are imple- mented. These data will help inform school districts and states about the potential improvements associated with change. To- gether we expect these efforts to provide a powerful motivation for change. But motivation is not itself enough, either for the higher- level change sought by SERP or the Tower-level change encour- aged through accountability standards. Motivated teachers still must have the support to change their practice. The idea of attending seriously to what it takes to use research-based prac- tices at the school level runs throughout the envisioned SERP program. Carrying out much of the work in a range of class- room settings will allow the problems of classroom use to them- selves be a subject of study and an issue for development and program design. Indeed, in the Oklahoma case, the combination of evidence of success and support from the school district and the program developer to change curriculum resulted in all teachers preferring the new program, making continued con- trolled experimentation difficult. It is the unique combination of attention to carefully measured outcomes and attention to the requirements for classroom use that leads the committee to be convinced that SERP can make the kind of change possible that has been so difficult to achieve in the past. 106 STRATEGIC EDUCATION RESEARCH PARTNERSHIP