Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6 Concluding Thoughts and Possible Next Steps At the end of the two-day program, a panel of participants was convened to synthesize what they had heard and try to identify some of the key messages. Referring back to the criteria the committee had asked the presenters to consider with regard to their programs, the discussants noted that it was clear that selecting any one of them as most important could not be the key to bridging the kinds of gaps that were discussed. Rather, the criteria emerged as important ways of considering the strengths and limitations of different approaches. Each of the programs presented was tailored to suit a particular set of circum- stances, and to address particular challenges, and some of the differences among them were striking. Some served students in relatively disadvantaged circum- stances; others served greater numbers of advantaged students. The programs ranged in scale, in their methods, and in the goals they were trying to meet. Thus no one starting point would make sense for all of them. The discussants also noted that few of the presenters provided much eve dence of the effectiveness of their programs.] Moreover, with a few exceptions, little effort has yet been made to transfer these programs to other settings with different characteristics. The discussants noted that such follow-up work is badly needed. The programs discussed for the most part struck them as very promising, but many are still in early stages of their development. It will be very important, . 1This may have been partly because the agenda was very full. Such evidence may be available for many of the programs, and interested readers are encouraged to seek it using the contact information in Appendix C. 42
CONCLUDING THOUGHTS AND POSSIBLE NEXT STEPS 43 the discussants agreed, to see which elements of them are of use beyond their own contexts. At the same time, however, several important common threads were apparent in the presentations and in the discussion. These are presented not as recommen- dations to those responsible for assessment programs, but rather as a distillation of the experiences described at the workshop which may be helpful to others: . . . . Responsibility lodged with teachers. In virtually every one of the programs, the responsibilities that devolve to teachers seem to be critical to the success of the enterprise. Many speakers were struck by the extent to which these programs were dependent on teachers who were prepared to change their thinking and their practice. Teachers were asked to master new concepts and techniques for assessment, and also, in many cases, to change other elements of their work as they adapted to the needs of the assessment program. Perhaps most important, teachers' judgments about student performance, how and when to assess, and many other issues are being sought and used in these programs to an extent not often seen. Commitment to professional development. Presenter after presenter spoke about how important an investment in professional development was to their programs. The developers of many of the programs that placed new responsibilities on teachers realized from the start that teachers had not had sufficient training in measurement to succeed with the new requirements without targeted training up front. Ongoing support of many kinds through summer workshops, shared websites, mentoring networks, and the like is another key element in many. Experienced teachers were enlisted in many cases to spread their knowledge to col- leagues, and teachers were offered opportunities to participate in test development and scoring sessions. Several presenters expressed concern that resources to maintain this level of commitment may be at risk but all seemed convinced that it was crucial. Clear descriptions of expectations for students. Many of the programs had in common descriptions of the expectations for students that are unusually concrete and detailed. Using frameworks, matrices, or some other structure, many of the examples that were discussed provide teachers with clear definitions of the stages students are likely to move through as they progress to mastery of chosen academic objectives. Breaking the learning process down in this way seemed to be a particularly useful way of meshing the goals of instructions and accountability. Plentiful feedback to teachers and students. In many of the programs, provision of usable feedback is built into the system, and often careful thought has gone to the form the feedback will take. Reports of assess- ment results often include analysis that breaks down the student work to reveal specific misconceptions and gaps in knowledge. The feedback is,
44 ASSESSMENT IN SUPPORT OF INSTRUCTION AND LEARNING in many cases, designed to be folded back into both the teachers' deci- sions and the students' thinking about where they stand and what they need to do. . . Summative assessments do not stand alone. Though none of the exam- ples discussed was perhaps initiated with the explicit goal of bridging a gap or, certainly, of meeting the criteria the committee has described- they do mostly share the notion that summative assessments ought not to be stand-alone exercises, but elements of an integrated system. While few would likely disagree with that notion, the programs here have taken a variety of specific steps to try to make it a reality. As each of the programs proceeds, evidence of their success may influence other states and districts that are recognizing the consequences of having a system that is not as coherent and integrated as it could be. Adherence to professional standards. In many of the programs dis- cussed, the explicit assistance of measurement experts was sought either to review new assessment plans or to work with and train the teachers and officials who would be developing and carrying out the program. The developers of these programs recognized that they were attempting some- thing ambitious and that taking particular care that the technical innova- tions passed professional muster would be important. At the same time, content specialists were often involved in developing the detailed expec- tations for students discussed above. The programs that were developed by researchers were of course also grounded in high professional stan- dards. Although high professional standards are important to any assessment program, the participation of experts is perhaps especially important where educators and administrators are trying to meet expecta- tions for accountability in new ways. This workshop is just a step toward the National Research Council's goal of fostering the understanding of and commitment to assessment for learning. The information presented here will be used in studies just being initiated by all three of the boards that sponsored the workshop. The Board on Testing and Assess- ment is overseeing a project that will help states design the science assessments that will be required under the No Child Left Behind Act. The Mathematical Sciences Education Board has a study of mathematics assessments underway, and the Committee on Science Education K-12 is conducting a study on science learning. The committee hopes that the examples presented here will stimulate the thinking of each of these committees as they consider the tensions presented by assessment systems with multiple goals.
