Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
ASSESSMENT IN SUPPORT OF INSTRUCTION A ' ~ r r A ~ ' ~ rim ,~ ~ ' rat Bridging the Gap Between Large-Scale and Classroom Assessment Workshop Report Committee on Assessment in Support of Instruction and Learning Board on Testing and Assessment Committee on Science Education K-12 Mathematical Sciences Education Board Center for Education Division of Behavioral and Social Sciences and Education NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Insti- tute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract/Grant No. ESI-0102582 between the National Acad- emy of Sciences and the National Science Foundation. Any opinions, findings, conclu- sions, or recommendations expressed in this publication are those of the authorts) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number 0-309-08978-6 (Book) International Standard Book Number 0-309-52616-7 (PDF) Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334- 3313 (in the Washington metropolitan area); Internet, http://www.nap.edu Copyright 2003 by the National Academy of Sciences. All rights reserved. Printed in the United States of America. Suggested citation: National Research Council (2003~. Assessment in support of instruc- tion and learning: Bridging the gap between large-scale and classroom assessment. Workshop report. Committee on Assessment in Support of Instruction and Learning. Board on Testing and Assessment, Committee on Science Education K-12, Mathematical Sciences Education Board. Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm. A. Wulf is president of the National Academy of . ~ . . . engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. Wm. A. Wulf are chair and vice chair, respectively, of the National Research Council. www. nationa l-academies.org
COMMITTEE ON ASSESSMENT IN SUPPORT OF INSTRUCTION AND LEARNING ,1. MYRON ATKIN (Chair), Center for Educational Research, Stanford University EVA L. BAKER, School of Education, University of California, Los Angeles ,IAN DE LANGE, Freudenthal Institute, Utrecht University, The Netherlands TOM KELLER, Maine Department of Education, Augusta ,IAMES MINSTRELL, Talaria, Inc., Seattle MARGE M. PETIT, National Center for the Improvement of Educational Assessment, Portsmouth, New Hampshire ANTHONY SCOTT, Chicago Public Schools LORRIE A. SHEPARD, School of Education, University of Colorado, Boulder GUADALUPE VALDES, Department of Spanish and Portuguese, Stanford University MERYL W. BERTENTHAL, Study Director ANDREW E. TOMPKINS, Research Assistant MICHAEL DECARMINE, Project Assistant v
BOARD ON TESTING AND ASSESSMENT EVA L. BAKER (Chair), The Center for the Study of Evaluation, University of California, Los Angeles LORRAINE MCDONNELL (Vice Chair), Departments of Political Science and Education, University of California, Santa Barbara LAURESS L. WISE (Vice Chair), Human Resources Research Organization, Alexandria, Virginia CHRISTOPHER F. EDLEY, ,IR., Harvard University Law School EMERSON ,1. ELLIOTT, Independent Consultant, Arlington, Virginia MILTON D. HAKEL, Department of Psychology, Bowling Green State University ROBERT M. HAWSER, Institute for Research on Poverty, Center for Demography, University of Wisconsin, Madison PAUL W. HOLLAND, Educational Testing Service, Princeton, New Jersey DANIEL M. KORETZ, Graduate School of Education, Harvard University EDWARD P. LAZEAR, Graduate School of Business, Stanford University RICHARD ,1. LIGHT, Graduate School of Education and John F. Kennedy School of Government, Harvard University ROBERT ,1. MISLEVY, Department of Measurement and Statistics, University of Maryland ,IAMES W. PELLEGRINO, University of Illinois, Chicago LORRIE A. SHEPARD, School of Education, University of Colorado, Boulder KENNETH I. WOLPIN, Department of Economics, University of Pennsylvania PATRICIA MORISON, Acting Director LISA ALSTON, Administrative Associate vim
COMMITTEE ON SCIENCE EDUCATION K-12 CARY SNEIDER (Chair), Boston Museum of Science CARLO PARRAVANO (Vice Chair), Merck Institute for Science Education, Rahway, New Jersey TANYA ATWATER, Department of Geological Sciences, University of California, S ante B arbara FRANCISCO AYALA, Department of Ecology and Evolutionary Biology, University of California, Irvine CAROL BREWER, Division of Biological Sciences, University of Montana, Missoula JUANITA CLAY-CHAMBERS, Detroit Public Schools KATHLEEN COMFORT, WestEd, San Francisco DAVID CONLEY, Center for Educational Policy Research, University of Oregon, Eugene ALAN FRIEDMAN, New York Hall of Science, Corona JEFFREY FRIEDMAN, Friedman Lab, Rockefeller University, New York BARBARA GONZALEZ, Department of Chemistry and Biochemistry, California State University, Fullerton LINDA GREGG, TERC, Cambridge, Massachusetts PATRICIA HARMON, San Francisco Unified School District, California ,IENIFER HELMS, Educational ConsultantlEvaluator, Denver ANNE JOLLY, The Regional Laboratory at SERVE, Mobile, Alabama JUDITH JONES, East Chapel Hill High School, North Carolina TOM KELLER, Maine Department of Education, Augusta OKHEE LEE, School of Education, University of Miami, Florida ,IAMES MINSTRELL, Talaria, Inc., Seattle MARY JANE SCHOTT, Charles A. Dana Center, Austin, Texas JERRY VALADEZ, Fresno Unified School District, California JEAN MOON, Study Director JULIE SCHUCK, Research Associate LASHAWN SIDBURY, Senior Project Assistant . . via
MATHEMATICAL SCIENCES EDUCATION BOARD JOAN LEITZEL (Chair), President Emerita, University of New Hampshire ,IERE CONFREY (Vice Chair), Department of Curriculum and Instruction, University of Texas, Austin JUDY ACKERMAN, Montgomery College, Rockville, Maryland DEBORAH LOEWENBERG BALL, School of Education, University of Michigan THOMAS BANCHOFF, Department of Mathematics, Brown University ,IAN DE LANGE, Freudenthal Institute, Utrecht University, The Netherlands LOUIS GOMEZ, School of Education and Social Policy, Northwestern University DOUGLAS A. GROUWS, Curriculum and Instruction, University of Iowa ARTHUR GAFFE, Department of Mathematics, Harvard University ERIC JOLLY, Education Development Center, Newton, Massachusetts DANIEL KENNEDY, The Baylor School, Chattanooga, Tennessee JIM LEWIS, Department of Mathematics and Statistics, University of Nebraska, Lincoln KAREN LONGHART, Flathead High School, Kalispell, Montana GEORGE MCSHAN, National School Boards Association, Harlingen, Texas KAREN MICHALOWICZ, The Langley School, McLean, Virginia JUDITH MUMME, WestEd, Camarillo, California CASILDA PARDO, Valle Vista Elementary School, Albuquerque SUE PARSONS, Department of Mathematics, Cerritos College, Norwalk. California MARGE PETIT, The National Center for the Improvement of Educational Assessment, North Fayston, Vermont DONALD SAARI, Distinguished Professor of Economics and Professor of Mathematics, University of California, Irvine RICHARD SCHEAFFER, Professor Emeritus, University of Florida WILLIAM STEENKEN, Hamilton, Ohio FRANCIS SULLIVAN, Center for Computing Sciences, Bowie, Maryland HUNG HIS WU, Department of Mathematics, University of California, Berkeley CAROLE LACAMPAGNE, Study Director VICKI STOHL, Research Associate DIONNA WILLIAMS, Senior Project Assistant . . . vail
Preface The National Research Council (NRC) Workshop on Bridging the Gap Between Large-Scale and Classroom Assessment was convened during a period of rising attention in education policy circles to matters of testing and assessment. At this juncture in American education history, the emphasis is increasingly on large-scale examinations developed outside the classroom to gauge what students know. Their aim is primarily to strengthen public accountability. This kind of assessment, which now is projected at orders of magnitude much greater than anything yet seen in this country, is already having profound effects. There are serious consequences financial and otherwise for students, parents, teachers, schools, and districts associated with the test results. Tests have also been shown to have powerful influences on curriculum and teaching methods. One problem with relying exclusively on tests designed to examine millions of students is that they do not easily conform to curricula devised to match state and national standards for mathematics or science. Nor do they do much to promote the kind of student learning that is reflected in those standards. Addi- tionally, these external assessments may have little relation to what students are learning and teachers are teaching in their classrooms. Most important, at present the system does not usually incorporate forms of assessment that have been shown, when done well, to have a direct and positive influence on how much students learn: specifically, the assessments that are part of a teacher's everyday classroom practice and that are integrated into instruction. To quote from a recent publication, Knowing What Students Know (NRC, 2001 c) from the Board on Testing and Assessment, one of three NRC standing boards and committees that joined to organize the present workshop, "The cur- ~x
x PREFACE rent imbalance of mandates and resources should be redressed by shifting from an emphasis on external forms of assessment to an increased emphasis on class- room formative assessment to assist learning" (p. 310~. The NRC workshop reported here addressed that gap between external and classroom assessment. During the workshop we heard about issues associated with designing an assessment system that meets the demands of public account- ability and, at the same time, improves the quality of the education that students receive day by day. The workshop focused on assessment that addresses both accountability and learning. What guidelines or criteria might be developed to take advantage of the strengths and potential inherent in large-scale examinations, on the one hand, and everyday assessment in the classroom on the other? How might steps be taken to minimize the sometimes counterproductive nature of some assessment practices- indeed, to maximize the potential of each practice? What are the challenges? What is gained and what is lost as the states and the nation try to create a coherent and integrated assessment system? These are some of the many questions raised. The heart of the workshop was an opportunity to learn about approximately a dozen programs in which attempts are being made to bridge the gap. It should be recognized that the workshop was exploratory. It was not a showcase. None of the programs that were described and discussed is perfect. Few are exemplary, except in the goals they are trying to accomplish. Most face serious challenges. The members of the committee that planned the workshop are deeply indebted to those who agreed to talk about the current state of their work in a setting that encouraged probing questions. All the participants recognized that it would take hard and steady effort to construct a high-quality system.* A further goal of the workshop was to establish clearer directions for specific NRC initiatives in the months and years ahead to inform the larger education community about issues associated with assessment, learning, and accountability. Therefore the genesis of the workshop is relevant. Three of the constituent bodies of the NRC's Center for Education joined to plan the two-day meeting. They will be involved in whatever initiatives grow out of the workshop delibera- tions. For ten years, the Board on Testing and Assessment has been producing insightful publications on improving large-scale examinations. It has helped the education community and the public to recognize the strengths and limitations of such examinations. It has led the way in synthesizing research on the topic, making recommendations, and pointing out areas that need additional serious study. The Mathematical Sciences Education Board and the Committee on Science Education K-12, while not inattentive to assessment issues, have focused primarily on matters of curriculum and teacher education. Bringing these three *See Appendix C for sources of further information about the programs discussed.
PREFACE Xt groups together to lead this workshop, and inform the NRC's future work in this arena, helps to ensure the kind of scope and comprehensiveness needed around the topic of assessment for both learning and accountability. In planning this workshop the Committee on Assessment in Support of Instruction and Learning benefited tremendously from the contributions and good- will of many people, and the committee is grateful for their support. First, we wish to acknowledge the National Science Foundation (NSF), which sponsored this workshop through a grant to the Center for Education. We particularly thank Janice Earle, who served as the link between the NSF and the committee. The Board on Testing and Assessment, the Committee on Science Education K-12, and the Mathematical Sciences Education Board the units within the National Research Council that launched this workshop were instrumental in shaping the project and in providing general guidance and support along the way. Within the NRC, a number of individuals supported the project. Michael Feuer, executive director of the Division of Behavioral and Social Sciences and Education; Patricia Morison, associate director of the Center for Education; and Jay Labov, deputy director of the Center for Education, provided support and encouragement along the way. The committee expresses particular gratitude to the members of the NRC project staff for contributing their intellectual and organizational skills throughout the life of the project. Meryl Bertenthal, the project's study director, helped to conceptualize the workshop and provided guid- ance and support to the committee. Judy Koenig was responsible for planning the committee's first meeting and, at the workshop, proved to be a skilled note taker and exacting timekeeper. Andrew Tompkins provided excellent research support and adeptly handled all of the logistics related to the workshop. We were particu- larly impressed by his knowledge and use of technology, which allowed us to feature more than twenty speakers and their slides without a single glitch. Michael DeCarmine ably assisted Andrew in ensuring that the committee's work pro- ceeded smoothly. The committee is extremely grateful to Alix Beatty for her skillful writing of this workshop summary. This report has been reviewed in draft form by individuals chosen for their Averse perspectives and technical expertise, in accordance with procedures approved by the NRC' s Report Review Committee. The purpose of this indepen- dent review is to provide candid and critical comments that will assist the institu- tion in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confiden- tial to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Paul J. Black, Department of Education and Professional Studies, King's College, London; Peggy Carlisle, Teacher, Pecan Park Elementary School, Jackson, Mississippi; Sharon Sikora, Center for Learning and Teaching of the West, Colorado State University; and
xt! PREFACE Gary Sykes, Education Administration and Teacher Education, Michigan State University Although the reviewers listed above have provided many constructive com- ments and suggestions, they were not asked to endorse the final draft of the report before its release. The review of this report was overseen by Marshall S. Smith, Education Program, The William and Flora Hewlett Foundation, Menlo Park, California. Appointed by the National Research Council, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were care- fully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. Finally, I would like to thank all of the committee members, who generously contributed their time and intellectual efforts to this project. The organization of such a large workshop and the conceptualization of the criteria for selecting programs to feature was an extraordinary challenge that they met extremely well. J. Myron Atkin, Chair
Contents Introduction The Criteria in Context The Ideal, 10 Large-Scale Assessments, 11 Classroom Assessments, 13 The Nature of the Gap Some International Examples Australia, 20 Queensland, 21 Great Britain: Enhanced Formative Assessment, 22 The International Baccalaureate (IB) Diploma Programme, 23 Programme for International Student Assessment (PISA), 23 Assessment to Improve Learning Nebraska: School-Based Teacher-Led Assessment Recording System, 26 Delaware: Comprehensive Science Assessment, 27 Vermont: The Vermont Assessment System and the Partnership for the Assessment of Standards-Based Science, 29 Wyoming: Body of Evidence System, 30 . . . x~ 5 16 19 26
xlv Maine: Comprehensive Assessment System, 32 Washington: Adapting a Traditional Assessment, 34 Berkeley Evaluation and Assessment Research System, 35 Northern California Mathematics Assessment Collaborative, 36 Facet-Based Assessment, 38 Model-Based Assessment, 39 Concluding Thoughts and Possible Next Steps References Appendices A Workshop Agenda B Workshop Participants C Resources for Further Information CONTENTS 42 45 47 51 55