BEST PRACTICES FOR STATE ASSESSMENT SYSTEMS, PART I

Summary of a Workshop

Alexandra Beatty, Rapporteur

Committee on Best Practices for State Assessment Systems: Improving Assessment While Revisiting Standards

Center for Education

Division of Behavioral and Social Sciences and Education

NATIONAL RESEARCH COUNCIL
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
BEST PRACTICES FOR STATE ASSESSMENT SYSTEMS, PART I Summary of a Workshop Alexandra Beatty, Rapporteur Committee on Best Practices for State Assessment Systems: Improving Assessment While Revisiting Standards Center for Education Division of Behavioral and Social Sciences and Education

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This project was sponsored by the James B. Hunt, Jr. Institute for Educational Leader- ship and Policy, with additional support from the Bill & Melinda Gates Foundation and the Stupski Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number-13: 978-0-309-15381-2 International Standard Book Number-10: 0-309-15381-6 Additional copies of this report are available from National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. Copyright 2010 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Research Council. (2010). Best Practices for State Assess- ment Systems, Part 1: Summary of a Workshop. A. Beatty, Rapporteur. Committee on Best Practices for State Assessment Systems: Improving Assessment While Revisiting Standards. Center for Education, Division of Behavioral and Social Sciences and Educa- tion. Washington, DC: The National Academies Press.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examina - tion of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1

OCR for page R1
COMMITTEE ON BEST PRACTICES FOR STATE ASSESSMENT SySTEMS: IMPROvINg ASSESSMENT WHILE REvISITINg STANDARDS Diana Pullin (Chair), Lynch School of Education, Boston College Joan Herman, National Center for Research on Evaluation, Standards, and Student Testing Scott Marion, Center for Assessment, National Center for the Improvement of Educational Assessment, Dover, New Hampshire Dirk Mattson, Research and Assessment Division, Minnesota Department of Education Rebecca Maynard, Graduate School of Education, University of Pennsylvania Mark Wilson, Graduate School of Education, University of California, Berkeley Judith A. Koenig, Study Director Stuart Elliott, Director, Board on Testing and Assessment Alexandra Beatty, Senior Program Officer Kelly Duncan, Senior Project Assistant Rose Neugroschel, Research Assistant 

OCR for page R1

OCR for page R1
Preface The idea that states might pool their resources and sign on to a common set of education standards has gone from a speculative concept to an emerg - ing reality. Forty-eight states have now signed on to the “Common Core State Standards Initiative.” States are also responding to the opportunity to compete for significant federal education funds through the Race to the Top Assessment Program, which calls on them to make improvements to their standards and assessments. Even before these recent developments, the con - text for decisions about assessment and accountability was shifting as federal and state policy makers began to take stock of the effects of the No Child Left Behind Act and to consider possibilities for changing it. At the same time, researchers—and a few states—have explored approaches to measur- ing student learning that are based on theoretical models distinctly different from those that have traditionally governed most state programs. Thus, states are reviewing their approaches to assessment and the role it can and should play in a standards-based accountability system in a complex environment of practical, political, theoretical, and technical questions. With funding from the James B. Hunt, Jr. Institute for Educational Leader- ship and Policy, as well as additional support from the Bill & Melinda Gates Foundation and the Stupski Foundation, the National Research Council (NRC) planned two workshops designed to explore some of the possibilities for state assessment systems. The workshops were designed to pull together data and perspectives on the challenges in current assessment and account - ability systems and on innovative assessment approaches that offer possibili - ii

OCR for page R1
iii PREFACE ties for reshaping the expectations educators and policy makers have for this educational tool. The Committee on Best Practices for State Assessment Systems was formed to plan the two workshops. The first workshop, held in December 2009, focused on lessons to be drawn from past experiments with innovative assessments, technical challenges, and the opportunities pre - sented by the current common standards movement; this report describes the presentations and discussions. The second one was held in spring 2010 and provided a more detailed look at possibilities for developing coherent assess - ment systems that incorporate innovative approaches. This report describes the presentations and discussions from the first workshop; Part II will be available in early summer 2010. These two workshops were designed to build on two previous ones that examined the possibilities for and questions about common standards for K-12 education. Although a separate committee was responsible for that series (Com- mon Standards for K-12 Education?: Considering the Eidence, Summary of a Workshop Series, National Research Council, 2008), we hope that the body of information produced by all of these workshops will provide useful guidance for policy makers in a very fast-changing educational context. But the broader goal for this two-stage activity is to pull together research and perspectives from diverse sources that can contribute to thoughtful deliberation about longer- term questions and goals. Many people contributed to the success of this first workshop. We would first like to thank the sponsors for their support of this work, particularly Judith Rizzo and Stephanie Dean with the James B. Hunt, Jr. Institute. The commit - tee also recognizes the scholars who wrote papers and made presentations at the workshop, including Steve Ferrara, CTB McGraw-Hill; Margaret Goertz, University of Pennsylvania; Brian Gong, National Center for the Improve - ment of Educational Assessment; Ron Hambleton, University of Massachusetts- Amhert; Laura Hamilton, RAND; Steve Lazer, Educational Testing Service (ETS); Lorraine McDonnell, University of California, Santa Barbara; Lorrie Shepard, University of Colorado; Brian Stecher, RAND; Shawn Stevens, Uni- versity of Michigan; Laurie Wise, HumRRO; and Rebecca Zwick, ETS and the University of California, Santa Barbara. We are also grateful to senior staff members of the NRC’s Division of Behavioral and Social Sciences and Education who helped to move this project forward. Michael Feuer, executive director, and Patricia Morison, associate executive director and acting director of the Center for Education, provided support and guidance at key stages in this project. Eugenia Grohman, associate executive director, and Kirsten Sampson Snyder, senior report review officer, offered their knowledge and experience with NRC procedures to guide the report through the NRC review process. Christine McShane, senior editor, pro- vided expert editing assistance, and Yvonne Wise, production editor, adeptly moved this report through the publication process.

OCR for page R1
ix PREFACE The committee also thanks the NRC staff who worked directly on this proj- ect. We thank Kelly Duncan, senior project assistant, and Rose Neugroschel, research assistant, both with the Board on Testing and Assessment, for their invaluable assistance in handling the logistical arrangements for the workshop. Their deft organizational skills and careful attention to detail helped to ensure the success of the workshop. We are especially grateful to Judy Koenig, staff director for this workshop project, who played an invaluable role in organizing an informative set of papers and presentations. Judy’s wisdom and political skills made certain that the workshop would be a useful contribution to the public discourse on these important topics. We are also grateful to Stuart Elliott, director, and Alix Beatty, senior program officer, of the Board on Testing and Assessment for their contributions in formulating the workshop design and making it a reality. We particularly wish to recognize Alix Beatty for her superb writing skills and ability to translate workshop presentations and discussions into a coherent, readable report. Finally, as chair of the committee, I wish to thank the committee members for their dedication and outstanding contributions to this project. They gave generously of their time in planning the workshop and actively participated in workshop presentations and discussions. Their varied experiences and perspec- tives contributed immeasurably to the success of the project and made them a delightful set of colleagues for this work. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with proce - dures approved by the Report Review Committee of the NRC. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectiv- ity, evidence, and responsiveness to the charge. The review comments and draft manuscript remain confidential to protect the integrity of the process. We thank the following individuals for their review of this report: Suzanne Lane, Research Methodology Program, School of Education, University of Pittsburgh; Pat Roschewski, Statewide Assessment, Nebraska Department of Education, Lincoln; Theresa Siskind, Division of Accountability, State Department of Education, Columbia, SC; Thomas Toch, Executive Director’s Office, Association of Independent Schools of Greater Washington; and Steven L. Wise, Research and Development, Northwest Evaluation Associa - tion, Lake Oswego, OR. Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the content of the report, nor did they see the final draft of the report before its release. The review of this report was overseen by Edward H. Haertel of the School of Education of Stan - ford University. Appointed by the NRC, he was responsible for making certain that an independent examination of this report was carried out in accordance

OCR for page R1
x PREFACE with institutional procedures and that all review comments were carefully con - sidered. Responsibility for the final content of this report rests entirely with the author and the institution. Diana Pullin, Chair Committee on Best Practices for State Assessment Systems: Improving Assessment While Revisiting Standards

OCR for page R1
Contents 1 INTRODUCTION 1 Context, 3 Recent Changes in Tests, 4 Interim Assessments, 6 Multiple Purposes for Assessment, 7 The Current System, 9 Strengths, 9 Weaknesses, 10 Challenges, 11 2 IMPROVING ASSESSMENTS— POSSIBILITIES AND QUESTIONS 13 Developing Standards That Lead to Better Instruction and Learning: An Example, 13 Developing Assessments That Lead to Better Instruction and Learning: An Example, 19 Technical Challenges, 22 Discussion Summary, 28 3 INNOVATIVE ASSESSMENT—LESSONS FROM THE PAST AND PRESENT 31 Looking Back, 32 Vermont, 32 Kentucky, 33 xi

OCR for page R1
xii CONTENTS Maryland, 33 Washington, 34 California, 35 NAEP Higher-Order Thinking Skills Pilot, 35 Lessons from the Past, 36 Current Innovations, 36 Performance Assessment, 36 Portfolios, 38 Technology-Supported Assessment, 38 Looking to the Future, 40 4 POLITICAL ExPERIENCES AND CONSIDERATIONS 41 Maryland, 41 Kentucky, 43 Minnesota, 45 Policy Implications, 46 5 OPPORTUNITIES FOR BETTER ASSESSMENT 49 Improvement Targets, 49 Cost Savings, 50 Improved Cognitive Analyses, 51 Cross-State Comparisons, 52 Perspectives: The Past and the Future, 55 6 RESEARCH NEEDS 59 Theory and Goals, 59 Research Priorities, 61 REFERENCES 65 APPENDIxES A Workshop Agenda 71 B Workshop Participants 77