BOARD BULLETIN

A Valedictory

Reflections on 60 Years in Educational Testing

Lee J. Cronbach

Board on Testing and Assessment

Commission on Behavioral and Social Sciences and Education

National Research Council

With special appendix:

Sampling and Statistical Procedures Used in the California Learning Assessment System

Lee J. Cronbach,

Norman M. Bradburn, and

Daniel G. Horvitz

National Academy Press
Washington, D.C.
1995



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
A Valedictory: Reflections on 60 Years in Educational Testing BOARD BULLETIN A Valedictory Reflections on 60 Years in Educational Testing Lee J. Cronbach Board on Testing and Assessment Commission on Behavioral and Social Sciences and Education National Research Council With special appendix: Sampling and Statistical Procedures Used in the California Learning Assessment System Lee J. Cronbach, Norman M. Bradburn, and Daniel G. Horvitz National Academy Press Washington, D.C. 1995

OCR for page R1
A Valedictory: Reflections on 60 Years in Educational Testing NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council. The work of the Board on Testing and Assessment is supported by the U.S. Department of Defense, Education, and Labor, through a grant administered by the Employment and Training Administration of the U.S. Department of Labor. The appendix reprinted in this report is not the work of the board. Additional copies of this report are available from: Board on Testing and Assessment National Research Council 2101 Constitution Avenue N.W. Washington, D.C. 20418 Printed in the United States of America Copyright 1995 by the National Academy of Sciences. All rights reserved.

OCR for page R1
A Valedictory: Reflections on 60 Years in Educational Testing BOARD ON TESTING AND ASSESSMENT Richard C. Atkinson (Chair), University of California, San Diego Constance B. Newman (Vice Chair), Smithsonian Institution, Washington, D.C. Richard J. Shavelson (Vice Chair), School of Education, Stanford University Laurie J. Bassi, Graduate Public Policy Program, Georgetown University David C. Berliner, College of Education, Arizona State University, Tempe Richard F. Elmore, Graduate School of Education, Harvard University Patricia M. Flynn, Graduate School of Business, Bentley College Edmund W. Gordon, Department of Psychology, City University of New York Sylvia T. Johnson, School of Education, Howard University Brigitte Jordan, Xerox Palo Alto Research Center and Institute for Research on Learning, Palo Alto, Calif. Carl F. Kaestle, Department of Education, University of Chicago Michael W. Kirst, School of Education, Stanford University Luis M. Laosa, Educational Testing Service, Princeton, N.J. Renee S. Lerche, Ford Motor Company, Dearborn, Mich. Alan M. Lesgold, Learning Research and Development Center, University of Pittsburgh Robert L. Linn, School of Education, University of Colorado, Boulder Miles A. Myers, National Council of Teachers of English, Urbana, Ill. James L. Outtz, Outtz and Associates, Washington, D.C. Neal W. Schmitt, Department of Psychology, Michigan State University, East Lansing Alan H. Schoenfeld, School of Education, University of California, Berkeley William L. Taylor, Attorney At Law, Washington, D.C. Ewart A.C. Thomas, Department of Psychology, Stanford University Michael J. Feuer, Director Holly Wells, Administrative Assistant Adrienne Crawford, Administrative Assistant

OCR for page R1
A Valedictory: Reflections on 60 Years in Educational Testing FOREWORD In the spring of 1994, when the Board on Testing and Assessment was planning a special program to mark its first anniversary, it sought the wisdom and counsel of a master. The choice was simple. Lee J. Cronbach, whose extraordinary career in psychological measurement has yielded some of the most important and original contributions to education, statistics, and the assessment of human performance, could provide a unique perspective on the pitfalls of testing. Lee graciously accepted an invitation to attend the board's first annual retreat, held in Monterey, California, and to offer reflections from his “60 years in the testing wars.” Part reminiscence, part cautionary tale, his account ranges over many of the problems associated with the use of educational tests in this country during the past three decades. Of particular interest to those concerned with the vision and ideals of school reform embodied in Goals 2000 and other federal legislation, Lee recounts some of his experiences in the past year with California's effort to institute standardized assessments in its elementary and junior high schools. In 1993, the California Learning Assessment System—known as CLAS—administered an examination of language arts and mathematics to nearly all California students in Grades 4, 8, and 10, using performance measures of achievement rather than relying solely on more traditional multiple-choice questions. Reports on the performance of schools and school districts raised public alarm over the condition of education in the state; the reporting of results for schools, sometimes based on small samples of pupils, caused concern over the management of the assessment system. The state superintendent of public instruction needed the best possible scientific advice, and so asked Lee to chair a small committee (with Norman Bradburn and Daniel Horvitz) to review the CLAS testing system and to comment on the methodology slated for implementation in 1994. The committee's report to the state of California, which applauds the objectives and accomplishments of CLAS while also identifying certain fundamental technical problems, was released shortly after Lee's visit with the board. The CLAS report is reproduced here as an appendix to Lee' s paper (with the kind permission of Dale Carlson, the director of CLAS), because the board felt that the pioneering experiences of California offer compelling lessons for the design and use of testing as a tool of educational assessment and school reform. The importance of these lessons is

OCR for page R1
A Valedictory: Reflections on 60 Years in Educational Testing implicit in the committee's encouraging conclusion that “all the shortcomings of CLAS-1993 can be remedied ....” In preparing his paper and the CLAS report for this publication, Lee has added some additional ideas he developed after the committee 's report was submitted, which appear following the body of the report. Indeed, his final words suggest that he has many more words ahead —Lee may again find it hard to slip into retirement completely—happy news to students and practitioners of education policy and to all who have come to rely on Lee's impeccable scholarship and good sense. Richard C. Atkinson, Chair Board on Testing and Assessment

OCR for page R1
A Valedictory: Reflections on 60 Years in Educational Testing BOARD ON TESTING AND ASSESSMENT PUBLICATIONS Anticipating Goals 2000: Standards, Assessment, and Public Policy Report of a Workshop Evaluation of the U.S. Employment Service Workplan for the GATB Improvement Project