Myths and Tradeoffs

The Role of Tests in Undergraduate Admissions

Steering Committee for the Workshop on Higher Education Admissions

Alexandra Beatty, M.R.C. Greenwood, and Robert L. Linn, editors

Board on Testing and Assessment

Commission on Behavioral and Social Sciences and Education
and
Office of Scientific and Engineering Personnel

National Research Council


NATIONAL ACADEMY PRESS
Washington, D.C.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Myths and Tradeoffs The Role of Tests in Undergraduate Admissions Steering Committee for the Workshop on Higher Education Admissions Alexandra Beatty, M.R.C. Greenwood, and Robert L. Linn, editors Board on Testing and Assessment Commission on Behavioral and Social Sciences and Education and Office of Scientific and Engineering Personnel National Research Council NATIONAL ACADEMY PRESS Washington, D.C.

OCR for page R1
National Academy Press 2101 Constitution Avenue, N.W. Washington, D.C. 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council. International Standard Book Number 0-309-06597-6 Additional copies of this report are available from: National Academy Press 2101 Constitution Avenue N.W. Washington, D.C. 20418 Call 800-624-6242 or 202-334-3313 (in the Washington Metropolitan Area). This report is also available on line at http://www.nap.edu Printed in the United States of America Copyright 1999 by the National Academy of Sciences. All rights reserved.

OCR for page R1
STEERING COMMITTEE FOR THE WORKSHOP ON HIGHER EDUCATION ADMISSIONS CHRISTOPHER F. EDLEY, JR., Harvard Law School, Harvard University M.R.C. GREENWOOD (Co-chair), Chancellor, University of California, Santa Cruz CARLOS G. GUTIERREZ, Department of Chemistry and Biochemistry, California State University, Los Angeles MICHAEL W. KIRST, School of Education, Stanford University ROBERT L. LINN (Co-chair), School of Education, University of Colorado JOHN D. WILEY, Provost and Vice Chancellor for Academic Affairs, University of Wisconsin, Madison ALEXANDRA BEATTY, Study Director DOROTHY MAJEWSKI, Senior Project Assistant

OCR for page R1
BOARD ON TESTING AND ASSESSMENT ROBERT L. LINN (Chair), School of Education, University of Colorado, Boulder CARL F. KAESTLE (Vice Chair), Department of Education, Brown University RICHARD C. ATKINSON, President, University of California PAUL J. BLACK, School of Education, King's College, London, England RICHARD P. DURÁN, Graduate School of Education, University of California, Santa Barbara CHRISTOPHER F. EDLEY, JR., Harvard Law School, Harvard University RONALD FERGUSON, John F. Kennedy School of Public Policy, Harvard University ROBERT M. HAUSER, Institute for Research on Poverty, University of Wisconsin PAUL W. HOLLAND, Graduate School of Education, University of California, Berkeley RICHARD M. JAEGER, Center for Educational Research and Evaluation, University of North Carolina BARBARA M. MEANS, SRI International, Menlo Park, California LORRAINE MCDONNELL, Department of Political Sciences, University of California, Santa Barbara KENNETH PEARLMAN, Lucent Technologies, Inc., Warren, New Jersey ANDREW C. PORTER, Wisconsin Center for Education Research, University of Wisconsin, Madison CATHERINE E. SNOW, Graduate School of Education, Harvard University WILLIAM L. TAYLOR, Attorney at Law, Washington, D.C. WILLIAM T. TRENT, Associate Chancellor, University of Illinois, Champaign VICKI VANDAVEER, The Vandaveer Group, Inc., Houston, Texas LAURESS L. WISE, Human Resources Research Organization, Alexandria, Virginia KENNETH I. WOLPIN, Department of Economics, University of Pennsylvania

OCR for page R1
MICHAEL J. FEUER, Director VIOLA C. HOREK, Administrative Associate LISA ALSTON, Administrative Assistant

OCR for page R1
OFFICE OF SCIENTIFIC AND ENGINEERING PERSONNEL ADVISORY COMMITTEE M.R.C. GREENWOOD (Chair), Chancellor, University of California, Santa Cruz JOHN D. WILEY (Vice Chair), Provost and Vice Chancellor for Academic Affairs, University of Wisconsin, Madison CARLOS GUTIERREZ, Department of Chemistry and Biochemistry, California State University, Los Angeles STEPHEN J. LUKASIK, Independent Consultant, Los Angeles, California TADATAKA YAMADA, Healthcare Services, SmithKline Beecham Corporation, Philadelphia RONALD G. EHRENBERG, Department of Industrial and Labor Relations and Economics, Cornell University CLAUDIA I. MITCHELL-KERNAN, Vice Chancellor of Academic Affairs and Dean, University of California, Los Angeles A. THOMAS YOUNG, Independent Consultant, North Potomac, Maryland WILLIAM H. MILLER, Department of Chemistry, University of California, Berkeley CHARLOTTE KUH, Executive Director MARILYN J. BAKER, Associate Executive Director

OCR for page R1
Preface More than 8 million students enrolled in 4-year, degree-granting postsecondary institutions in the United States in 1996. The multifaceted system through which these students applied to and were selected by the approximately 2,240 institutions in which they enrolled is complex, to say the least; for students, parents, and advisers, it is often stressful and sometimes bewildering. This process raises important questions about the social goals that underlie the sorting of students, and it has been the subject of considerable controversy. The role of standardized tests in this sorting process has been one of the principal flashpoints in discussions of its fairness. Tests have been cited as the chief evidence of unfairness in lawsuits over admissions decisions, criticized as biased against minorities and women, and blamed for the fierce competitiveness of the process. Yet tests have also been praised for their value in providing a common yardstick for comparing students from diverse schools with different grading standards. Two units of the National Research Council (NRC), the Office of Scientific and Engineering Personnel (OSEP) and the Board on Testing and Assessment (BOTA) of the Commission on Behavioral and Social Sciences and Education, have been following this discussion and have been concerned about several aspects of it. We have worried not only about the sense of conflict and crisis that pervades the discussion, but also about the many misconceptions regarding standardized test scores that

OCR for page R1
seem to have fueled some of the conflict. OSEP's mission includes assessment of the human resources needed to support science and engineering in the United States, and the office's interests in recruitment, graduate education, and employment have led it to consider broader questions about higher education. BOTA's mission is to provide scientific expertise regarding issues of testing and assessment in education and the workplace, and the board has had particular interest in the uses of tests as policy tools and the civil rights implications of tests. The somewhat different missions of these two NRC units come together in a joint concern about the role of tests in higher education admissions. BOTA and OSEP agreed to collaborate on an exploratory investigation of the issues involved in tests in the admissions process for higher education. This work builds not only on the interests and concerns of BOTA and OSEP, but also on a history of studies on testing by the National Research Council. Most recently, BOTA's Committee on Appropriate Test Use offered a broad review of the uses—and misuses—of tests in schools. The steering committee's investigation was strongly supported and encouraged by the NRC leadership, in recognition of the unique place of tests in many aspects of students' education and of the intense public debate about testing. To carry out this work, BOTA and OSEP formed a small steering committee, drawn from the membership of the two units and co-chaired by the units' chairs. We organized a workshop to review basic technical information about the two standardized tests that are most widely used in the undergraduate admissions process (the SAT and the ACT) and to explore a variety of perspectives on questions about their use. This workshop, which took place on December 17–18, 1998, brought together researchers who had focused on particular questions about the tests, admissions officers and other administrators from a variety of institutions who discussed pressures on the college admissions system, and thoughtful observers who reflected on key questions from a variety of perspectives. This report is based on the information presented and discussed at the workshop and the steering committee's deliberations. It has three purposes: to identify and correct some persistent myths about standardized admissions tests and highlight some of the specific tradeoffs that decisions about the uses of tests entail; to present the steering committee's conclusions and recommendations about the role of tests in college admissions; and to lay out several issues about which information would clearly help decision makers, but about which the existing data are either

OCR for page R1
insufficient or need synthesis and interpretation. We believe it will benefit a broad audience of college and university officials, state and other officials and lawmakers, and others who are wrestling with decisions about admissions policies, definitions of merit, legal actions, and other issues. The workshop and deliberations summarized in this report were held during a time of particularly rapid change in the landscape of collegiate admissions policy, public opinion about fairness and merit in American society, and legal actions regarding racially conscious admissions practices. The steering committee recognized, therefore, that a necessarily brief investigation could not allow us to address all of the issues surrounding the history and current status of test use in higher education. For example, our limited time and resources prevented a detailed and exhaustive examination of test use practices in the vast and complex array of American institutions of higher learning; nor was there time or resources to explore rigorously the strengths and weaknesses of various alternatives to tests, a number of which are at early stages of development and experimentation. Thus, the steering committee has no illusions that a single report such as this one will settle definitively a debate of such intensity and duration. But we do hope—and believe—that a reminder, in lay terms, of the purposes, capacities, and limitations of the tests will help to clarify the terms of a discussion that has frequently been very acrimonious. We also hope to wave a bright yellow flag of caution in front of those who are making weighty decisions on sometimes shaky technical grounds. The steering committee is particularly grateful to the six scholars who wrote papers for the workshop. Hunter Breland, Richard Jaeger, Sylvia Johnson, Samuel Lucas, Linda Wightman, and Warren Willingham worked on a tight deadline to provide targeted examinations of key issues we wanted to explore. Their papers were very valuable during the deliberations that resulted in this report. The success of the workshop also depended on the efforts of a number of other scholars and college and university officials. Their thoughtful presentations reflected a variety of important perspectives. Many admissions officers took the time to assist us during a very busy time of the year, and their insights were invaluable. The steering committee extends its heartfelt thanks to them and to the other discussants, presenters, and panelists, who helped to lay out a wide range of issues and contributed to a lively and substantive discussion. Several members of BOTA who did not serve on the steering committee offered advice and assistance at several points and contributed most help-

OCR for page R1
fully at the workshop; we extend our thanks to Richard Atkinson, William Taylor, William Trent, and Lauress Wise. The steering committee also thanks Michael Feuer and Charlotte Kuh, the directors of BOTA and OSEP, respectively, who provided leadership in the conception and execution of this project, and Alix Beatty, who guided the project throughout and drafted this report. Dorothy Majewski's able administrative assistance with both the workshop and the report are gratefully acknowledged as well. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRCs' Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making the published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their participation in the review of this report: John A. Blackburn, Admissions Office, University of Virginia; Lloyd Bond, Department of Educational Research and Methodology, University of North Carolina, Greensboro; David W. Breneman, School of Education, University of Virginia; Daryl Chubin, Office of Research, Evaluation, and Communication, National Science Foundation; Jonathan R. Cole, Provost and Dean of Faculties, Columbia University; Gene Maeroff, Teachers College, Columbia University; Willie Pearson, Department of Sociology, Wake Forest University; David Pilbeam, Peabody Museum, Harvard University; Henry W. Riecken, Professor of Behavioral Sciences, University of Pennsylvania School of Medicine (emeritus); and Rebecca Zwick, Department of Education, University of California, Santa Barbara. Although the individuals listed above have provided constructive comments and suggestions, it must be emphasized that responsibility for the final content of this report rests entirely with the authoring committee and the institution. M.R.C. GREENWOOD, CHAIR, OFFICE OF SCIENCE AND ENGINEERING PERSONNEL ROBERT L. LINN, CHAIR, BOARD ON TESTING AND ASSESSMENT

OCR for page R1
Contents     Executive Summary   1     Context   3     History,   3     The Tests,   4     Persistent Controversies,   6     What Do Colleges Really Do?   10     What Do Test Scores Really Mean?   17     The SAT,   17     The ACT,   19     Benefits,   20     Limitations,   22     Other Uses of Tests,   26     Tradeoffs   28     Topics for Further Study   32     References   34

OCR for page R1
Appendix A:   Agenda, Workshop on the Role of Tests in Higher Education Admission   37 Appendix B:   Participants, Workshop on the Role of Tests in Higher Education Admission   41

OCR for page R1
This page in the original is blank.

OCR for page R1
This page in the original is blank.