|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
FAIRNESS IN
EMPLOYMENT TESTING
Validity Generalization,
M~rityIssues, and the
General Aptitude
Test Battery
John A. Hartigan and Alexandra K. Wigdor,
Eclitors
Committee on the General Aptitude Test Battery
Commission on Behavioral and Social Sciences and Education
National Research Council
NATIONAL ACADEMY PRESS
Washington, D.C. 1989
OCR for page R2
NATIONAL ACADEMY PRESS 2101 Constitution Avenue, NW
.
- Washington, DC 2W18
NOTICE: The project that is the subject of this report was approved by the Governing Board of the
National Research Council, whose members are drawn from the councils of the National Academy of
Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the
committee responsible for the report were chosen for their special competences and with regard for
appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by
a Report Review Committee consisting of members of the National Academy of Sciences, the National
Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished
scholars engaged in scientific and engineering research, dedicated to the furtherance of science and
technology and to their use for the general welfare. Upon the authority of the charter granted to it by the
Congress in 1863, the Academy has a mandate that requires it to advise the federal government on
scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National
Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its
administration and in the selection of its members, sharing with the National Academy of Sciences the
responsibility for advising the federal government. The National Academy of Engineering also sponsors
engineering programs aimed at meeting national needs, encourages education and research, and
recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National
Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the
services of eminent members of appropriate professions in the examination of policy matters pertaining
to the health of the public. The Institute acts under the responsibility given to the National Academy of
Sciences by its congressional charter to be an adviser to the federal government and, upon its own
initiative, to identify issues of medical care, research, and education. Dr. Samuel O. Thier is president of
the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to
associate the broad community of science and technology with the Academy's purposes of furthering
kn`)wledPe and advising the federal government. Functioning in accordance with general policies
determined by the Academy, the Council has become the principal operating agency of both the National
Academy of Sciences and the National Academy of Engineering in providing services to the government,
the public, and the scientific and engineering communities. The Council is administered jointly by both
Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice
chairman, respectively, of the National Research Council.
rely ~ i-~~ ~ my, EVE..= ~ of
This project was supported by the Employment and Training Administration of the U.S. Department of
Labor.
Library of Congress Cataloging-in-Publication Data
National Research Council (U.S.). Committee on the General Aptitude Test Battery.
Final report: fairness in employment testing: validity generalization, minority issues, and the General
Aptitude Test Battery/John A. Hartigan and Alexandra K. Wigdor, editors; Committee on the General
Aptitude Test Battery, Commission on Behavioral and Social Sciences and Education, National
Research Council.
p. cm.
Bibliography: p.
Includes index.
ISBN 0-309-04030-2 (paper); ISBN 0-309-04033-7 (cloth)
1. General aptitude test battery-Evaluation. 2. Employment tests-United States-Evaluation. 3.
Employment tests-Law and legislation-United States. I. Hartigan, John A., 1927- . II. Wigdor,
Alexandra K. III. Title.
HE 5549.5.E5N38 1989
153.9'4-dc20
89-3284 1
CIP
Copyright C) 1989 by the National Academy of Sciences
No part of this book may be reproduced by any mechanical, photographic, or electronic process, or in the
form of a phonographic recording, nor may it be stored in a retrieval system, transmitted, or otherwise
copied for public or private use without written permission from the publisher, except for the purposes
of official use by the U.S. government.
Printed in the United States of America
First Printing, May, 1989
Second Printing, August 1990
OCR for page R3
Committee on the General Aptitude Test Battery
JOHN A. HARTIGAN (Chair), Department of Statistics, Yale University
LORRIE A. SHEPARD (Vice Chair), School of Education, University of
Colorado, Boulder
MARCUS ALEXIS, Dean, College of Business Administration,
University of Illinois, Chicago
MANFRED EMMRICH, North Carolina State Employment Service,
Raleigh
LARRY V. HEDGES, Department of Education, University of Chicago
IRA J. HIRSH, Department of Psychology, Washington University, and
Central Institute for the Deaf, St. Louis, Mo.
RICHARD M. JAEGER, School of Education, University of North
Carolina at Greensboro
STEPHEN P. KLEIN, The Rand Corporation, Santa Monica, Calif.
ROBERT L. LINN, School of Education, University of Colorado, Boulder
JOHN M. RAUSCHENBERGER, Employee Development Office, Ford
Motor Company, Dearborn, Mich.
MICHAEL ROTHSCHILD, Department of Economics and Dean, Social
Sciences, University of California, San Diego
PAUL R. SACKETT, Industrial Relations Center, University of
Minnesota
O. PETER SHERWOOD, Solicitor General, New York State
HOWARD F. TAYLOR, Department of Sociology, Princeton University
ALEXANDRA K. WIGDOR, Study Director
HILDA WING, Research Associate
DIANE L. GOLDMAN, Administrative Secretary
. . .
OCR for page R4
OCR for page R5
Liaison Group, Members
ROBERT BOLDA, Personnel Research Division, General Motors
Corporation, Detroit, Mich. (Retired)
RALPH G. CANTRELL, Virginia Employment Commission, Richmond
WAYNE F. CASCIO, Graduate School of Business Administration,
University of Colorado, Denver
SUZAN CHASTAIN, Office of Civil Rights, Office of the Solicitor, U.S.
Department of Labor
ROBERT DELAHUNTY, Office of the Assistant Attorney General (C)vi!
Rights Division), U.S. Department of Justice
CONSTANCE L. DUPRE, U.S. Equal Employment Opportunity
Commission (Retired)
PATRICIA J. DYER, IBM Corporate Employment and Placement,
Armonk, N.Y.
KENNETH EDWARDS, Skill Improvement Department, International
Brotherhood of Electrical Workers, Washington, D.C.
MANFRED EMMRICH, North Carolina State Employment Service,
Raleigh
BOB FUNSTON, Oklahoma Employment Security Commission,
Oklahoma City
JOHN E. HUNTER, Department of Psychology, Michigan State
University
HENRY LEVIN, Departments of Education and Economics, Stanford
University
PHILIP B. LYONS, U.S. Equal Employment Opportunity Commission
CHARLES F. NIELSON, Texas Instruments, Inc., Dallas, Tex.
EDWARD E. POTTER, McGuiness & Williams, Washington, D.C.; Equal
Employment Advisory Council
v
OCR for page R6
Vi LIAISON GROUP, MEMBERS
MARJORIE RAGOSTA, Educational Testing Service, Princeton, N.J.;
Committee on Disabilities and Handicaps, American Psychological
Association
NAMBURY S. RAJU, Department of Psychology, Illinois Institute of
Technology, Chicago
DENNIS K. RHOADES, National Economic Commission, American
Legion, Washington, D.C.
WILLIAM L. ROBINSON, Lawyers' Committee for Civil Rights Under
Law, Washington, D.C.
WILLIAM W. RUCH, Psychological Services, Inc., Glendale, Calif.
ROBERT A. SCHAERFL, U.S. Employment Service, U.S. Department of
Labor
FRANK L. SCHMIDT, College of Business Administration, University of
Iowa
NEAL W. SCHMITT, Department of Psychology, Michigan State
University
DONALD J. SCHWARTZ, Office of Research and Analytic Services, U.S.
Equal Employment Opportunity Commission
RICHARD T. SEYMOUR, Lawyers' Committee for Civil Rights Under
Law, Washington, D.C.
JAMES C. SHARE, Career Entry Group, U.S. Office of Personnel
Management
WILLIAM TRACEY, Employment Services, New Jersey Department of
Labor, Trenton (Retired)
DENNIS L. WARMKE, Philip Morris U.S.A., Richmond, Va.
RAUL YZAGUIRRE, National Council of La Raza, Washington, D.C.
OCR for page R7
Preface
The Department of Labor is considering whether to promote the use of
its General Aptitude Test Battery (GATB) throughout the U.S. Employ-
ment Service to screen many of the 19 million people who pass through
the system annually in search of private- and public-sector jobs. This
study was undertaken at the agency's request because system-wide use of
the test battery for referral purposes raises important questions of public
policy. In order to provide employers with the maximum benefits of
testing, the Employment Service would need to refer applicants in order
of test score. Such a policy, however, would severely reduce the
employment opportunities of lower-scoring applicants, particularly of
minority job seekers, who have lower average test scores as a group than
the majority.
What is the appropriate balance between anticipated productivity gains
from better employee selection and the well-being of individual job
seekers? Can equal employment opportunity be said to exist if screening
methods systematically filter out very large proportions of minority
candidates? Such an outcome would leave employers and the Employ-
ment Service vulnerable to the charge of discrimination under Title VII
of the Civil Rights Act of 1964.
In pilot projects of the test-based referral system, conducted since
1980, Department of Labor officials have adopted a score-adjustment
strategy in which each applicant's test score is computed as a percentile
score within his or her own racial or ethnic group (black, Hispanic, and
other). By combining within-group percentile scores and top-down
. .
V11
OCR for page R8
viii PREFACE
selection of the applicants to be referred, the Department of Labor
sought a workable compromise between productivity goals and federal
equal employment opportunity policy. However, the within-group
scoring strategy created its own problem: the Department of Justice
questioned its legality and constitutionality on grounds of reverse discrim-
ination.
As a consequence of this challenge, the Department of Labor sought
guidance from the National Academy of Sciences, which, through the
National Research Council, has convened a committee of experts to
conduct a thoroughgoing evaluation of the plan to use the GATB as the
primary tool for deciding which applicants to refer to employers. The
Department of Labor asked the committee to address a number of
important technical questions to establish the appropriateness of using a
single general aptitude test to predict performance in a large number of
very different jobs. In addition, recognizing that these technical issues are
surrounded by a complex web of governmental policies and legal require-
ments and have serious economic and social implications, the Department
of Labor asked the committee to consider the possible effects of wide-
spread adoption of testing in the Employment Service-effects on em-
ployers, on various categories of job seekers, and on the economic health
of the country.
This study is intended to help policy makers decide whether the GATB
should be given a primary role in the Employment Service referral system
and, in that event, to offer guidance on methods for assembling the pool
of job candidates and for reporting test scores. The policy context is
described in Chapters 1 and 2. Chapter 3 presents an overview of the U.S.
Employment Service and its activities. In Chapter 4 we look at the quality
of the GATB, and in Chapter 5 discuss critical weaknesses in the test and
in the GATB research program that must be overcome if it is to be used
as a primary referral tool.
The principal scientific underpinnings of the Department of Labor's
plan to use the GATB for referrals to all kinds of jobs, contained in the
theory of validity generalization, are examined in Chapters 6 through 9.
Chapter 8 looks at the accumulated body of GATB validity research and
presents the committee's judgments about the degree of predictive
accuracy that can reasonably be assumed for jobs not studied.
Chapter 10 describes the referral system- which we call the VG-GATB
Referral System to signal its dependence on validity generalization as it
has been conceptualized by the research staff of the U.S. Employment
Service. Chapter 11 discusses the potential effects of the system on the
various Employment Service clients, and Chapter 12 analyzes the claims
about economic benefits said to accrue from referring job candidates in
order of test score.
OCR for page R9
PREFACE 1X
Based on the technical findings presented earlier, Chapters 13 and 14
present the committee's recommendations to policy makers. In Chapter
13 the committee presents its conclusions about the fair use of employ-
ment tests and makes recommendations about adjusting the test scores of
minority job seekers. Chapter 14 presents the committee's most impor-
tant recommendations for the use of the GATB and the design of the
VG-GATB Referral System.
In providing independent advice to the government on matters of
science policy, the National Research Council depends on committees of
volunteers chosen for their expertise, together with members of the
permanent staff, to carry out its work. The members and staff of the
Committee on the General Aptitude Test Battery include experts in
statistics and meta-analysis, psychometrics, industrial and personnel
psychology, economics, sociology, policy analysis, law, and the Employ-
ment Service the expertise needed to address the broad range of
technical and policy questions raised in this study. Brief biographies of
committee and staff appear in Appendix C.
During the course of its study, the committee has called on a great
many people, who gave generously of their time, their expertise, and their
insights. Because the policy decisions ultimately made by the Department
of Labor with regard to the GATB will have an impact, perhaps a great
impact, on the interests of a variety of individuals, groups, and institu-
tionsj a carefully selected liaison group was appointed to ensure that the
committee would be conversant with all relevant policy perspectives and
areas of expertise. Our particular thanks go to the 28 members of this
group who met with us on three occasions and provided needed informa-
tion and position papers throughout. Individual members also presented
occasional special briefings on specific issues before the committee.
We received assistance from other quarters as well. In response to
questions from the committee about employers' reactions to the VG-GATB
system and within-group scoring, the Employers' National Job Service
Committee developed and circulated a questionnaire that elicited over 500
responses. Our work also benefited from the cooperation of the National
Rehabilitation Association and the major veterans organizations.
We owe a great deal to John Hawk, personnel research psychologist in
the test research division of the U.S. Employment Service, and his
colleagues in the central and regional offices. In response to our sheer
need to be educated about the Employment Service system and the
GATB, they provided helpful briefing documents. As we came to grips
with the very large undertaking required by our charge, they remained
helpful and forthcoming in the face of frequent requests for data,
documents long buried in the files, information about day-to-day opera-
tions, and myriad other questions that came up. Staff members of the
OCR for page R10
X PREFACE
Northern Test Development Field Center provided the tapes of the GATB
data base and arranged for the collection of a sample of item-Ievel test
data for our item-bias analysis. The Southern Test Development Field
Center supplied useful information about the development of the GATB
and the operations of the Employment Service system.
We would also like to recognize the contributions of several consultants
who helped with our data analysis: Douglas Weeks assisted committee
member Robert Linn in the differential validity and differential prediction
analysis; Laura Burris and Victoria Crawshaw assisted Paul Sackett with
the meta-analysis of the entire GATB data base; Robin Corley assisted
Lorrie Shepard with a study of item bias; Anita Tesh assisted Richard
Jaeger with a synthesis of the literature on GATB properties. Glen
Sueyoshi contributed a background paper on the economic effects of
improved employee selection as the question is addressed in the econom-
ics literature.
Our acknowledgments would not be complete without special thanks to
staff members who worked with the committee: Hilda Wing, who assisted
with the research and writing; Diane Goldman and Carolyn Sax, who
provided administrative support and kept control of the evolving manu-
script; and Christine McShane, whose graceful editing defied a hectic
schedule.
JOHN A. HARTIGAN, Chair
ALEXANDRA K. WIGDOR, Study Director
Committee on the General Aptitude Test Battery
OCR for page R11
Contents
Summary
PART I
BACKGROUND AND CONTEXT
1 The Policy Context ......................
2 Issues in Equity and Law.................
3 The Public Employment Service...........
PART II
. .17
.29
.. . . . .52
ANALYSIS OF THE GENERAL APTITUDE TEST BATTERY
4 The GATB: Its Character and Psychometric Properties . . .73
5 Problematic Features of the GATB: Test Administration,
Speededness, and Coachability 99
PART III
VALIDITY GENERALIZATION AND GATB VALIDITIES
6 The Theory of Validity Generalization .................
7 Validity Generalization Applied to the GATB.
8 GATB Validities.....................................
9 Differential Validity and Differential Prediction.
X1
119
. . .134
149
.172
OCR for page R12
xii CONTENTS
PART IV
ASSESSMENT OF THE VG-GATB PROGRAM
10 The VG-GATB Program: Concept, Promotion, and
Implementation
I! In Whose Interest: Potential Effects of the VG-GATB
Referral System ................
.191
.209
|2 Evaluation of Economic Claims 235
PART V
CONCLUSIONS AND RECOMMENDATIONS
)3 Recommendations for Referral and Score Reporting
14 Central Recommendations .......................
References ....
APPENDIXES
A A Synthesis of Research on Some Psychometric
Properties of the GATB
Richard M. Jaeger, Robert It inn, and Anita S. Tesh
B Tables Summarizing GATE Reliabilities 325
C Biographical Sketches, Committee Members and Staff 329
.251
. .281
. .291
.303
Index.....
.335
OCR for page R13
FAIRNESS IN
EMPLOYMENT TESTING
.~
OCR for page R14
