Psychological Testing
in the Service of
Disability Determination
Committee on Psychological Testing, Including Validity Testing,
for Social Security Administration Disability Determinations
Board on the Health of Select Populations
INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract/Task Order No. SS00-13-60048/0003 between the National Academy of Sciences and the U.S. Social Security Administration. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
International Standard Book Number-13: 978-0-309-37090-5
International Standard Book Number-10: 0-309-37090-6
Additional copies of this report are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.
Copyright 2015 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2015. Psychological testing in the service of disability determination. Washington, DC: The National Academies Press.
“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
—Goethe
INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES
Advising the Nation. Improving Health.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council.
COMMITTEE ON PSYCHOLOGICAL TESTING, INCLUDING VALIDITY TESTING, FOR SOCIAL SECURITY ADMINISTRATION DISABILITY DETERMINATIONS
HERBERT PARDES (Chair), Executive Vice Chairman of the Board, New York-Presbyterian Hospital, New York, New York
ARTHUR J. BARSKY III, Professor of Psychiatry, Harvard Medical School, and Vice Chair for Psychiatric Research, Brigham and Women’s Hospital, Boston, Massachusetts
MARY C. DALY, Senior Vice President and Associate Director of Economic Research, Federal Reserve Bank of San Francisco, California
KURT F. GEISINGER, W. C. Meierhenry Distinguished University Professor of Educational Psychology and Director, Buros Center for Testing, University of Nebraska–Lincoln
NAOMI LYNN GERBER, University Professor, Center for the Study of Chronic Illness and Disability, George Mason University, Fairfax, Virginia
ALAN M. JETTE, Professor of Health Policy & Management, Boston University School of Public Health
JENNIFER I. KOOP, Associate Professor, Department of Neurology, Medical College of Wisconsin, Milwaukee
LISA A. SUZUKI, Associate Professor of Applied Psychology, New York University Steinhardt School of Culture, Education, and Human Development, New York, New York
ELIZABETH W. TWAMLEY, Associate Professor of Psychiatry, University of California, San Diego
PETER A. UBEL, Madge and Dennis T. McLawhorn University Professor of Business, Fuqua School of Business, and Professor of Public Policy, Sanford School of Public Policy, Duke University, Durham, North Carolina
JACQUELINE REMONDET WALL, Professor, School of Psychological Sciences, University of Indianapolis, Indiana, and Director, Office of Program Consultation and Accreditation, American Psychological Association, Washington, DC
Liaison to IOM Standing Committee of Medical Experts to Assist Social Security on Disability Issues
HOWARD H. GOLDMAN, Professor Psychiatry, University of Maryland School of Medicine, Baltimore
IOM Project Staff
CAROL MASON SPICER, Study Director
FRANK R. VALLIERE, Associate Program Officer
ALEJANDRA MARTÍN, Research Associate (since January 2015)
NICOLE GORMLEY, Senior Program Assistant (since December 2014)
JONATHAN PHILLIPS, Senior Program Assistant (April to November 2014)
JON SANDERS, Program Coordinator (through January 2015)
PAMELA RAMEY-McCRAY, Administrative Assistant
FREDERICK ERDTMANN, Director, Board on the Health of Select Populations
Reviewers
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
David Autor, Massachusetts Institute of Technology Economics
Leighton Chan, National Institutes of Health
Allen W. Heinemann, Northwestern University
Anita Hubley, University of British Columbia, Vancouver Campus
Michael Kirkwood, Children’s Hospital Colorado-Aurora
Glenn J. Larrabee, Clinical Neuropsychology
Brian Levitt, Kaplan Psychologists
Patricia Owens, Health and Disability Policy Programs
Stephen M. Raffle, Forensic and Clinical Psychiatry
Jerry Sweet, University of Chicago, Pritzker School of Medicine
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions
or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Nancy Adler, University of California, San Francisco, and Randy Gallistel, Rutgers University. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Preface
The U.S. Social Security Administration (SSA) disability programs provide important, sometimes vital, benefits to millions of adults and children annually in the United States. The programs are an expression of the nation’s principle of caring for individuals who need support from the larger community. Within the confines of SSA policy, the state Disability Determination Services (DDS) agencies, which implement the policy, have the latitude to do so in whatever way they deem fit. It is not surprising that in a country as diverse as the United States we would find geographic variations in the style and methods with which that process is undertaken.
One element of such variation is the use or not of standardized psychological tests during the disability determination process, other than the use of intelligence tests in determinations of intellectual disability in children and adults. SSA asked the Institute of Medicine (IOM) to review selected psychological tests and to provide guidance on the use of psychological testing in SSA disability determinations.
SSA and the DDS agencies have the critical task of determining which applicants qualify for disability benefits, a task complicated by the lack of direct correlation between the presence of an impairment and disability, which SSA defines as the inability to work. DDS examiners undertake the very complex task of reviewing and developing applicants’ files to determine which requests for disability benefits are justified. As described in the report, the committee felt that it was worth considering whether increased systematic use of standardized psychological testing in specific circumstances would strengthen the current process for disability determination.
The committee thanks colleagues, organizations, and agencies that were willing to share their expertise, time, and information during the committee’s information-gathering meetings. The names of the speakers are included in the meeting agendas provided in Appendix A. The committee is grateful to the authors of the two commissioned papers, Erin Bigler, David Freedman, and Jennifer Manly, for the in-depth analyses they provided. The study sponsor, SSA, gladly provided information and data and responded to questions. We also thank Howard Goldman, chair of the IOM Standing Committee of Medical Experts to Assist Social Security on Disability Issues, who served as a consultant to the committee and provided valuable insight. The contributions from all of these sources informed the committee deliberations and enhanced the quality of this report.
I want also to pay tribute to and thank the expert members of our committee. A diversity of views, at times a difference of views, all contributed to generating a consensus about issues important to SSA and to the country. Throughout the project, they put in an enormous amount of time and effort; contributed their experience, knowledge, and perspective; listened to contending arguments; and ultimately generated the recommendations in this report. It is heartening to me and the other committee members to experience the excellence and the commitment of so many good colleagues. I trust this report will be helpful to and well received by SSA.
Finally, the committee thanks the IOM staff members who contributed to the production of this report, including Frederick “Rick” Erdtmann (board director), Carol Mason Spicer (study director), Frank Valliere (associate program officer), Alejandra Martín (research associate), Nicole Gormley (senior program assistant), Jonathan Phillips (senior program assistant), Jon Sanders (program coordinator), Julie Wiltshire (financial associate), and other staff of the Board on the Health of Select Populations and the IOM, who provided support. Research assistance was provided by Daniel Bearss, Rebecca Morgan, and Catherine van der List.
Herbert Pardes, Chair
Committee on Psychological Testing, Including
Validity Testing, for Social Security Administration
Disability Determinations
Contents
Committee’s Approach to Its Charge
2 DISABILITY EVALUATION AND THE USE OF PSYCHOLOGICAL TESTS
Social Security Administration Disability Determination Process
Composition of SSA Beneficiaries
Psychological Testing in SSA Disability Evaluations
Use of Psychological Tests in Non-SSA Disability Evaluations
3 OVERVIEW OF PSYCHOLOGICAL TESTING
Psychometrics: Examining the Properties of Test Scores
Psychological Testing in the Context of Disability Determinations
4 SELF-REPORT MEASURES AND SYMPTOM VALIDITY TESTS
Assessing Self-Report of Symptoms
Psychological Self-Report Measures and Disability Evaluation
Administration and Interpretation of Non-Cognitive Psychological Measures
Assessing the Validity of Non-Cognitive Symptom Report
Use of Non-Cognitive Measures with Specific Populations
5 COGNITIVE TESTS AND PERFORMANCE VALIDITY TESTS
Administration of Cognitive and Neuropsychological Tests to Evaluate Cognitive Impairment
Psychometrics and Testing Norms for Cognitive Tests
Interpretation and Reporting of Test Results
Assessing Validity of Cognitive Test Performance
Applicant Populations for Whom Performance-Based Tests Should Be Considered or Used
Costs of Psychological Testing
Assessing the Benefits of Psychological Testing
Estimates of Cost Savings from Psychological Testing
7 CONCLUSIONS AND RECOMMENDATIONS
Role of Psychological Testing in Social Security Administration Disability Programs
Standardized Non-Cognitive Psychological Measures and Symptom Validity Tests
Standardized Cognitive Tests and Performance Validity Tests
Qualifications for Test Administration and Interpretation
Boxes, Figures, and Tables
BOXES
1-2 Major Concepts in the International Classification of Functioning, Disability and Health
3-1 Descriptions of Tests by Four Areas of Core Mental Residual Functional Capacity
4-1 SSA Definitions of Symptoms, Signs, and Laboratory Findings
4-2 SSA Definitions of Relevant Mental Disorders
4-3 Definitions of Relevant Disorders with Disproportionate Somatic Symptoms
4-4 SSA Proposed Functional Domains
4-5 Embedded/Derived SVTs for Negative Self-Presentation
4-6 Stand-Alone SVTs for Negative Self-Presentation
FIGURES
S-1 Components of psychological assessment
2-1 Overview of the SSA disability process
2-2 Disability determination process for adults by the numbers
2-3 Disability determination process for children by the numbers
2-4 Filing rates by state, fiscal year 2013
2-5 Allowance rates by state, fiscal year 2013
2-6 Composition of new beneficiaries in 2013 for SSDI and SSI adults and children
3-1 Components of psychological assessment
4-1 Psychological versus nonpsychological self-report measures
TABLES
1-1 Characteristics of SSDI and SSI Beneficiaries, 2012
1-2 SSDI and SSI Beneficiaries by Diagnostic Category, 2012
1-3 Definitions of Psychological Terms
2-2 Summary of Reported Base Rates of Malingering
2-3 Psychological Testing in Different Settings
3-1 Listings for Mental Disorders and Types of Psychological Tests
6-1 Costs of Psychological and Neuropsychological Testing Services
6-2 Estimated Costs of Testing
6-3 Calculation of 2011 SSDI Costs for Each Level of Malingering of Mental Disorders
6-4 Calculation of 2011 SSI (Adult) Costs for Each Level of Malingering of Mental Disorders
6-5 Estimated Annual Savings of Testing New Disability Awardees
6-6 Estimated Lifetime Spending on an Individual Disability Awardee, 2 Percent Annual Discounting
AACN | American Academy of Clinical Neuropsychology |
AADEP | American Academy of Disability Evaluating Physicians |
ABCN | American Board of Clinical Neuropsychology |
ABIME | American Boards of Independent Medical Examiners |
ADL | activity of daily living |
AFB | Ability-Focused Neuropsychological Test Battery |
ALJ | administrative law judge |
AMA | American Medical Association |
APA | American Psychological Association |
ASAPIL | Association for Scientific Advancement in Psychological |
Injury and Law | |
BDI | Beck Depression Inventory |
BLS | Bureau of Labor Statistics |
BPRS | Brief Psychiatric Rating Scale |
BSI | Brief Symptom Inventory |
BVMT-R | Brief Visuospatial Memory Test-Revised |
C&P | compensation and pension |
CASL | Comprehensive Assessment of Spoken Language |
CBCL | Child Behavior Checklist |
CDMI | Composite Disability Malingering Index |
CE | consultative examination |
CELF-4 | Clinical Evaluation of Language Fundamentals-4 |
CFS | chronic fatigue syndrome |
CIDI | Composite International Diagnostic Interview |
CPP | Canada Pension Plan |
CRPS | complex regional pain syndrome |
CVLT-II | California Verbal Learning Test—second edition |
DDS | Disability Determination Services |
DIF | differential item functioning |
DOM | Depression Outcomes Module |
DSM | Diagnostic and Statistical Manual of the American |
Psychiatric Association | |
GAF | Global Assessment of Functioning Scale |
GAO | Government Accountability Office |
HVLT-R | Hopkins Verbal Learning Test-Revised |
ICF | International Classification of Functioning, Disability and Health |
ID | intellectual disability |
IDES | Integrated Disability Examination System |
IME | independent medical examination |
IOM | Institute of Medicine |
IQ | intelligence quotient |
IRT | item response theory |
MDI | medically determinable impairment |
MEDCOM | U.S. Army Medical Command |
M-FAST | Miller Forensic Assessment of Symptom Test |
MINI | Mini International Neuropsychiatric Interview |
MMPI | Minnesota Multiphasic Personality Inventory |
MMY | Mental Measurements Yearbook |
MRFC | Mental Residual Functional Capacity |
MSVT | Medical Symptom Validity Test |
NAN | National Academy of Neuropsychology |
NIH | National Institutes of Health |
NIM | Negative Impressionism |
NPP | negative predictive power |
NPRM | Notice of Proposed Rulemaking |
NRC | National Research Council |
OIDAP | Occupational Information Development Advisory Panel |
OIS | Occupational Information System |
OTSG | Office of the Surgeon General |
P-3 | Pain Patient Profile |
PAI | Personality Assessment Inventory |
PCE | psychological consultative examination |
PDRT | Portland Digit Recognition Test |
PHQ | Patient Health Questionnaire |
POMS | Program Operations Manual System |
PPP | positive predictive power |
PTSD | posttraumatic stress disorder |
PVT | performance validity test |
RAVL | Rey Auditory Verbal Learning |
RDS | Reliable Digit Span |
RMFIT | Rey Memory for Fifteen Items Test |
RMT | Recognition Memory Test |
RMTF | Warrington Recognition Memory Test for Faces |
SCAN | Schedule for Clinical Assessment in Neuropsychiatry |
SCL-90-R | Symptom Checklist 90-Revised |
SDM | single-decision-maker |
SGA | substantial gainful activity |
SIMS | Structured Inventory of Malingered Symptomology |
SIRS | Structured Interview of Reported Symptoms |
SSA | U.S. Social Security Administration |
SSDI | Social Security Disability Insurance |
SSI | Supplemental Security Income |
SVT | symptom validity test |
TBI | traumatic brain injury |
TMJ | temporomandibular joint disorder |
TOMM | Test of Memory Malingering |
VA | U.S. Department of Veterans Affairs |
VBA | Veterans Benefits Administration |
VHA | Veterans Health Administration |
WAIS | Wechsler Adult Intelligence Scale |
WHO | World Health Organization |
WISC | Wechsler Intelligence Scale for Children |
WMS | Wechsler Memory Scale |
WMT | Word Memory Test |