This report summarizes the presentations and discussions at the Workshop on Integrating New Measures of Specific Mental Illness Diagnoses with Functional Impairment into the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Data Collection Programs, which was held in Washington, D.C., in September 2015. The workshop was organized as part of an effort to assist SAMHSA and the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services (HHS) in their responsibilities to expand the collection of behavioral health data in several areas. The workshop was structured to bring together experts in the measurement of specific mental illness diagnoses in adults, related functional impairment, and health survey methods to facilitate discussion of measures and mechanisms most promising for expanding SAMHSA’s data collections in this area.
The overall effort is being overseen by the Standing Committee on Integrating New Behavioral Health Measures into SAMHSA’s Data Collection Programs. 1 In addition to the topics covered by this workshop, SAMHSA and ASPE are interested in expanding data collection on serious emotional disturbance in children, on trauma, and on recovery from
________________
1 For a description of the overall study, see http://sites.nationalacademies.org/DBASSE/CNSTAT/Behavioral_Health_Measures_Committee/index.htm [October 2015].
substance use or mental disorder. Workshops on all four topics are being convened as part of the overall effort.
Neil Russell of SAMHSA described his agency’s goals in exploring how to best measure and expand SAMHSA’s data collection programs to include specific mental illness diagnoses with functional impairment and the inherent challenges in this effort. He first explained that the parameters of the expanded data collection are (1) to produce direct national estimates and state estimates for adult mental disorders with functional impairment, (2) to collect data on a wider variety of disorders than would be needed to estimate the prevalence of “serious mental illness,”2 and (3) to collect these data at a frequency of not less than every 5 years.
Russell next summarized a previous SAMHSA effort to collect data on adult mental health disorders through adding a module to the National Survey of Drug Use and Health (NSDUH): that module was the Mental Health Surveillance Study (MHSS). The purpose of the MHSS was to produce national and state-level estimates of the prevalence of serious mental illness in accordance with SAMHSA’s legislative mandate.
Russell further explained that the MHSS covered noninstitutionalized civilians aged 18 years and older who completed the NSDUH questionnaire in English. There was no Spanish version of the MHSS. NSDUH respondents were sampled and recruited for a one-time follow-up clinical interview that was administered following the NSDUH main study. The MHSS was fielded for 5 years, between 2008 and 2012. At its conclusion, a total of 5,653 respondents had participated, for an overall weighted response rate of 64.6 percent. Data from the MHSS were used to develop a model to estimate serious mental illness and apply it to the full sample of NSDUH participants.
Russell then described the interview process for the MHSS: on average, the clinical telephone interview was 72 minutes. The instrument used was the Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID-I)–Non-patient Edition (SCID-I/NP), which was administered by interviewers who had undergone extensive training. The SCID includes standardized questions that are read verbatim and sequentially, followed
________________
2 On May 20, 1993, SAMHSA’s Center for Mental Health Services (CMHS) published its definition of serious mental illness in the Federal Register (58 FR 29425): Persons aged 18 and over, who currently or at any time during the past year, have had diagnosable mental, behavioral, or emotional disorder of sufficient duration to meet diagnostic criteria specified within DSM-III-R [Diagnostic and Statistical Manual of Mental Disorders, III, Revised] that has resulted in functional impairment, which substantially interferes with or limits one or more major life activities.
by unstructured follow-up questions that the interviewers tailor to each respondent on the basis of clinical judgment and respondent answers. Clinical judgment was used to code each item in the SCID as “1” (absent or false), “2” (subthreshold), “3” (threshold or true), or “?” (inadequate information).
Data were collected on the following disorders: past year mood disorders (including major depressive disorder, bipolar disorder-manic episode, dysthymic disorder); past year anxiety disorders (including specific phobia, social phobia, generalized anxiety disorder, panic disorder with and without agoraphobia, agoraphobia without history of panic disorder, obsessive compulsive disorder, posttraumatic stress disorder); past year substance use disorders (including alcohol abuse or dependence, and drug abuse or dependence); past year eating disorders (including anorexia nervosa, bulimia nervosa); past year adjustment disorders; past year impulse control disorders (including intermittent explosive disorder); and past year psychotic symptoms (including delusions or hallucinations).
Russell pointed out that several mental disorders were excluded because some types of disorders are not amenable to the structure of the MHSS, including bipolar II disorder, personality disorders, other disorders typically identified in childhood, schizophrenia, and other psychotic disorders. However, a screener for two psychotic symptoms was included in the assessment. Developmental disorders were also excluded because they are excluded from the definition of serious mental illness.
The MHSS also included the Global Assessment of Functioning (GAF), a global measure of functional impairment. With scores ranging from 1 to 100, the GAF scale is a measure of global functional impairment rather than functional impairment specific to an individual mental disorder.3
Russell described challenges SAMHSA faces in considering how to collect data on a wider range of disorders while factoring in impairments related to those disorders. The first challenge is measuring disorder-specific functional impairment when there are multiple disorders and medical conditions. It is not clear whether respondents can accurately attribute functional impairment to a specific mental disorder in the presence of two or more mental disorders (including substance use disorders) and medical conditions, such as stroke or heart disease.
The second challenge is identifying measures of functional impairment. SAMHSA used a global measure of impairment in the MHSS,
________________
3 For a report that provides a global overview of the data, see Center for Behavioral Health Statistics and Quality. (2014). 2012 National Survey on Drug Use and Health: Methodological Resource Book (Section 16a, 2012 Mental Health Surveillance Study: Design and Estimation Report). Rockville, MD: Substance Abuse and Mental Health Services Administration.
but the current question is whether data on impairment for a particular disorder can be measured and collected. The GAF was omitted from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), because it lacked conceptual clarity and had questionable psychometric properties in routine practice. DSM-5 advises clinicians to use the World Health Organization Disability Assessment Scale (WHODAS, 2.0), either the 12-item or 36-item version; SAMHSA has been using a truncated, 8-item version in the NSDUH since 2008.
Third, Russell underscored that assistance is needed in identifying instruments to measure more DSM-5 disorders than are covered in the MHSS; considering the issues and instruments related to measuring functional impairment; and determining the most suitable approach for collecting the data. He listed SAMSHA’s main options for data collection, noting that some of these options could require guidance on implementing model-based estimation procedures:
- Using the NSDUH, which would require a major redesign effort to incorporate the collection of data on specific mental health diagnoses with functional impairment.
- Reinstating the MHSS to accommodate more disorders and functional impairment, perhaps as was previously done by gathering data from small subsamples over time.
- Developing a new data collection program.
- Using existing data sources if they are representative at the national and state levels and the questions used have good psychometric properties.
D.E.B. Potter of the Office of the Assistant Secretary of Planning and Evaluation of HHS, cosponsor of the study, extended her appreciation for the expertise of the standing committee, workshop steering committee, and workshop presenters. She emphasized that the workshop would be informing not only SAMHSA, but also other surveys that HHS administers. She encouraged discusion that could inform multiple surveys; multiple purposes, including epidemiological and policy; and short-term as well as longer-term solutions.
The specific statement of task for the workshop (shown in Box 1-1) was developed on the basis of the charge for the overall project, which was to expand data collections on several behavioral health topics. The main goals of the workshop were to discuss options for collecting data and producing estimates on specific mental illness diagnoses with func-
BOX 1-1
Statement of Task
A steering committee will organize a public workshop that will feature invited presentations and discussions on options for expanding SAMHSA’s behavioral health data collections to include measures of specific mental illness diagnoses with functional impairment. The discussion will explore new measures and efficient mechanisms for collecting the data. Possibilities include adding new measures to existing surveys, initiating new data collections, or implementing model-based estimation procedures that take advantage of existing data sources, in the event that primary data collection methods are cost prohibitive or not necessary. Survey and questionnaire design tradeoffs, as well as the potential impact of any changes to existing surveys, will also be discussed. An individually authored summary of the presentations and discussions at the workshop will be prepared by a designated rapporteur in accordance with institutional guidelines.
tional impairment, including available measures and associated possible data collection mechanisms.
This summary describes the workshop presentations and the discussions that followed each topic: see the workshop agenda in Appendix A. Biographical sketches of the presenters and of the steering committee members are in Appendix B.
Chapter 2 covers two topics that were used to set the stage for later presentations: the historical context of collecting data on mental illness diagnoses and functional impairment in the United States, which has been driven by evolving federal definitions of serious mental illness over the last 60 years, and a new study that is presently being planned by the National Institute of Mental Health (NIMH) to collect prevalence data on a range of mental illness diagnoses.
Chapter 3 describes studies that have been conducted to estimate specific mental illness diagnoses with functional impairment, and instruments that are available for this purpose.
Chapter 4 looks at existing data and data collection methods for measuring disorders, severity, and impairment, including the approaches used by the Global Burden of Disease study and the National Health Interview Survey, as well as the strengths and weaknesses of the administrative and other data sources available for potentially estimating the prevalence of mental disorders. Chapter 5 discusses innovative approaches to measure-
ment: computerized adaptive testing and the Patient Reported Outcome Measurement System.
Chapter 6 covers the final workshop discussions, summarizing the major themes and implications for SAMHSA’s planning efforts.
This report has been prepared by the workshop rapporteurs as a factual summary of what occurred at the workshop. The steering committee’s role was limited to planning and convening the workshop. The views contained in the report are those of individual workshop participants and do not necessarily represent the views of all workshop participants, the steering committee, or the National Academies of Sciences, Engineering, and Medicine.
