Measuring Specific Mental Illness Diagnoses with Functional Impairment: Workshop Summary (2016)

3

Instruments Available for Measuring
Specific Mental Illness Diagnoses
with Functional Impairment

OVERVIEW OF EPIDEMIOLOGICAL STUDIES

Darrel Regier (Uniformed Services University) began his presentation by saying that he is pleased to see the very impressive efforts of the SAMHSA team to research the history of surveys and measures and reevaluate how to move forward with their data collections. This work builds on a rich tradition of updating epidemiological studies, which started with the advent of the DSM-III. Regier also commented that it was good to see that the survey program, which started with NIMH’s Epidemiological Catchment Area (ECA) study and continued through the National Comorbidity Study (NCS) and the National Comorbidity Study Replication (NCS-R), has continued to develop.

Regier provided an overview of prevalence rates of different disorders across some of the earlier epidemiological studies that were also mentioned by Ron Manderscheid (see Chapter 2). The ECA study found that the diagnostic criteria were not congruent with what would be identified as treatment need and treatment use. As defined by the DSM-III at that time, about 28 percent of people had a disorder, and about 15 percent received some mental or addiction services: approximately 6 percent received specialty mental health services, 5 percent received general medical services, and 4 percent received other services. However, about one-half of those who were receiving services were not identified as having a mental or addictive disorder. Among the 28 percent of the population that had mental or addictive disorders in the ECA, the rate of those

with a mental disorder only was 19 percent; comorbid mental and addictive disorders was 3 percent; and addictive disorder only was 6 percent.

Regier also discussed the Marshfield Primary Care Study, which predates the ECA and used the Research Diagnostic Criteria (RDC) and the Global Assessment Scale (GAS), which later became the Global Assessment of Functioning (GAF). This study found that about 28 percent in the primary care population had mental or addictive disorders, and a little over half of those, 15 percent, had a score of 70 or less on the GAS, which indicates minimal impairment. Excluding people with minimal impairment and those with less than minimal impairment from the analysis reduced the number of those who met criteria for an RDC disorder by half. Lowering the threshold for the GAS score to less than 60 resulted in an estimate of 10 percent, and lowering it to less than 50 resulted in an estimate of about 2 percent.

The results from the ECA were not consistent with data from the NCS, which estimated the annual prevalence rate of any mental or addictive disorder to be around 38 percent. Regier said that the two surveys were reconciled through the addition in the DSM-IV of the clinical significance criteria that are required for any mental disorder.¹ Scoring individual symptom areas in terms of their clinically significant distress involved asking several questions, including: Did it interfere with your life a lot? Did you ever take any medication for it? Did you ever talk with anybody about these symptoms? With these criteria in the DSM-IV, the prevalence rates in the ECA and NCS dropped to 18.5 percent for any mental or substance use disorder and 14.9 percent for any mental disorder.

Regier recounted that Manderscheid’s earlier reference to the definition of severe mental disorders was occasioned by the National Advisory Mental Health Council being asked by Senator Pete Domenici to develop a study of the cost of parity insurance coverage for the severely mentally ill. In response to this, the council looked at the ECA data and specifically at disorders with psychotic symptoms (i.e., schizophrenia, schizoaffective disorder, manic depressive disorder, and autism) and severe forms of other disorders, including major depression, panic disorder, and obsessive compulsive disorder. Personality disorder was not included because it was not part of the Senate definition of severe mental disorders at that time. The council found the prevalence of those disorders to be 2.8 percent. However, those receiving any services in the ECA were only 1.7 percent, and those with a disorder lasting 1 year (the duration criterion that was mentioned by Manderscheid) were only 0.8 percent.

________________

¹ Narrow, W.E., Rae, D.S., Robins, L.N., and Regier, D.A. (2002). Revised prevalence estimates of mental disorders in the United States: Using a clinical significance criterion to reconcile two surveys’ estimates. Archives of General Psychiatry, 59(2), 115-123.

Regier said that they also looked at other sources of information, such as Social Security Administration (SSA) data for the Supplemental Security Income (SSI) and the Social Security Disability Insurance (SSDI) programs. At that time, 0.5 percent of the population was receiving SSI or SSDI for severe mental disorders, based on the SSA definition. Using the Wisconsin and New Hampshire definitions of the severely mentally ill, the data showed 0.4 percent under treatment for intermittent care and 0.1 percent for continuous care. For nursing home long-term hospitalization, data from the Center for Mental Health Services Client/Patient Sample Survey showed 0.05 percent for mental illness. In other words, the different ways of defining severe mental disorders varied and resulted in different prevalence rates.

One of the conclusions drawn from the ECA, the Marshfield Primary Care Study, and earlier research was that the GAF was the best predictor of service use, when compared with any specific diagnosis, even a diagnosis of schizophrenia. Consequently, the GAF was adopted for DSM-III-R, and then it remained as Axis 5 for the DSM-IV. Regier said that at the time it was the best measure of functioning, although it somewhat conflated symptoms, suicide risk, and impairment. The GAF was also widely used by insurance companies as a criterion for hospitalization: a GAF score of less than 60 was needed for admission, and a score of more than 60 for discharge. However, there were reliability challenges with the GAF, and it required a lot of training for administration.

Regier also discussed the developments of the Kessler six-item, eight-item, and ten-item scales (K6, K8, and K10), which were used in the National Comorbidity Study and other studies as a screener for psychopathology. He noted that these measures are more accurately described as distress measures, because the items assess anxiety and depression. Since the DSM-IV introduced the idea of clinically significant distress in its additional criteria, these measures have become the most well-validated distress measures in the field. They were also used by SAMHSA to obtain an estimate of severe mental illness based on various cut points, rather than administering the Composite International Diagnostic Interview to the entire sample, and to determine the prevalence of severe mental illness on the basis of the Senate definition.

Regier then discussed more recent developments associated with the DSM-5 and the need to develop dimensional measures. The two-question Patient Health Questionnaire (PHQ-2) had been recommended by the federal task force on prevention as a screener for depression. Based on the PHQ-9 screener, the PHQ-2 includes items on mood and interest. But the goal was to assess a range of domains besides depression, such as mood, anxiety, sleep disturbance, substance use, and suicide. The researchers

also wanted cross-cutting Level 1 and Level 2 measures and a dimensional severity rating to be freely available for download online.²

In describing the cross-cutting measures, Regier said that they call attention to symptoms that are relevant to most psychiatric disorders, such as mood, anxiety, sleep disturbance, substance abuse, and suicide. These measures are self-administered and include 13 symptom domains for adults and 12 for children. The measures are brief, with one to three questions per symptom domain: they screen for important symptoms but are not specific screens for individual disorders. The Level 2 items are completed when the corresponding Level 1 item is endorsed as mild or greater for most but not all items. The Level 2 measures provide more detailed assessment of symptom domains, and they are largely based on long-standing, well-validated measures including the revision of the Swanson, Nolan, and Pelham [SNAP] Questionnaire (SNAP-4) for inattention; the National Institute on Drug Abuse (NIDA)-modified Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) for substance abuse, and the Patient Reported Outcomes Measurement Information System (PROMIS) forms for anger, sleep disturbance, and emotional distress.

For documenting the severity of a specific disorder, the frequency and intensity of its component symptoms are assessed for individuals with either a diagnosis, those meeting full criteria, or an “other” specified diagnosis, especially a clinically significant syndrome, that does not meet diagnostic threshold. Some of the severity measures are clinician rated and some are patient rated.

THE WORLD HEALTH ORGANIZATION
DISABILITY ASSESSMENT SCALE

Based on the work that has already been completed on the World Health Organization Disability Assessment Schedule (WHODAS), Regier said that this measure became the recommended assessment for disability in the DSM-5. The WHODAS corresponds to the disability domains of the International Classification of Functioning, Disability and Health, is developed for use in all clinical and general population groups, and was tested worldwide and in the DSM-5 field trials. The team that worked on the development of the WHODAS was composed of researchers at different WHODAS centers around the world. Researchers from NIMH, the National Institute on Alcohol Abuse and Alcoholism, and NIDA collaborated closely with researchers at the World Health Organization.

________________

² See http://www.psychiatry.org/psychiatrists/practice/dsm/dsm-5/online-assessment-measures [January 2016].

Regier said that there are compelling arguments for using a measure of this type. Studies have shown that diagnosis alone fails to predict service needs,³ length of hospitalization,⁴ outcome of hospitalization,⁵ receipt of disability benefits or work performance,⁶ and social integration.⁷ In contrast, diagnosis combined with disability can predict health service utilization,⁸ outcome after hospitalization,⁹ and work performance,¹⁰ among other positive outcomes. The WHODAS has the advantage of being an internationally recognized classification of functioning, disability, and health that can be used for physical and mental disorders. It also has cross-cultural comparability, good psychometric properties, ease of use, and availability.

The WHODAS captures functioning in the domains of cognition, mobility, self-care, getting along, life activities, and participation. There are six questions in each domain, which produce a score and a disability profile. The full version is 36 items and provides the most detail, but there is also a 12-item version, which is used for brief assessments. There is also a hybrid version with 12 items to screen for problematic domains of functioning: on the basis of positive responses to those 12 items, respondents may be asked up to 24 additional questions. The WHODAS can be administered by interview or computerized adaptive testing (discussed below).

Regier commented that computerized adaptive testing will eventually enable researchers to develop efficient surveys using these measures. Large pools of data can be used to standardize the approach for different population groups, which is likely to be the direction of research in the future. There is a lot more work to be done, however. For example, the

________________

³ National Advisory Mental Health Council. (1993). Healthcare reform for Americans with severe mental illness: Report of the National Advisory Mental Health Council. American Journal of Psychiatry, 150, 1447-1465.

⁴ McCrone, P., and Phelan, M. (1994). Diagnosis and length of psychiatric inpatient stay. Psychological Medicine, 24, 1025-1030.

⁵ Rabinowitz, J., Modai, I., and Inbar-Saban, N. (1994). Understanding who improves after psychiatric hospitalization. Acta Psychiatrica Scandidiavica, 89, 152-158.

⁶ Massel, H.K., Liberman, R.P., Mintz, J., and Jacobs, H.E. (1990). Evaluating the capacity to work of the mentally ill. Psychiatry: Journal for the Study of Interpersonal Processes, 53, 31-43.

⁷ Ormel, J., Oldehinkel, T., Brilman, E., and van den Brink, W. (1993). Outcome of depression and anxiety care: A three wave 3~HF year study of psychopathology and disability. Archives of General Psychiatry, 50, 759-766.

⁸ Ormel, J., Oldehinkel, T., Brilman, E., and van den Brink, W. (1993). Outcome of depression and anxiety care: A three wave 3~HF year study of psychopathology and disability, Archives of General Psychiatry, 50, 759-766.

⁹ Rabinowitz, J., Modai, I., and Inbar-Saban, N. (1994). Understanding who improves after psychiatric hospitalization. Acta Psychiatrica Scandidiavica, 89, 152-158.

¹⁰ Massel, H.K., Liberman, R.P., Mintz, J., Jacobs, H.E., Rush, T.V., Giannini, C.A., and Zarate, R. (1990). Evaluating the capacity to work of the mentally ill. Psychiatry: Journal for the Study of Interpersonal Processes, 53, 31-43.

hybrid version of the WHODAS is not yet in the DSM-5, and neither are the adaptive testing versions of the measures in PROMIS.

Using the WHODAS, meaningful distinctions have been found among subgroups of people with mental health problems, alcohol problems, drug problems, physical health problems, and the general population. For example, people with mental health problems have greater disabilities on the domain of “understanding and communicating” in comparison with people who have physical health problems. People with mental health problems also show high levels of disabilities in the domains of “getting along with people,” “work,” “household functioning,” and “participation with society.”

Regier commented that it will be valuable to start to disaggregate the WHODAS into the different subscales and start associating these with the specific disorders because of the different profiles for different disorders. Exactly how these are going to inform clinical judgments and how to go forward is something that no one has studied yet. But it is an important developmental area that needs attention.

Several papers have been published that examine the WHODAS as part of the DSM-5 field trials. The January 1, 2013, volume of the American Journal of Psychiatry contains several papers on methodology and design,¹¹ reliability of the findings,¹² and outcomes from dimensional measures.¹³ There have also been results published from the routine clinical practice field trials with participation by more than 600 psychiatrists, psychologists, social workers, counselors, and psychiatric nurses.¹⁴

Findings from the field tests at one of the sites in Houston underscored the problem with relying only on diagnoses. There was a very high proportion of comorbidity in persons with major depressive disorder, posttraumatic stress disorder, alcohol use disorder, and generalized anxiety disorder, which accounted for almost 70 percent of the patient

________________

¹¹ Clarke, D.E., Narrow, W.E., Regier, D.A., Kuramoto, S.J., Kupfer, D.J., Kuhl, E.A., Greiner, L., and Kraemer, H.C. (2013). DSM-5 field trials in the United States and Canada, Part I: Study design, sampling strategy, implementation, and analytic approaches. American Journal of Psychiatry, 170(1), 43-58.

¹² Regier, D.A., Narrow, W.E., Clarke, D.E., Kraemer, H.C., Kuramato, S.J., Kuhl, E.A., and Kupfer, D.J. (2013). DSM-5 field trials in the United States and Canada, Part II: Test-retest reliability of selected categorical diagnoses. American Journal of Psychiatry, 170(1), 59-70.

¹³ Narrow, W.E., Clarke, D.E., Kuramoto, S.J., Kraemer, H.C., Kupfer, D.J., Greiner, L., and Regier, D.A. (2013). DSM-5 field trials in the United States and Canada, Part III: Development and reliability testing of a cross-cutting symptom assessment for DSM-5. American Journal of Psychiatry, 170(1), 71-82.

¹⁴ Moscicki, E.K., Clarke, D.E., Kuramoto, S.J., Kraemer, H.C., Narrow, W.E., Kupfer, D.J., and Regier, D.A. (2013). Testing DSM-5 in routine clinical practice settings: Feasibility and clinical utility. Psychiatric Services, 64(10), 952-960.

population at this site. Diagnoses other than those four accounted for 27 percent of that patient population.

CHALLENGES AND OPTIONS FOR SAMHSA

One question of interest to SAMHSA is whether disability can be measured for specific disorders: Regier said that this cannot be done. If someone has an impairment, especially across several physical and mental disorders, trying to disentangle the attribution—whether using WHODAS or any other measure—is very problematic. Neither an individual nor a clinician would know which diagnosis is causing the problems. After years of testing with the DSM-III, DSM-IV, and more recently with DSM-5, it has been found that there are no firm boundaries between disorders.

Genetic studies have revealed that there are common genetic vulnerabilities across the first four major disorder groups of neurodevelopmental disorders, including autism spectrum and attention deficit hyperactivity disorder, schizophrenia, bipolar disorder, and depression. There are a large number of genes, perhaps 1,000, for a disorder like schizophrenia, but they contribute a very small vulnerability risk for an individual getting schizophrenia. Whatever the combination of the genetic risk is with environmental exposures, the epigenetic influences that turn these genes on and off can produce an almost infinite number of combinations that are not going to break cleanly across the diagnostic boundaries.

The introduction to the DSM-5 manual advises looking at diagnoses primarily as central tendencies. The challenge for population surveys is to figure out how best to use a combination of the dimensional profiles, categorical diagnostic criteria, disability impairment measures, and severity measures in order to get at information that is clinically relevant and that will provide predictability in terms of clinical course, response to treatment, need for disability insurance coverage, or disability payments for the individual.

In his conclusion, Regier noted that WHODAS has the potential to draw attention to disability concepts in clinical settings and to better integrate them into routine practice. He also pointed out that preliminary analysis of mean WHODAS scores indicates some interesting age, diagnosis, and informant effects. He said that the self-report WHODAS appears to be very reliable, but the clinician assessment that uses the six-item WHODAS is not reliable. More items are needed in order to get a reliable estimate of the six domains than just one question for each domain. He also said that there is evidence for the validity of the WHODAS based on disability scores that are higher for patients with two disorders than for patients with one disorder, and based on total disability scores that are higher for specific disorders than for “not otherwise specified” disorders.

Regier emphasized that the field needs more epidemiological studies. He suggested that if NIMH and SAMHSA could meld this into their ongoing research programs, it would be possible to produce population rates for some of the disability measures for the full range of disorders. Future research is also needed to understand what the potential of the WHODAS is to affect clinical care, assist clinical decision making, improve patient care outcomes, and enhance patients’ involvement in their own care.

In relation to SAMHSA’s challenge to link impairment with specific diagnoses, Mark Olfson (Columbia University) asked about whether statistical methods, such as factor analysis or path analysis, could be used in population studies to examine the extent to which the variation in impairment or disability is accounted for uniquely by disorders. Regier replied that there is value in understanding the statistical associations between the disorders and the level of disability. However, analysis is also needed for assessing the severity of those disorders. It would be useful to understand how much of the disability is associated with the severity of the individual disorders and how much of the disability is associated with the comorbidity with other disorders. He said that if all of the variables are available in a dataset, then statistically it would be possible to examine what happens to disability when severity or comorbidity are added to the analysis. Dean Kilpatrick (Medical University of South Carolina) agreed that asking someone to attribute their disability to a specific disorder may not be feasible, and that it is best to measure disability in functional areas. He also agreed that it may be possible to sort some of this out in the analysis stages.