To address its charge, the committee developed a framework for the development of standards for psychosocial interventions that can improve the quality and delivery of those interventions. Figure 2-1 depicts this framework. Adapted from Pincus (2010), the committee’s framework identifies the key steps in successfully bringing an evidence-based psychosocial intervention into clinical practice: it highlights the need to support research on the efficacy and effectiveness of interventions, the need to understand the key elements that drive the interventions’ effects (Chapter 3), the need to develop a systematic and uniform method for appraising the evidence for the effectiveness of interventions (Chapter 4), the need to develop methods for measuring the quality and outcomes of interventions (Chapter 5), and the need to establish methods for successfully implementing and sustaining these interventions in regular practice (Chapter 6). Central to the framework is the consumer perspective in informing this process.
The framework cycle begins with strengthening the evidence base in order to identify effective psychosocial interventions and their elements. As described in Chapter 1, many evidence-based psychosocial interventions currently exist. While it was beyond the scope of this study to provide a comprehensive review of these interventions, they include a number of psychotherapies, including (but not limited to) interpersonal psychotherapy, dialectal behavioral therapy, cognitive processing therapy, eye movement desensitization and reprocessing, psychodynamic therapy, behavioral couples therapy, problem solving therapy, cognitive-behavioral therapy, social skills training, family-focused therapy, behavioral activation, relaxation training, parent skills training, and motivational interviewing. Evidence-
based interventions also include behavioral interventions such as contingency management, community reinforcement approach, and exposure and response prevention. The list includes as well ecological interventions such as assertive community treatment, peer-operated support services, peer recovery support services, wellness planning, supported employment, and housing first (IOM, 2010; WHO, 2010).
The data on these interventions are compelling. A number of meta-analyses have established the effects of psychosocial interventions on mental health and substance abuse problems. Psychotherapies in particular have been subject to numerous meta-analyses. In a recent meta-analysis of psychotherapy, the mean effect size across a broad array of mental disorders in 852 trials (137,000 participants) was slightly higher than the corresponding effect size for pharmacotherapies (mean effect size = 0.58 [95 percent confidence interval (CI) = 0.4-0.76] versus 0.40 [95 percent CI =
0.28-0.52]) (Huhn et al., 2014).1 The effect sizes for psychotherapies varied across mental disorders. The largest effect sizes were for bulimia nervosa (SMD2 = 1.61, CI3 = 0.96-2.29), obsessive compulsive disorder (SMD = 1.37, CI = 0.64-2.24), trichotillomania (SMD = 1.14, CI = 0.38-1.89), anorexia nervosa (SMD = 0.99, CI = 0.38-1.6), and binge eating disorder (SMD = 0.86, CI = 0.42-1.3). The effect sizes were still moderate or greater (SMD >0.5) for major depressive disorder, generalized anxiety disorder, social anxiety disorder, posttraumatic stress disorder, and insomnia. The lowest effect sizes were for schizophrenia with psychodynamic therapy (SMD = −0.25, CI = −0.59-0.11) and alcohol use disorders (SMD = 0.17, CI = 0.08-0.26) (Huhn et al., 2014). These effect sizes are based on a variety of different psychotherapies from different theoretical orientations. Several other meta-analyses have been conducted for specific psychotherapies (e.g., cognitive-behavioral therapy, interpersonal psychotherapy, and problem solving therapy), indicating that some therapies are specifically indicated for particular disorders, while others appear to be effective for many different disorders.
Few meta-analyses exist for other types of psychosocial interventions, such as suicide prevention programs, vocational rehabilitation, and clinical case management. However, these interventions have been subjected to randomized controlled trials (RCTs) and have been shown to have positive effects on the intended intervention target.
Although meta-analyses support the use of psychosocial interventions in the treatment of mental health and substance use problems, other studies are needed to further determine the utility of these interventions in different populations and settings. An argument can be made for emphasizing new study designs that yield immediately actionable results relevant to a variety of stakeholders. Tunis and colleagues (2003) describe the need for “practical clinical trials” that address issues of effectiveness—whether interventions work under real-world conditions—as a second step following efficacy studies under the ideal circumstances of an RCT. Pragmatic or practical trials focus on engaging stakeholders in all study phases to address questions related to intervention effectiveness, implementation strategies, and the degree to which an intervention can be conducted to fidelity in a variety of service settings. These studies also address the resources required
1 The effect size is the difference between treatment and control groups and is expressed in standard deviation units. An effect size of 1 indicates that the average treated patient is 1 standard deviation healthier than the average untreated patient. An effect size of 0.8 is considered a large effect, an effect size of 0.5 is considered a moderate effect, and an effect size of 0.2 is considered a small effect.
2 Huhn and colleagues (2014) measured standardized between-group mean differences (SMDs).
3 Reported data include CIs.
to implement an intervention. At times, pragmatic trials take advantage of data from electronic health records (EHRs) and insurance claims (Krist et al., 2013). Thus, while a number of psychosocial interventions are supported by existing evidence, many questions remain to be answered with regard to their effectiveness across settings: who is best able to deliver them, what their limitations are, and how they are best implemented.
Given the rigor and time involved in conducting a systematic review of the evidence for psychosocial interventions, this task is beyond the purview of the committee. Chapter 4 provides recommendations for how such systematic reviews should be conducted. The committee refers the reader to published reports from organizations that have reviewed evidence in accordance with the IOM (2011a) report Finding What Works in Health Care: Standards for Systematic Reviews (e.g., the Agency for Healthcare Research and Quality, the U.S. Department of Veterans Affairs [VA], and the U.K. National Health Service’s [NHS’s] National Institute of Heath and Care Excellence). While the reviews discussed here focus on treatment recommendations for specific disease and problem areas, they all include psychosocial/behavioral interventions (when appropriate). The VA and NHS, based on these reviews, have implemented large-scale provider trainings in a number of evidence-based psychosocial interventions, and the authoring organizations note that psychosocial interventions are critical options in treating mental health and substance use disorders.
The next step after expanding the evidence base for psychosocial interventions is to standardize them and identify the important elements that drive their effects. These elements, as defined briefly in Chapter 1 and at greater length in Chapter 3, may be either nonspecific (common to all effective psychosocial interventions) or specific to a particular theoretical model or psychosocial approach. Most evidence-based psychosocial interventions are standardized, and these standards are detailed in treatment manuals. Indeed, without these manuals, the implementation of standards for psychosocial interventions would be complicated. For example, the National Registry for Evidence-based Programs and Practices (NREPP) of the Substance Abuse and Mental Health Services Administration (SAMHSA) requires that interventions have a manual or set of guidelines, as well as a training program and a means for measuring quality that tracks to the core competencies in the manuals (SAMHSA, 2015).
Treatment manuals operationalize interventions by describing the theory, procedures, techniques, and strategies they entail. The procedures are detailed with scripts and case examples. Manuals were developed initially to facilitate efficacy studies of psychotherapy, to ensure that therapists
were carrying out the treatment under study with fidelity, and to ensure consistency among therapists in how the treatment was delivered. However, manuals have become an important aspect of the implementation of interventions. Without a manual, a guideline, or documentation of how an intervention works, the intervention cannot be deployed as it was developed.
Most manuals have been adapted for different age groups, cultures, disorders, and delivery formats. Sometimes the adaptations have been newly tested, but often they have not. The proliferation of manuals has caused some confusion, and as a result, the manuals often are not widely accepted in clinical practice (Addis and Waltz, 2002). Among the reasons for limited acceptance is the view that the manuals are overly prescriptive and too complicated to follow, and most are not accompanied by evidence-based trainings. When providers are properly trained and supported in an intervention, however, manuals can be useful resources.
Standardization of psychosocial interventions provides an opportunity for identifying the potential nonspecific and specific elements of these treatments. As discussed in Chapter 3, a process for specifying elements will be necessary to improve the impact of psychosocial interventions.
Once standardized evidence-based psychosocial interventions and their elements have been identified, systematic reviews can be conducted to inform clinical practice guidelines through a methodical, transparent process (IOM, 2011b). As discussed in Chapter 4, centralization of systematic reviews to support the development of guidelines has the potential to minimize the current confusion over which interventions are evidence based and under what circumstances they are most effective. Furthermore, existing standards for systematic reviews may need to be modified for psychosocial interventions to include methods for determining the limits of the interventions, who can be trained to deliver them, and what supports are needed to sustain their quality. Given the cost and time involved in conducting these reviews, innovations from the fields of engineering (e.g., natural-language processing) could be used to expedite the review process.
Guidelines based on systematic reviews support decision making among providers and consumers and also form the basis for the development of quality measures that can be used to monitor and evaluate the quality of care in real-world clinical practice settings and ultimately the impact of interventions in improving patient outcomes. Currently, there exist measures
of provider competencies, often referred to as fidelity measures, for many evidence-based psychosocial interventions. Like treatment manuals, fidelity measures were developed for use in RCTs to ensure that participants enrolled in a study are actually receiving the treatment under study, but also are not receiving elements from different interventions. A good fidelity tool measures not only providers’ adherence to an intervention’s strategies and processes, but also the degree to which providers conduct the intervention to competence. It is not enough to know the steps in a treatment; it is important as well to know how to adjust the treatment to meet the needs of the individual consumer without completely abandoning the therapeutic elements that drive the intervention’s effect.
A comprehensive quality framework must consider the context in which interventions are delivered. This context includes characteristics of the consumer and the qualifications of the provider. A means for training and credentialing providers in evidence-based treatment is critical to support providers in the use of these interventions. The context for the delivery of interventions also includes the clinic or specific setting in which care is rendered, the health system or organization in which the setting is embedded, and the regulatory and financial conditions under which it operates. Stakeholders in each of these areas can manipulate levers that shape the quality of a psychosocial intervention; shortfalls in the context of an intervention and in the manipulation of those levers can render a highly efficacious intervention unhelpful or even harmful.
An evidence base demonstrates that consumers bring important perspectives on and knowledge of mental health and substance use problems to psychosocial research and intervention development (Beinecke and Delman, 2008; Berwick, 2009; Deegan, 1993). Their active participation in this process can lead to interventions that address outcomes of most importance to them, improving both adherence and effectiveness (Graham et al., 2014). Consumers are active participants when they offer perspectives and take actions that influence the process of developing and assessing interventions (Checkoway, 2011). As it applies to the committee’s framework, consumer involvement is important to identify and formulate research questions for systematic review, help develop guideline recommendations, inform the development of quality measures, and monitor the implementation of interventions.
Active consumer participation has been implemented most compre-
hensively through a community-based participatory action research (PAR) framework. PAR is a process through which professionals and disadvantaged community members work collaboratively to combine knowledge and action for social change, with community members being able to participate in every stage of the project (Israel et al., 2003).
Implicit in consumer engagement is a thorough consideration of the context for psychosocial interventions, including existing diagnoses, comorbidities, risk factors, social determinants of health, and personal values and preferences. The framework for psychosocial interventions is a complex process, and the committee encourages a broad bio-psychosocial perspective that avoids a siloed approach.
As more evidence emerges from research trials as well as from practical trials based on real-world experience, the cycle of the framework begins anew. Each step in the cycle generates additional research questions and can provide additional evidence. The data systems created for monitoring quality and improving care, for example, can be used in identifying new knowledge about the effectiveness of psychosocial interventions and their elements in different settings or for different populations. Thus, the framework is envisioned as a continuous, iterative process, with each step in the cycle expanding the knowledge base for the development of new and improved standards for psychosocial interventions that can improve patient outcomes.
The committee drew the following conclusions about the need for a framework:
The mental health and substance use care delivery system needs a framework for applying strategies to improve the evidence base for and increase the uptake of high-quality evidence-based interventions in the delivery of care.
Broad stakeholder involvement is necessary to develop effective interventions that will lead to improved outcomes for individuals with mental health and substance use disorders.
Recommendation 2-1. Use the committee’s framework for improving patient outcomes through psychosocial interventions to strengthen the evidence base. The U.S. Department of Health and Human Services
should adopt the committee’s framework to guide efforts to support policy, research, and implementation strategies designed to promote the use of evidence-based psychosocial interventions. Steps in this iterative process should focus on
- strengthening the evidence base for interventions,
- identifying key elements of interventions,
- conducting independent systematic reviews to inform clinical guidelines,
- developing quality measures for interventions, and
- implementing interventions and improving outcomes.
This is a complex process, and the framework is intended to be used to guide a continuous progression. At each step in the process, systematic research and evaluation approaches should be applied to iteratively expand the knowledge base for the development of new and improved standards for psychosocial interventions that will improve patient outcomes.
Recommendation 2-2. Require consumer engagement. The U.S. Department of Health and Human Services and other public and private funding agencies should ensure that consumers are active participants in the development of practice guidelines, quality measures, policies, and implementation strategies for, as well as research on, psychosocial interventions for people with mental health and substance use disorders, and provide appropriate incentives to that end. In addition, family members of consumers should be provided with opportunities to participate in such activities.
Addis, M. E., and J. Waltz. 2002. Implicit and untested assumptions about the role of psychotherapy treatment manuals in evidence-based mental health practice. Clinical Psychology: Science and Practice 9(4):421-424.
Beinecke, R., and J. Delman. 2008. Commentary: Client involvement in public administration research and evaluation. The Innovation Journal: The Public Sector Innovation Journal 13(1). http://www.innovation.cc/peer-reviewed/beinicke_7_commenta-_client_public_admin.pdf (accessed February 18, 2009).
Berwick, D. M. 2009. What “patient-centered” should mean: Confessions of an extremist. Health Affairs 28(4):w555-w565.
Checkoway, B. 2011. What is youth participation? Children and Youth Services Review 33(2):340-345.
Deegan, P. E. 1993. Recovering our sense of value after being labeled mentally ill. Journal of Psychosocial Nursing and Mental Health Services 31(4):7-11.
Graham, T., D. Rose, J. Murray, M. Ashworth, and A. Tylee. 2014. User-generated quality standards for youth mental health in primary care: A participatory research design using mixed methods. BMJ Quality & Safety 10.1136/bmjqs-2014-002842.
Huhn, M., M. Tardy, L. M. Spineli, W. Kissling, H. Forstl, G. Pitschel-Walz, C. Leucht, M. Samara, M. Dold, J. M. Davis, and S. Leucht. 2014. Efficacy of pharmacotherapy and psychotherapy for adult psychiatric disorders: A systematic overview of meta-analyses. JAMA Psychiatry 71(6):706-715.
IOM (Institute of Medicine). 2010. Provision of mental health counseling services under TRICARE. Washington, DC: The National Academies Press.
_____. 2011a. Finding what works in health care: Standards for systematic reviews. Washington, DC: The National Academies Press.
_____. 2011b. Clinical practice guidelines we can trust. Washington, DC: The National Academies Press.
Israel, B. A., A. J. Schulz, E. A. Parker, A. B. Becker, A. J. Allen, and J. R. Guzman. 2003. Critical issues in developing and following community-based participatory research principles. In Community-based participatory research for health, edited by M. Minkler and N. Wallerstein. San Francisco, CA: Jossey-Bass. Pp. 53-76.
Krist, A. H., D. Shenson, S. H. Woolf, C. Bradley, W. R. Liaw, S. F. Rothemich, A. Slonim, W. Benson, and L. A. Anderson. 2013. Clinical and community delivery systems for preventive care: An integration framework. American Journal of Preventive Medicine 45(4):508-516.
Pincus, H. A. 2010. From PORT to policy to patient outcomes: Crossing the quality chasm. Schizophrenia Bulletin 36(1):109-111.
SAMHSA (Substance Abuse and Mental Health Services Administration). 2015. NREPP reviews and submissions. http://www.nrepp.samhsa.gov/ReviewSubmission.aspx (accessed May 28, 2015).
Tunis, S. R., D. B. Stryer, and C. M. Clancy. 2003. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. Journal of the American Medical Association 290(12):1624-1632.
WHO (World Health Organization). 2010. mhGAP intervention guide. http://www.paho.org/mhgap/en (accessed January 6, 2015).
This page intentionally left blank.