Page 1 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

Summary

The opioid epidemic, now several decades in the making, continues to cause pain and suffering for millions of Americans. Each year, thousands of individuals die from overdose, and thousands more grieve from these losses. Opioid use disorder (OUD) can lead to a complete interruption of day-to-day activities, including caring for one’s family, maintaining a job or career, or keeping track of basic necessities, such as health care and finances.

The government’s response to this epidemic is manifold. This report, an offshoot of that response, is the first of three reports requested by the U.S. Congress and commissioned by the Substance Abuse and Mental Health Services Administration (SAMHSA). This series of reports focuses on four of SAMHSA’s grant programs that help alleviate suffering due to opioids and improve treatment quality and access. Following directly from a 2016 law titled the Comprehensive Addiction and Recovery Act (CARA), two of these grant programs focus on preventing overdose: the First Responder Training (FR-CARA) and Improving Access to Overdose Treatment (OD Treatment Access) programs. An additional two grant programs focus on treatment: the Building Communities of Recovery (BCOR) and Pregnant and Postpartum Women Pilot (PPW-PLT).

In the text that follows, an expert committee convened by the National Academies of Sciences, Engineering, and Medicine provides recommendations to SAMHSA to improve the outcomes and metrics associated with these four grant programs. The committee offers recommendations about the existing reporting tools used by these programs, which includes the

Page 2 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

Division of State Programs Management Reporting Tool, the Center for Substance Abuse Treatment Government Performance Results and Modernization Act (GPRA) tool, and tools unique to individual programs. The committee also proposes additional metrics and outcomes that should be considered. Recognizing that the grantees under each of these programs may be limited in resources or staff, it is the committee’s hope that the following recommendations represent practical, feasible steps toward reducing the harm associated with opioids and increasing engagement and retention in life-saving treatments.

RECOMMENDATIONS

The following outcomes and metrics may assist SAMHSA in achieving success with respect to the two prevention grant programs, FR-CARA and OD Treatment Access.

Recommendation 2-1: The Substance Abuse and Mental Health Services Administration (SAMHSA) should ensure that in education and outreach efforts across SAMHSA programs, including activities funded by the First Responder Training program and the Improving Access to Overdose Treatment program, a key outcome is the degree to which providers, first responders, and members of the public understand the danger associated with fentanyl and its analogues. SAMHSA should facilitate the distribution of fentanyl test strips along with naloxone—especially to regions with above-average overdose rates and in regions with relatively low penetration of fentanyl already—and should collect data on how many test strips were distributed and used.

Recommendation 2-2: The Substance Abuse and Mental Health Services Administration (SAMHSA) should support grantees in collecting and reporting data on the number of overdose survivors who initiate and are engaged in evidence-based opioid use disorder treatment. With technical assistance on evidence-based practices offered by SAMHSA, grantees should include information about the characteristics of the programs providing treatment and the evidence on which their treatment is based.

Recommendation 2-3: The Substance Abuse and Mental Health Services Administration should collect information that would provide a full understanding of the opioid use disorder treatment programs to which patients are referred, focusing on whether the treatments delivered are evidence based. Additionally, it would be important

Page 3 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

to understand the time between overdose, referral, and initiation of treatment. For grantees that cannot capture such data due to resource or technical constraints, a random sampling approach—or a systematic sampling approach if more appropriate for any given grantee or population—should be used to track these metrics over time and identify where referral and engagement processes can be improved.

Recommendation 2-4: The Substance Abuse and Mental Health Services Administration should provide support to grantees to engage survivors of overdose in a full range of harm-reduction services (including syringe services programs and naloxone distribution services) and collect data on how many of these individuals end up engaging in harm-reduction services (e.g., number of individuals receiving a naloxone kit following an overdose event). Grantees could accomplish this by developing new services or by partnering with existing harm-reduction organizations.

Recommendation 2-5: The Substance Abuse and Mental Health Services Administration should require information on insurance coverage and housing (or lack of coverage and housing) as additional metrics to be reported in grantees’ disparity impact statements in the Division of State Programs Management Reporting Tool. Grantees should collect and report repeated overdoses in all populations, with a particular focus on reducing overdoses among those on Medicaid and those who are uninsured or unstably housed.

Recommendation 2-6: The Substance Abuse and Mental Health Services Administration (SAMHSA) should support grantees in collecting geographic locations of overdoses (such as zip codes) to identify gaps in services and prevent future overdoses. These data should be used to help grantees identify the reach of their activities and provide a benchmark to improve on. In addition, SAMHSA should support grantees in collecting data on nonfatal overdoses in coordination with first responders where transportation to the hospital does not occur (including on how many times naloxone was administered and who administered it).

Recommendation 2-7: In addition to measuring the number and characteristics (e.g., friend/family member of a person who uses drugs) of trained individuals who feel confident administering naloxone in the case of an overdose, the Substance Abuse and Mental Health Services Administration should support grantees that focus on training laypeople or a community organization/agency/staff in measuring

Page 4 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

the extent to which individuals actually administer naloxone when needed and whether their knowledge and skills in administering naloxone has been improved by the program’s activities.

Recommendation 2-8: The Substance Abuse and Mental Health Services Administration should develop efforts within these grant programs to include medical, nursing, social work, and pharmacy students in training sessions on prescribing and co-prescribing of naloxone and should collect data and report on the number of students trained.

Recommendation 2-9: The Substance Abuse and Mental Health Services Administration (SAMHSA) should assess first responder job satisfaction and rates of compassion fatigue or turnover due to overdose prevention as key outcomes of the First Responder Training program. SAMHSA should routinely assess progress toward these outcomes among first responders who are beneficiaries of the grant. Surveys of first responders can be used to elucidate job satisfaction and feelings of compassion fatigue and to assess the extent to which opioid-related events impact their sentiments about their work (surveys, however, should be as brief as possible to elicit the necessary information, thereby reducing undue administrative burden). Rates of first responder turnover can be assessed through reviews of administrative data at the state and local levels. Furthermore, SAMHSA should support grantees in establishing mechanisms for maintaining job satisfaction and preventing compassion fatigue among first responders.

The following outcomes and metrics may assist SAMHSA in achieving success in the second set of grant programs (BCOR and PPW-PLT).

Page 5 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

ing tools to include a more diverse set of evidence-based treatment options for opioid use disorder (OUD), including medications for OUD.¹

Recommendation 3-3: The Substance Abuse and Mental Health Services Administration should implement measures for grantees to report whether and how patients are treated in an integrated manner and how the targeted service systems have progressed in becoming more integrated.

Recommendation 3-4: The Substance Abuse and Mental Health Services Administration should implement a validated and psychometrically sound tool for assessing recovery among clients of its grant programs, as the Center for Substance Abuse Treatment Government Performance Results and Modernization Act tool does not elicit adequate data on the process of recovery.

Recommendation 3-5: The Substance Abuse and Mental Health Services Administration should include measurements about unintentional and intentional overdose events in its grant reporting tools (e.g., number of events in a given period, outcome of overdose events, first or non-first overdose event) as well as information about fentanyl and concurrent use of opioids and benzodiazepines.

Recommendation 3-6: The Substance Abuse and Mental Health Services Administration (SAMHSA) should use or develop additional data-collection tools to measure alcohol and drug use in more realistic ways than past 30-day abstention and frequency of use, which do not necessarily capture a clear picture of drug use behavior. Additionally, SAMHSA should encourage grantees to adopt a harm-reduction approach when interacting with clients, emphasizing that relapse is common and not a signal of personal failure.

Recommendation 3-7: The Substance Abuse and Mental Health Services Administration should use or develop self-report measures for its grant programs that minimize confusion and risk of alienating patients and clients. This is especially important when asking about sensitive topics, such as substance use, child custody, income, and sexual behavior.

___________________

¹ The committee notes that in the final stages of preparing this report, the GPRA tool underwent an update. Now, the revised tool includes a question specifically about medications for OUD, including methadone, buprenorphine, and naltrexone.

Page 6 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

Recommendation 3-8: The Substance Abuse and Mental Health Services Administration should incorporate measures of living arrangement safety and perceptions of safety in data-collection tools.

Recommendation 3-9: The Substance Abuse and Mental Health Services Administration should support grantees in providing and measuring the success of activities related to job skills and readiness in the Building Communities of Recovery and State Pilot Grant Program for Treatment for Pregnant and Postpartum Women programs.

Recommendation 3-10: The Substance Abuse and Mental Health Services Administration should collect additional program-level data in order to better describe and understand the strengths and needs of funded programs and to determine the evidence-based factors that lead to improved outcomes and reduced disparities for diverse beneficiaries of the Building Communities of Recovery and State Pilot Grant Program for Treatment for Pregnant and Postpartum Women programs.

Recommendation 3-11: The Substance Abuse and Mental Health Services Administration should implement additional measurements to assess the system-level change that grantees are making using the Building Communities of Recovery and State Pilot Grant Program for Treatment for Pregnant and Postpartum Women grant funding. This could include surveys of stakeholders impacted by the grant or use of administrative data to determine changes made at the local and state levels in response to grantee activities.

Recommendation 3-12: The Substance Abuse and Mental Health Services Administration should support grantees in measuring and reporting how peer recovery services have impacted the overall recovery process for clients served by the Building Communities of Recovery grant.

Recommendation 3-13: The Substance Abuse and Mental Health Services Administration should support programs in measuring the following:

Change or increase in the number of pregnant and postpartum women (PPW) served in all state licensed substance use disorder (SUD) treatment programs in years prior to, during, and after implementation of the State Pilot Grant Program for Treatment for Pregnant and Postpartum Women (PPW-PLT) grant program and their referral sources. This would help determine whether the

Page 7 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

PPW-PLT grantee served as a significant facilitator of enhancing treatment access.
Data on the trimester of pregnancy when women enter treatment (to see if the PPW-PLT program is able to reach women early in pregnancy and increase the number of women entering treatment earlier in pregnancy) as well as outcomes of child delivery.
The number and percent increase in number of child protective services referrals to a PPW treatment case manager.
The number of family members served and average number of services received by extended family members. Recognizing that relationships between patients with SUD and their families are often complex, engagement of family members who support recovery, who do not use drugs, and who provide positive support is likely to assist in engagement and retention in treatment.
Number of PPW who drop out of treatment early and at what stage. These data could be used to employ quality improvement strategies to decrease early dropout and to assess if the PPW-PLT program improved treatment retention since grant implementation.

It is the committee’s hope that these recommendations, if implemented, could reduce overdose rates, increase access to and engagement in treatment, and reduce the individual and societal suffering stemming from OUD.

Page 8 Cite

Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2020. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement. Washington, DC: The National Academies Press. doi: 10.17226/25745.

×

This page intentionally left blank.

Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement (2020)

Chapter: Summary

Summary

RECOMMENDATIONS

Welcome to OpenBook!

Get Email Updates