4
Intermediary Organizations and Scale-Up
Sometimes scale-up and implementation of prevention programs are aided by intermediary entities. In the case of family-focused prevention programs, such entities have included Invest in Kids, the Evidence-Based Prevention and Intervention Support (EPIS) Center, the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) Strategic Prevention Framework, Project LAUNCH (Linking Actions for Unmet Needs in Children), PROSPER (PROmoting School-community-university Partnerships to Enhance Resilience), the New York State Office of Mental Health’s Clinical Technical Assistance Center (CTAC), and the REACH Institute. Representatives of these organizations presented at the workshop, and their remarks are summarized in this chapter. This chapter also includes a description of the Washington State Institute for Public Policy which has conducted economic cost–benefit analyses of evidence-based programs Washington State is considering funding. The chapter concludes with a table (see Table 4-1) that summarizes strategies used by the intermediary entities to aid scale-up and implementation of family-focused prevention programs.
Invest in Kids is a nonprofit organization founded in 1998 by a group of lawyers and community leaders in the Denver, Colorado, area. The mission of this organization is to improve the health and well-being of vulnerable young children and families throughout Colorado. Working in partnership with local communities, Invest in Kids identifies, introduces,
implements, and tries to ensure the long-term success of research-based and proven prevention programs.
Invest in Kids meets with local community leaders throughout Colorado, ascertains what their greatest needs are for services, and then uses that information to do a national targeted search for appropriate prevention programs. Lisa Hill, executive director of Invest in Kids, explained that the search is restricted to programs that have been found not only to be effective, but to show a cost–benefit return to society. These programs also must serve predominantly low-income children, prenatal to age 8 and their families. “We bridge the gap between research and practice,” Hill said.
Hill believes the broad-based community support and involvement they seek in pursuit of their goals is critical because it improves coordination and avoids duplication of services, as well as promotes the long-term sustainability of the programs both through political support and through invested local leadership.
Once Invest in Kids identifies programs for specific communities, it lobbies state legislators, providing data about both the need for the programs and the expected outcomes based on previous research, as well as the costs and accountabilities of those programs. Invest in Kids then helps implement family-based programs in Colorado through agency partnership and community collaboration. One of its partners is the National Implementation Research Network of the University of North Carolina, whose formula for garnering socially significant outcomes is that it depends not just on the effectiveness of interventions, but effective implementation methods.
A key activity of Invest in Kids is ensuring ongoing consultation and support after workers implementing prevention programs have been trained, according to Hill. Invest in Kids also ensures ongoing program success through measurement of results of the programs it supports (Hicks et al., 2008). The first program Invest in Kids promoted in Colorado was the Nurse–Family Partnership (NFP). This program is now offered in 60 of 64 counties in Colorado with a total capacity of more than 3,000 mothers, which is about 20 percent of the penetration rate of first-time Medicaid mothers in Colorado. Nearly 17,000 first-time, low-income families have enrolled in NFP since 2000. This has resulted in less tobacco and alcohol use and less domestic violence during pregnancy, a 20 percent greater immunization rate of 2-year-olds, and fewer mothers experiencing second pregnancies by the time their child is 2 years old.
Another program Invest in Kids has promoted and supported is The Incredible Years® (IY). More than 35,000 young children and 3,200 low-income parents have enrolled in the IY in Colorado since 2002. According to Hill, the program has improved children’s ability to solve problems, control anger, self-monitor their emotions, make friends, and follow verbal instructions from their teachers. In addition, parents have improved their
positive parenting practices, such as setting clear expectations for not fighting, giving brief time outs, and more frequently discussing problems with children, with fewer parents resorting to spankings or empty threats of punishment. Concerned about maintaining the quality of staffing for the IY program, Invest in Kids employed 17 peer coaches around the state who are “moving the quality in the direction we need it to go,” Hill said.
Invest in Kids also provides ongoing consultation and quality improvement for all the programs it supports by identifying gaps at the local level and determining ways to fill them. For example, the NFP in Colorado was suffering from a lack of nurse retention, and Olds found that clients are seven to eight times more likely to drop out of the program when their nurses leave. So Invest in Kids created the Nurse Practice Council, which is based on the magnet hospital concept, and brought 14 nurses for 2-year terms together to improve their outcomes. Since the Council has been operating there has been an increase in nurse retention. “We’re seeing better client outcomes as a consequence of their sense of buy-in,” Hill said.
EVIDENCE-BASED PREVENTION AND INTERVENTION SUPPORT (EPIS) CENTER
The EPISCenter is a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD) and Pennsylvania State University, with funding and support from the PCCD and the Pennsylvania Department of Public Welfare. The EPISCenter supports the dissemination, quality implementation, sustainability, and impact assessment of a menu of proven, effective prevention and intervention programs, and conducts original translational research to advance the science and practice of evidence-based prevention.
Brian Bumbarger of Pennsylvania State University, founding director of EPISCenter, started this unique partnership between policy makers, researchers, and communities in 2008 after recognizing that despite grants and a list of effective prevention programs, communities trying to replicate them struggled with their implementation. The three main goals of EPISCenter are to
- Mobilize and support prevention infrastructure at the community level to collect and utilize diagnostic epidemiological data for strategic prevention planning,
- Support and provide training and technical assistance for a specific menu of evidence-based interventions the state is providing grants to communities to implement, and
- Improve the quality of juvenile justice programs and practices, such as ensuring they are grounded in the latest research about child development and other relevant areas.
According to Bumbarger, EPISCenter tries to meet these goals by building general prevention science knowledge and capacity as well as program-specific capacity, in addition to facilitating interactions and communications between relevant systems (e.g., service providers, policy makers, and researchers/program developers). He noted that EPISCenter acts as an intermediary between developers of programs and organizations in Pennsylvania that want to adopt them. For example, EPISCenter helps schedule trainings for the IY program and makes sure that communities have not only built their capacity and are ready for the program before they adopt it, but that they understand what is needed to evaluate the implementation and effectiveness of the program.
In addition to building program-specific capacity in service providers, EPISCenter tries to build their general capacity to be data driven, and to understand the importance of research, quality, fidelity, and sustainability planning. EPISCenter also acts as a conduit between program providers, funders, and developers, and is a mediator between these stakeholders. EPISCenter has developed state-specific evaluation tools to meet state funders’ requirements, and has partnered with the state to add a quality assurance requirement into the grants they fund. When the state funds a community to adopt a program, by the end of the second year, the community has to provide implementation data to the developer to demonstrate that the community is implementing the program with sufficient quality and fidelity.
SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION STRATEGIC PREVENTION FRAMEWORK
Recognizing that the prevention funds it provides would probably be more efficiently and effectively used if they were distributed according to a strategic prevention plan, SAMHSA began offering Strategic Prevention Framework State Incentive Grants. These grants provide states, jurisdictions, and tribes with funding to develop an infrastructure to support the development of a coordinated strategic plan for substance abuse and mental health prevention programs (SAMHSA, 2014a,b,c). Clarese Holden from SAMHSA’s Division of State Programs in the Center for Substance Abuse Prevention reported that such plans must
- Assess prevention needs based on data;
- Detail how prevention capacity will be built and effective community prevention programs, policies, and practices implemented; and
- Evaluate prevention efforts for outcomes.
Every state, 9 territories, and 16 tribes have received these grants and created strategic framework plans. Holden explained that to create these plans, state governors built advisory councils comprising many of the partners, organizations, and agencies within the state that focused on prevention. These councils then constructed plans that detailed how they would institute five steps—assessment, capacity building, planning, implementation, and evaluation—while encouraging cultural competence and sustainability. “States are different, as are communities, so we cannot have a cookie-cutter approach to all of them. They are building their own strategic prevention plans and taking ownership of those plans so they are able to do what they need to do for their communities,” Holden emphasized.
The primary objective of Project LAUNCH is to promote the social, emotional, behavioral, and physical health and cognitive development of young children from birth to 8 years of age. Under a federal grant program administered by SAMHSA, Project LAUNCH grantees work to reach the objective through five core strategies: developmental assessment, integration of behavioral health into primary care settings, home visiting, mental health consultation, and family strengthening and parent skills training (OPRE, 2014).
Project LAUNCH is a 5-year grant program that has been applied in 40 states, tribes, and communities since 2008, Holden reported. The program has a dual focus on systems improvement and implementation of evidence-based prevention and promotion practices. The program also fosters collaboration across sectors and works to enhance and expand programs rather than reinvent them. “State, territorial, tribal, and local partnerships are forming, and we are infusing mental health knowledge and expertise into all of our early childhood settings,” Holden said. In Maine, this program targeted a rural and highly impoverished county by supporting families with high-risk pregnancies. Data suggest Project LAUNCH increased the rates of adequate prenatal care for teen mothers (12-19 years old) in Washington County, Maine, from 62 percent to 85 percent within 4 years.
PROSPER is a scientifically proven system for facilitating sustained, high-quality delivery of evidence-based programs that reduce risky youth behaviors, enhance positive youth development, and strengthen families. This delivery system links university-based prevention researchers with
two established program delivery systems within a state—the Cooperative Extension System at the Land Grant University and the public school system. The public school system offers access to youth in the community, while the Cooperative Extension System offers knowledge of community programming needs and experience in disseminating educational programs (PROSPER, 2014; Spoth and Greenberg, 2011). In this way, the delivery system entails a partnership-based approach to evidence-based programming, called the PROSPER Partnership Model (see Figure 4-1).
Using this model, explained Richard Spoth of the Partnerships in Prevention Science Institute at the Iowa State University, small and strategic community teams have access to the latest intervention outcome research from university scientists and benefit from the expertise and consistent support of the Cooperative Extension System. The PROSPER delivery system also ensures that programs for youth and their parents are implemented properly, are supported in the community, and can be sustained over time. Critically important in this effort are prevention coordinators, who provide ongoing proactive technical assistance, Spoth reported. These coordinators serve as a liaison between a university-based team and local community-based teams.
FIGURE 4-1 PROSPER evolving community partnership sustainability model.
NOTE: EBI = evidence-based intervention.
SOURCE: Spoth et al., 2004.
PROSPER Community Teams start with between 8 and 10 members, including a family and/or youth Community Extension–based team leader, a school-based co-team leader, and community volunteers, who include local mental health or public health representatives, local substance abuse agency representatives, parents, or youths. These teams expand as they mature, while being guided by technical assistance. PROSPER educates and trains its members to rigorously monitor their programs and provides them with a system for continuous as well as annual monitoring of their implementation quality. PROSPER also encourages teams to sustain their own programs with their own fundraising ventures.
Studies show that PROSPER significantly reduces substance misuse, negative peer influences, and other problem behaviors in youth, while strengthening parenting, family relationships, and youth skills. Research also reveals that PROSPER effectively mobilizes community teams, many of whom have sustained their programming efforts for as long as 10 years and have had high recruitment rates compared to other approaches for delivering school-based substance use interventions (Spoth, 2012; Spoth et al., 2013). In addition, PROSPER appears to be more cost efficient and effective than regular programming, with one study finding it reduces implementation costs by more than half (Crowley et al., 2012). “There are efficiencies gained with sustainability of an effective system. As organizations learn how to implement better, they become more efficient over time so the benefits of implementing through this delivery system offset the costs to some extent,” Spoth explained.
NEW YORK STATE OFFICE OF MENTAL HEALTH CLINIC TECHNICAL ASSISTANCE CENTER
The New York State Office of Mental Health’s (NYSOMH’s) Clinic Technical Assistance Center (CTAC) aids providers in building their capacity to provide higher quality services for children and families. CTAC provides training and consultation in business practices, organizational and leadership support, and evidence-informed clinical practices. CTAC provides webinars, in-person, and learning collaboratives on quality improvement strategies to all 350 of the NYSOMH-licensed clinics serving children and families. “We are responsible for finding out what kinds of training clinics need and how we can best provide that support,” said CTAC director Kim Hoagwood from the New York University School of Medicine.
CTAC helps clinics make the business case for evidence-based prevention and treatment programs before adopting them. “We work with them to ascertain how adoption will affect their productivity and volume and how they will bill for it. We do a lot of pre-rollout work with them to help them think it through, but it pays off,” Hoagwood said. She added, “We
have a group of clinics that are teetering in a fiscal sense. We are working intensively with them over a 2-year period using a learning collaborative model to help them stay in business.” CTAC also helps providers collect continuous quality improvement data on the programs they implement and use those data for supervision, Hoagwood said.
Thomas Dishion from Arizona State University reported that the recently created REACH Institute of Arizona State University (ASU) offers infrastructure support to providers of programs. The REACH Institute of ASU is developing the online capacity for low-cost training, education, and support for global implementation of the Family Check-Up program model as well as other ASU interventions. These services are expected to reduce the upfront expenses for agencies as they adopt new programs. The Institute is also developing digital platforms for data collection and delivery of program protocols, and it will also redesign programs to better suit an adopting organization’s needs.
WASHINGTON STATE INSTITUTE FOR PUBLIC POLICY
The mission of the Washington State Institute for Public Policy (WSIPP) is to carry out practical, nonpartisan research, at legislative direction, on issues of importance to Washington State (WSIPP, 2014). The institute has been instrumental at conducting innovative economic cost–benefit analyses of evidence-based programs Washington State is considering funding.
The institute, which was created in 1983, operates as an independent ratings agency and is often asked to evaluate what makes sense economically to implement, given restricted budgets. Topic areas that the institute frequently evaluates include crime, education, child welfare, and behavioral health, according to Stephanie Lee. “The legislature asks us to assess if we can get a positive return on investment with some of these evidence-based programs and policies,” she said. The results of the institute’s analyses are posted regularly on their website.
The institute takes a three-step approach to evaluating programs, Lee reported:
- Meta-analyze all rigorous evaluations of policies to improve public outcomes of legislative interest.
- Calculate the return on investment for various programs and policies by computing benefits, costs, and risk to the people of Washington State using a consistent crosscutting framework that can be applied to diverse areas. By considering the risk of errors in
-
measurement that would cause variance from the means used in cost benefit analyses, the institute estimates what proportion of the time one could expect an intervention to at least pay for itself or break even.
- Calculate how specific portfolios of program investments would affect statewide outcomes and the risks involved in those programs.
“You don’t want to put all of your eggs in one basket as a personal investor or as a government entity that’s paying for programs. Instead, you want to spread your risk across a number of different populations,” Lee noted. How long it will take to reap the benefits of various programs is also factored in such that there is a balance of long-term and short-term investments in the portfolio.
For example, Lee showed the factors the Institute considered as it calculated the return on investment in the NFP program. For each of these factors, the institute assigned appropriate dollar figures per family served in the program. The likely higher earnings of more highly educated women and children in the program translated into a $24,000 benefit, whereas the reduced criminal behavior reduced state costs by $5,000 per family. Lower health care and public assistance costs and benefits were also factored in, leading to overall benefits of about $27,000 per family and overall costs of around $10,000 per family.
The final return on investment figure of $17,000 per family participating in the NFP would not only be returned to tax payers but to a broad sector of society, including the mothers and children who participate in the program and the victims of crime that would be avoided by such participation, Lee noted. The institute estimated there would be a 76 percent chance of at least breaking even if the program was implemented in Washington State and a 24 percent chance that the program would not pay for itself. “So given the average return of about $17,000 per family and 76 percent breakeven point, we said this is probably a good investment for Washington State,” Lee noted. She concluded, “When we think about creating a portfolio of programs or policies, these are the kinds of things we think about—how many different populations can we target, what are the time-lines where we would expect those programs to pay off, and how can we best spread our investment across a number of different options.”
The institute is currently being supported by the McArthur Foundation and Pew Charitable Trusts Results First Initiative to develop its cost–benefit model so other states can put their specifics into the model and have it churn out a tailored analysis. The WSIPP cost–benefit model is currently undergoing testing in 15 states, according to Lee.
SUMMARY OF INTERMEDIARY STRATEGIES TO AID SCALE-UP
The following table (see Table 4-1) summarizes the strategies to support the scale-up of family-focused prevention programs utilized by the intermediary organizations discussed at the workshop.
| Program Name | Strategies to Aid Scale-Up |
| Invest in Kids |
• Partner with the local community to ascertain what services best match the needs of the community • Conduct a national search for the most appropriate evidence-based and cost-effective programs • Lobby state and local legislators, using data, to demonstrate the need for and expected outcomes of selected programs • Build political support and investment by local leadership • Provide ongoing support and monitoring of program implementation |
| EPISCenter |
• Build general prevention science knowledge as well as program-specific capacity • Provide technical assistance to service providers implementing a specific menu of evidence-based interventions • Facilitate communication among stakeholders and systems, as well as between the program developer and organization implementing the program • Support providers before a program is adopted to build capacity for implementation and knowledge of what will be needed to evaluate program implementation and effectiveness • Partner with states to include quality assurance requirement in grants • Develop state-specific evaluation tools to meet state funders’ requirements • Mobilize and support prevention infrastructure at the community level to collect and utilize diagnostic epidemiological data for strategic prevention planning |
| SAMHSA (Substance Abuse and Mental Health Services Administration) Strategic Prevention Framework |
• Grants to states, jurisdictions, and tribes to create strategic plans for prevention programs, including support for coordination and infrastructure building, that are specific to the needs of the given community • Encourage assessment, capacity building, planning, implementation, evaluation, cultural competence, and sustainability |
| Program Name | Strategies to Aid Scale-Up |
| Project LAUNCH (Linking Actions for Unmet Needs in Children) |
• Dual focus on systems improvement and implementation of evidence-based prevention programs • Foster collaboration across sectors • Enhance and expand programs rather than reinvent them • Infuse mental health knowledge and expertise into all early childhood settings |
| PROSPER (PROmoting School-community-university Partnerships to Enhance Resilience) |
• Facilitate sustained, high-quality delivery of evidence-based programs • Partnership between small, strategic community teams and university prevention researchers • Prevention coordinators provide ongoing, proactive technical assistance, and act as a liaison between community teams and university researchers • Provide training on rigorous and continuous monitoring or program implementation quality • Efficiency gained through program sustainability |
| New York State Office of Mental Health’s (NYSOMH’s) Clinic Technical Assistance Center |
• Determine the most appropriate training topics and formats for each NYSOMH clinic • Technical assistance offered through webinars, in-person consultation, and learning collaboratives on business practices, organizational capacity, leadership support, and evidence-informed practices • Work with clinics to ascertain how adoption of a program will impact productivity and budget (e.g., billing for services) • Support continuous collection of quality improvement data on implemented programs |
| REACH Institute |
• Online capacity for low-cost training • Digital platforms for data collection and delivery of program protocols • Offer infrastructure support to providers of programs • Reduce upfront expenses for agencies as they adopt new programs |
| Washington State Institute for Public Policy (WSIPP) |
• Conduct nonpartisan, innovative economic cost–benefit analyses of evidence-based programs • Apply a consistent, cross-cutting approach to computing benefits, costs, and risk • Assess whether an evidence-based program will yield a positive return on investment, if implemented • Evaluate economic costs and benefits of implementing an evidence-based program |
Crowley, D. M., D. E. Jones, M. T. Greenberg, M. E. Feinberg, and R. L. Spoth. 2012. Resource consumption of a diffusion model for prevention programs: The PROSPER delivery system. Journal of Adolescent Health 50(3):256-263.
Hicks, D., C. Larson, D. L. Olds, and E. Johnston. 2008. The influence of collaboration on program outcomes: The Colorado Nurse-Family Partnership. Evaluation Review 32(5):453-477.
OPRE (Office of Planning, Research and Evaluation). 2014. Cross-site evaluation of SAMHSA’s Project LAUNCH initiative: Key findings in the first year. http://www.acf.hhs.gov/programs/opre/resource/cross-site-evaluation-of-samhsas-project-launch-initiative-key-findings-in (accessed June 5, 2014).
PROSPER. 2014. PROSPER vision and overview. http://www.prosper.ppsi.iastate.edu/default.asp?whatsprosper (accessed June 5, 2014).
SAMHSA (Substance Abuse and Mental Health Services Administration). 2014a. About the Strategic Prevention Framework (SPF). http://captus.samhsa.gov/access-resources/about-strategic-prevention-framework-spf (accessed June 5, 2014).
SAMHSA. 2014b. Strategic Prevention Framework. http://beta.samhsa.gov/spf (accessed June 5, 2014).
SAMHSA. 2014c. Strategic Prevention Framework State Incentive Grant (SPF SIG). http://captus.samhsa.gov/grantee/capt-clients/spfsig (accessed June 5, 2014).
Spoth, R. 2012. Moving toward population-level impact with community-based prevention: PROSPER project findings, lessons, big questions, future directions. Presentation at a National Institute of Drug Abuse, Division of Epidemiology, Services, and Prevention Research seminar session, Bethesda, MD. http://www.prosper.ppsi.iastate.edu/publications/Spoth_PROSPER_Population_Impact.pdf (accessed June 5, 2014).
Spoth, R., and M. Greenberg. 2011. Impact challenges in community science-with-practice: Lessons from PROSPER on transformative practitioner-scientist partnerships and prevention infrastructure development. American Journal of Community Psychology 48(1-2):106-119.
Spoth, R., M. Greenberg, K. Bierman, K., an C. Redmond. 2004. PROSPER community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science 5(1):31-39.
Spoth, R., C. Redmond, C. Shin, M. Greenberg, M. Feinberg, and L. Schainker. 2013. PROSPER community-university partnership delivery system effects on substance misuse through 6½ years past baseline from a cluster randomized controlled intervention trial. Preventive Medicine 56(3-4):190-196.
WSIPP (Washington State Institute for Public Policy). 2014. About WSIPP. http://www.wsipp.wa.gov/About (accessed June 29, 2014).
