7
Design and Delivery of Services
Key Points Raised by Individual Speakers
• Effective parenting interventions are available that can reduce child maltreatment, but they need to be more broadly disseminated and implemented.
• Similarly, great strides have been made in developing treatments for children who suffer from abuse, violence, and neglect, but a large research-to-practice gap continues to hinder the delivery of evidencebased treatments.
• Research has demonstrated the value of brief, single-focus techniques even with multiproblem families, which are the norm rather than the exception. These brief, focused interventions may or may not be less effective than more intensive approaches that seek to address all problems present, but are likely to reach more at-risk families.
• Research on the dissemination and implementation of evidencebased practices can help convert new understandings to interventions that can change lives.
The session on the design and delivery of services at the workshop picked up on many of the themes from the previous session on prevention. Multiple effective service models exist, many of which are adapted from other service sectors (e.g., services to help parents of children with specific behavioral disorders). But a variety of impediments keep evidence-based treatments from being widely used. Research therefore needs, several speakers said, to examine the dissemination and implementation of evidence-based treatments even as it continues to examine the evidentiary basis for those treatments.
A multitude of effective preventive and treatment models exist for parenting interventions, said John Landsverk, director of the Child and Adolescent Services Research Center at Rady Children’s Hospital in San Diego. The primary problem is the translation of those interventions into child welfare service systems and other systems such as child mental health care.
Effective parent-mediated interventions have many elements, including nonharsh methods such as timeouts, consistent consequences for behavior problems, homework, and in vivo practice. Also, a wealth of data is available to support these interventions. A meta-analysis of child psychotherapy trials between 1963 and 2002 grouped trials by issue, finding 94 publications focused on anxiety or fears, 23 dealing with depression, 135 on conduct-related disorders and problems, and 35 on parent-focused or parent-mediated interventions (Weisz et al., 2004). Many issues have been well studied using randomized controlled trials, said Landsverk.
Mental Health Services for Children
Traditionally, child welfare is organized around three mission elements: safety, permanence or stability, and well-being. Child welfare systems typically have been comfortable taking responsibility for the first two, but considerable ambivalence surrounds well-being. It is the most difficult to assess. It also is seen as requiring expertise and resources of sister agencies such as child mental health, developmental services, health, and education.
Parenting interventions can be focused either on the abusive and neglectful behaviors that put children at risk or on the behavior resulting from abusive and neglectful parenting behaviors, said Landsverk. In the latter category, not much diagnostic information exists about externalizing problems. But a San Diego study of more than 400 children dependent on the child welfare system found 42 percent with a diagnosis of moderate impairment, including attention-deficit/hyperactivity disorder (ADHD) and oppositional defiant disorder (Garland et al., 2001). Similarly, a National Survey of Child and Adolescent Well-Being (NSCAW) study on mental health service use found that 45 percent of the sample population met Child Behavior Checklist criteria for behavior problems (Orton et al., 2009).
Current data show that the deeper children get into the child welfare system, the more likely they are to be referred to specialty mental health
services. Even when the system just investigates a family, the children are more likely to end up with a referral. Moreover, NSCAW data show no sharp drop-off in mental health services once children leave the welfare system, contradicting the hypothesis that there is little continuity of care.
Children suffering from neglect are less likely to be referred to mental health services than those in sexually or physically abusive homes, even if they present with behavioral and conduct problems, Landsverk said, suggesting that bias or assumptions sometimes outweigh clinical decisions when referring children for care. Neglect is rarely present without harsh parenting, which suggests that parenting interventions developed for externalizing behaviors could be applied to neglect situations. “There is now experimental evidence and conceptual evidence to change our thinking about the viability of parenting interventions for neglectful behaviors,” Landsverk stressed.
Future Opportunities for Service Design and Delivery
Many parenting interventions borrowed from other disciplines have strong evidence of effectiveness. They are appropriate for different ages of children, problems of varying severity, and diverse populations. “We don’t so much need more interventions,” Landsverk pointed out. “We need to know how to move them and place them in service systems that can pay for them and deliver them in an effective way. It becomes a problem of dissemination and implementation.” Borrowing from other disciplines is a wonderful resource, but it creates the challenge of fitting those interventions into the welfare system, when the interventions were not designed with that framework in mind.
The use of these interventions to deal with neglect poses some special problems, but a body of recent work has demonstrated a variety of ways to approach the problems caused by neglect. “Parenting interventions that were developed for externalizing behaviors may now be a real possibility for use with the largest population in child welfare—namely, neglect.”
Child welfare managers need decision tools that they can use to select age-and condition-appropriate parenting interventions and link parenting intervention outcomes to child welfare outcomes. In addition, parenting interventions should be extended downward in age, Landsverk said, with adaptation that makes them suitable for parents with younger children.
Finally, a number of promising dissemination and implementation studies are under way, some with promising published results where well-being outcomes are associated with the safety and permanence outcomes at the core of the child welfare mission. This dissemination and implementation research should continue to be emphasized, Landsverk said. (Dissemination and implementation studies are discussed in greater detail in the next chapter.)
Since the 1993 NRC report, great strides have been made in developing treatments for children who suffer from abuse, violence, and neglect, said Shannon Dorsey, assistant professor in the Department of Psychiatry and Behavioral Sciences at the University of Washington. Many therapies can be delivered in 12 to 20 sessions in a range of settings. Most draw on cognitive behavioral therapy and are effective as both group and individual approaches. Research supports the effectiveness of these interventions across cultural and ethnic groups and demonstrates that, in most cases, the evidence-based approach works better than usual care. Sources for learning about effective interventions include the CEBC and the Kauffmann Best Practices Project to Help Children Heal from Child Abuse.
Cognitive behavioral therapies tend to be effective for PTSD, depression, and anxiety, Dorsey said. Adolescent depression also responds well to Interpersonal Therapy. Many name-brand approaches target only one disorder, she pointed out. “But when you think about kids exposed to child abuse and neglect, comorbidity is more the rule than the exception.” A modularized approach is frequently needed to address the issue of children with multiple disorders. (Children with multiple disorders are discussed later in this chapter.)
An increasing amount of data is also available on psychopharmacological approaches, which can work in concert with evidence-based psychotherapies. A Treatment for Adolescents with Depression Study that was focused on severe refractory depression found a combined approach to be more effective than therapy or medication alone, though Dorsey pointed out that children with comorbidities may not have made it into the trial and other studies have found cognitive behavioral therapies to be more effective than medication. Less evidence exists for the effectiveness of medication with children suffering from PTSD, complex trauma,
or emotional dysregulation. With younger children, the goal is to avoid medication.
From Research to Practice
Repeating a point made by Landsverk, Dorsey emphasized that there is a large research-to-practice gap with evidence-based treatments. Efficacy trials show large effect sizes, compared to effect sizes near zero for some usual care practices, but implementation proceeds too slowly. The availability of evidence-based practices for children also is lacking, partly due to the slow acceptance and spread of the methods. “Training in graduate school isn’t necessarily consistent with the types of interventions that we teach when we think about evidence-based therapies or evidence-based practices,” Dorsey said. In contrast, she pointed to a growing body of work in low-and medium-income countries teaching counselors with little or no training to deliver evidence-based interventions. “They need more training and more supervision, but there is less resistance to this type of approach.” Some schools, such as the University of Maryland and University of Washington, are changing their curriculums and training graduate students to be open to these types of interventions.
Part of the problem, Dorsey pointed out, is a lack of incentives for delivering certain types of services. The system rewards any kind of treatment, regardless of type. “Currently we are spending a lot of money on interventions that don’t have a lot of evidence of working at all. The cost of doing business the way we are is pretty high.”
The system also lacks ways to link adolescents to evidence-based treatments. Child welfare does not mandate evidence-based interventions, and often child welfare workers choose services without much prior information about evidence-based practices. She lamented the lack of training and lack of monitoring to ensure the use of the best available therapies. New approaches need to monitor both outcomes and the adherence to effective therapies, she said.
A small randomized controlled trial in Washington state, which trained case workers in how to refer children to evidence-based practices, saw increased awareness as workers became more familiar with the options available to them (Dorsey et al., 2012). Referrals did not necessarily increase, but the study provided information on how to train brokers to make referrals in light of evidence-based case planning.
Even if new interventions are not necessary, the field needs more research on the effectiveness of interventions and on borrowing interven-
tions from other areas. Existing evidence-based interventions for children with high emotional dysregulation can be restrictive, and children often end up in residential or inpatient settings where little data exist on the use of effective practices. Also, the use of less-intense interventions could benefit children with high dysregulation, Dorsey said. Dialectical behavioral therapy (DBT), which is used in some programs, shows promise, but there are no trials of DBT with adolescents. For young children, most interventions are parent-mediated, but strong evidence supports the effectiveness of programs like Attachment and Biobehavioral Catch-up and Multidimensional Treatment Foster Care for Preschoolers (MTFC-P).
Dorsey also pointed to the need for improving the provision of services in residential and inpatient settings and for transitioning children from foster care to home placement when they are receiving intense therapies. Also, reliable methods are needed for getting services to children with subclinical levels of internalizing disorders such as PTS who may not be able to get coverage through Medicaid.
Child abuse and neglect require comprehensive approaches, she said. “How do we make sure that families and children get validation, acknowledgment of what happens, psychoeducation, and support for parents … in an intervention of a limited nature—only 1 to 3 hours?” This type of comprehensive service has no obvious provider. Child advocacy centers might be a reasonable place to start, but they are not available everywhere. Technology-based approaches and solutions outside traditional delivery models could provide answers, Dorsey suggested.
Trauma-informed systems could be useful as well, but it is important to be clear about what those systems need to do for children and adolescents. Screening, identification of children with externalizing and internalizing disorders, and possibly a comprehensive response are all functions that could be expected of these systems. But trauma-informed systems need to move beyond educating people that trauma is common and has severe consequences for development and brain functioning, to include screening, feedback on screening results, referral, and—when appropriate—treatment provision.
Future Research on Service Design and Delivery
Dorsey addressed several problems in existing research. She called for improvement of sample issues in medication studies and oversight of prescribing practices for psychotropic drugs. In addition, she pointed out that more research on treating grief and loss in children who are cut off
from their parents due to termination of parental rights would be beneficial.
She echoed other presenters in calling for better use of short-term, evidence-based interventions rather than process-oriented, extendedduration approaches. Research that can examine how to implement evidence-based treatments in community settings is also crucial.
Discussion
Osofsky brought up the issue of support and supervision for counselors, who deal with difficult and draining situations. Dorsey suggested that counselors be trained in exposure, cognitive reprocessing, and other skills for addressing anxiety disorders and trauma.
Angela Diaz from the Mount Sinai School of Medicine pointed out that many children have a history of abuse that is not known to the welfare or legal system, but is often discovered by primary care providers. Lucy Berliner from the University of Washington said that such cases need to be validated, acknowledged, and provided with opportunities for treatment. Dorsey added that parents do not need to be as involved for internalizing disorders, so children who do not want their parents to be part of the process, or are estranged from their family, can still receive services that will improve their situation.
FAMILIES DEALING WITH MULTIPLE PROBLEMS
Multiple problems are the norm in child maltreatment cases, said Steven Ondersma, clinical psychologist and associate professor in the Department of Psychiatry and Behavioral Neurosciences of the Wayne State University School of Medicine. The question is what to do about it.
The assumption that practitioners must make progress in all areas to have any effect “is almost a guaranteed way of inducing hopelessness,” he said, “if not in ourselves, then certainly among folks on the front line.” Child welfare workers feel they can never provide enough highquality services and see families as overwhelmed with everything they are asked to do.
Clinical trials have demonstrated the success of some multifocal treatment plans, he acknowledged. “I am not going to make the point that this assumption that doing things together in an integrated way is always wrong, because it is not. I am going to make the point that perhaps it is
not always right, and we need to attend to that and think carefully about it.”
Integrated Versus Nonintegrated Treatment
A meta-analysis looking at integrated and nonintegrated treatment found no difference in their effectiveness in treating substance use, depression, and other issues in people with co-occurring disorders (Tiet and Mausbach, 2007). This study also demonstrated that focusing on one disorder in individuals with co-occurring disorders was just as effective at treating that disorder as with a person only suffering from a single disorder. A trial on Seeking Safety, a popular cognitive behavioral approach for women with a history of trauma and substance use, showed no difference between women assigned to that approach or a women’s health education group (Hien et al., 2009).
His own failure in this area was instructive, Ondersma added. When he and two colleagues ran a demonstration program for mothers of drugexposed infants designed to include every possible treatment option, they found no association between program participation and outcome (Mullins et al., 2005). The do-it-all approach derives from the best of intentions, he pointed out, but it does not always work.
Future Opportunities: The Promise of Brief Interventions
“We have to think carefully about what we are doing and what we are getting for it,” Ondersma stated. Researchers tend to focus on treatments with the largest possible effect size. But the community may be better served by a stronger emphasis on distribution and interventions designed for ease of implementation. “If we want to have a broad effect, we have to start thinking more about starting with reach and then, within that constraint, making something as efficacious as we possibly can.”
In this respect, findings that brief and single-focus techniques can work well with multiproblem families are promising, said Ondersma. The Family Check-Up is one example. It shows excellent effects on child externalizing problems with a couple of sessions at birth, a session at 12 months, and another at 24 months. The SEEK program by Dubowitz et al. (2009) is another good example of a brief approach with good results. A meta-analysis by Bakermans-Kranenburg and colleagues (2003) showed that interventions on parental sensitivity and child attachment had better results when families received less than 5 sessions and worse results when the number of sessions went above 16. Indeed, Ondersma
successfully replicated a study using a 20-minute computer-delivered intervention with high-risk parents to decrease postpartum drug use.
“Random assignment to brief interventions very often yields similar effects as to the more extended interventions,” he said. Equally interesting, the effects of a brief intervention appear to be more pronounced among people with more severe substance use disorders.
The evidence suggests that parenting interventions may be sufficient even for families with multiple risk factors, Ondersma said. “When cooccurring risks are present, we should think carefully about the possibility of brief interventions with a single focus.” Stepped and long-term episodic approaches, such as recovery management checkups and motivational checkups used in the substance abuse field, could also be a useful tool. Judicious use of technology also could greatly improve reach.
In the future, Ondersma concluded, researchers should pay more attention to reach, which demands consideration of nontraditional approaches. In addition, research on stepped, sequential care and proactive identification of families at risk should be a high priority.
Discussion
Given that effective interventions exist but are not widely available, asked Lucy Berliner from the University of Washington, who moderated the session on the design and delivery of services, how can attitudes and outlooks that are impeding the use of evidence-based practices be changed? “Part of the answer is designing things differently,” Ondersma replied. Interventions need to be “designed from the start to be implementable given the system that we have.” Technology is also an important tool, he added, both for training service providers and for delivering services.
Landsverk pointed out that commercializing interventions makes them difficult to modify. “There’s something about product development that gets in the way of doing lots of really interesting adaptations.” He added that targeting services correctly with regard to developmental stage could help conserve resources while producing good results. Finally, he suggested an incentive system for developing treatments that function with the current delivery system. Dorsey agreed, adding that efforts to create change especially need to focus on graduate schools and on providers.
IMPLEMENTATION OF EVIDENCE-BASED PRACTICE
In his review of dissemination and implementation research, Gregory Aarons, a clinical and organizational psychologist and professor of psychiatry at the University of California, San Diego, School of Medicine, began with several definitions. “Dissemination is the targeted distribution of information and intervention materials to a specific public health or clinical practice audience. The intent is to spread knowledge and the associated evidence-based interventions” (HHS, 2010). He defined implementation as the use of strategies to introduce or change evidencebased health interventions within specific settings. Scale-up is a type of implementation with the goal of spreading evidence-based practices broadly across a service system. Finally, sustainment is the continued use of an evidence-based practice with fidelity and with sufficient spread for public health impact.
A Conceptual Model for Implementation
Bearing these definitions in mind, Aarons presented a conceptual model that illustrates the complexity inherent in implementation. The model divides implementation into four phases: exploration, adoption decision and preparation, active implementation, and sustainment. All of these phases must consider both the outer context and the inner context. Outer context is sociopolitical and involves such factors as funding, legislation, and interorganizational relationships within a service system. Inner context encompasses organizational characteristics such as openness to evidence-based practice, skills and expertise, and goals. Both of these contexts can either support or hinder implementation.
The exploration phase involves consideration of implementing an evidence-based practice and about the “fit” of a given evidence-based practice with outer context (i.e., service system) and inner context (e.g., service organizations, providers, and clients). Once an adoption decision has been made, planning for implementation begins. This phase entails consideration of what factors in the outer (e.g., policies, funding, leadership) and inner contexts (e.g., organizational leadership, workforce) need to be in place prior to implementation. Active implementation involves moving an evidence-based practice into the field and, in the process, problem solving unanticipated issues that may arise. Finally, the sustainment phase is characterized by having the appropriate supports in place to maintain continued use of the evidence-based practice with fidelity.
Examples of Dissemination and Implementation
Aarons used this model to explore dissemination and implementation experiences with several specific programs. One was a statewide implementation trial in Oklahoma of the child neglect intervention SafeCare (Aarons et al., 2009). With support from the National Institutes of Health, Aarons and colleagues investigated such factors as program impacts on the workforce, the therapeutic process, the relationship of the case manager with clients, and organizational processes. “What implementation research tries to do is say that these things that are nuisance variables, we want to take them head on and understand what the concerns are,” Aarons said. “Then we want to design future studies to go in and really improve those processes.”
As an example of this work, Aarons described a finding that workforce retention was increased by the program. This result was unexpected because the program had the effect of reducing job autonomy. “I’ve never been happier to be wrong.” Those doing the evidence-based practice were more likely to stay in their organizations, at least partly because of less stress and burnout, according to Aarons. A follow-up study is now looking at the effects of team stability on team climate.
Another study involved an intervention in rural Appalachia known as Availability, Responsiveness, and Continuity (ARC) that works to improve the culture and climate that children’s services workers experience (Glisson et al., 2010). The study looked at four conditions: usual care, the ARC organizational intervention only, multisystemic therapy (MST) only, and ARC combined with MST. The study found that ARC alone and MST alone lead to decreased placement changes. It also found that youth receiving MST plus ARC entered out-of-home placements at a significantly lower rate (16 percent) than youth in the control condition (34 percent) and had better 6-month behavior problem outcomes. “This is an example of going in and trying to affect the organizational inner context prior to and during the active implementation phase,” said Aarons.
Aarons also mentioned several studies that are in earlier phases. In one—a program to scaleup SafeCare in San Diego County—teams of staff from different agencies are working together to implement evidencebased practice. The San Diego County child welfare system worked with the United Way to develop a seed team trained in SafeCare, which allowed the program to move away from reliance on the National SafeCare Training and Research Center. The structure allows new staff and replacement staff to be drawn from multiple organizations, which improves the fit with the SafeCare model and provides for local and ongoing quali-
ty assurance. It also may create a pathway for people to gain expertise, become coaches, and expand the program. “The idea is to spread that expertise efficiently throughout the service system and then support it with the seed team. This is ongoing now, and we will be following these teams for the next 4 years to see if they maintain fidelity and if we get successive reductions in child maltreatment reports and recidivism,” said Aarons.
Future Opportunities to Enhance Dissemination
and Implementation
Both the outer system and the inner organizational context need to be improved to enhance receptivity to evidence-based practice, said Aarons. In particular, understanding of evidence-based practices can be increased in both the outer and inner contexts.
Stakeholder collaboration and partnerships also need to be maximized to support the implementation of evidence-based practice, as does leadership coordinated across the outer and inner contexts.
Methodological innovation in research design and in the methods and measures of implementation is needed. Examples include roll-out designs, system dynamics, network analyses, decision science, and developmental measures of implementation climate. Technological innovations also can serve as implementation methods.
Finally, models of sustainment need to be developed and tested. “Once we have the practice in [i.e., implemented], what do we need to know to effectively sustain evidence-based practice in the outer and inner contexts of child welfare?” Aarons asked.
