National Academies Press: OpenBook
« Previous: 6 Policy Strategies
Suggested Citation:"7 Assessing the Evidence on Interventions." National Academies of Sciences, Engineering, and Medicine. 2019. Fostering Healthy Mental, Emotional, and Behavioral Development in Children and Youth: A National Agenda. Washington, DC: The National Academies Press. doi: 10.17226/25201.
×
Page 171
Suggested Citation:"7 Assessing the Evidence on Interventions." National Academies of Sciences, Engineering, and Medicine. 2019. Fostering Healthy Mental, Emotional, and Behavioral Development in Children and Youth: A National Agenda. Washington, DC: The National Academies Press. doi: 10.17226/25201.
×
Page 172
Suggested Citation:"7 Assessing the Evidence on Interventions." National Academies of Sciences, Engineering, and Medicine. 2019. Fostering Healthy Mental, Emotional, and Behavioral Development in Children and Youth: A National Agenda. Washington, DC: The National Academies Press. doi: 10.17226/25201.
×
Page 173
Suggested Citation:"7 Assessing the Evidence on Interventions." National Academies of Sciences, Engineering, and Medicine. 2019. Fostering Healthy Mental, Emotional, and Behavioral Development in Children and Youth: A National Agenda. Washington, DC: The National Academies Press. doi: 10.17226/25201.
×
Page 174

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Prepublication Copy, Uncorrected Proofs Chapter 7 7 Assessing the Evidence on Interventions Part II of this report examines a wide range of evidence that has expanded understanding of the efficacy of many sorts of interventions designed to promote healthy mental, emotional, and behavioral (MEB) development and prevent disorder. In many areas, findings that have emerged since 2009 provide the basis for more productive directions and shifts in emphasis for policymakers and others who are tackling specific MEB-related challenges. The evidence for some interventions is quite well established. Much of it has been synthesized in meta-analyses of research including randomized controlled trials and other strong study designs, and in many cases research that has tracked potential effects across long time periods. Looking across Chapters 3 through 6, we note, for example, strong evidence for the efficacy of programs to support and promote effective parenting and family bonding, screen pregnant women and mothers for depression and offer effective treatments, and teach children from preschool through grade 12 social and emotional skills and mindfulness. Emerging evidence is the basis for recommendations regarding the integration of MEB health promotion and disorder prevention into primary health care. On the policy front, we have seen evidence that programs of long standing, such as Medicaid and the Earned Income Tax Credit, can have direct benefits for the MEB health of children and youth. For other interventions—such as parent training to reduce substance use disorders or prevent child abuse, whole-school approaches to establishing a positive school climate, or prenatal parenting education to reduce pregnant women’s substance use—researchers are still exploring mechanisms for achieving desired outcomes and other questions. Existing work, however, points to significant potential benefits. This body of evidence is foundational—the necessary basis for selecting programs for dissemination and implementation on a broad scale that are the path for achieving meaningful improvements in MEB health for U.S. children and youth. We close Part II with our reflections on the nature of the research on interventions relevant to MEB health and the broad implications of this body of work. We begin this discussion by noting that it is common to speak of the value of “evidence- based” interventions without necessarily defining what “evidence” means in general or in specific contexts. The research reviewed in Part II on the primary points of access for improving MEB health—families and caregivers, schools, the health care system, and government policy— includes randomized controlled trials, quasi-experimental studies, and qualitative studies. We have described many individual studies to illustrate the sorts of interventions that are available and some of the ways researchers have tried to identify the components of effectiveness. Where large-scale studies with strong experimental designs or meta-analyses are available, we have highlighted those studies, and we have also highlighted available meta-analyses. Evidence is often reported in terms of effect size, reflecting the degree to which a hoped- for outcome was achieved. Looking across the research we have reviewed, it consists primarily of efficacy studies (which examine whether an intervention can produce the intended result when   171

Prepublication Copy, Uncorrected Proofs Chapter 7 the study circumstances are tightly controlled) and limited numbers of effectiveness studies (which examine whether an intervention can produce the intended result in more complex, real- world circumstances) (Gartlehner et al., 2006). In efficacy and effectiveness studies related to MEB outcomes, effect sizes vary, but average approximately 0.3. A robust effect size would be 0.5 or above, which means that many study participants did not show an expected benefit, even when the outcome for the study group was statistically significant. Effect sizes are usually lower in effectiveness and dissemination than in efficacy studies, which suggests that in scaled interventions, expectations for effect sizes should be relatively low, even if the intervention is faithfully implemented. Obviously, effect size is an important consideration when selecting an intervention for implementation at a population level, but other factors are also important. One is that for many reasons, people eligible to participate in an intervention often decline to do so. For example, refusal rates for home visiting can be as high as 50 percent, and those who agree to participate are often those who least need the intervention (Ammerman et al., 2015). Most reports of outcomes from efficacy and effectiveness studies do not include data on enrollment rate, although this situation may be changing. To the extent that rates of program dropout are reported, they are also high, as discussed in Chapter 8. There is little available data about outcomes when participation is limited. In practice, an intervention that has a favorable effect size of 0.5 in efficacy studies, participation and retention rates of 50 percent each, and an effect size in effectiveness studies that is 75 percent of that found in efficacy studies, the calculation might actually show benefit for fewer than 10 percent of families or children who were projected to benefit. These considerations mean that it is important also to report and consider the number of eligible participants who enroll in a study, the number of enrollees who complete the study, and what the benefit might be for partial participation, as well as other factors that affect the potential for successful implementation, such as program complexity, cost, and the infrastructure and workforce needed to create and sustain the program. If these factors were routinely reported and discussed in published studies, it would be easier to consider them carefully when planning scale-up, organizing quality improvement efforts to achieve best outcomes, and assessing the population-level value of an intervention. For example, benefit/cost calculations have been based on outcomes of efficacy studies and have not routinely factored in these considerations; they therefore may overestimate potential benefits. We also note that many randomized controlled trials of the sorts of interventions discussed in this report have a risk of bias because the study conditions are not blinded, or for other reasons (e.g., Allen et al., 2016; Yap et al., 2016). Blinding is not always feasible in the context of intervention trials, particularly those delivered in real-world settings. It is also worth noting that strictly controlled laboratory experiments often lack generalizability to real-world settings (external validity), whereas intervention trials conducted in unmodified settings (pragmatic trials) that serve children and families can provide valuable information to support implementation and dissemination. This discussion of methodological issues reveals not weaknesses in the interventions studied but limitations on the certainty provided by even the most rigorous research because of the complexity of factors that influence developmental outcomes for children and youth. In short, it is important to look beyond the expectation that “evidence-based” programs, as defined in the past, if widely adopted, can achieve the desired benefits for all or even a substantial proportion of children in need.   172

Prepublication Copy, Uncorrected Proofs Chapter 7 Without a doubt, research is needed to continue building on this foundation. But the problem that has kept a body of knowledge that was already strong 10 years ago from bringing meaningful improvement at the population level, in the committee’s judgment, is not with whether and how interventions work in an experimental setting, but with how to effectively reach and benefit children and families who could benefit at a population level, the subject of Part III.   173

Prepublication Copy, Uncorrected Proofs Chapter 7 REFERENCES Allen, M.L., Garcia-Huidobro, D., Porta, C., Curran, D., Patel, R., Miller, J., and Borowsky, I. (2016). Effective parenting interventions to reduce youth substance use: A systematic review. Pediatrics, 138(2). Ammerman, R.T., Altaye, M., Putnam, F.W., Teeters, A.R., Zou, Y., and Van Ginkel, J.B. (2015). Depression improvement and parenting in low-income mothers in home visiting. Arch Womens MentArchives of Women’s Mental Health, 18(3), 555-–563. Gartlehner G., Hansen, R.A., Nissman, D., Lohr, K., and Carey, T.S. (2006). Trials in Systematic Reviews. Rockville, MD: Agency for Healthcare Research and Quality. Available: https://www.ncbi.nlm.nih.gov/books/nbk44029/. Yap, M.B.H., Morgan, A.J., Cairns, K., Jorm, A.F., Hetrick, S.E., and Merry, S. (2016). Parents in prevention: A meta-analysis of randomized controlled trials of parenting interventions to prevent internalizing problems in children from birth to age 18. Clin Psychol Rev, 50, 138-158.   174

Next: Part III: Implementation and Scale-Up of Effective Interventions »
Fostering Healthy Mental, Emotional, and Behavioral Development in Children and Youth: A National Agenda Get This Book
×
Buy Paperback | $75.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Healthy mental, emotional, and behavioral (MEB) development is a critical foundation for a productive adulthood. Much is known about strategies to support families and communities in strengthening the MEB development of children and youth, by promoting healthy development and also by preventing and mitigating disorder, so that young people reach adulthood ready to thrive and contribute to society. Over the last decade, a growing body of research has significantly strengthened understanding of healthy MEB development and the factors that influence it, as well as how it can be fostered. Yet, the United States has not taken full advantage of this growing knowledge base. Ten years later, the nation still is not effectively mitigating risks for poor MEB health outcomes; these risks remain prevalent, and available data show no significant reductions in their prevalence.

Fostering Healthy Mental, Emotional, and Behavioral Development in Children and Youth: A National Agenda examines the gap between current research and achievable national goals for the next ten years. This report identifies the complexities of childhood influences and highlights the need for a tailored approach when implementing new policies and practices. This report provides a framework for a cohesive, multidisciplinary national approach to improving MEB health.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!