National Academies Press: OpenBook

Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop (2021)

Chapter: Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts

« Previous: Setting the Stage: Evaluating Efforts to
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

3

Using Prevention Science and Implementation Science to Better Evaluate Sexual Harassment Prevention Efforts

This chapter covers the content presented on the principles of prevention science and implementation science, including how they might be applied to the evaluation of sexual harassment prevention efforts.

APPLYING PREVENTION SCIENCE

Cindy Crusto, Yale University School of Medicine, and Lisa Hooper, University of Northern Iowa, presented key tenets of prevention science and program evaluation as a proposed framework for preventing sexual harassment in higher education. This information was based on their paper outlining how prevention science has been and is currently being applied to sexual harassment evaluation in higher education, commissioned by the workshop planning committee.1 The paper also offers an organizing pre-

___________________

1 Available at: https://www.nap.edu/catalog/26279.

Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

vention evaluation framework for sexual harassment for use in diverse higher education contexts.

Hooper noted that prevention science focuses on the development, implementation, and evaluation of evidence-based programs and strategies that reduce risk factors and enhance protective factors to improve the health and well-being of individuals, families, communities, and organizations. Drawing from a diverse range of disciplines, prevention science aims to understand the determinants of societal, community, and individual level problems, such as sexual harassment. Prevention science also examines risk factors, which are associated with increasing the likelihood of developing the priority problem; protective factors, which are associated with reducing the likelihood of developing the problem; and promotive factors, which are associated with optimizing health rather than protecting health.

To select and develop a prevention framework, Hooper discussed the need to consider:

  • accumulated evidence, by identifying prevention programs and interventions with empirical support that meet the organizational need to prevent the problem;
  • conceptual fit, through a review of available programs and interventions that appear to meet the institutional need to prevent the problem; and
  • practical fit, by reviewing available programs and interventions that appear to align with the targeted population and type of institution.

Commonly used principles to guide prevention program development and process include:

  • assessing available empirical support,
  • understanding historical efforts,
  • establishing a theory of change or logic model,
  • clearly identifying the priority problem,
  • identifying risk and protective factors and prevention targets, and
  • developing short- and intermediate-term outcomes.

Hooper presented on a prevention science program development framework developed by the Substance Abuse and Mental Health Services Administration (see Figure 3-1). The agency describes two relevant guiding

Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Image
FIGURE 3-1 Prevention science program development framework.
SOURCE: Cindy Crusto and Lisa Hooper (2021). Workshop presentation. Adapted from the Substance Abuse and Mental Health Services Administration (2019).
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

principles for the framework: (1) cultural competence, defined as “the ability of an individual or organization to understand and interact effectively with people who have different values, lifestyles, and traditions based on their distinctive heritage and social relationships,” and (2) sustainability, defined as “the process of building an adaptive and effective system that achieves and maintains desired long-term results.”

Hooper also presented another prevention framework for sexual harassment (see Figure 3-2). Factors that form the foundation of the framework include a focus on well-being and a culturally responsible, safe, and supportive climate and organization. Other key features include assessment, evaluation, prevention, and response practices; leadership and organizational support; and stakeholders who are trained to be culturally responsive, among other areas.

Image
FIGURE 3-2 Prevention framework for sexual harassment.
SOURCE: Cindy Crusto and Lisa Hooper (2021). Workshop presentation.
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

The program evaluation process should begin at program development and continue through program implementation and beyond, stated Crusto. Developing appropriate outcomes is a critical part of the evaluation process. Outcomes can also reflect changes in learning, action, and condition. Not all outcomes occur at the same time, and some are necessary before others can happen. There are, for example, short-term, intermediate, and long-term outcomes.

To gain buy-in from participants during an evaluation, Crusto recommended including participants in the process from the beginning and asking evaluation questions that are important to them. The goal is to build an ongoing learning program or culture; the capacity to engage participants is key to achieving this goal. It is also critical to build in equity from the beginning of the program. Crusto added that the prevention lens can be helpful for understanding protective factors, including when to intervene, which can occur at the individual or systems level, and can help shape action on how to modify or adapt.

The Community Readiness Model was also discussed as a tool to assess whether a community is prepared to take action on an issue. Crusto described nine stages of community readiness ranging from no awareness (Stage 1) to high level of community ownership (Stage 9). There are also opportunities to “create” readiness by carrying out community training before conducting the intervention.

Crusto also presented the application of a logic model to an initiative at Rutgers University aimed at significantly increasing faculty and staff education and skill development.2 The case study captured the project’s goals, outputs, and outcomes. Considering inputs or outputs can support training that is trauma-informed and is faculty, staff, and student-focused. As was

___________________

2 Available at: https://www.nap.edu/catalog/26279.

Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

discussed, the case study clarifies the importance of clearly communicating expectations through university policies.

Regarding developing training programs, we know what works in prevention—a “one and done” model does not suffice in terms of resulting in long-term knowledge or behavior change, stated Crusto. At a minimum, booster sessions and follow-up assessment are needed over time.

APPLYING IMPLEMENTATION SCIENCE

Raechel Soicher and Kathryn Becker-Blease, Oregon State University, provided an overview of the field of implementation science, including how it can be applied to sexual harassment prevention efforts in institutions of higher education. Their presentation was based on a paper commissioned by the workshop planning committee.3 The paper includes a summary of the organizational barriers to preventing sexual harassment; defines implementation science, including a comparison to other intervention research fields; outlines a subset of the research methods, designs, and models used in implementation science; and provides examples of models of implementation science that may be relevant to evaluating sexual harassment prevention efforts. The paper also includes an overview of potential barriers and next steps for approaching sexual harassment prevention from an implementation science lens.

Soicher discussed multiple outcomes of interest related to implementation, including how they could be applied to the evaluation of sexual harassment prevention, which include acceptability, adoption, appropriateness, feasibility, cost, reach, sustainability, and fidelity. (See Appendix E for worksheets that demonstrate how these outcomes of interest can be applied to the evaluation of sexual harassment prevention efforts.) In fact, fidelity, she noted, is at the forefront of implementation science. Implementation science can be used to understand implementation processes, identify contextual influences, and assess external validity. It can also be used to identify critical components of a program and describe what factors may influence successful implementation. Implementation science design methods, including those used for within-site and between-site analyses, are also discussed in the commissioned paper.

___________________

3 Available: https://www.nap.edu/catalog/26279.

Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

One of the strengths of the implementation science approach is that it can support and facilitate evaluation and be easily adapted and applied in varying contexts. Foundational to this approach, Soicher stressed, is the need to engage the community in the evaluation. Partnerships with the community and between researchers and practitioners are critical. Soicher noted that effective practices, implementation, and enabling contexts, for example, collaborating with teams, can result in improved outcomes (see Figure 3-3).

Soicher presented the Practical, Robust Implementation and Sustainability Model (PRISM) as it applies to Vanderbilt University’s effort to alter departmental admission policies. The effort was designed to diffuse dependent relationships between graduate students and their advisors (see Box 3-1 and Figure 3-4; also see Case Study C in Appendix D). PRISM is an extension of a widely used planning and evaluation framework known as Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM). Soicher noted that this model can be used to improve implementation efforts as well as develop potential intervention and implementation questions.

Becker-Blease provided an overview of the Consolidated Framework for Implementation Research (CFIR) as applied to an initiative of the

Image
FIGURE 3-3 Formula for success and active implementation frameworks.
SOURCE: Raechel Soicher and Kathryn Becker-Blease (2021). Workshop presentation. Adapted from Fixsen et al., 2005, 2015.
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

Massachusetts Institute of Technology to offer lab-based inclusive culture workshops (see Figure 3-5 and Box 3-2; also see Case Study E in Appendix D). This framework synthesizes common constructs from across multiple implementation theories and provides a consistent taxonomy for building a knowledge base around what works where and why. The CFIR can outline detailed construct definitions and steps for research processes and consists of five major domains including:

  • intervention characteristics, or the core components and aspects which should be preserved to maintain the effectiveness of the intervention;
  • outer setting, or economic, political, and social contexts that influence implementation of and intervention within an organization;
  • inner setting, or the local culture, climate, and structure of the organization which affects implementation;
  • individual characteristics, which refer to recipients of the intervention and their knowledge and beliefs about the intervention; and
  • process, or details of the active change process.

The key challenge, stated Becker-Blease, is assessing whether gaining buy-in from faculty will ultimately have an impact on the effectiveness of the program. The framework may be used to assess training impact for this case example. (See Chapter 6 and the associated commissioned paper4 for a discussion of resources to support the implementation of this framework.)

___________________

4 Available: https://www.nap.edu/catalog/26279.

Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Image
FIGURE 3-4 Practical, Robust Implementation and Sustainability Model (PRISM).
SOURCE: Raechel Soicher and Kathryn Becker-Blease (2021). Workshop presentation. Adapted from Feldstein & Glasgow 2008.
Image
FIGURE 3-5 Consolidated framework for implementation research.
SOURCE: Raechel Soicher and Kathryn Becker-Blease (2021). Workshop presentation. Adapted from Damschroder et al., 2009, 2011.
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×

This page intentionally left blank.

Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 13
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 14
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 15
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 16
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 17
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 18
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 19
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 20
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 21
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 22
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 23
Suggested Citation:"Using Prevention Science andImplementation Science to Better EvaluateSexual Harassment Prevention Efforts." National Academies of Sciences, Engineering, and Medicine. 2021. Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26279.
×
Page 24
Next: Challenges and Limitations thatArise When Evaluating SexualHarassment Prevention Efforts »
Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop Get This Book
×
Buy Paperback | $50.00 Buy Ebook | $40.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Rising awareness of and increased attention to sexual harassment has resulted in momentum to implement sexual harassment prevention efforts in higher education institutions. Work on preventing sexual harassment is an area that has recently garnered a lot of attention, especially around education and programs that go beyond the standard anti-sexual harassment trainings often used to comply with legal requirements.

On April 20-21, 2021, the National Academies of Sciences, Engineering, and Medicine hosted the workshop Developing Evaluation Metrics for Sexual Harassment Prevention Efforts. The workshop explored approaches and strategies for evaluating and measuring the effectiveness of sexual harassment interventions being implemented at higher education institutions and research and training sites, in order to assist institutions in transforming promising ideas into evidence-based best practices. Workshop participants also addressed methods, metrics, and measures that could be used to evaluate sexual harassment prevention efforts that lead to change in the organizational climate and culture and/or a change in behavior among community members. This publication summarizes the presentations and discussion of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!