Communication and Dissemination Plan
Evaluation and Refinement Plan
The committee’s statement of task included developing a plan for communicating and disseminating the framework proposed in this report and specifying a plan for evaluating and refining the framework in the context of current decision-making processes (see Chapter 1). As a result of its deliberations, the committee determined
that the plans for communication and dissemination and for evaluation and refinement specified in its statement of task would best be developed in the context of other aspects of the framework’s practical application as defined by its potential users—decision makers and researchers—who will identify, evaluate, generate, incorporate, or use evidence to inform their selection, implementation, and evaluation of obesity prevention efforts. To guide future efforts, this chapter outlines key considerations for the development of these plans.
COMMUNICATING AND DISSEMINATING THE FRAMEWORK
If the L.E.A.D. framework is to be communicated and disseminated successfully to its target audiences, three major objectives need to be met. First, an environment has to be created in which the framework and its elements are known and understood. Second, a culture of shared beliefs about what constitutes evidence and how it ought to be used to make decisions should be created in accordance with the framework. Finally, to develop support for implementation of the framework, strong champions need to be recruited to lay the foundation for its integration into public health decision making and related research. To develop a communication and dissemination plan that meets these objectives, it will be necessary to define and learn about the target audiences, explore the channels that are best suited to reaching those audiences, identify potential partnerships, determine the messages to be delivered and the materials that will best convey them, and decide how and when to implement the plan and how to monitor its implementation (NCI, 2008).
The target audiences for the L.E.A.D. framework are decision makers (e.g., policy makers and their intermediaries, program planners, and practitioners) and researchers working on obesity prevention and other complex public health problems. Research funders and publishers of research results are also important audiences for the framework.
Decision Makers and Intermediaries
A variety of interested individuals and organizations help identify public health issues and potential strategies for addressing these issues through policy decisions. They include public health departments; local, state, and federal policy makers; health plans; employers; and professional and trade organizations. They also include the staff of decision makers, consultants, and firms that specialize in gathering and synthesizing research, as well as academic and private foundation policy think tanks and advocacy and stakeholder groups. These intermediaries may be involved in or belong to professional organizations and associations and network with those who have simi-
lar professional interests (e.g., employee business groups; state, local, or community legislation–based groups). They vary considerably in ho, why, and when they use and incorporate evidence to help them develop strategies for addressing a particular problem.
Evidence is generated by individuals and organizations that conduct research. Researchers do their work in both the public and private sectors, including agencies within local and federal government, academia, foundations, and industry. Many are affiliated with the setting where they perform the research and use their work-affiliated networks, as well as professional organizations and associations to which they belong, to obtain and circulate information. The research they generate is influenced by the resources available to them or the resources they themselves can garner to support their work.
Other Important Audiences
Research Funders. Research is supported by funders that supply the necessary equipment, money, facilities, tools, and other materials. Funders of research also exist throughout the public and private sectors. The main supporters of research in the public sector include local and federal government agencies that focus on public health issues. These agencies, as well as large organizations in the private sector, support the majority of public health–related research. Their decisions to support research on particular topics or groups are based on a variety of protocols and policies their leaders have developed to meet their organizations’ objectives. State or federal legislatures may mandate that certain agencies produce reports on topics of particular importance to their constituencies and may have specific restrictions on how and for what purpose the grants they offer can be used. However, many of the factors that affect decisions to support research are not apparent. For example, private organizations may have policy restrictions on the research they will support, but also have the flexibility to change the focus of their support to meet the needs of their targeted communities or staff members.
Many federal, state, and local government staff members belong to and attend the meetings of a number of field-specific professional organizations. They may also network among themselves through interagency task forces and with the rest of the community through regular meetings with their grantees. A number of private organizations and industry groups are members of or support coalitions and associations as well. For example, the Healthy Eating Active Living Convergence Partnership (HEALCP) comprises a number of private organizations and a federal agency that have come together “with the shared goal of changing policies and environments to better achieve the vision of healthy people living in healthy places” (HEALCP, 2008).
Publishers of Research Results. Once the research is complete, its results are published by various public and private entities. Publishers of research results play an important role in determining what information becomes available; without a venue for publication, researchers’ findings and conclusions may not reach the public. A number of public and private organizations and academia help distribute the results of research studies, including the funders of the research or those with an interest or specialization in the topic. Peer-reviewed journals, professional organizations (and their associated publications), foundations, media, industry, and the public sector (local, state, and federal government) all publish research results. Some apply a strict set of policies to establish standards for the research they publish, while others may not have such policies in place. Additionally, publishers establish the accessibility of the research they publish. They determine whether it is free to the public, whether it costs a nominal amount, or whether only their associated members can access it. Before the advent of the Internet and powerful search engines, research reached the public primarily through the medium of scientific journals. Now, web-based venues are accessible to search engines, and researchers have several channels through which to communicate their findings. However, peer-reviewed journals have much more credibility, at least to scientists and decision makers, than other Internet sources.
A variety of channels can be used for communicating and disseminating the L.E.A.D. framework, each of which has both benefits and drawbacks. For example, websites can reach a large number of people but require maintenance over time.
Individualized approaches may be necessary to help meet the objectives of the communication and dissemination plan. Although the time and resources required for these approaches may be prohibitive, individualized messaging and materials are among the most effective ways to affect attitudes, skills, and behavior and are more likely to be trusted and influential than other information channels (NCI, 2008). Key researchers with a major influence on the practice and policy environment should be identified and individually targeted to gain their support for integrating the framework into their current practices. For example, informal discussions and training sessions could be held for these key audiences.
At the same time, organizational-level approaches will be required to help meet the objectives of the framework’s communication and dissemination strategy. Established meetings and conferences are a familiar and trusted source of information, training, and best practices for many members of the target audiences for the framework. This venue can reach a large audience in one place cost-effectively. For example, each year groups such as the U.S. Centers for Disease Control and Prevention (CDC) and the National Chronic Disease Directors sponsor meetings that are attended by key personnel from state programs. These meetings often offer training sessions for public health professionals (including program and/or evaluation staff), partners, and stake-
holders and would be ideal venues for communicating the framework. Additionally, research funders and decision makers may convene or sponsor conferences and workshops to help disseminate research findings and identify priority areas for future research. Conferences and workshops that are sponsored by well-respected supporters of particular fields of public health are especially well attended. These meetings may be either open to the public (usually government-related), attended by affiliated members of the sponsoring organization (trade or professional), or attended only by selected stakeholders invited by the sponsoring organization. Journal editors may attend such meetings as well to identify research results they want to publish and to contact the researchers about submitting an article.
Writing articles for professional peer-reviewed journals or newsletters is another way to reach a large proportion of the framework’s target audiences. Although the writing and publication process may be time-consuming, using such trusted sources of information to convey the messages of the framework can be particularly influential. Another approach to delivering the messages of the framework is through funders of obesity prevention research, which can support the use, understanding, and evaluation of the framework by incorporating it into requests for proposals.
Although individual and organizational channels may be more effective, mass media channels should also be incorporated into the communication and dissemination strategy for the L.E.A.D. framework. Through the use of interactive media such as websites, webinars, CD-ROMs, and e-mail mailing lists, it is possible to reach a large number of people quickly. Through the development of interactive and graphically appealing materials, messages can be tailored to meet the needs of and appeal to multiple members of the framework’s target audiences. On the other hand, these channels can be expensive to develop and maintain over time; effort is also required to reach an audience with these messages, instead of depending on the audience to search actively for them.
Working with other organizations can be a cost-effective way to extend the reach of the framework, enhance the credibility of its messages, and create support for implementing and sustaining the concepts it embodies. Through such partnering, it is possible to tap into an organization’s resources, expertise, and networks. However, it is important to develop a specific plan with potential partners to ensure that all involved have the same goals and to specify the role each partner will take in communicating and disseminating the framework.
Several public–private partnerships specifically address obesity prevention–relate research and activities. They include HEALCP and the National Collaborative on Childhood Obesity Research (NCCOR). Knowing the mission and objectives of such collaborations can help in identifying their potential role in the communication and dissemination of the framework. HEALCP is a collaboration of funders whose goal
is to change policies and environments to achieve healthy people and healthy places, while NCCOR is focused on improving the efficiency, effectiveness, and application of obesity research (specifically for children) (HEALCP, 2008; NCCOR, 2009). Such existing collaborations are important sources of support for integrating the framework into public health practices.
Individually, public and private organizations and agencies also have existing networks and expertise that may be important sources of partners for communicating and disseminating the framework. For example, the scope of work in a number of federal agencies within the Department of Health and Human Services and the Department of Agriculture includes support for research on a wide range of public health issues, including obesity prevention. In the private sector, a number of large funders and professional organizations increasingly focus on strategies and policies to prevent obesity in specific demographically defined groups, including children and low-income families. Partnering with public and private organizations and individuals that have supported other multifaceted public health initiatives, such as smoking cessation programs, may also guide or support the crafting of effective communication and dissemination strategies for the framework.
Messages and Materials
The content of messages about the framework—and therefore the materials used to communicate those messages—should take into account the objectives of the communication and dissemination plan, the target audiences, the possible channels for reaching and engaging those audiences, and the resources and goals of potential partners in the communication and dissemination process. The messages delivered should reflect an understanding of the framework’s purpose, sensitivity to the current decision-making environment, and understanding of the importance of integrating the framework into current practice. Some materials may be detailed enough to serve as a reference (e.g., this report). Others may briefly underscore the major messages of this report (e.g., its summary or a general report brief). Other materials can target a specific audience; these include targeted report briefs, presentations, training materials, and webinars. These different types of materials can each be customized for a specific audience (e.g., researchers, decision makers, or a general audience) with a specific message (e.g., increase the knowledge base) through a specific channel (e.g., conference, presentation).
Implementation and Monitoring
A detailed implementation plan is essential for effective communication and dissemination of the framework. For example, it is important to indicate how and when resources will be needed, when specific events will occur, and how the communication and dissemination efforts will be assessed. It is also important to determine whether messages are being delivered appropriately, effectively, and efficiently; whether
materials are being distributed to the right people and in the right quantities; and whether the intended activities are occurring. Such “process evaluation” helps identify needed corrections to the messages or materials and determine whether the plan is meeting its objectives.
Specific measures can be used to monitor the implementation of the communication and dissemination plan. Such measures could include the following questions: Is the framework or discussion of the framework appearing in peer-reviewed journal articles, grant applications, meeting or conference proceedings, or websites? If so, which ones, and what is their audience? What organizations or associations are using the framework, how are they using it, and who is in their audience network? In what current decision-making activities is the framework being used? How is it being used in the public policy arena? Answers to these and similar questions can be used to modify the messages and materials employed and identify the settings, channels, networks, and members of the target audience that have not been reached.
Finally, to implement and monitor the communication and dissemination plan, it may be necessary to develop a depository, or virtual storehouse, to help track relevant events. It may also be helpful to convene an advisory committee at regular intervals to review the progress being made and communication efforts and their audiences.
EVALUATING AND REFINING THE FRAMEWORK
Changes that occur or are suggested as a result of implementing the framework need to be measured and evaluated to determine how well the framework is meeting its objectives in supporting evidence-generation and decision-making processes and how it could be improved, refined, or integrated to make it more effective.
The evaluation and subsequent refinement of the framework have three major objectives. First, it is important to assess the acceptance of the framework in the current decision-making environment by measuring its adoption and utilization by the target audiences. Assessing the acceptance of the framework will help determine the degree to which the communication and dissemination plan has achieved its objectives. Second, it is important to measure the impact of integrating the framework into the current environment by examining its effect on and thus its relevance to the generation of research and public health policies and programs. Finally, it is essential to sustain the use of the framework in current and future evidence-generation and decision-making activities. The results of these evaluations should be used to refine and adapt the framework to incorporate real-world lessons and feedback and sustain its usefulness.
To meet the above objectives, the development of a strategic evaluation and refinement plan should start with defining key questions to address (such as those listed in the next section) and the specific information needed to answer them. These
questions could include the outcomes of integrating the framework into evidence-generation and decision-making processes, as well as those of implementing the communication and dissemination plan; exploring the methods and opportunities best suited to capturing and measuring these outcomes; and determining how and when to evaluate these findings and integrate them into the framework.
Key outcomes to be evaluated include the framework’s utilization, adoption, acceptance, and impact. The following specific questions should be considered:
Utilization and adoption
Is the framework being incorporated into research?
Is it being used by the target audiences or incorporated into their decision-making activities?
Is it being used in obesity- and other public health–related prevention research?
Is the framework usable (effective, efficient, and satisfying) for the target audiences (International Organization for Standardization, 1998)?
What aspects of the framework are unclear or require additional expertise to understand?
Are major concepts of the framework inhibiting its acceptance?
Is it feasible for the framework to be used to meet the needs of the target audiences?
Is the framework changing the way people identify, collect, and generate evidence?
Is it changing the way people make decisions?
Data Collection Methods and Opportunities
After specifying the desired outcomes, the next step is to select data collection methods that can best be used to measure them. These methods will most likely include direct observation, questionnaires, or personal interviews. Specific tools may include survey questionnaires, interview guides, training kits, structured interview templates, and tally sheets. The methods and tools used should be based on access to the intended audiences and resources at hand.
A number of opportunities for data collection can be considered. For example, funders of research could request research proposals to study the effects of using the framework. This would have the benefit of encouraging researchers to critique and publish on the progress of the framework’s acceptance, use, and integration into public health–related evidence generation and policy making. Activities employed to train
the target audiences in the use and integration of the framework could be used to gather immediate feedback, and serve as a source of information over a longer period of time as well. Another opportunity for collecting data on the desired outcomes is monitoring the literature for articles citing the framework.
Integration of Evaluation Findings into the Framework
Evaluating the impact of the framework on public health–related evidence generation and policy making will highlight the framework’s strengths and weaknesses. These lessons learned should then be incorporated into the framework to increase its value and sustain its impact.
HEALCP (Healthy Eating Active Living Convergence Partnership). 2008. Home. http://www.convergencepartnership.org (accessed August 21, 2009).
International Organization for Standardization. 1998. Ergonomic requirements for office work with visual display terminals (VDTS) part II: Guidance on usability. ISO9241-11. Geneva, Switzerland: International Organization for Standardization.
NCCOR (National Collaborative on Childhood Obesity Research). 2009. Mission. http://www.nccor.org/about_mission.html (accessed August 27, 2009).
NCI (National Cancer Institute). 2008. Making health communication programs work. Bethesda, MD: U.S. Department of Health and Human Services.