Research and Advocacy Groups: How Does Evidence Inform Policy?
As described by moderator Sarah Samuels, President, Samuels and Associates, the organizations represented on the second panel serve as middlemen. They work with community groups, such as those represented on the first panel (see Chapter 5), to produce and provide evidence for program development and evaluation. They also marshal the evidence and serve as advocates to help policy makers adopt, endorse, and support policies and programs aimed at preventing obesity.
The messages that emerged from the four presenters reflect their experiences in presenting evidence to very different audiences, including grassroots organizations, health economists, policy makers with varying levels of familiarity with obesity-related policies, and others. These messages include the following:
Decision makers have different frames of reference in dealing with obesity prevention. While some easily grasp the connection between obesity and community environments, others frame the issue solely as a matter of individual choice. Some see a clear role for government; others remain skeptical. Some expect to see clear financial benefits to obesity-related and other preventive health expenditures. Organizations must find ways to deal with these different frames of reference.
Research is only one type of evidence upon which policy makers draw. Only in rare cases do research findings alone, however definitive, move policy; the expectation that science alone will lead to policy change is unrealistic. Policy makers are confronted with
political considerations, views of trusted advisors and constituents, their own experiences, economic analyses, and numerous other inputs. A related point is the need to collect consistent health and economic data from those obesity programs that are in place to permit clearer conclusions about what is working and the cost implications.
The way the evidence is presented is critical. Policy makers are barraged with information. The organizations represented on the panel continually seek the most effective ways to build on the evidence to draw the attention of policy makers, the media, other opinion leaders, and the public. These methods include mapping and other visuals, easy-to-grasp metrics, brief summaries of relevant research, and personal stories and testimonials.
Research, action, and policy can be linked to increase impact. The presenters shared examples in which communities, instead of just serving as the subject of a research study, participated in collecting, analyzing, and disseminating the evidence. The community thus becomes more engaged in the research, and the result is often tangible changes in policies or programs.
The panel consisted of representatives from three policy and advocacy organizations and a community organizer. Jeffrey Levi, Executive Director, Trust for America’s Health, Washington, DC, identified five challenges to communicating with federal policy makers about obesity prevention. Allison Karpyn, Director of Research and Evaluation, The Food Trust, Philadelphia, Pennsylvania, focused on the policy-making process and what it means for advocacy organizations that present evidence to policy makers. According to Rebecca Flournoy, Associate Director, PolicyLink, Oakland, California, convincing policy makers about the role of community environments in childhood obesity means communicating research findings in compelling ways; indeed, if the issue is compelling enough, policies may be enacted even absent the most definitive evidence, as occurred with tobacco use. Derek Birnie, Executive Director, Delridge Neighborhoods Development Association, Seattle, Washington, described how, as a community organizer, he serves as a bridge to grassroots groups, researchers, and policy makers in his work with the King County Food and Fitness Initiative.
UNDERSTANDING CHALLENGES TO ADVOCACY FOR OBESITY PREVENTION
Dr. Levi explained that the Trust for America’s Health focuses on reaching federal policy makers on a range of public health issues, includ-
Challenges to Advocacy for Obesity Prevention
ing changes to reverse the obesity epidemic. He structured his presentation around five challenges the Trust encounters in communicating with policy makers on these issues. Four of the challenges, he said, are external to those doing obesity prevention work and research, while the fifth is internal.
Definitions of Prevention
To Levi, the first and probably most significant challenge is that every policy maker has a different definition of prevention, which creates enormous confusion. While the Trust and other groups talk about true primary prevention to avoid the development of disease, others think of prevention as disease management, clinical interventions, or a host of other variations. Compounding the confusion, policy makers like simple answers, which do not exist for this issue.
Perceptions About the Role of Government
Policy makers also have differing concepts of the appropriate role for government in fighting obesity. Many people, including policy makers, frame the issue in terms of personal responsibility: individuals can solve the problem by eating less and exercising more. Levi suggested that an effective response is to acknowledge personal responsibility but then to question the environmental obstacles to healthier individual behaviors. In many cases, policy makers will then agree on a role for government. Levi termed this a stepwise progression; by comparison, policy makers instinctively recognize a government role in combating swine flu.
Focus on Saving Money
Policy makers support prevention that saves money, Levi stated, particularly within a certain window of time. He argued that improved health outcomes, quality of life, and productivity should also matter, as they do when one is considering the effectiveness of biomedical interventions. Yet because so much of the debate about prevention is centered around money, the Trust undertook a study to model the savings associated with community-level obesity prevention. The study, funded by The Robert Wood Johnson Foundation and The California Endowment and conducted in collaboration with other organizations, yielded findings that were released in the report Prevention for a Healthier America (Trust for America’s Health, 2008).
For certain conditions, physical activity, better nutrition, and smoking cessation result in a savings of $5.60 for every $1 invested. Levi cautioned against extrapolating these findings to all kinds of prevention and conditions for two converse reasons: some supporters have exaggerated the cost-benefits, and on the other hand, some skeptics may now argue that only interventions that meet this standard of a large return on investment should be funded.
Convincing Health Economists
Health economists represent a formidable group whose views are considered in policy choices. An impetus behind producing Prevention for a Healthier America was to start to engage health economists in looking at different models. According to Levi, the report has initiated a dialogue with the health economics community and with the Congressional Budget Office and the Office of Management and Budget.
The questions raised by health economists that must be considered include the following:
What is the standard of evidence? Levi said health economists want to understand the link between community-level interventions that increase physical activity and better health outcomes, and between those outcomes and cost savings.
How do we know an intervention works? The health economics community, Levi asserted, has a bias toward randomized controlled trials, while community prevention interventions vary from place to place. Levi suggested a dialogue between evaluators, who are used to this complexity, and health economists.
What is the scalability of interventions? Will interventions that are successful in individual communities work when scaled up, and will they be sustainable? Levi urged learning to measure answers
to these questions, although he observed that a double standard exists in that clinical trials for drugs need only show a benefit for the period of the trial itself. “Scalability is a two-edged sword,” he said. “I think we need to have answers to some of these questions, but I’m not sure they should be barriers per se.”
“We are not that young of a field that we should not be able to decide what are the measures that should be consistently collected across programs.”
What is the time frame for showing a return on investment? In the case of childhood obesity, the time frame may be even longer than the 5 to 10 years considered for other types of health interventions.
Consistency of Data
Levi directed his fifth and final challenge to those working on community-level interventions. He urged more consistency in the measures collected across programs to permit measurement of cumulative impact. Developing Prevention for a Healthier America underscored how inconsistency in data collection makes it difficult to report on effectiveness. Levi urged the National Institutes of Health (NIH) and CDC to recommend consistent collection of certain data to enable better comparison of programs. He also urged ready availability of information about program costs.
The bottom line, Levi concluded, is that many policy makers, including President Obama, support community-level obesity prevention. There is still a long way to go, however, in reaching other influential policy makers who are less receptive. It is important to utilize the diversity of community experiences by tailoring the message to the policy maker. The prevention community must also define and articulate its standards of evidence.
ESTABLISHING THE BURDEN OF EVIDENCE
Dr. Karpyn centered her presentation on the level of evidence needed to inform policy makers. By way of introduction, she quoted a former Robert Wood Johnson Foundation staff member who was looking back on his work with the foundation. He wrote, “Journals favored by policymakers were filled with thought-provoking articles containing the kind of pertinent information that ought to fire them up for change and told them ways to do it … [but the studies] didn’t move the needle one bit” (Newbergh, 2009).
Karpyn suggested that many in the audience could relate to the notion that compelling research findings are not effecting policy changes. The issue is not just informing policy makers, but doing so in a way that causes them
Traditional Levels of Evidence Valued by Researchers (in descending order of impact)
SOURCE: Sackett et al., 2000.
to take action. The issue comes down to how policy is made. According to John Kingdon’s Multiple Streams Model, three “streams”—a problem, a policy that might address it, and politics that will affect the inclusion or exclusion of a topic on the policy agenda—must be aligned (Kingdon, 1984). Policies are not the product of rational actions because policy makers generally do not evaluate alternatives systematically. Researchers focus more on the problem, but they must acknowledge the role of the other two streams to see movement on a problem.
Evidence has several roles to play in this model. It must make policy makers understand that a problem is urgent and that members of the public (including constituents) care about it. The challenge is to align the problem and the evidence with a politically feasible solution. Organizations like The Food Trust try to bridge the gap between research and policies.
“Agencies like ours start to bridge the gap between the reearch body and policies. Again, it is about aligning [the problem, policy, and politics] together before you get a policy movement.”
The research community’s traditional framework of levels of evidence has at its pinnacle the systematic review of randomized controlled trials, followed by other types of controlled studies (see box above and, for example, Sackett et al., 2000). Policy makers favor a very different hierarchy, Karpyn said, often relying on expert opinion (at the bottom of the researchers’ traditional hierarchy) and polls (which did not make the Sackett et al. list). The level of evidence needed to move an issue forward varies: When the multiple
streams of problem, policy, and politics align, such as with tobacco or obesity prevention, the burden of evidence can be much lower than otherwise.
A crucial aspect of establishing the burden of evidence is communicating the evidence. It is not enough to generate data and hope that policy makers will use them. Instead, Karpyn suggested a different paradigm in which researchers and advocates collaborate on effective ways to present research results so they will lead to desired policy changes.
Presenting evidence in easily understood, visual ways is necessary to reach policy makers. Examples include maps, poll results and other media attention–getters, visuals, and one-page summaries with bulleted points. Highly technical reports alone are too dense, although they serve as the basis for these more easily digested presentations. Karpyn’s examples included the well-known animated state-by-state map of obesity prevalence, based on research from CDC, and a map of the “grocery gap” in Philadelphia, based on research conducted by The Food Trust (Figure 6-1). A jar of sugar sent
to city officials, which represented the amount a Philadelphia child ingests from soft drinks every week, helped change the availability of these beverages in schools (Figure 6-2).
Presenting data is a science and an art. The data must be based on reputable research, but they also must affect the policy maker’s district or state, be easily understood, and clearly address a proposed policy solution.
CONNECTING RESEARCH AND ACTION
Ms. Flournoy explained that PolicyLink is a national research and action institute that advances policies aimed at achieving economic and social equity by “lifting up what works.” Its Center for Health and Place works to create neighborhood conditions that encourage health.
Evidence for Policy Making
Flournoy began by stepping back from the obesity issue to ask a broader question about the level of evidence needed to spur policy makers to address other public health issues. For example, it was not until the 1990s that researchers identified the biochemical mechanisms linking cancer with cigarette smoke. Policy makers did not wait for this definitive evidence before enacting measures to curb tobacco use, nor would the public have wanted them to.
As observed during the previous panel (see Chapter 5), many community residents state that researchers enter their communities to study their neighborhoods and populations, but nothing changes as a result. Flournoy urged researchers to think about how to connect their research with action. PolicyLink applies some of the principles of community-based participatory research, such as sharing preliminary findings at community meetings and implementing aspects of programs and then evaluating them. Researchers, said Flournoy, need to think about the policy implications of what they study.
PolicyLink has developed a model of policy change in which research and information gathering are part of a larger advocacy process (see Figure 6-3). Research alone is rarely sufficient to effect a policy change, although it can contribute to an atmosphere in which the climate is ready for change.
The Retail Food Environment Index (RFEI)
RFEI calculations from different communities dramatically show the link between the retail food environment and health. To calculate the RFEI:
SOURCE: PolicyLink, the California Center for Public Health Advocacy, and UCLA Center for Health Policy Research, 2008.
Policy makers are used to thinking about individual behaviors that affect health. Making the leap to the role played by community environments requires that organizations like PolicyLink communicate research findings in compelling ways, such as packaging quantitative data; illustrating with maps and images; and recounting stories, especially successes.
For example, PolicyLink, the California Center for Public Health Advocacy, and the Center for Health Policy Research at the University of California, Los Angeles (UCLA) developed the Retail Food Environment Index (RFEI). The RFEI shows the effect of the retail food environment on obesity rates by comparing a community’s numbers of grocery stores and produce vendors with its numbers of convenience stores and fast food restaurants. On average, the RFEI of lower-income communities in California is 20 percent higher than that of their higher-income counterparts, with obesity rates and diabetes prevalence showing similar trends. Publicizing the RFEI generated media attention in the state and nationwide. Policy makers asked about their area’s RFEI and how to improve it. Producing the index was, Flournoy concluded, an effective way for policy makers to consider how community environments affect health.
“Maps can be incredibly powerful in making the case.”
Flournoy agreed with other presenters about the power of maps. When used in Louisville, Kentucky, and New York City, for example, mapping revealed disparities in the food environment and the effect of those disparities on health in an easily grasped and compelling way (see Figure 6-4).
Policy makers also respond to images and stories. As noted earlier, PolicyLink emphasizes “lifting up what works”—highlighting cases in
which people in low-income communities recognize a problem, figure out how to address it, and see some success. These stories are powerful because they provide concrete examples of success that can be adapted or replicated elsewhere and because they provide cause for optimism.
ORGANIZING THE COMMUNITY IN SUPPORT OF OBESITY PREVENTION
Mr. Birnie was the final presenter in this session. In his role with the King County Food and Fitness Initiative (KCFFI), in which the Delridge Neighborhoods Development Association is a partner, he serves as a bridge between grassroots leaders, who are passionate about what they are experiencing in their everyday lives, and the more systematic approaches of researchers and policy experts.
The King County Food and Fitness Initiative
KCFFI is one of many initiatives the W.K. Kellogg Foundation is supporting in communities across the country. These initiatives are based on the premise that health is a product of the systems surrounding an individual, which can either promote or harm health. While the Delridge and nearby White Center communities feature many positive local efforts connected to food and fitness, they are also characterized by systemic characteristics that have resulted in the highest prevalence of diabetes and overweight in the city. Through a cross-sectoral approach involving many partners, KCFFI is building healthy communities through healthy food and safe places to play. Birnie shared KCFFI’s vision and values, echoing comments made earlier in the workshop about building a movement rather than just a program.
The initiative is at the end of a 2-year planning phase conducted to guide the next 8 years of collective policy collaboration. The three goals of its implementation are to (1) create policy and system changes, (2) engage in strategies to make the initiative’s vision a sustainable reality, and (3) create community environments that support access to healthy food and safe places for physical activity. The goals do not currently include measuring outcomes; however, Birnie said that, based on the workshop discussions, this is something he will discuss when he returns to Seattle.
Use of Experience to Inform Policy
The experience of coalition members, including grassroots organizers and leaders, advocates, public health practitioners, academics, and service providers, informs policy. Separately, the members have achieved much, including neighborhood and comprehensive plans, financing tools, school
Benefits of Participatory Research
nutrition policies, and land use policies. An initial success for KCFFI was farm-to-school legislation that allows school districts to buy locally grown produce, even if it is more expensive.
The research that informs KCFFI’s policy initiatives is documented in the report Food for Thought (University of Washington, 2008). The developers of that report drew on community-based participatory research carried out by youth, coalition members, and others. They investigated grocery store offerings, accompanied by elected officials and reporters. The result, according to Birnie, was “great data that [could] be used for advocating the policies, but also informed, invested leaders who [would] do the advocating and an audience that [was] a little bit more familiar with what [was] going to be represented at the end of the research.”
The case for the problem has been made, Birnie said. KCFFI must now strengthen the analysis that connects health impacts with historical policies and systemic causes. The initiative needs more information about best practices and case studies, analysis to determine the long-term value of the investments under consideration, and information about the intersection of public policy and private-sector decision making.
Policy makers have been enthusiastic; as Birnie said, “the Seattle way” embraces study, discussion, and community engagement. Funding for commitments and regulatory changes are more difficult to obtain. Enthusiasm is strongest when the KCFFI strategy is linked with other regional initiatives focused on smart growth and economic development.
Birnie closed by summarizing some lessons he has learned through his work:
Producing convincing data should be a priority, but community leaders must find effective ways to convey the data to decision makers.
Various stakeholders and constituencies value different qualities and sources of data.
Community-based participatory research produces a wide range of results, such as partnerships, data, and momentum.
Organizers’ work involves translating between constituencies as much as conducting research and analysis.
The need is well documented, but still lacking are data-based projections for the outcomes of strategies.
Discussion following the panel presentations revolved around two principal topics: (1) results when evidence does inform policy and (2) conversely, any evidence on the effects of inaction.
Examples of evidence that resulted in policy change. Over the past 5 years, the focus on obesity prevention has shifted from individual responsibility alone to strategies aimed at changing food and physical activity environments and policies to support those changes. Samuels said the work of many members of the audience had helped provide the evidence to effect those changes. She asked the panelists to identify one piece of evidence that has either been particularly effective or backfired.
Karpyn replied that The Food Trust’s graphing linking data on diet-related mortality to poverty and supermarket access helped move policy forward. Flournoy cited data from the Fresh Food Financing Initiative, first developed by The Food Trust, which showed benefits in terms of jobs created, new retail space, and housing values when supermarkets in underserved areas were supported.
Levi said that the Trust’s report, which he had discussed in his presentation, had resonated with policy makers who want to believe prevention saves money. He warned participants, however, not to assume that all policy makers have made the transition from framing obesity in individual to environmental terms. Birnie concurred. He said that when he originally brought KCFFI to the Delridge board for consideration, the first response was to question how the initiative related to Delridge’s mission to support affordable housing. Another response was wariness, expressed as, “Who are you to tell me what I should be feeding my kids, or that I am not healthy?” How the issue is framed from the outset and the evidence used to support it is important in engaging community groups and policy makers.
Data on the cost of doing nothing. A participant asked whether data exist to demonstrate the cost of not taking action to address the obesity problem, which would resonate with some policy makers. Levi said The Food Trust’s report contains some such data, and Flournoy referred to research conducted by Manuel Pastor on how the conditions in a specific community can have an impact on the economy of a larger region.
Organizations like those represented on the panel link community programs and policy makers in developing approaches to reduce obesity. They