Summary1

The obesity epidemic in the United States has serious health, economic, and social consequences for individuals and society at large. The recognition of these consequences has accelerated efforts to characterize and understand the problem more fully, to take evidence-based and innovative actions, and to assure progress in obesity prevention. Recognition of the large number and variety of environmental and policy strategies being implemented across the country, and the need to understand whether the actions are having an impact in preventing obesity, have created a demand for timely and meaningful data to inform and improve these efforts.

Rigorous scientific evaluation can offer this information to various stakeholders—legislators responsible for amending or creating policies; funders deciding where to invest; elected local officials developing a blueprint for change; and administrators accountable for the stewardship of resources, program management, and policy implementation. Evaluation can provide information on how well programs and policies are being implemented, on which interventions work best in varied real-world contexts, and for rapid course correction. Evaluation can also offer longer-term evidence that interventions are achieving the intended outcomes, and identify emerging issues to investigate.

This report aims to increase the likelihood that (1) obesity prevention efforts will be evaluated appropriately; (2) the results of these evaluations will inform and improve decision making in all sectors; (3) progress will be made in monitoring the adoption, implementation, and maintenance of tested interventions; and (4) the most promising approaches for accelerating the prevention of obesity will be disseminated widely.

STUDY APPROACH AND SCOPE

With funding from the Michael & Susan Dell Foundation, the Institute of Medicine (IOM) Committee on Evaluating Progress of Obesity Prevention Efforts was formed to develop a concise and actionable plan for measuring progress in obesity prevention efforts for the nation. The Committee was asked to focus the scope of its evaluation plan on assessment of the policy and environmental strategies recommended in the IOM report Accelerating Progress in Obesity Prevention: Solving the Weight

_____________

1 This summary does not include references. Citations to support statements made herein are given in the body of the report.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary 1 T he obesity epidemic in the United States has serious health, economic, and social consequences for individuals and society at large. The recognition of these consequences has accelerated efforts to char- acterize and understand the problem more fully, to take evidence-based and innovative actions, and to assure progress in obesity prevention. Recognition of the large number and variety of environmental and policy strategies being implemented across the country, and the need to understand whether the actions are having an impact in preventing obesity, have created a demand for timely and meaningful data to inform and improve these efforts. Rigorous scientific evaluation can offer this information to various stakeholders—legislators respon- ­ sible for amending or creating policies; funders deciding where to invest; elected local officials develop- ing a blueprint for change; and administrators accountable for the stewardship of resources, program management, and policy implementation. Evaluation can provide information on how well programs and policies are being implemented, on which interventions work best in varied real-world contexts, and for rapid course correction. Evaluation can also offer longer-term evidence that interventions are achieving the intended outcomes, and identify emerging issues to investigate. This report aims to increase the likelihood that (1) obesity prevention efforts will be evaluated appropriately; (2) the results of these evaluations will inform and improve decision making in all sectors; (3) progress will be made in monitoring the adoption, implementation, and maintenance of tested inter- ventions; and (4) the most promising approaches for accelerating the prevention of obesity will be dis- seminated widely. Study approach and Scope With funding from the Michael & Susan Dell Foundation, the Institute of Medicine (IOM) Committee on Evaluating Progress of Obesity Prevention Efforts was formed to develop a concise and actionable plan for measuring progress in obesity prevention efforts for the nation. The Committee was asked to focus the scope of its evaluation plan on assessment of the policy and environmental strate- gies recommended in the IOM report Accelerating Progress in Obesity Prevention: Solving the Weight 1  This summary does not include references. Citations to support statements made herein are given in the body of the report. 1

OCR for page 1
of the Nation (APOP) (IOM, 2012a), rather than on clinical interventions for individuals. The charge ­ to the Committee was further delineated as follows: (1) develop a plan for evaluating national obesity prevention efforts; (2) develop a community-level measurement plan that adds detail and support to the n ­ ational-level plan; and (3) identify measurement ideas that can determine the specific impact of the Home Box Office (HBO)/IOM campaign The Weight of the Nation (TWOTN). The intended audiences for the plans and measurement ideas in this report are simple: individuals with an interest in obesity prevention. These “evaluation users” include policy makers, government agency staff, nongovernmental organizations at all levels, advocates, local coalitions, researchers and evaluators, businesses, media, and the public. A note on terminology: within the literature, terms such as assessment, surveillance, monitoring, ­ and evaluation are often used interchangeably or with different meanings that vary among professions, disciplines, and settings. For consistency throughout this report the Committee uses these terms as described in Box S-1. Within this collection of terms, evaluation is widely, and sometimes in this report, used to refer to all four or some combinations of these functions. Additionally, the objects of evaluations can encompass programs, systems, policies, environmental changes, services, products, or any combina- tion of these multifaceted aspects of initiatives. The Committee will refer to these collectively and in their various combinations as interventions. BOX S-1 A Note on Terminology • Assessment is an effort to use data on the community or other jurisdiction to characterize the prob- lem, its distribution, and efforts to address it. • Monitoring is the tracking of the implementation of interventions* compared to standards of performance.  • Surveillance is the ongoing systematic collection, analysis, and interpretation of data tracked over time to detect patterns, disparities, and changes that may be associated with interventions or other causes. • Summative Evaluation is the effort to detect changes in output, outcomes, and impacts associated with interventions and to attribute those changes to the interventions. • Evaluation refers to all four or various combinations of these functions (assessment, monitoring, sur- veillance, and summative evaluation). * In this report, interventions refer to programs, systems, policies, environmental changes, services, products, or any combina- tion of these multifaceted initiatives. 2 Evaluating Obesity Prevention Efforts

OCR for page 1
The report contains (1) a conceptual evaluation framework to consider when evaluating progress of obesity prevention efforts; (2) broad conclusions from discussion with evaluation users of their needs and a review of existing evaluation efforts and infrastructure; (3) model (flexible) evaluation plans (i.e., a framework and suggested approaches and methods) for national, state, and community stakeholders; (4) indicators of progress and existing data sources for measuring these indicators to complement targeted new evaluations; (5) recommendations for evaluation infrastructure changes to encourage and enhance the extent and effectiveness of obesity prevention evaluations; and (6) measurement ideas to determine the impact of the HBO/IOM TWOTN campaign. In developing the tools, guides, and recommendations found in the report, the Committee consid- ered the following key material: (1) recommended environmental and policy strategies outlined in the APOP report; (2) the components and evaluation efforts to date of the HBO/IOM TWOTN campaign; (3) a review of literature on stakeholder perspectives and evaluation approaches and methods; (4) views of representatives from selected evaluation stakeholder/user groups at a public workshop on October 12, 2012, and through interviews; and (5) the context (i.e., What to evaluate?, How to evaluate?, Who will be doing the evaluation?, and By what timing or interval should the evaluation be done?) and resources that are available for evaluating interventions. More specifically the APOP report identifies 20 environmental and policy strategies that hold the most promise for accelerating progress in preventing obesity. The strategies are organized into five envi- ronments: (1) the physical activity environment, which includes the built environment as well as norms and processes that increase opportunities for, access to, and social reinforcement of physical activity; (2) the food and beverage environment, including support for increased availability and affordability of healthful foods; (3) the message environment that encompasses media and marketing; (4) the health care and worksite environments in which promotion of healthful foods and physical activity can be supported and arranged; and (5) acknowledgment of the school environment as an important hub of health promo- tion. The APOP report stresses that the recommended strategies are interrelated and that, consistent with a systems science approach, successful implementation of the strategies will require engagement across all levels and sectors of society. These APOP strategies serve as the focus of the Committee’s evaluation plans and recommended actions for implementing the plan in this report. An Evaluation Framework To guide future obesity evaluation efforts, the Committee developed a vision statement and a frame- work of an evaluation process that can lead to the achievement of this vision. This evaluation framework lays out the needs, inputs, resources, activities, outputs, outcomes, and impacts that need to be considered when planning and implementing the evaluation of progress in obesity prevention efforts (see Figure S-1). The Committee’s vision is to assure collection and analysis of timely and meaningful data or informa- tion to inform and improve obesity prevention efforts at national, state, and community levels. The Committee’s evaluation framework especially highlights the context, activities, and intended outcomes of obesity prevention efforts and provides guidance for assuring the availability of data to inform progress in these efforts from the community to the national levels. Summary 3

OCR for page 1
1. Inputs • User/Stakeholder Needs • Existing Objectives Strategies • Context for Evaluation a • Guiding Principles for Evaluation a • Resources 5. Intended Impacts/ Improvements 2. Activities Evaluation • Identify Indicators/Measures of Success • Partnerships, Environments, Policies, Behaviors, Norms Framework • Develop Evaluation a Plans and Infrastructure • Energy Expenditure/Intake • Overweight/Obesity • Develop Resources for Training, Technical • Population Health and Well-Being; Equity Assistance, and Dissemination 4. Outcomes 3. Outputs • Short-term: Improved Evaluationa Capacity and Training • Core Indicators and Measures • Intermediate-term: Increased • Recommendations and Guidance Evaluationa Activities • Support for Implementation • Long-term: Enhanced Data Use FIGURE S-1  Framework to guide the work of assuring collection and analysis of data to inform progress of obesity prevention efforts. Figures S-1, 4-1 and 10-1.eps a Evaluation refers to assessment, monitoring, surveillance, and summative evaluation activities. Conclusions Based on its review of existing evaluation efforts, the Committee identified key findings on dimen- sions of national and community-level evaluation. This includes aspects of information/data needs of those interested in obesity prevention and its results, indicators that can act as markers for assessing the progress of obesity prevention efforts recommended in the APOP report, the infrastructure and capacity to support evaluation of APOP recommended strategies, and methods and protocols for conducting evaluation. Evaluation users operate at federal, state, and community levels in at least three contexts: the policy making process; dissemination and diffusion of obesity prevention strategies; and local implementation, quality improvement, and sustainability of policies and programs. Across all of the information sources and the various kinds of users the Committee consulted, the highest-priority questions were (1) “Why is 4 Evaluating Obesity Prevention Efforts

OCR for page 1
obesity prevention important to me?” (2) “What works to prevent obesity?” and (3) “How are we doing in preventing obesity?” In obesity prevention program development and implementation, evaluation is most commonly used to clarify the dimensions of the problem of obesity, inform the operation of a pro- gram or policy, and better understand the mechanisms of the effects of the intervention. The current col- lection, packaging, and dissemination of data and information, then, often is not responsive to the second ­ and third priority questions posed by evaluation users. Indicators that measure the progress of obesity prevention efforts recommended in the APOP report can be found in a wide variety of existing data sources (government, academia, private sector, commer- cial). However these indicators are not compiled in one easily accessible place. A list of indicators from available and ongoing data sources can provide a menu of possible indicators for use by evaluators, be a starting point for identifying a set of common core indicators for use at the national and community level, and identify gaps in the current collective data and information system. Although a large number of indicators exist related to strategies recommended in the APOP report, data and information gaps remain. There is a need for indicators that assess partnerships, leadership, health equity issues, and more generally data that can be used at the community level. The current national monitoring and surveillance system (infrastructure) for obesity and related risk factors provides a history of tracking key impact indicators, valid and reliable measures, and sample sizes that provide population-level estimates for various subgroups. Healthy People 2020, the U.S. Dietary Guidelines, and Physical Activity Guidelines for Americans provide a framework of key indicators to inform assessment, monitoring, surveillance, and summative evaluation efforts related to APOP strategies. The majority of these indicators have data available at the national level, but little available at the com- munity level. However current monitoring and surveillance systems that sample selected regions allow the use and comparison of national, state, and some city/county levels for selected communities. The cur- rent collective system for measuring progress of APOP strategies: (1) lacks focus on monitoring of policy, environmental, and systems-level efforts important for obesity prevention, for a variety of settings, and for certain populations; (2) lacks dedicated leadership for coordinating efforts; and (3) lacks resources for assessment, monitoring, surveillance, and summative evaluation, and timely reporting of results. Additionally, common guidance for relevant core indicators, common measures, methods, and protocols to use in obesity prevention evaluation has not been agreed upon for use at the national and community levels. And there is an evident need to assure a competent workforce to collect and use evaluation data. The seven broad conclusions that emerged from these findings (see Box S-2) serve as context for the development and guidance provided in the recommended plans, supporting actions, and measurement ideas that follow. Obesity Evaluation Plans Based on its review of current evaluation efforts and infrastructure and the components identified in the evaluation framework, the Committee developed national and community evaluation plans for mea- suring the progress of obesity prevention efforts identified in the APOP report. These evaluation plans— among the key actions to improving evaluation efforts—contain guidance for organizing and implement- ing evaluation-related efforts (a framework and suggested approaches and methods) to achieve the plan’s ­intended outcomes. Summary 5

OCR for page 1
BOX S-2 Broad Conclusions Regarding Existing Evaluation Efforts • There is a pressing need to act on the problem of obesity, but there are gaps in the certainty of the effectiveness of actions or mixture of actions being implemented across the country. Systematic and comprehensive evaluations along with more routine assessments, monitoring, and surveillance offer valuable guidance for improving the quality and outcomes (or impact) of the actions being implement- ed and for defining the direction of further basic and implementation research. • Information generated from current obesity prevention evaluation efforts, other than assessment of needs at the national and state levels, does not always address the needs and interests of the users of this information, often because of limited or outdated data (especially at the community level) and few presentations of the data in useful and timely formats. • Current data (monitoring) systems do not adequately track progress of environmental and policy- related obesity prevention actions or systems changes recommended in the Accelerating Progress in Obesity Prevention report (IOM, 2012a). Such monitoring is needed at both the national and communi- ty levels, especially for populations at greatest risk for obesity. These limitations exist primarily because monitoring systems have traditionally focused on measuring individual behaviors, energy expenditure/ energy intake, and overweight and obesity. • Current investment in evaluation is too low and sporadic, presenting serious barriers to understanding the impact of and need for future investments in implementing interventions. • A systems science approach (i.e., interactions and connectedness of components in a whole system) to evaluation can help evaluation users identify and select combinations of actions and strategies to implement in multiple sectors, and at multiple levels, with available resources. • Although many data systems exist, the current national systems for monitoring progress of recommend- ed obesity prevention actions and for surveillance of their effects on obesity lack adequate leadership, coordination, infrastructure, guidance, accountability, and capacity. • Communities lack adequate guidance, capacity, data, and resources necessary for assessing the status of obesity and its determinants, identifying prevention needs, monitoring obesity prevention actions, evaluating their short-term outcomes, and tracking (through surveillance) their long-term association with obesity reduction in the aggregate and differences among population segments. The National Obesity Evaluation Plan U.S. efforts lag behind international efforts to provide common guidance, support, and the appro- priate infrastructure to nurture evaluation of obesity prevention efforts. The recommended National Obesity Evaluation Plan (see Box S-3) is designed to organize the planning, implementation, and evalu- ation of obesity prevention and related policies and programs recommended in the APOP report at the national level. The National Obesity Evaluation Plan integrates existing national surveys, evaluation 6 Evaluating Obesity Prevention Efforts

OCR for page 1
BOX S-3 Core Components of the National Obesity Evaluation Plan Purpose: To evaluate the progress at the national level in implementing strategies in the Institute of Medicine Accelerating Progress in Obesity Prevention report and in achieving its intended outputs, out- comes, and impacts. 1. Identify leadership, infrastructure, resources, priorities, and timeline for implementing the plan. 2. Identify current national efforts for evaluation, including indicators, and incorporate them selectively into national monitoring, surveillance, and summative evaluation data systems that are responsive to the needs of data users. 3. Propose data and infrastructure to add to existing monitoring and surveillance systems to fill gaps and facilitate community obesity evaluation plans. 4. Propose additional assessment, monitoring, surveillance, and summative evaluation activities; new measures and innovative strategies to implement in the future. 5. Outline mechanisms for feedback to data users, assuring accessibility, privacy, and cost-efficiency. 6. Detail adaptations of the plan at the state level, with further applications at the regional level. s ­ tudies, and monitoring and surveillance systems that currently focus primarily on individual-level mea- sures with recommendations for new infrastructure, indicators, and data that would capture environ­ mental and policy changes. The evaluation plan is intentionally broad to provide the flexibility necessary for meeting the needs and resources of the evaluators. Framed in a systems approach, the evaluation plan includes implementation across multiple sectors and use of variables that address health equity. It can also be used as a model for state and multi-state regional evaluations. In this report, the Committee provides detailed activities, support, and guidance for addressing each component identified in the National Obesity Evaluation Plan. The Committee stresses the need to pri- oritize the activities of the National Obesity Evaluation Plan to leverage existing resources that maximize efficiency of data collection and avoid duplication of efforts. The Community Obesity Evaluation Plan The Community Obesity Evaluation Plan provides an actionable framework for evaluation at the community level that can be adapted for local needs and resources. It includes suggested indicators avail- able for measuring progress of APOP-related strategies. The Community Obesity Evaluation Plan includes two distinct sets of activities—community assessment and surveillance (see Box S-4) and community pro- gram and initiative (or intervention) monitoring and summative evaluation (see Box S-5). For purposes of this report, community assessments describe the current state of obesity-related and contextual indicators ­ Summary 7

OCR for page 1
BOX S-4 Components of a Community Obesity Assessment and Surveillance Plan Purpose: To provide accurate and timely knowledge of local obesity-related conditions and relevant changes or trends over time as a result of implementing strategies in the Institute of Medicine Accelerating Progress in Obesity Prevention report. 1. Define community boundaries. 2. Engage community members and other key stakeholders in as many of these steps as feasible. 3. Plan assessment/surveillance. 4. Collect data. 5. Analyze and make sense of the data. 6. Disseminate findings. BOX S-5 Components of a Community-Level Obesity Intervention Monitoring and Summative Evaluation Plan Purpose: To guide local action and to inform national choices about the most effective and cost-effective strategies in the Institute of Medicine Accelerating Progress in Obesity Prevention report for funding, dis- semination, and uptake by other communities. 1. Design stakeholder involvement. 2. Identify resources for the monitoring and summative evaluation. 3. Describe the intervention’s framework, logic model, or theory of change. 4. Focus the monitoring and summative evaluation plan. 5. Plan for credible methods. 6. Synthesize and generalize. 8 Evaluating Obesity Prevention Efforts

OCR for page 1
and tracks them over time (surveillance). Community intervention evaluations seek to establish and share what is being implemented (monitoring) and “what works” (summative evaluation). Together, these activ- ities provide baseline data and “diagnostic” data on the state of obesity and related “determinants” or conditions in the community (e.g., community assessment) and then measure progress in accelerating the reduction of obesity and its determinants (e.g., community program initiative summative evaluation). Box S-3 and Box S-4 identify key components for developing and implementing community-specific obesity assessment/surveillance and intervention monitoring/summative evaluation, respectively. Although not shown in the boxes here, the report provides detailed support and guidance for implementing each component, including recommended indicators and methods for collecting and reporting on progress for APOP strategy–focused community assessment/surveillance and intervention monitoring/summative evalu- ation that can be applied to communities with varying skills and resources. Considerations for Implementing the Obesity Evaluation Plans Most community health efforts are under resourced, and current support and resources for evalua- tions are limited. This means that some of the Committee’s recommendations to support implementation of the evaluation plans call for leadership and expenditures that will require government, organizations, and the private sector to make trade-off decisions. To be mindful of available resources, address the cur- rent status of APOP strategies, and provide baseline data for future evaluation activities, the Committee’s recommendations for supporting the obesity evaluation plans would ideally be implemented with short-, intermediate-, and long-term perspectives and astute use of existing resources and prioritization of other necessary actions. The recommended national and community evaluation plans provide a framework for obtaining end-user input; for choosing indicators, measures, and designs focused on APOP strategies; for data col- lection and analysis; and ultimately for improving the evaluation infrastructure to support evaluation efforts. To support these actions, the Committee (1) summarized the needs of a diverse set of stakeholders/ users of evaluation information; (2) identified existing indicators of progress for APOP-related strategies that can be incorporated into the recommended plans, help to identify gaps in existing data and informa- tion systems, and be used as examples of indicators for evaluators of obesity prevention interventions; and (3) recommended actions that will improve leadership and coordination, guidance, capacity, infrastruc- ture, systems orientation, and help to prioritize actions for evaluation efforts. These actions to support the implementation of the plans will improve evaluation capacities for all users in the short term (e.g., use of a core set of existing indicators), increase evaluation activities in the intermediate term (e.g., improve capacity and guidance), and enhance data use in the long term to assess the population-level changes and improvements that can result from widespread implementation of evidence-based obesity prevention interventions (i.e., intended outcomes). Finally, the National Obesity Evaluation Plan and the Community Obesity Evaluation Plan are interdependent. The two plans have the potential to provide essential support and feedback to each other. Successful implementation of the Community Plan is supported by the components of the National Plan, using indicators, sources of data, resources, and methodologies coordinated and developed with leader- ship at the national level. However, the Community Plan also provides an additional level of detail and Summary 9

OCR for page 1
local context-specific information that the National Plan cannot measure, including the appropriate mix- ture of strategies to implement, their feasibility, and ease of their implementation. Indicators of Progress One clear gap in evaluation efforts is a lack of recognition and consensus among users and evalua- tors about a set of core indicators that could be used at the national and community levels for measuring ­ progress in obesity prevention. Consensus is needed to assure a degree of uniformity of measurement that would enable comparative analyses of evaluation across jurisdictions and time periods. These i ­ndicators can guide the collection of baseline data to more comprehensively and comparatively assess the ­ besity prevention actions already being implemented. As a key first step in identifying this core set, o the Committee identified indicators that currently exist. Based on available and ongoing data sources, the Committee identified 83 indicators that were best aligned with the recommendations in the APOP report. These indicators provide a menu of possible indicators for use by evaluators and offer a starting point for the development of core indicators and related measures. This process also enabled the identification of gaps in existing data systems to be filled and provide a focus for the Committee’s proposed evaluation plans (national and community levels) and of potential improvements to long-term evaluation infrastruc- ture and capacities. In the short-term, evaluators of obesity prevention programs, policies, and environ- ments can use the indicators identified by the Committee. Of particular importance to the Committee was recognition that evaluating progress for the nation as a whole, and for regions and communities, requires special attention to the disparities that appear to be associated with the obesity epidemic. Although numerous challenges remain, the Committee found a small yet growing literature on tools and methodologies for monitoring progress toward obesity prevention among racial and ethnically diverse and disadvantaged populations. Taking Action to Support the National and Community Obesity Evaluation Plans Using the considerable number of indicators identified in this report, and guided by methodolo- gies and protocols outlined in the plans, stakeholders can take immediate action to begin comprehen- sive assessment of the obesity prevention efforts recommended in the APOP report and already under way. The Committee realizes that its obesity evaluation plans will not be fully implemented without organizational changes across multiple federal, state, and local government agencies and departments in collaboration with other nonfederal partners responsible for obesity prevention–related activities. Implementation of the plans will require adequate resources, but expenditure decisions should consider leveraging of existing resources and prioritization of necessary actions. The following recommenda- tions support the successful implementation of all of the components of the obesity evaluation plans. (Potential actions and actors to guide the implementation of recommendations are detailed in the report.) Improve Leadership and Coordination for Evaluation The Committee believes that centralized leadership is necessary to coordinate the planning, imple- mentation, and evaluation of obesity prevention efforts across the country. Most of the existing data 10 Evaluating Obesity Prevention Efforts

OCR for page 1
­ ollection and support for evaluation exists across multiple federal agencies.2 The current de­ entralized c c structure provides limited authority, responsibility, support, and coordination for these efforts at the national level. The Committee views the lack of empowered leadership to coordinate resources at the fed- eral level as a major obstacle to measuring obesity prevention efforts. Progress could be made if a federal entity would take a leadership position in this coordination effort. A number of relevant entities could serve in this coordination role. The Committee believes that one or a combination of these entities would be the best option for overseeing and implementing the National Obesity Evaluation Plan and reporting to whatever agency is leading these coordination efforts. Alternatively, the appointment of a new task force could also successfully address the need for improved leadership and coordination of evaluation, but the Committee does not view it as necessary. It was not in the Committee’s charge or in its expertise to analyze various options and then recommend a specific entity to take on this responsibility. Recommendation 1: An obesity evaluation task force or another entity should oversee and implement the National Obesity Evaluation Plan and provide support for the Community Obesity Evaluation Plan and should coordinate with federal, state, and local public- and private-sector groups and other stake­ olders h who support, use, or conduct evaluations. The taskforce/entity could be a new or existing entity or a combination of existing entities. Improve Data Collection for Evaluation Recommendation 2: Using the recommended indicators and gaps identified in this report as guides (i.e., related to Accelerating Progress in Obesity Prevention report strategies), all federal agencies3 and state and local health departments responsible for collecting data relevant to obesity prevention efforts, in coordination with relevant private partners, should identify, coordinate, and maximize current efforts for ongoing collection of recommended indicators and, according to the priorities identified, should address existing evaluation gaps at the national and local levels. Provide Common Guidance for Evaluation Recommendation 3: Relevant federal agencies (e.g., in the U.S. Departments of Agriculture, Commerce, Health and Human Services, Labor, and Transportation) and state and local health departments, in c ­ ollaboration with nonfederal partners, should standardize the collection and analysis of data, including common indicators, measures, methods, and outcomes used for assessment, monitoring, surveillance, and summative evaluation to assure aggregation among localities and back to the National Obesity Evaluation Plan. 2  Includes,but is not limited to, the following federal agencies: Corporation for National and Community Service; Domestic Policy Council; Environmental Protection Agency; Federal Trade Commission; General Services Administration; Office of Management and Budget; and U.S. Departments of Agriculture, Commerce, Defense, Education, Health and Human Services, Interior, Labor, Transportation, and Veteran Affairs. 3  Agricultural Research Service, Economic Research Service, and Food and Nutrition Service of the U.S. Department of Agriculture; Census Bureau of the U.S. Department of Commerce; Agency for Healthcare Research and Quality, Centers for Disease Control and Prevention, Health Resources and Services Administration, and National Institutes of Health of the U.S. Department of Health and Human Services; Bureau of Labor Statistics of the U.S. Department of Labor; and Federal Highway Administration of the U.S. Department of Transportation. Summary 11

OCR for page 1
Improve Access to and Dissemination of Evaluation Data Recommendation 4: Relevant federal agencies (e.g., in the U.S. Departments of Agriculture, Commerce, Health and Human Services, Labor, and Transportation), in collaboration with academics, non­ governmental organizations, and state and local health departments, should coordinate existing efforts to ensure that federal, state, and local assessment, monitoring, surveillance, and summative evaluation sys- tems include a mechanism for feedback to users of evaluation data. In addition, local evaluations should continue to build the evidence base for the Accelerating Progress in Obesity Prevention report strategies; be stored, curated, synthesized, and shared to improve generalizable knowledge about implementation barriers and opportunities; and clarify “what works” in different contexts. Improve Workforce Capacity for Evaluation Recommendation 5: The Centers for Disease Control and Prevention, National Institutes of Health, and the U.S. Department of Agriculture, through the National Collaborative on Child Obesity Research4 and other nongovernmental and professional organizations, should build on their existing evaluation resources to assure support for the diverse and interdisciplinary workforce engaged in conducting assess- ments, surveillance, monitoring, and summative evaluation activities. Improve Evaluations to Address Disparities and Health Equity Recommendation 6: The U.S. Department of Health and Human Services, in collaboration with other federal and nonfederal partners, should increase its capacity to address health equity by practicing partici- patory and culturally competent evaluation, and it should standardize the collection, analysis, and report- ing of data targeting disparities and health equity and improve the accessibility of tools and methods for measuring social determinants that place populations at elevated risk for obesity. Support a Systems Approach in Evaluation Recommendation 7: Evaluators, government, and private funders should incorporate a systems approach to evaluating obesity prevention efforts into their research-related activities through leadership, funding, and training support. The Weight of the Nation Measurement Ideas Finally, the Committee identified ways to evaluate the impact of the HBO/IOM TWOTN campaign, launched in 2012. This multi-media, multi-organizational campaign was designed to help create aware- ness, inform, and motivate action to combat obesity. The Committee reviewed the campaign’s components and evaluation efforts to date, and it offered ideas for future measurement for both the national- and community-level components of the campaign. After reviewing the evaluation literature and current national- and community-level evaluation efforts of TWOTN, the Committee concludes that further national-level evaluation of this specific cam- 4  One of the goals of the National Collaborative on Childhood Obesity Research, a private-public collaboration, is to improve the ability of obesity researchers and program evaluators to conduct research and program evaluation. 12 Evaluating Obesity Prevention Efforts

OCR for page 1
paign is not warranted at this time. In this report, the Committee presents some methods for national evaluation of future campaigns, but it concludes that it would be unproductive to disentangle the effects of media campaign activities from other national and community activities that have employed policy and environmental strategies to raise awareness and engage stakeholders in obesity prevention. Further sum- mative evaluations of community-level interventions related to this campaign should emphasize (1) the use of strong theoretical or logic models; (2) the assessment of reach or dosage, which is a critical step in the logic model for any health promotion program or mass media campaign; and (3) the use of multiple waves of measurement preferably before, during, and after a campaign. final thoughts The Committee offers an evaluation framework to guide future efforts to inform and improve o ­ besity prevention efforts at national, state, and community levels. The national and community Obesity Evaluation Plans, stakeholder perspectives, indicators of progress, and existing data sources will provide guidance for improving targeted new evaluations of the collective strategies recommended in the APOP report. From the beginning recommendation for national leadership and infrastructure to the last recom- mendation for innovation in developing a systems approach to obesity prevention in general and in spe- cific communities, the Committee’s recommendations offer a series of logical and cyclical paths to support the implementation of its Obesity Evaluation Plans. The recommendations range from federal to local ­ and back, from use of selected existing indicators to consensus on a set of expanded indicators, from suc- cesses for whole populations to successes in populations facing health disparities, and from development of evidence from existing projects to dissemination, adaptation, and evaluation of the strategies in other communities. Summary 13

OCR for page 1