Summary¹

The ability to answer research questions about measures of obesity prevalence in the population and trends in obesity among population subgroups depends on skillful research design. Consideration also must be given to the range of methodological approaches available and how such approaches apply to a given problem. The conduct of a study focusing on any research question encompasses a range of methodologies, including randomized controlled trials and observational research: prospective and retrospective cohort, case-control, and cross-sectional studies. Each approach has strengths and weaknesses and the application of a given approach to a specific research question will affect how research findings are interpreted and applied.

The current literature of reports on obesity includes estimates of prevalence and trends for various population groups throughout the United States. The data sources from which these estimates are derived range from local initiatives to national surveillance programs, differing from each other in terms of resources, intent, funding, and approach. Data from population surveys, school-based assessments of body mass index (BMI), electronic health records (EHRs), and cohort studies have all been used to describe the status of obesity in the population. Differences across analyses are driven by the specific question(s) being asked, the quantity and quality of the data, and the approach used by the investigators.

Challenges exist in the collection and analysis of data across population groups and subgroups within the population. Many relate to epidemiologi-

___________________

¹ This summary does not contain references. Citations to support statements made herein are given in the body of the report.

cal and statistical issues, but not to the age of participants. Populations that include children, however, require distinct considerations. These involve changes in body composition due to growth, small measurement errors that can affect weight status classification for young children, and the need to collect information from the child’s parent or guardian, among others. These considerations must be factored into the study design, the data collection procedures, and the analytic approach. How investigators overcome such methodological challenges affects the estimate that is produced.

Accurate and meaningful population estimates of obesity prevalence and trends are fundamental to understanding and describing the scope of the issue. Policy makers, program planners, and other stakeholders at the national, state, and local levels are among those who search for reports relevant to their population(s) of interest to inform their decision making. The differences in the data collection, analysis, and reporting, up to this point, have produced a body of evidence that is inconsistent. As a result, those who use estimates of obesity prevalence or trends are challenged with interpreting and appropriately applying information derived from reports.

STUDY TASK AND APPROACH

To better assess and apply published reports on obesity prevalence estimates and trends and to consider strategies for future research on these issues, the Robert Wood Johnson Foundation (RWJF) asked the National Academies of Sciences, Engineering, and Medicine to convene an expert committee. It requested that the committee examine the approaches to data collection, analysis, and interpretation that have been used in recent reports on obesity prevalence and trends at the national, state, and local levels, particularly among U.S. children, adolescents, and young adults (see Box S-1).

The committee comprehensively reviewed and assessed sources directly relevant to its task. To be inclusive, it considered a wide range of information from the peer-reviewed literature, along with publicly available national, state, and local research and surveillance sources. The review of the evidence allowed the committee to broadly examine the landscape of data collection, analysis, and reporting related to obesity prevalence and trends. In addition to reviewing the literature, the committee held a public workshop that included the perspectives of investigators who collect and analyze obesity data, along with stakeholders who rely on reports of such analyses to inform decision making. The committee also considered public comments received through an online submission system. From these activities, the committee developed a framework for assessing and interpreting reports on obesity prevalence and trends and recommendations for evaluating published reports, improving future data collection efforts, and filling data gaps.

THE LANDSCAPE OF DATA COLLECTION, DATA SOURCES, AND ANALYTIC APPROACHES

BMI, calculated from an individual’s height and body weight, is the prevailing measure used to determine whether a person has obesity and to assess obesity prevalence and trends. Common data sources that capture height and weight data include population surveys, school-based assessments, clinical and public health administrative data, and cohort studies. From a design perspective, these data sources differ from each other in the use of and approach to sampling, the population represented, the setting in which data are collected, and the total sample size.

Height and weight data can be obtained through direct measure, proxy-report, or self-report. Various protocols are used to directly measure height and weight, and differences exist in the questions used to collect reported data. Obesity prevalence estimates based on proxy- and self-reported data are typically not equivalent to estimates derived from directly measured data. This affects the comparability of estimates based on data sources

using different approaches to collecting height and weight data. Despite this limitation, specific data sources that use proxy- or self-report are filling data gaps that would otherwise exist, especially across states and select localities.

Demographic data are used to assess the representativeness of a sample and often serve as the basis for creating subgroups for analysis. Subgroups analyses provide insight into who is affected, to what extent, and if trends differ between groups. Although not all comparisons are evaluations of health disparities, the assessment of health disparities typically rely on demographic characteristics. The characteristics that are captured, the measures used to capture the data, and the level of specificity of the measures vary across data sources. The differences in measures and methods make comparisons challenging. Beyond subgroup analyses, demographic data can also be used to help researchers and stakeholders recognize and account for demographic shifts in a population that can affect the interpretation of trends analyses. This consideration will continue to play an important role as the demographic composition of the United States changes.

In analyzing data, investigators select what criterion will be used to classify obesity status. For adults, the standard cut point is a BMI of 30 kg/m² or greater. For children, adolescents, and young adults, classification requires comparison to a reference population. Although the 2000 Centers for Disease Control and Prevention (CDC) sex-specific BMI-for-age growth charts are most commonly used, others exist. Use of different reference populations can lead to estimates of prevalence that differ from each other, and as such are not interchangeable. The way in which extreme height, weight, and/or BMI values are identified in a dataset and subsequently handled can also affect the prevalence estimate.

When assessing the data, investigators, policy makers, and other stakeholders must apply a number of considerations, including an assessment of the response rate, evaluation of missing data, and, if applicable, the weighting of the sample. The bounds of statistical analyses are determined, in part, by the sample size, which affects how the sample and the time periods are grouped in the analysis. For trend analyses, considerations include the beginning and end dates and time intervals used to define the trend.

FRAMEWORK FOR ASSESSING PREVALENCE AND TRENDS IN OBESITY

The variations in the methods, data sources, and analytic approaches used to estimate obesity prevalence and trends has made navigating and understanding the literature challenging. Interpreting an estimate requires attention to the details, nuances, and caveats of published reports. Evaluating studies for the purpose of informing a decision requires more than interpreting the statistical analysis. Appropriate application of reports involves

consideration of how the parameters of the estimate align with a user’s specific information need. A wide range of policy makers, program planners, and others use or seek to use reports on obesity to inform decision making (hereafter referred to as “end users”; see Box S-2). To help end users interpret and apply estimates, the committee offers the Assessing Prevalence and Trends (APT) Framework (see Figure S-1). The proposed framework provides a conceptual process for how end users can approach published reports, consider the strengths and weaknesses of obesity data estimates, and synthesize the information for the purposes of decision making.

The Three Phases of the Assessing Prevalence and Trends (APT) Framework

The assessment process is separated into three phases: (1) identification of goal or information need, (2) assessment of published report(s), and (3) synthesis to inform decision making. Driving the assessment are questions related to each of the framework elements. An expanded list of questions is provided in the body of the report.

Phase 1: End User’s Identification of Goal

In the first phase of the framework, end users identify their goal for assessing the report(s). This includes a consideration of the decision to be made as well as the need for additional information to fill gaps in the evidence. Achieving clarity on their goal helps end users determine the utility of the report findings relative to their unique information needs. End user goals will vary in depth, complexity, and specificity.

Phase 2: Assess Published Report(s)

In the second phase of the assessment process, end users evaluate the published report(s). The three core components of a report (population, methodology, analysis) inform the interpretation of the estimate. The dynamic assessment of these three components in the context of each other provides a means for end users to consider how the approach taken in one may have benefited or limited one or both of the other components. By the end of the assessment of the report(s), end users should clearly understand the parameters associated with the estimate.

Phase 3: End User’s Synthesis to Inform Decision Making

In the final phase of the assessment process, end users turn back to their goal in order to synthesize and interpret the report(s) findings in the context of their information needs for decision making. To accomplish this, end users first weigh the strengths and weaknesses of each report, initially in consideration of the report’s strengths and weaknesses generally, then in relation to their specific information needs and decision-making priorities.

End users will categorize a given feature of a report differently, depending on their overall objective and the data gap they are trying to fill. For example, some would consider state-representative samples a strength of a report, while others at the community level may view this parameter as a weakness for their particular decision-making process. Once end users have established what the strengths and weaknesses are, they then determine how the findings inform their decision.

UNDERLYING PRINCIPLES OF THE FRAMEWORK

The APT Framework is intended to serve as a starting point for those who wish to better understand and apply published reports. It is grounded in six underlying principles discussed below and summarized in Box S-3.

The APT Framework Can Be Used Both for Assessing Individual Reports and for Synthesizing Multiple Reports

The number of reports assessed and how they are evaluated through the APT Framework will depend on the goal of the end user and the availability of reports related to the goal. In some instances, the framework would be used to assess a single report. In other instances, an end user can start by using the framework to identify the parameters that define the individual estimates, and then progressively consider them together. In practice, a broader assessment of multiple reports would bolster the decision making. However, given the limitations of the current state of the literature, the committee recognizes that pertinent information may reside only within a single report for certain end users’ goals.

A Variety of End Users Can Use the APT Framework

It is anticipated that end users will have various backgrounds and expertise. The intent of the framework is to provide sufficient guidance on the elements that affect the interpretation and application of estimates, while remaining general enough to accommodate the diverse literature and the broad array of decisions that may be informed by obesity prevalence and trends reports.

An End User’s Goal Informs the Application of Any Report or Reports

Reports on obesity prevalence and trends are generally designed to address a specific set of questions within a defined set of parameters. The estimates presented in reports are guided not only by the question the report’s authors sought to answer, but also by the methodologies used and

analytic limits of the collected data. The APT Framework directs the end user to further this line of thinking and reflect on the type of information that is needed to inform their decision making.

The Three Core Components of a Published Report Are Interdependent

Three core components in a report inform the interpretation of an obesity prevalence or trend estimate: the population assessed, the methodologies employed, and the analysis approach used. These components do not exist or operate in isolation, but are interdependent. The assessment of reports necessitates a fundamental understanding of each of these components in relation to the each other. As such, appraisal of the population, methodology, and analysis is not a linear process; rather, it is dynamic and iterative.

Questions Lead the End User Through the Assessment Process

In each of the phases of the APT Framework, the end user is prompted to consider pertinent guiding questions to ask, as represented by the lower portion of the visual. The questions that appear on the figure itself serve as a starting point, with an expanded list of potential questions to guide the assessment provided in the body of this report. End users may find some questions more relevant to their specific information needs than others, and as they are thinking through the evaluative process, they may develop questions of their own.

The APT Framework Facilitates an Assessment of the Evidence to Inform the Decision-Making Process

In the first phase of the framework, identification of the end user’s goal directs the assessment with the specific information need. The second phase of the framework guides identification of the bounds of an estimate. The final phase brings the first two phases together, and the end user considers how that estimate compares to the information being sought. By using the end user’s goal to contextually frame the assessment process, the APT Framework highlights the interface between reading a report for meaning and using the findings for a specific application.

CONCLUSIONS

The interpretation of obesity prevalence and trends estimates is contingent on considerations specific to the assessment of obesity status, principles that are founded in epidemiology and concepts that are fundamental to

statistics. This interplay of elements, which span from general to specific, is reflected in the committee’s conclusions. Much of the available evidence indicates that at the core of several of the current limitations are seemingly basic challenges faced by any population-based prevalence or trends evaluation. The fact that these issues exist, however, underscores the challenges of finding viable solutions. The committee’s conclusions focus on key domains that cut across the broad literature base. These include current sources of data, data for specific population groups, measured versus reported data, estimates of changes and trends over time, and interpretation of estimates.

Current Sources of Data

A wide variety of data sources capture height and weight data, but differ from each other by design, sample size, target population(s), geographic representation (e.g., national; multiple states and localities; rural and urban regions), and collection method. Few data sources are designed to generate estimates of multiple states and those that do tend to describe select population groups.

Conclusion 1: The committee concludes that existing data sources used to estimate prevalence and trends in obesity vary by factors, including study design, geographic representation, data collection methodologies, and overall intent. Each offers specific and distinct information about the state of obesity. The differences between data sources, however, can limit the comparability of reports.

Data for Specific Population Groups

Investigators divide analytic samples into subgroups to determine the extent to which obesity prevalence and trends vary within a broader population. The factors defining the groups (e.g., span of ages, race and ethnicity categories) can vary widely across reports, even those that analyze data from the same data source. Published reports often cite inadequate sample size as the reason for omitting one or multiple subgroups from an analysis or combine heterogeneous groups into a single category.

Conclusion 2: The committee concludes that inclusion of subgroups in data sources provides essential insight into how obesity prevalence and trends estimates vary within and between population groups. However, insufficient sample size is a primary limitation to generating reliable estimates.

Measured Versus Reported Data

Height and weight data used to calculate BMI are collected through direct measurement, proxy-report, and self-report. Within each collection approach, variability exists in the specific data collection protocol. Proxy- and self-reported height and weight questions have been incorporated into various population surveys in which factors such as the overall design and mode of delivery (e.g., phone interview) do not allow for direct measurement. These surveys often have large sample sizes and some have been used to generate estimates that are compared across states and select localities. Evidence indicates that use of proxy-reported data for young and school-aged children generally does not lead to accurate estimates of obesity prevalence. Limited evidence, based on different nationally representative surveys, suggests that trends in obesity estimated from self-reported and directly measured heights and weights among high school-aged individuals exhibit similar patterns, albeit at different values.

Conclusion 3: The committee concludes that although all measures have limitations, directly measured height and weight data collected using a standardized protocol provide the best estimates of obesity prevalence. Self- and proxy-reported height and weight data can be used to fill data gaps and provide insight into overall obesity trends, although these data collection methods do not produce prevalence estimates comparable to those based on direct measure.

Estimates of Changes and Trends Over Time

Published reports assessing obesity prevalence over time have presented findings as change (the difference between two time points) or trends (the difference over three or more time points). Such estimates pertain only to the specific time points included in the analyses. Trend estimates typically become more precise and nuanced as the number of time points increases. However, the number of time points is dependent, in part, on the reliability of the prevalence estimates. Investigators often combine multiple years or cycles of data to increase the reliability of the estimates used to determine the trend, thereby reducing the number of data points. Changes to the time interval included in the trend analyses directly affect the estimate and its interpretation.

Conclusion 4: The committee concludes that comparability of trend reports is enhanced when analyses use similar start and end dates and time intervals to define the trend.

Interpretation of Estimates

Factors that affect the meaning of obesity prevalence and trends estimates not only include characteristics of a data source, but also encompass decisions made during analysis. Data sources differ with respect to who the sample is designed to represent and who contributes data. Changes to the sampling or data collection procedures over time affect what data are available for trend analyses. The portion of the overall sample that is used for analysis varies across published reports for a number of reasons, including what question(s) is being asked of the data, how the data were prepared for analysis, and whether the sample size led to reliable estimates of prevalence. The statistical analyses are varied and are guided by the intent of the specific report, the quality control measures taken during data collection, the study design from which the data were derived, and the amount of data available.

Conclusion 5: The committee concludes that appropriate interpretation of estimates of obesity prevalence and trends requires consideration of the population in the sample, the data collection methodologies, and the analytic procedures together in a guided way.

RECOMMENDATIONS

Data sources that capture height and weight largely operate in isolation or within a single surveillance system, resulting in designs and protocols that differ from each other. Although these differences often limit comparability of prevalence and trend estimates, their existence underscores the diverse context in which decisions and compromises have to be made in the design, collection, and analysis of the data. The committee offers recommendations in three areas: assessing published reports on obesity prevalence and trends; improving future data collection efforts; and conducting research to address data gaps.

Assessing Obesity Prevalence and Trends Reports

Because understanding and appropriately applying estimates of obesity prevalence and trends is a complex process, the committee provides the APT Framework as a conceptual guide for those who seek to better understand and use reports. The framework draws on the committee’s synthesis of key considerations related to inconsistencies that exist in the literature while simultaneously drawing on fundamental principles of epidemiology and statistics. The committee considers determining the utility of estimates presented in published reports a highly individualized process, guided by the end user’s overall goal and specific information needs.

Recommendation 1: The committee recommends that stakeholders who use or seek to use estimates of obesity prevalence and trends to inform policy making, program planning, and decision making follow the Assessing Prevalence and Trends (APT) Framework to guide their interpretation of published reports.

The committee recognizes that end users who operate at the national, state, and local levels often have different information needs. The extent to which available analyses meet those needs varies considerably. Individual end users are therefore likely to have different priorities when it comes to the strengths and weaknesses of published reports. In order to be adaptable to a range of possible applications, the APT Framework integrates consideration of the end user’s context to guide the assessment.

Future Data Collection

This report serves as an important starting point for moving toward comparable, more unified data collection, analysis, and reporting practices. Current practices, however, are determined by more than just the analytic and scientific rationale presented in a published report. Factors such as cost, existing infrastructure, and available resources play a role in the selection of a study design and the success of its implementation. Explicit, prescriptive guidance on specific methodologies to be used by the research and public health surveillance communities requires consideration of the experiential knowledge of those who fund, develop, and carry out such activities.

Recommendation 2: The committee recommends that an organization with a track record of cross-sector leadership in the field of obesity, such as the National Collaborative on Childhood Obesity Research or the Robert Wood Johnson Foundation, convene relevant stakeholders to examine and identify feasible and practical approaches to standardizing methodologies for data collection and reporting, appropriate for application at the national, state, and local levels to enhance comparability of obesity prevalence and trend analyses.

To include a range of participants, the committee recommends that the organization(s) sponsoring the proposed activity not only be nationally prominent, but also have strong ties to diverse stakeholders who operate at the state and local levels. The organizations included in the recommendation—the National Collaborative on Childhood Obesity Research (NCCOR) and the Robert Wood Johnson Foundation (RWJF)—have missions and experience that are aligned with the recommendation goals. NCCOR and RWJF are two examples of potential conveners. How-

ever, the committee notes that other conveners or collaborators may enrich the proposed activity as well.

The committee further recommends that a broad range of stakeholders who operate at the national, state, and local levels participate in this activity, including, but not limited to

• Local and state public health agencies;
• Federal governmental agencies (e.g., CDC, U.S. Department of Agriculture, Agency for Healthcare Research and Quality);
• Community-based organizations;
• School officials (e.g., state Departments of Education; superintendents; school nurses and physical education teachers);
• Academic researchers;
• Research organizations;
• Research funders;
• Obesity-oriented and public health professional organizations; and
• Other decision makers who operate at the national, state, and local levels.

Research to Address Gaps

The assessment of obesity prevalence and trends estimates continues to change with technological, methodological, and statistical advancements. Some of the inconsistencies and limitations that currently exist in the literature represent prime opportunities for improvement and progress.

Recommendation 3: The committee recommends that the research community design and conduct studies to strengthen the evidence base and improve methodological approaches to assessing obesity.

Because opportunities for improvement encompass a wide range of disciplines, the research community described in the recommendation includes, but is not limited to, federally-funded researchers, clinical researchers, social scientists, and engineers.

Specific research initiatives could include

• Evaluating how the 2000 CDC BMI-for-age growth charts can better provide continuity to obesity classification across the life course. The committee acknowledges that the 2000 CDC BMI-for-age growth charts are the predominant reference currently in use in the United States for children ages 2 years and older and

• recognizes the importance of its continued use as a platform for comparing estimates of obesity among children and adolescents. The committee anticipates findings from current and future initiatives (e.g., the Dietary Guidance Development Project for Birth to 24 Months and Pregnancy [B-24/P], the INTERGROWTH-21st Fetal and Newborn Growth Consortium, and the Environmental Influences on Child Health Outcomes [ECHO] Program) will inform an evidence-based consensus on how weight status should be classified for children younger than age 2 years. Additionally, opportunities exist to clarify when and how to best transition young adults to the adult criterion for obesity classification.

• Identifying appropriate measures of core demographic variables—including but not limited to race and ethnicity, socioeconomic status, and rurality—that can be captured in a consistent manner across various data collection efforts at the national, state, and local levels. As the demographic landscape of the country continues to change, it will become increasingly vital to characterize populations in ways that capture the existing diversity.
• Developing innovative, practical, and accurate tools for assessing adiposity. Although BMI is the predominant measure of relative weight used to classify obesity status, it is not without limitations. For the purposes of population-based assessments, a new measure of obesity will need to be a simple alternative that provides comparable or improved predictive ability, can be measured accurately in a variety of settings, and is relevant to diverse population groups across the life course.
• Preventing the misclassification of data from individuals with severe obesity as biologically implausible values. Technology-based systems that are used for direct data entry often have features that automatically detect extreme values in height and weight. Identification at the time of measurement allows for the values to be corrected or properly documented. Data collection procedures that first record measurements on paper or in systems without automatic detection of extreme values often have limited ability to check the quality of data until entry into a database or analysis. Opportunities exist to expand the use of technology in data collection to enhance the accuracy of recorded measurements.
• Identifying innovative opportunities to capture longitudinal data throughout childhood. A variety of data sources—including EHRs and school-based BMI assessments—are primed to be used in novel ways to serve as the basis for or supplement longitudinal evaluations.

CONCLUDING REMARKS

This report evaluates the strengths and weaknesses associated with existing approaches to collecting obesity data, creating estimates of obesity prevalence, and assessing trends. It also recommends ways to systematically assess obesity-related reports, given these strengths and weaknesses, in order to understand and interpret the information the reports provide.