Executive Summary
The statistics on food insecurity and hunger in U.S. households, published annually by the U.S. Department of Agriculture (USDA), are based on a survey measure developed by the U.S. Food Security Measurement Project, an ongoing collaboration among federal agencies, academic researchers, and private organizations. The measure was developed over the course of several years in response to the National Nutrition Monitoring Act of 1990. One of the objectives of the development of the food security measure was to create a measure with generally agreed-on concepts, definitions, and measurement methodologies that could be used to estimate a standard and consistent indicator of the frequency and severity of problems regarding access to food in this country.
These estimates are based on data collected annually in the Food Security Supplement to the Current Population Survey (CPS). On the basis of the number of food insecure conditions that households report, respondents are classified into one of three categories for monitoring and statistical analysis of the food security status of the population: (1) food secure, (2) food insecure without hunger, and (3) food insecure with hunger.
The USDA estimates, published in a series of annual reports, are widely used by government agencies, the media, and advocacy groups to report the extent of food insecurity and hunger in the United States, to monitor progress toward national objectives, to evaluate the impact of particular public policies and programs, as a standard by which the performance of USDA programs is measured, and as a basis for a diverse body of research relating to food assistance programs. Despite the extensive use of the measure, some major questions continue to be raised regarding the underlying concepts, the methodology, and their use.
PANEL CHARGE
The USDA requested the Committee on National Statistics (CNSTAT) of the National Academies to convene a panel of experts to undertake a two-year study in two phases to review the concepts, methodology for measuring food insecurity and hunger, and the uses of the measures. The charge specifies that during Phase 1 of the study a workshop will be held to address the key issues laid out for the study and a short report will be prepared based on the workshop discussions and preliminary deliberations of the panel. The specific tasks to be addressed in Phase 1 of the study are:
-
the appropriateness of a household survey as a vehicle for monitoring on a regular basis the prevalence of food insecurity among the general population and within broad population subgroups, including measuring frequency and duration;
-
the appropriateness of identifying hunger as a severe range of food insecurity in such a survey-based measurement method;
-
the appropriateness, in principle and in application, of item response theory and the Rasch model as a statistical basis for measuring food insecurity;
-
the appropriateness of the threshold scores that demarcate food insecurity categories—particularly the categories “food insecure with hunger” and “food insecure with hunger among children”—and the labeling and interpretation of each category;
-
the applicability of the current measure of the prevalence of food insecurity with hunger for assessing the effectiveness of USDA food assistance programs, in connection with the Government Performance Results Act performance goals for the Food and Nutrition Service;
-
future directions to consider for strengthening measures of hunger prevalence for monitoring, evaluation, and related research purposes.
In Phase 2 of the study the panel will consider in more depth the issues raised in the workshop relating to the concepts and methods used to measure food security and make recommendations as appropriate. In addition, the panel will address and make recommendations on:
-
the content of the 18 items and the set of food security scales based on them currently used by USDA to measure food insecurity;
-
how best to incorporate and represent information about food security of both adults and children at the household level;
-
how best to incorporate information on food insecurity in prevalence measures;
-
needs and priorities for developing separate, tailored food security scales for population subgroups, for example, households versus individuals, all individuals versus children, and the general population versus homeless persons;
-
future directions to consider for strengthening measures of food insecurity prevalence for monitoring, evaluation, and related research purposes throughout the national nutrition monitoring system.
This report addresses the panel’s mandate for Phase 1 only and provides the panel’s preliminary assessment of the food security measure and interim guidance for the continued production of the food security estimates. A final report with the panel’s conclusions and recommendations also will be prepared.
PRELIMINARY ASSESSMENT
Appropriateness of the Definition of Hunger
Perhaps the most controversial aspect of the measurement of food security is the identification of persons as food insecure with hunger. Hunger is a politically sensitive word that conjures images of severe deprivation. The question of whether it is appropriate to identify hunger as a category at the severe end of the range of food insecurity is a conceptual one. The panel thinks that a clear conceptualization of resource-constrained hunger—both a physiological and socioeconomic construct—is not evident in the current measure of food insecurity with hunger.
The physiological aspect of hunger is an individual experience, and questions about the experience of hunger should be asked at the individual and not the household level. The socioeconomic aspect of hunger may follow from the economic resources of the household. However, it is not directly linked to an individual’s experience of hunger because it is not clear how household-level resources translate into individual-level eating and hunger. The definition of hunger as both a physiological and socioeconomic concept is not made clear. USDA needs a better definition and method for measuring the concept of hunger as well as an improved measure of food insecurity.
In the panel’s judgment, until further work is completed in Phase 2 to refine the concept and measurement of hunger and how it relates to food insecurity, USDA should continue the current survey but may want to use the categories of food insecurity as currently reported without using the label of hunger. This area needs more development, and the panel hopes to provide USDA with specific guidance on this subject.
Item-Response-Theory Models
USDA uses item-response-theory (IRT) models to estimate food insecurity experienced by households in the United States. The Rasch model, a specific type of IRT model, is used to estimate the food insecurity of survey respondents. This model has some attractive proper-
ties if the data fit the model’s assumptions. One property of the Rasch model is that each item contributes the same amount of information to the household’s propensity for food insecurity.
The use of IRT and specifically the Rasch model to measure food insecurity has been challenged in several settings. Some have questioned whether the assumptions of IRT models are violated given the data generated from the current food insecurity instrument and in particular how well the data fit the Rasch model.
The current concept of food insecurity with hunger is based on the definition of hunger as part of a continuum of food insecurity. If this is the concept that USDA is measuring, it is appropriate to consider hunger as a latent, continuous occurrence that can be measured using IRT models.
The key factor is how the construct is defined and whether this definition can be validated. To the panel’s knowledge, no studies have tried to validate whether households classified as food insecure with hunger did indeed really experience hunger. A separate question about the use of IRT models is whether or not data generated from the current food insecurity supplement fit the assumptions of IRT models and the specific assumptions of the Rasch model. There is some evidence that some of these assumptions may not hold, but further research is necessary.
Given the current definition of hunger, IRT models are appropriately suited to estimate levels of food insecurity. While there is evidence that the Rasch model may not be the best model for these data, the use of other IRT models should be explored.
Threshold Scores
USDA totals the sum of affirmative responses to the food security scale questions and uses threshold scores to classify households as either food secure, food insecure without hunger, or food insecure with hunger. It is common and accepted practice to use such thresholds with IRT models. The more controversial aspect of using thresholds is how they are labeled—particularly the labeling of the most severe
threshold, food insecure with hunger. The panel has concerns that a clear conceptual basis for measuring hunger has not been articulated.
Appropriateness of a Household Interview Survey
Although a household survey may be appropriate for measuring food insecurity, the current set of questions used for these concepts combines individual-level experiences with household-level experiences. While it seems reasonable to address some of these questions only to the household respondent (e.g., were you ever hungry?), it is not clear why other questions also ask about the other adults in the household. The problem is not just the use of a household survey, but issues of questionnaire design and the selection of respondents in the participating household also need to be considered.
Theoretically, it is also reasonable to consider questions of the frequency and duration of food insecurity using a household survey. The current version of the 18-item food security scale used in the CPS does not collect much information relevant to frequency and duration. If the goal is to obtain better information about the frequency and duration of food insecurity, USDA might consider using questions on these topics from the full Food Security Supplement and not just the 18-item scale. In addition to the issue of measuring the frequency of food insecurity, there is interest in measuring the duration of food insecurity, as well as changes over time, at the individual and household levels. Including a food security supplement in a longitudinal survey, such as the Survey of Income and Program Participation, which interviews households every 4 months for 2 to 4 years, would facilitate the analysis of duration and change over time. The design of the CPS could also allow for some longitudinal analysis of food security of some households.
A household-based survey, however, is limited with respect to coverage of the U.S. population. In general, such surveys do not include persons living in group quarters, those who are institutionalized, or the homeless. There is reason to believe, therefore, that household-based surveys may not adequately cover individuals who are food insecure yet do not live in households.
Applicability for Assessing Effectivenss of Programs
As required by the Government Performance and Results Act of 1993, the 2000-2005 strategic plan of the Food and Nutrition Service—the agency with responsibility for the major food assistance programs in the United States, including the Food Stamp Program, the Special Supplemental Nutrition Program for Women, Infants, and Children, and the National School Lunch Program—states a goal for the agency, in delivering the food assistance programs, to reduce the prevalence of food insecurity with hunger among households with income under 130 percent of the federal poverty standard. At the present time, USDA uses prevalence estimates of food security to report annual performance in the execution of the strategic plan.
These prevalence estimates are not well designed for use in measuring progress toward meeting the goals of the Government Performance and Results Act for food assistance programs. Evaluating the efficacy of food assistance programs by examining fluctuations in prevalence of food insecurity has little meaning. The estimates do not measure anything directly tied to the food assistance programs (such as improved nutritional status because of program participation). Thus, effective performance of the programs cannot be directly linked to improved food security status, nor can a deterioration of food security be attributed to failure of these programs.
CONCLUSIONS AND INTERIM RECOMMENDATIONS
This interim report provides USDA with the panel’s preliminary guidance on how the food security measure can be improved, based on discussions during the workshop and panel deliberations. On the basis of the panel’s findings and conclusions, it presents interim recommendations for improving the food security measure during the period until completion of work in Phase 2 of the study and the completion of additional recommended research by USDA. Full discussion of the panel’s conclusions and interim recommendations appears in Chapter 3.
Conclusions Conclusion 1: The concept and definition of hunger as measured in the Food Security Supplement, and how they relate to food insecurity, are not clear. In addition, it is not clear whether hunger is appropriately identified as the extreme end of the scale. Conclusion 2: Food insecurity is important to measure. It is a multifaceted concept, each facet of which is appropriate to consider as latent and continuous. It is appropriate to use item response theory models to measure these dimensions. However, the Rasch IRT model may not be appropriate in the current application. If the Rasch model is not appropriate, then using the sum scores of the items also is not appropriate. Conclusion 3: Threshold scores applied to estimates provided by IRT models can be used to categorize households into levels of food insecurity. However, the appropriate categories and labels need to be examined further. Conclusion 4: A household interview survey may be one appropriate vehicle to query households about their food security experiences and to measure the prevalence of food insecurity among households. Conclusion 5: Prevalence estimates of food insecurity as currently obtained are not well suited for evaluation of the effectiveness of food assistance programs. It is unclear that monitoring the prevalence of food insecurity at national and sub-national levels would be suitable for evaluation of these programs. |
Interim Recommendations Interim Recommendation 1: Because the problem of hunger is important and should be measured, the USDA should refine its definition and measurement of hunger and how, and if, it relates to the concept of food insecurity. Interim Recommendation 2: In presenting the data in the annual food security reports, USDA should prominently report frequencies of the individual items that make up the scale. Interim Recommendation 3: Given that the concept of food insecurity is multifaceted, the USDA should consider which specific facets should be measured. Interim Recommendation 4: USDA should explore the use of alternative or additional surveys to estimate the national prevalence of food insecurity. In the meantime, USDA should continue to measure food insecurity as currently conducted using the Food Security Supplement of the CPS. |