3
Preliminary Assessment

This chapter addresses the key questions posed to the panel for Phase 1 of the study. It presents the panel’s preliminary findings to date on concepts and definitions, the questions used to measure food security or food insufficiency, and the design and methodology for measuring these concepts. It provides the panel’s comments, conclusions, and interim recommendations based primarily on workshop discussions and panel deliberations during Phase 1 of the study, while the panel pursues these and other issues in more detail in the Phase 2 of the study.

CONCEPTS, DEFINITIONS, AND THEIR MEASUREMENT

Concepts and Definitions

The 10- and 18-item set of questions used in the Food Security Supplement of the Current Population Survey (CPS) (shown in Box 2-1) covers a range of experiences, perceptions, and behaviors concerning the adequacy of food in the household and on an individual level. The questions range in severity from anxiety about being able to buy



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report 3 Preliminary Assessment This chapter addresses the key questions posed to the panel for Phase 1 of the study. It presents the panel’s preliminary findings to date on concepts and definitions, the questions used to measure food security or food insufficiency, and the design and methodology for measuring these concepts. It provides the panel’s comments, conclusions, and interim recommendations based primarily on workshop discussions and panel deliberations during Phase 1 of the study, while the panel pursues these and other issues in more detail in the Phase 2 of the study. CONCEPTS, DEFINITIONS, AND THEIR MEASUREMENT Concepts and Definitions The 10- and 18-item set of questions used in the Food Security Supplement of the Current Population Survey (CPS) (shown in Box 2-1) covers a range of experiences, perceptions, and behaviors concerning the adequacy of food in the household and on an individual level. The questions range in severity from anxiety about being able to buy

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report the desired types of food to actual shortage leading to the experience of hunger as defined by USDA. This range of severity is intended to measure and assign people to the three categories of food security, food insecurity without hunger, and food insecurity with hunger described in Chapter 2. The concepts underlying these categories are complex and multifaceted, which is why a scale based on multiple questions was chosen as the method to estimate the prevalence of food security. In the judgment of the panel, a clear conceptual basis does not exist for some of these concepts and the questions may not be well suited to measure these concepts. Measurement of the Concepts The measurement of food security (and insecurity) as currently defined includes three separate concepts: Uncertainty about being able to obtain food in socially acceptable ways due to a lack of resources, causing worry and mental, emotional, and physical stress. This worry and uncertainty may also result in changes in behavior—for example, changes in the allocation of time and resources. Insufficiency in (or lack of access to) the quantity and quality of nutritionally adequate and safe foods. This concept includes two ideas: lack of access to the nutritionally appropriate foods and lack of access to desired types of foods. It is separate and different from worrying about food, since there is an actual reduction in the quantity or quality of foods—or both. Whereas uncertainty about obtaining food in socially acceptable ways may or may not lead to changes in behaviors, insufficiency results in an actual reduction in the quantity and or quality of foods. Insufficiency does not necessarily imply hunger because one could lack access to nutritional or desired foods and still not experience hunger. The definition of the concept of hunger used in the current food security measure incorporates both a physiological component—“the uneasy, painful sensation caused by a lack of food”

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report and a socioeconomic component—“the recurrent and involuntary lack of access to food.” The current set of questions used in the survey to measure food uncertainty includes questions relating to each of these concepts. Question 1 in Box 2-1 asks about food insecurity—that is, it directly asks whether the respondent worried about the food running out before there was money to pay for more. Questions 2-6 imply food insufficiency (e.g., cutting the size of meals, couldn’t afford to eat balanced meals). Finally, question 7 asks directly about hunger. The remaining questions (8-10) do not specifically ask about hunger. These questions could be considered as indicating “food insufficiency” because they imply an insufficient quantity of food. They may also imply hunger. Another important concern with food insecurity measures is that all three concepts are measured in a household survey and households are classified into the three dimensions of food security, even though the concepts themselves may not be appropriately measured at the level of the household. The concepts of food uncertainty and food insufficiency are really household-level concepts. Each implies decisions about household resource allocation (e.g., how much of a limited budget can be spent on food compared with other goods and how much of the food budget is spent for food for different household members). Worrying about having enough money to pay for food is a response that considers constraints on the household’s resources. Cutting meal size and not being able to afford a balanced meal are also adaptations made with consideration of the entire household’s resources. In contrast, hunger is experienced by individuals, not households, although everyone in the household could individually experience hunger. Food uncertainty, food insufficiency, and hunger are different and separate concepts, although they are certainly related. The appropriate questions and methods to measure these concepts therefore may be quite different as well. The current method used to estimate the prevalence of food security status does not delineate these concepts, that is, responses to questions about food uncertainty and food insufficiency are totaled with responses to questions about hunger, and each response contributes equally to the estimates of the prevalence of food insecurity.

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report The panel is concerned that this lack of conceptual clarity contributes to controversy surrounding the use of the terms, especially the use of the term “hunger.” The panel will review and comment further on these concepts in its final report at the end of Phase 2. USDA, however, needs to make important decisions now on some of the basic issues. Timely but preliminary assessment therefore is appropriate at this time and interim recommendations for improvements are needed now. Appropriateness of Defining Hunger as a Severe Range of Food Insecurity Perhaps the most controversial aspect of the measurement of food security is the identification of persons as food insecure with hunger. Hunger is a very politically sensitive word that conjures images of severe deprivation. Much of the criticism about the current food security measurement project is targeted toward this classification, specifically the cut points, and the questions to which responses confirm or deny the hunger cut point that are used to classify people as food insecure with hunger (see Bavier, 2004). The question of whether it is appropriate to identify hunger as a category at the severe end of the range of food insecurity is a conceptual one. The panel thinks that a clear conceptualization of resource-constrained hunger—both a physiological and socioeconomic construct—is not evident in the current measure of food insecurity with hunger. The physiological aspect of hunger is an individual experience, and questions about the experience of hunger should be asked at the individual and not the household level. The socioeconomic aspect of hunger may follow from the economic resources of the household. However, it is not directly linked to an individual’s experience of hunger because it is not clear how household-level resources translate into individual-level eating and hunger. As it is currently measured in the 10-item scale, one question (question 7) asks directly about hunger—In the last 12 months, were you ever hungry, but did not eat, because you couldn’t afford enough food? (yes/no). Most people would interpret this question to reflect a

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report physiological aspect of hunger—the uneasy feeling caused by a lack of food. The socioeconomic aspect of the concept is not a part of this question. Other questions in the supplement ask about experiences like weight loss and the skipping of meals. These behaviors suggest hunger but they do not necessarily mean a person experienced hunger. Some of these questions are directed to the respondent only (questions 8-10), but others ask if other adults in the household also had that experience (question 6). It is not clear how each of these other questions relates to the conceptual definition of hunger on which the measure is based. Conclusion 1: The concept and definition of hunger as measured in the Food Security Supplement, and how they relate to food insecurity, are not clear. In addition, it is not clear whether hunger is appropriately identified as the extreme end of the food security scale. USDA has stated that the goal of food security measurement is not solely to estimate the prevalence of hunger, but rather, to obtain estimates of the prevalence of the uncertainty of having enough food or the inadequacy of the food that is available—that is, the prevalence of food insecurity and food insufficiency. The panel concurs with USDA that the goal should be to measure the broader concept of food insecurity. However, if key policy questions revolve around the issue of hunger, then the current food insecurity measure may not be appropriate. USDA needs a better definition and method for measuring the concept of hunger. The panel’s conclusion is based on the fact that, although a strong theoretical and research base exists for the conceptualization and measurement of food insecurity, we do not have a correspondingly strong base for either the conceptualization of hunger or its measurement. That is, there is now ample theoretical, conceptual, ethnographic, and quantitative work done to justify the measurement of the experience of food insecurity using a questionnaire. For the measurement of the experience of hunger to be equally credible, there needs to be a stronger base than we currently have in developing clear concepts for how we should think about hunger and in tested means to accurately elicit

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report information from survey respondents about whether they have experienced hunger. The panel will address this issue in depth in Phase 2 of the study. Interim Recommendation 1: Because the problem of hunger is important and should be measured, USDA should refine its definition and measurement of hunger and how, and if, it relates to the concept of food insecurity. In the panel’s judgment, until further work is completed in Phase 2 to refine the concept and measurement of hunger and how it relates to food insecurity, USDA should continue the current survey but may want to use the categories of food insecurity as currently reported without using the label of hunger. This area needs more development and the panel hopes to provide USDA with specific guidance on this subject. Moreover, when the conceptual basis of hunger is better developed, USDA should also evaluate and test questions geared toward the measurement of hunger. ITEM-RESPONSE-THEORY MODELS AS A STATISTICAL BASIS FOR MEASUREMENT OF FOOD INSECURITY USDA uses item-response-theory (IRT) models to estimate food insecurity experienced by households in the United States. IRT models are a class of statistical models used to describe the responses to a set of categorical items. In the case of food insecurity, the responses to the 10 questions (18 for households with children) of the Food Security Supplement questionnaire are used to estimate the propensity of households to experience various levels of food insecurity. IRT models have commonly been used in educational testing, and the parlance about them is geared toward this—an individual’s ability is the estimated propensity measure, as is the difficulty of each item on the test. IRT models rely on three assumptions: unidimensionality, conditional independence, and monotonicity. Unidimensionality assumes that each survey respondent in the sample has a one-dimensional, latent quantity that describes the respondent’s propensity to endorse the item on the survey, where pro-

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report pensity in this case is food insecurity. Conditional independence assumes that the items in the survey are independent of each other, given a respondent’s propensity. Monotonicity assumes that the probability that an item is endorsed is a nondecreasing function of a respondent’s propensity—that is, respondents with a high propensity (more food insecure) are more likely to endorse items than those with a low propensity (less food insecure). The Rasch model, a specific type of IRT model, is used by USDA to estimate the level of food insecurity of survey respondents. (For more detail the reader is referred to the paper prepared by M.S. Johnson for the workshop in 2004; see Hamilton et al., 1997b, for a description of the application of IRT to food insecurity and how this approach was chosen.) This model has some attractive properties if the data fit the model’s assumptions. One property of the Rasch model is that each item contributes the same amount of information to the household’s propensity for food insecurity. In other words, under the Rasch model assumptions, the raw score over all items (i.e., the sum of all the items) is a minimal sufficient statistic for the individual’s propensity. USDA uses the Rasch model assumptions and the sum of the raw scores to estimate propensity for food insecurity. To make interpretation of this propensity more easily understood, USDA uses cut points of these propensities to classify households as either food secure, food insecure without hunger, or food insecure with hunger. The use of IRT and specifically the Rasch model to measure food insecurity has been challenged in several settings. Bavier (2003, 2004) argues that hunger is a discrete, observable phenomenon that is really a consequence of food insecurity rather than a severe range of food insecurity and that it is not appropriate to use IRT models to measure hunger. Others have questioned whether the assumptions of IRT models are violated given the data generated from the current food insecurity instrument and in particular how well the data fit the Rasch model (Froelich, 2002; Johnson, 2004; Opsomer, Jensen, and Pan, 2003; Opsomer et al., 2002; Wilde, 2004b). The panel was asked to consider the appropriateness, in principle and in application, of IRT and the Rasch model as a statistical basis for measuring food insecurity. This section first asks whether hunger is a discrete, observable phenom-

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report enon and then considers how well the assumptions of IRT models and the Rasch model in particular fit the data from the current food security questionnaire. Is Hunger an Observable Phenomenon? The panel has distinguished three separate (though related) measurement concepts—food uncertainty, food insufficiency, and hunger. The panel has concerns that the current instrument may not be a good measure of any of these concepts and that more work is needed to elucidate the concepts to be measured and to develop the instruments to measure them. The panel’s argument about whether hunger is or is not a discrete, observable phenomenon is not specific to the current distinction of food insecure with hunger, but to any other conceptualization of the three experiences. The current concept of food insecurity with hunger is based on the definition developed by the Life Sciences Research Office (see Chapter 2) adopted by the original expert panel charged with developing a food insecurity measure. This concept is based on the definition of hunger as part of a continuum of food insecurity. If this is the concept that USDA is measuring, it is appropriate to consider hunger as a latent, continuous occurrence that can be measured using IRT models. The key factor is how the construct is defined and whether this definition can be validated. To the panel’s knowledge, no studies have tried to validate whether households classified as food insecure with hunger did indeed really experience hunger. Theoretically, such a validation study might be conducted by examining the caloric and nutrient intake of individuals in households that also responded to the Food Security Supplement questions. Practically, however, such a study would face severe data requirements—for example, data on food consumption and diet would need to be collected for the same individuals over a 12-month period (since the food security survey reference period is over 12 months), and it would have to be conducted for all household members, since the questions refer to households and individuals in the households. If such a study could be conducted, one might learn how well the definition and methodology of food insecu-

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report rity with hunger predict actual hunger. It would also give an assessment of how well the methodology actually fits the concept of actual hunger.1 Do the Data Fit the Model Assumptions? A separate question about the use of IRT models is whether or not data generated from the current Food Insecurity Supplement fit the assumptions of IRT models, and the specific assumptions of the Rasch model. There is some evidence that some of these assumptions may not hold, but further research is necessary. Opsomer et al. (2002) found that the assumption of the Rasch model that each item contributes equally to a household’s food insecurity propensity may not be met with the given data. This study found that some demographic characteristics of respondent groups were significantly associated with different responses for particular items. Wilde (2004b), using 2000 Food Security Supplement data, found results consistent with this finding, namely that those households with and without children responded differently to the adult-referenced food security items. This result means that raw threshold scores do not have equivalent meaning across the two different types of household, the method currently used to estimate the prevalence of food insecurity. Froelich (2002) found that the assumption of unidimensionality may not hold for households with children. Using data from the 18 items of the 1995 Food Security Supplement, this study found perhaps two dimensions according to whether the item was an adult/household item or an item for children. Nord and Bickel (2002) also found 1   Bhattacharya, Currie, and Haider (2004) examined how well the food security questions predict nutritional outcomes of individuals. Although the outcomes they examined are not actual hunger, one would expect some relationship between hunger and poor nutritional outcomes. The authors found that for children ages 2-5, the food security questions are poor predictors of child health; for adults ages 18-64, the food security questions can predict some nutritional outcomes as expected; and for adults over the age of 65, the food security measure is a very good predictor of nutritional outcomes. Other research has studied outcomes of food insecurity and could be evaluated to help address this issue.

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report evidence that the child and adult items are not on the same dimension and that this could result in an underestimation of the prevalence of hunger among children by about 20 percent at the national level. Johnson (2004) fit a 2-parameter logistic model instead of a Rasch model to the data from the 2002 Food Security Supplement. This model allows the discrimination parameter to vary across items. He found that the discrimination parameter is largest for the question on whether the respondent was ever “hungry, but didn’t eat,” suggesting that assuming that every item contributes the same amount of information on the household’s propensity for food insecurity (which the Rasch model assumes) may be not be appropriate, depending on the consequences of violating the assumption, given the intended purpose. Given the current definition of hunger, IRT models are appropriately suited to estimate levels of food insecurity. While there is evidence that the Rasch model may not be the best model for these data, the use of other IRT models should be explored. Even if further efforts are made to make clear any definition of hunger, it is still appropriate to use IRT models to probabilistically link the propensity for hunger (and food insecurity) to responses to a questionnaire on hunger (and food insecurity). Any definition and its measurement through a survey will not be an absolutely perfect set of questions, and respondents will interpret the same questions differently. Measurement error, the problem of recall error, and the social stigma of reporting a lack of access to food will also result in imprecision of any estimate of hunger. This is far from a fatal weakness in the use of a survey to measure food insecurity and hunger—many other concepts are measured this way (e.g., intelligence). Furthermore, any alternative to measuring hunger in a more direct way would be prohibitively costly and invasive and still would not address the socioeconomic component of hunger. Conclusion 2: Food insecurity is important to measure. It is a multifaceted concept, each facet of which is appropriate to consider as latent and continuous. It is appropriate to use item-response-theory models to measure these dimensions. However, the Rasch IRT model may not be appropriate in the current application. If the Rasch model is not appropriate, then using the sum scores of the items also is not appropriate.

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report Appropriateness of Threshold Scores to Demarcate the Categories of Food Security USDA totals the sum of affirmative responses to the food security scale questions and uses threshold scores to classify households as either food secure, food insecure without hunger, or food insecure with hunger. The thresholds for these scores are shown in Box 2-2. It is common and accepted practice to use such thresholds with IRT models. Johnson (2004) raises concerns specific to the use of the Rasch model for the use of these thresholds. He also explains that other IRT models can be used to generate scores to use with thresholds. The more controversial aspect of using thresholds is how they are labeled—particularly the labeling of the most severe threshold, food insecure with hunger (Bavier, 2003, 2004). The panel has concerns that a clear conceptual basis for measuring hunger has not been articulated. Conclusion 3: Threshold scores applied to estimates provided by IRT models can be used to categorize households into levels of food insecurity. However, the appropriate categories and labels need to be examined further. The panel has examined the use of IRT models to generate scores to use with thresholds of food insecurity. The more controversial aspect of using thresholds is how they are labeled—particularly the labeling of the most severe threshold, food insecurity with hunger. Interim Recommendation 2: In presenting the data in the annual food security reports, USDA should prominently report frequencies of the individual items that make up the scale. APPROPRIATENESS OF A HOUSEHOLD INTERVIEW SURVEY TO ESTIMATE THE PREVALENCE OF FOOD INSECURITY Food uncertainty and insufficiency are household-level concepts. It is appropriate to ask one household respondent regarding worry about food running out or whether members of the household had ever cut the size of meals. These questions can be asked of a large representative sample of households to provide estimates of food security

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report for the entire population and for subgroups of the population. The agencies that developed the questionnaire were diligent in developing a relatively short instrument that could be attached to a number of household surveys, including the Current Population Survey. Other major household surveys also have included versions of the instrument. Some examples are the National Health and Nutrition Examination Survey, the Survey of Program Dynamics, the Early Childhood Longitudinal Survey, and the Panel Study of Income Dynamics.2 Conclusion 4: A household interview survey may be one appropriate vehicle to query households about their food security experiences and to measure the prevalence of food insecurity among households. Theoretically, it is also reasonable to consider questions of the frequency and duration of food insecurity using a household survey. The current version of the 18-item food security scale used in the CPS does not collect much information relevant to frequency and duration. Respondents are generally asked whether they experienced an event or perception “often, sometimes, or never” over the past 12 months. These broad categories do not provide enough detail to give a clear picture of frequency. Also, these broad categories are not used when the answers are scaled to classify individuals as food secure, food insecure without hunger, or food insecure with hunger. Instead, responses are coded as either affirming the question or not. The only question in the instrument that does ask for more detail concerning how often the respondent or other adult in the household did not eat for the whole day (question 10 for households without children and question 17 for households with children; see Box 2-1) is also limited to broad response categories concerning the number of months in which this occurred. Responses are coded as affirming if it is “almost every month” or “some months but not every month” and not affirming if it is only “only 1 or 2 months.” These responses are used in the scale. The full Food Security Supplement containing 54 questions in- 2   See Wilde (2004a) for a more thorough list of surveys to which the Food Security Supplement has been attached.

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report cludes additional questions not used to determine food security status. It includes questions about behavior and experiences related to food security in the past 30 days and questions about how often and the number of days that it happened. This information has been used by some researchers to give a better sense of the frequency of food insecurity (Nord, Andrews, and Winicki, 2002). If the goal is to obtain better information about the frequency and duration of food insecurity, USDA might consider using questions from the full supplement and not just the 18-item scale. In addition to the issue of measuring the frequency of food insecurity, there is interest in measuring the duration of food insecurity, as well as changes over time, at the individual and household levels. Including the supplement in a longitudinal survey, such as the Survey of Income and Program Participation, which interviews households every 4 months for 2 to 4 years, would facilitate the analysis of duration and change over time. In addition, the design of the CPS could also allow for some longitudinal analysis of the food security of some households. The survey interviews the same households for 4 months, does not interview them for the next 8 months, and again interviews them for 4 months following the 8-month break; a portion of the households could receive the Food Security Supplement twice while they are part of the CPS sample. The panel intends to explore these and other options for assessing the frequency and duration of food security in Phase 2 of the study. Also, a household-based survey is limited with respect to coverage of the U.S. population. In general, such surveys do not include persons living in group quarters, those who are institutionalized, or the homeless. There is reason to believe, therefore, that household-based surveys may not adequately cover individuals who are food insecure yet do not live in households. The homeless population is an important group to include in estimates of the prevalence of food insecurity. Although USDA has attempted to compensate by including in its latest annual report of food security estimates of the use of food pantries and emergency kitchens (Nord, Andrews, and Carlson, 2004), much work in this area needs to be done. Although a household survey may be appropriate for measuring

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report food insecurity, the current set of questions used for these concepts combines individual-level experiences with household-level experiences. For example, questions 4 and 9 ask whether the respondent or other adults in the household ever cut the size of or skipped meals or did not eat for a whole day, while questions 6-8 ask only whether the respondent ate less, went hungry, or lost weight and do not ask about other adults in the household (see Box 2-1). While it seems reasonable to address some of these questions only to the respondent (e.g., were you ever hungry?), it is not clear why other questions also ask about the other adults in the household. The problem is not just the use of a household survey, but issues of questionnaire design and the selection of respondents in the participating household also need to be considered. The panel has raised a number of issues about the questions in the CPS Food Security Supplement. For example, these questions mix references to the household, adults in the household, and the individual respondent. Furthermore, the 12-month reference period is too long and may make accurate respondent recall difficult. The panel urges USDA to build on the evaluation of these questions that was conducted at the onset of the food security measurement project and initiate a program to further test and improve the currently used questions. Interim Recommendation 3: Given that the concept of food insecurity is multifaceted, USDA should consider which specific facets should be measured. APPLICABILITY OF THE FOOD SECURITY MEASURE FOR ASSESSING THE EFFECTIVENESS OF FOOD ASSISTANCE PROGRAMS The Government Performance and Results Act of 1993 requires that federal departments and agencies within departments develop a strategic plan for multiple years and annual performance plans with specific targets, and then report annually on the agency’s success in meeting those targets. The 2000-2005 strategic plan of the Food and Nutrition Service—the agency with responsibility for the major food assistance programs in the United States, including the Food Stamp

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report Program, the Special Supplemental Nutrition Program for Women, Infants, and Children, and the National School Lunch Program—states a goal for the agency, in delivering the food assistance programs, to reduce the prevalence of food insecurity with hunger among households with income under 130 percent of the federal poverty standard.3 At the present time, USDA uses prevalence estimates of food security to report annual performance in the execution of the strategic plan. The panel was asked to comment on the applicability of the food security measure for such purposes. Estimates of the prevalence of food insecurity status are best used as a tool for monitoring the well-being of a population of interest. These estimates can show how a population is faring over time, whether its food insecurity status is improving, deteriorating, or remaining the same. These estimates can also serve as an important surveillance tool for identifying whether specific subgroups are doing poorly (e.g., the elderly, rural versus urban groups, regions and states, family and household structures). Such monitoring efforts are important because they may help identify where additional assistance is needed or where further investigation is needed to understand why the program or policies may or may not be working. These prevalence estimates, however, are not well designed for use in measuring progress toward meeting the goals of the Government Performance and Results Act for food assistance programs. Evaluating the efficacy of food assistance programs by examining fluctuations in prevalence of food insecurity has little meaning. The estimates do not measure anything directly tied to the food assistance programs (such as improved nutritional status because of program participation). Thus, effective performance of the programs cannot be directly linked to improved food security status, nor can a deterioration of food security be attributed to failure of these programs. Many factors can result in a change in food security status (e.g., changes in the economy of the nation, other programs or policies, demographic changes). For ex- 3   See Wilde (2004a) for a discussion of the use of the food insecurity measure for other performance assessment tools—for example, use with Health People 2010 goals.

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report ample, if USDA’s goal is to promote the food security of households below a certain level of poverty, and if the economy goes into a serious recession so that the food security of these households decreases, then USDA may not meet its goal of reducing food security even if the existing food assistance programs were very effective. The panel recognizes that finding appropriate measures of performance is difficult in general, but as currently fielded in national surveys, the food security estimates are not well suited for this purpose. Conclusion 5: Prevalence estimates of food insecurity as currently obtained are not well suited for evaluation of the effectiveness of food assistance programs. It is unclear that monitoring the prevalence of food insecurity at national and subnational levels would be suitable for evaluation of these programs. With the appropriate evaluation design, estimates of the prevalence of food insecurity could potentially be used as an outcome measure in studies assessing the effect of participation in food assistance programs, although they may not fully capture the effects that programs may have. But such studies would require finding a suitable comparison group to which such an outcome of program participation could be compared. Wilde (2004a) details some of the results of studies that have attempted to set up comparison groups or exploit longitudinal data to attempt to control for nonrandom participation in food assistance programs. If an experimental design was chosen so that participants in the food assistance program were randomly assigned to a new program or component of a program, then the food insecurity measure could potentially be used as an outcome. To date, however, such an experimental design has not been implemented in evaluation of food assistance programs. SURVEY OPTIONS FOR MEASURING FOOD INSECURITY As stated earlier, USDA bases its annual report and estimates of the prevalence of food insecurity on data collected from the Food Security Supplement to the CPS. The food security questions that are included in the CPS also appear on a number of other nationally rep-

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report resentative surveys, for example the National Health and Nutrition Examination Survey, the Survey of Program Dynamics, and the Early Childhood Longitudinal Survey. Attaching the food security questions to the CPS for the official estimates has several advantages. The CPS is the largest of these surveys and comes closest to fulfilling one of the key needs for the project—state-level monitoring of food security. Data from the CPS are produced regularly and released on a timely basis. There are however, reasons to consider other surveys as the primary vehicle for, or to augment, a food security supplement. For example, the National Health and Nutrition Examination Survey (NHANES) collects detailed anthropometrical, medical, and nutritional information on all sample persons. These data are very valuable for understanding the links between food insecurity and health and food insecurity and diet. Food security and sufficiency have been measured in NHANES since NHANES III. From 1999 to 2001 the food sufficiency question was expanded to include food adequacy. The 18-item household food security survey module used in the CPS—which was developed by USDA in collaboration with an expert panel and a federal interagency working group that included NCHS—has been included in NHANES since 1999. This module has been used in a number of other surveys. Beginning in 2000, NHANES included questions about the individual-level hunger of participants age 16 and over and of a proxy regarding children under age 12. Beginning in 2005, NNHANES will ask 12–16 year olds these questions. These are questions that ask about the individual’s experience, in contrast to the household level questions, which just ask about anyone in the household. These individual level responses can then be assessed in relation to individual measures from other examination components. NHANES staff has been involved in developing a short version of the module—six items rather than 18 items. NHANES used to be conducted on a periodic basis, but it is now conducted on a continuing basis and two years of data are cumulated and published every two years The NHANES samples 5,000 households annually, so it is not large enough for annual estimates or for subgroup or state analysis. The National Health Interview Survey collects basic health infor-

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report mation for monitoring the health of the population. It has a larger sample of 40,000 households, approximately 100,000 people. Most of the information, however, is obtained on a sample adult or child. It could also be considered as a potential vehicle for a food security supplement, although the data that it collects on income and demographic factors are somewhat different from those in the CPS. In Phase 2 of the study, the panel will further consider whether the CPS is the most appropriate vehicle to attach the food security supplement or whether other surveys should be used instead of, or in conjunction with it. An issue related to the design of the survey used to measure food security is the measurement of the frequency and severity of episodes of food insecurity and their duration. As noted earlier, it is difficult to assess the frequency, severity, and duration of episodes of food insecurity by using the CPS data. Only a few of the questions ask how often events related to food insecurity happen, and those questions ask very little detail about how intense they were or how long they lasted. Such data would be important to understand the mechanisms that cause and may help reduce food insecurity. It is possible that the questions could be better designed for assessing frequency and duration. The use of longitudinal surveys, such as the Survey of Income and Program Participation, could also be used to assess frequency and duration. The panel urges USDA to conduct further work to develop ways to measure the frequency and duration of food insecurity. Interim Recommendation 4: USDA should explore the use of alternative or additional surveys to estimate the national prevalence of food insecurity. In the meantime, USDA should continue to measure food insecurity as currently conducted using the Food Security Supplement of the Current Population Survey. Such exploration should include the assessment of the extent of coverage bias in estimates of food insecurity and hunger based on a household sample frame. Issues of the level of accuracy and precision also should be explored. Discussion of these issues, including ample size, periodicity of survey and therefore the estimates, and response

OCR for page 30
Measuring Food Insecurity and Hunger: Phase 1 Report burden, will be addressed in Phase 2 of the study before any specific recommendations can be made. CONCLUDING REMARKS The panel commends USDA and DHHS for the careful and extensive work that has gone into the development of the food security measure. The panel further recognizes USDA’s continuing efforts to evaluate and improve the measure. Overall, the panel thinks that the highest research priority is to develop a clear conceptual definition of hunger and to continue and expand the evaluation and testing of the questions in the current CPS Food Security Supplement. This Phase 1 report provides USDA with the panel’s preliminary guidance, based on discussions during the workshop and panel deliberations, for improving the food security measure. It also has made interim recommendations as guidance to USDA for the interim period until completion of the panel’s work in Phase 2 of the study and the completion of additional recommended research by USDA. The panel in its final report will examine in more depth the issues raised in the workshop relating to the concepts, definitions, measurement issues, and analytical methods used to measure food security; possible alternative survey vehicles for measuring the concepts instead of or supplemental to the CPS; and the problems of special populations. In addition, the panel will address and make recommendations as appropriate on the tasks specified for Phase 2 and listed in Chapter 1.