Appendix D
Approach to Literature Review
LITERATURE SEARCH STRATEGY
To identify primary literature and review the most relevant scientific publications available, staff initially conducted general searches on topics relevant to subject areas identified in the committee’s statement of task. Using the results of this primary search, staff then developed key search terms based on relevance to the study objectives. Searches were limited to English-language publications. After the initial search, staff designed a comprehensive search strategy in consultation with librarians at the George E. Brown Jr. Library of the National Academies. Search terms incorporated relevant MeSH (Medical Subject Headings) terms, as well as terms from the EMBASE thesaurus. Data-bases searched included Academic Search Premier, Agricola, Congressional Research Service, EconLit, ERIC, MEDLINE, NTIS, PsycINFO, and Web of Science (Science Citation Index and Social Science Citation Index). Database searches also included publications from the National Bureau of Economic Research and the U.S. Department of Agriculture’s Economic Research Service. Table D-1 provides an example of how the searches were conducted; only a subset of terms from the overall search are shown because including the entire search was impractical.
Staff limited the searches to publications dated 2000 and later. The initial search retrieved more than 1,100 citations, which were then sorted into predefined topics identified by the committee. The topical search terms included
• access to food,
• cost of food,
TABLE D-1 Example of Searches Using Key Words to Identify Relevant Literature
Search Number |
Search Terms |
Number of Hits |
1 |
food stamp program |
360 |
2 |
(“food stamp?” or “supplemental nutrition assistance” or “thrifty food plan”),sh,de,hw,to,ab. |
1,663 |
3 |
(SNAP adj3 {benefit? or assistance or secur* or insecure* or food)). ti,ab. |
16 |
4 |
or/1-3 |
1,666 |
5 |
limit 4 to (English language and yr=*2000 –Current*) |
377 |
6 |
remove duplicates from 5 |
366 |
7 |
food availability/ |
680 |
8 |
food security/ |
1,221 |
9 |
((food adj access*) or (food adj2 (insufficien* or insecure* or secur*))). Ti,ab. |
2,319 |
10 |
Or/7-9 |
3,429 |
11 |
6 and 10 |
55 |
12 |
income/ or household income/ or net cash income/ |
3,253 |
13 |
exp social welfare/ |
825 |
14 |
welfare.ti,ab,de,hw,sh. |
13,865 |
• economic insecurity,
• food choice and cultural preferences,
• food policy,
• health and nutrition effects, and
• program access.
ORGANIZATION OF THE LITERATURE
Relevant references obtained from the initial search were screened and categorized according to the research taxonomy shown in Box D-1. The committee organized the publications obtained from its search into three focus areas:
1. content area,
2. design of research study, and
3. source of paper/publication.
The topics within each focus area of the taxonomy were then expanded. Key citations were selected by committee members and annotated by staff. Reference lists of key citations were provided in tabulated format to facilitate the committee’s review and selection of critical publications for inclusion in this report.
Content Area
A. Access to food
a) Physical access/transportation
b) Financial access
c) Total resources available and resources available for food
B. Cost of food
a) Regional differences
b) Urban/suburban/rural differences
c) Nutrient density (less vs. more healthy foods)
C. Food choice/cultural preferences
a) Effect of individual and group characteristics (e.g., age, race/ethnicity, chronic illness, individual/group preferences)
b) Food choices related to the Dietary Guidelines for Americans (DGA) and nutrient density
c) At home/away from home
D. Food policy
a) SNAP program design (program characteristics)
• % income spent on food, nutrition education within SNAP, allowed retail outlets, restrictions and incentives
b) Other food policies as they impact SNAP
• DGA, other food assistance programs (Special Supplemental Nutrition Program for Women, Infants and Children [WIC]; National School Lunch Program [NSLP]/School Breakfast Program [SBP]); cash-out states, access to farmers’ markets and farm stands
E. Program access/participation
F. Health and nutrition effects of SNAP participation or nonparticipation
a) Health (effect on chronic and acute illness, e.g., diabetes, immunity)
b) Nutrition and dietary intake
• Nutrition and dietary intake
• Nutrition and dietary intake
c) Obesity
G. Economic/food insecurity effects of SNAP participation or nonparticipation
a) Measures of economic/food insecurity
b) Incidence and prevalence of economic/food insecurity
Design of Research
A. Descriptive/observational study
a) Quantitative reports (including secondary data analysis)
b) Qualitative reports
B. Intervention impact study
a) Quasi-experimental design (including secondary data analysis)
b) Experimental design
C. Literature review or policy brief
a) Systematic reviews
b) Meta-analyses
Source of Paper/Publication
A. Journal article
B. Government report
C. Other (e.g., nonpublished or working paper, report for advocacy group or foundation)
To evaluate the results of the literature search, the committee first used the organizational scheme described above to consider (1) the validity and (2) the generalizability of the studies and publications identified in its research taxonomy. Although the committee appreciates the range of evidence types and the strengths and weaknesses of various methodological approaches, it determined that the most useful approach was to examine research questions relevant to its statement of task with observational studies conducted in a real-world context. From this perspective, the quantitative approach of the randomized controlled trial was deemed limited because of the use of an artificial setting and the need to control variables that may have application to understanding and interpreting consumer behavior. Relevant experimental studies were not identified in the search and thus were not included in the evidence review.
The types of research studies that provided the most valid evidence for examining the behavior of participants in food assistance programs were field experiments and survey-based studies. The committee identified a range of observational evidence, including population-based surveys and impact studies based on secondary data analysis, that reflected the complexity of variables relevant to the outcomes of interest, as well as the generalizability of the evidence to pertinent research questions. The committee considered the merits of these studies on the basis of the methodological approach used and the overall quality of the research. Additional support for the findings from observational studies was obtained from evidence-based reviews, meta-analyses, and qualitative studies.
Finally, in addition to the body of peer-reviewed evidence from journal publications, the committee considered peer-reviewed evidence from government reports, in particular, relevant reports from the U.S. Department of Agriculture’s (USDA’s) Economic Research Service, as well as nonreviewed publications from stakeholder and nongovernmental organizations. Although government reports undergo a rigorous peer review, the process differs from that for peer-reviewed journal articles in that the review is not blinded. Specifically, the Office of Management and Budget Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies state: “In general, an agency conducting a peer review of a highly influential scientific assessment must ensure that the peer review process is transparent by making available to the public the written charge to the peer reviewers, the peer reviewers’ names, the peer reviewers’ report(s), and the agency’s response to the peer reviewers’ report(s).”1 While
____________________
1Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, Federal Register, Volume 67, Issue 24, p. 5465 (February 5, 2002).