Closing Session: Wrap-Up and Methodological Issues and Data Considerations
The final session of the workshop included summaries of key points made during the eight sessions, two presentations related to methodological issues and data considerations, an open discussion, and a closing statement from the chair.
HIGHLIGHTS OF PREVIOUS SESSIONS
Presenters: Workshop Moderators
Each moderator presented a brief summary that focused on the research priorities identified in his or her session. Unless indicated otherwise, the research topics are not listed by order of priority in the session summaries. In no instance do they represent group consensus.
Gail G. Harrison
A key message, said Harrison, was to “put the ‘W’ back in WIC”—that is, focus research on the preconceptional and interconceptional periods. She mentioned biases that are linked with studies of the effects of the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) on low birth weight and prematurity and the probability that WIC would have a small impact on those outcomes. Maternal health may be viewed as a pathway to improved outcomes for the baby, such
as size at birth that is appropriate for gestational age, and the mother. Thought needs to be given to methods for studying how WIC interventions can affect the woman’s risk of such outcomes as gestational diabetes and obesity, and especially how WIC could help reduce risks during the first pregnancy. Currently, the interconceptional period provides an opportunity to conduct research on how WIC can influence maternal health and behaviors. Another key message was that researchers should study long-term health outcomes.
Suggested methods for studying birth outcomes included collaboration with the National Children’s Study and setting priorities with the goal of gaining the greatest benefit from the research. One way to gain benefit would be to focus on women with prior adverse outcomes or on communities with marked disparities. In a study of relationships of pregnancy outcomes with vitamin D intakes, for example, one could select, as subjects, women at risk for vitamin D deficiency because of dark skin color or lack of exposure to sunlight. Harrison closed her summary by showing selected slides used by the presenters (also see Chapter 2) and reemphasizing the following types of studies:
Observational studies on the effect of interconceptional nutrition on birth and long-term child health outcomes.
Nutritional intervention studies that begin at interconception.
Nutritional research that focuses on women’s health before, between, and beyond pregnancy.
Studies designed to examine the effects of interventions to reduce preconceptional smoking and obesity, to achieve prenatal weight gain that falls within Institute of Medicine recommendations, and to support breastfeeding.
Overweight and Obesity
Patricia B. Crawford
Obesity has become the foremost health problem of children, said Crawford, and WIC is well positioned to address the problem. A number of risk factors for obesity have been identified, and they are applicable to the diverse population served by WIC. The speakers agreed that studies should focus on messages aimed at reducing those risk factors, and they agreed on study design. In particular, the speakers proposed a multi-stage approach that would involve many community partners as messengers. The first step would be to conduct formative research to develop the preventive messages and to consider the context in which they will be received. The process would include selecting the message that has the most promise for reducing
the risk of obesity and also the least potential for harm. Other steps include determining the target behaviors and other positive effects they might have, carrying out the intervention, and examining the fidelity of the intervention. For the types of studies discussed, outcome measures would be behavior changes, not changes in body mass index. Crawford gave the example of developing and testing messages to reduce the excessive consumption of sugar-sweetened beverages.
With respect to Whitaker’s mention of the value of working with the messenger as well as addressing the message, Crawford said that her studies indicate that working with the messenger can change the way he or she imparts the message and how the message is received. Adding to the speakers’ emphasis on the use of a coordinated community approach and on expanding communication with such other partners as Head Start and pediatricians, Crawford said that it would be very appropriate to include other partners as well.
After briefly discussing what is already known about breastfeeding and should therefore be excluded from research priorities (breastfeeding benefits, adverse effects of formula distribution, the importance of no formula during the first month after delivery, a continuum of sensitive individualized breastfeeding care), Labbok first focused on the research topic emphasized during the session on breastfeeding, that is, staffing issues related to peer counselors (their training, competencies, and roles) and lactation consultants and their effects on breastfeeding outcomes. The research would benefit from a phased approach with concurrent elements and program monitoring. Ecological and qualitative studies should be included to identify the issues that need to be studied quantitatively and to gain a better understanding of what had been learned. Perhaps the central piece of research would be group randomized controlled trials that address the peer counseling questions.
Labbok also mentioned three breastfeeding-related research topics that emerged strongly during discussion periods: (1) methods to achieve continuity of care and linkages to the health-care system; (2) perceptions and use of the new WIC food packages and key data needed to examine this issue; and (3) measures to address the inverted economic pressures that the provision of formula causes, perhaps starting with the testing of generic labeling of WIC-provided formula. She encouraged the U.S. Department of Agriculture to consider them for the WIC research agenda as well.
Food Insecurity and Hunger
Black began her summary by reminding the audience that there are more poor households now than when WIC started and by restating Weill’s question, “Is WIC a public health program or an anti-hunger program or an anti-poverty program?” After emphasizing the national importance of reducing food insecurity, its serious consequences for the population served by WIC, and WIC’s key role in reducing disparities early in life, Black spoke about the importance of considering the multiple pathways that connect food insecurity with children’s well-being. Promising research topics raised during the food insecurity session include the following:
Does WIC reduce the likelihood of food insecurity?
What effect does WIC have on families that are food insecure?
Does WIC affect the stress associated with food insecurity?
Does WIC reduce the effects of food insecurity on outcomes?
In examining associations among food insecurity, access to WIC, and WIC’s effects on children, it may be useful to consider Frongillo’s stress elimination model, compensation model, and buffering model.
Black ended by reiterating Neuberger’s concern about how the growing national deficit may lead to efforts to reduce funding for WIC, which could have serious consequences for the nation’s children.
Dietary Intake and Nutritional Status
According to Devaney, the basic general question from the session was, “Does WIC participation lead to better diets for women, infants and children?” Specific research priorities proposed during the session included the following:
The effect of the new food package on breastfeeding and the timing of complementary feeding.
Comparative studies of the consumption of foods provided in the WIC food packages among different groups and under different conditions: specifically, by WIC participants and non-participants at the present time, and by WIC participants prior to and following implementation of the new WIC food packages. Data from the National Health and Nutrition Examination surveys could be used
for these purposes and also to compare nutrient intakes pre- and post-implementation.
Useful data collection and evaluation efforts were identified:
Expansion of the types of WIC administrative data to be used for evaluation purposes to include voucher redemption data and certification/recertification data;
The use of the administrative data for longitudinal analyses as well as for cross-sectional analyses; and
A periodic national survey of WIC participants (and perhaps of non-participants), which, if it had been initiated earlier, would have provided data to answer many of the research questions raised during this workshop.
Shannon E. Whaley
Whaley began by saying that a considerable amount is known about nutrition education in WIC and that evidence shows that nutrition education can lead to behavior change. She then said, however, that new data collection strategies are needed to study the effects of the nutrition education that WIC provides. One useful approach would be more consistent reporting of evaluations using non-randomized designs to allow datasets from small studies to be merged.
Using information from the nutrition education session, Whaley proposed a three-step nutrition education research agenda:
Conduct qualitative or survey work on how nutrition education is being delivered by WIC across the nation.
Determine the outcome measures of highest interest and greatest relevance. What is it that participants want to learn? For example, would obesity prevention messages be of highest priority, and would the messages be meaningful to WIC mothers post partum?
Study the comparative effectiveness of various approaches, including, for example, a group approach, a one-on-one approach, an online approach, and a social networking approach. Participants should be assigned to a strategy, probably through the random assignment of WIC sites. The research plan would include delivering the education using the specified approaches, measuring the outcomes, and replicating the studies at different sites.
In addition, Whaley suggested carrying out studies to determine which nutrition education strategies will be most effective for participants who must change the way they shop for WIC foods because of the transition from vouchers to the electronic benefits transfer system.
Health Care and Systems Costs, Benefits, and Effectiveness
Devaney suggested that the logic model presented by Findley (see Chapter 8) was very comprehensive and could provide a useful approach for examining the various research priorities suggested in all the workshop’s sessions. In conducting health risk–benefit assessment, Jensen had emphasized that careful attention should be given to identifying and characterizing health effects, to assessing dose–response relationships, and characterizing benefits. Devaney said that the key message was to carefully determine which health outcomes should be investigated.
After considering the presentations by Findley and Jensen and the response by Beuscher, Devaney identified the following research priorities related to health care and systems costs, benefits, and effectiveness:
Analyses of WIC’s long-term (5-year and 10-year) effects on such aspects of children’s health as relationships among breastfeeding, body mass index, health care utilization, and cost, perhaps using data from the National Survey of Children’s Health;
Analyses concerning maternal health, such as relationships among dietary changes, weight loss counseling, and obesity and between gestational weight gain and the risk of type 2 diabetes; and
Updating of some of the early WIC evaluations using more types of administrative data and improved statistical methods.
The Reach of WIC
Jackson P. Sekhobo
Sekhobo emphasized that research on the reach of WIC should collect information related to how well WIC is meeting its mission of providing supplemental foods, nutrition education, and referrals to its clients—that is, to pregnant women, breastfeeding and non-breastfeeding postpartum women, and to infants and children up to the age of 5 years. In order to claim that WIC is having beneficial effects on health, it is essential to document that WIC is reaching people, especially through nutrition education. Documenting the reach of WIC might make it possible to determine the
proportion of the population-based health improvements that can be attributed to WIC for WIC-eligible populations.
Two overarching research questions were highlighted by Bartlett: (1) How well is WIC working? and (2) How can WIC work better? To address these broad issues, a number of specific research topics were suggested during the session, including analyses of WIC participation, potential improvements to the program, evaluation of the new food package, determination of the benefits and drawbacks of expensive ingredients added to infant formulas, and evaluation of the impact of other public assistance programs (such as the Supplemental Nutrition Assistance Program (SNAP) on participation in WIC. Useful methods that were identified include the REAIM (Reach, Efficacy, Adoption, Implementation, and Maintenance) model for evaluation research and the application of the logic model for effects. Research on the effects of the new food package on redemption would benefit from the use of a pending database that lists state policies before and after the introduction of the new food packages.
METHODOLOGICAL ISSUES AND DATA CONSIDERATIONS
Dietary and Nutrition-Related Issues
Presenter/Discussant: Philip Gleason
Two major challenges in studying the dietary and nutrition-related impacts of WIC are (1) determining how best to measure key outcomes (dietary intake, food insecurity/hunger, biomarkers of nutritional status, and anthropometric outcomes such as body mass index and obesity) and (2) developing a study design that can account for participants’ selection into WIC.
Measurement issues include identifying the best method to use, deciding on the timing of the reference period, and determining the amount of time required for WIC services to influence outcomes. The measurement of dietary intake typically depends upon 24-hour recalls, which are reports of intake over one particular day rather than a subject’s usual intake and which may be subject to misreporting. Using methods developed at Iowa State University and the National Cancer Institute, usual nutrient intake can be estimated for groups if at least two 24-hour recalls are available for a subsample of the population. Estimating the usual food intake is more challenging and may require at least 2 days of recalls for the entire sample. Regression analyses can be used to estimate the effect of WIC on mean
usual intake, but they cannot be used to estimate WIC’s impact on measures of the distribution of usual intake (such as the proportion of participants with inadequate intake of a particular nutrient).
Timing may be especially important in evaluating such outcomes as the change in iron status (which requires sufficient information about WIC participation over time) and food insecurity (for which the reference period is 12 months).
When selecting among feasible design options, researchers should be careful to take both selection bias and representativeness into account. The options that compare WIC participants with some set of non-participants include random assignment and four non-experimental designs: (1) regression discontinuity, (2) comparison group, (3) instrumental variables, and (4) fixed effects.
Random assignment It is likely that random-assignment studies of WIC will only be possible on a small scale. Possible studies include the different approaches to nutrition education mentioned earlier, isolated situations of oversubscription, and the testing of aspects of WIC policy.
Regression discontinuity Regression discontinuity compares people who are just below the income eligibility threshold with those who are just above it. This design requires using the measure of income that is used by the program rather than survey data. Although it provides an unbiased estimate of the impact of WIC, it is not necessarily applicable to the WIC participants with the most need.
Comparison group The comparison-group design usually compares WIC participants with income-eligible non-participants. This design can be applied to groups of the neediest WIC participants, but it has a major drawback in that it is impossible to know if the analysis has accounted for all the non-observable differences between the two groups.
Instrumental variable The instrumental-variables approach uses comparison groups whose participation decision was based on a factor that is strongly correlated with WIC participation but cannot be directly related to the outcome, such as distance of residence from the WIC clinic. Although the method is a sound one, it often has low external validity; and researchers attempting to use instrumental variables to estimate the impact of WIC have generally concluded that they were unable to identify an instrument that works well.
Fixed effects The fixed-effects approach uses variation in WIC participation across time or across siblings within a family, or both, to estimate impacts. Among its disadvantages are a lack of control for unobserved factors that change over time and a lack of sufficiently detailed longitudinal data about participation over time and outcomes.
Selection bias is a particular problem for WIC research. Because the most rigorous designs are likely to have limited external validity, Gleason recommended that the research agenda rely on multiple methods. Timing issues are especially important to consider in studying WIC, in large part because participants’ status changes frequently, some outcomes may require that interventions occur over an extended period, and some long-term outcomes are of interest. Large-scale longitudinal data on both WIC participants and non-participants would be especially helpful.
Presenter/Discussant: Theodore Joyce
This presentation by Joyce included a critique of recent studies with large sample sizes that used administrative data, and it provided examples of the four non-experimental study designs described by Gleason (see the preceding section). In his critiques, Joyce emphasized four points:
WIC research requires observational studies.
Large studies are not necessarily better than smaller ones.
The magnitude of the effect must be both plausible and clinically meaningful.
Sound research requires appropriate outcomes with confirmatory evidence of causal pathways.
Comparison Group Studies
A large study conducted in Washington state (El-Bastawissi et al., 2007) reported that the rate of preterm births among WIC mothers was 2.7 percentage points lower than the rate among non-WIC mothers, but there was no difference in the rate of low birth weights among full-term babies. Furthermore, the investigators reported that WIC was protective for those who enrolled late in pregnancy (which Joyce pointed out is a measure of inadequate prenatal care). Joyce considered the difference to be implausible and suggestive of gestational age bias (see Chapter 2).
A large study conducted in Florida (Gueorguieva et al., 2009) used redeemed food vouchers to measure the intensity of WIC participation. The researchers found that a 10 percent increase in participation intensity was accompanied by a modest 2.5 percent decrease in small-for-gestational-age births. The matching algorithm was sophisticated, and Joyce considered the outcome to be appropriate and the findings plausible.
Joyce provided the following hypothetical example to illustrate the use of an instrumental variable to simulate a randomized experiment:
In state X, prenatal WIC funds were depleted several months before the end of the fiscal year. Thus, the eligible pregnant women who conceived near the end of fiscal year would be unable to enroll early. Assuming that the month of depletion is known, this change in funding would provide a natural experiment that would allow investigators to compare outcomes of pregnant women before and after the cutoff of funds.
Using the instrumental variables approach, Figlio et al. (2009) compared rates of low birth weight among women who were marginally income eligible before federal income reporting requirements were made stricter with women who were marginally income eligible after the change. These investigators reported what Joyce considered to be an implausibly large decrease—13 percentage points—in low birth weight among the mothers who met the threshold compared with those who were above it.
Joyce offered two examples of the application of fixed effects that used discordant pairs of infants, one of whom was born while the mother was on WIC and the other born when the mother was not. Both studies (Foster et al., 2010; Kowaleski-Jones and Duncan, 2002) reported what Joyce said were implausibly large improvements in outcome for the WIC mothers—a 6.6 percentage point decrease in low-birth-weight births in the first, and a 9 percentage point decrease in the second.
Propensity Score Matching
Propensity score matching has become popular in studies of WIC, but Joyce said it is more suited for use as a diagnostic. That is, propensity score matching can be used to check on the suitability of non-participants as a comparison group. The matching technique is more likely to be useful if the
WIC participants and non-participants have similar propensity scores (i.e., scores that reflect the probability of participating in WIC).
Methods for Improving Studies
Joyce said that it is important to conduct some randomized design studies because they control variation in the intervention and they are useful for testing the observational methods. That is, they provide information on how well regression, propensity score matching, and instrumental variables control for unobservable variables. He suggested that the research agenda include a small number of randomized studies of augmented care (called WIC Plus by earlier presenters). In addition, researchers could exploit natural sources of exogenous variation, such as twinning (if naturally occurring) and changes that provide sharp, transparent breaks in the availability of WIC services (e.g., the end of infant formula feeding at age 1 year, incidents that cause extensive damage to WIC offices and cause their closure, or variation in the timing of the rollout of a programmatic change). Regression discontinuity designs also can be helpful. Joyce encouraged researchers to think creatively, saying that WIC is paying women not to breastfeed and asking why it wouldn’t be possible to pay them to breastfeed.
Joyce concluded by saying that all studies are biased until proven otherwise, sources of variation must be transparent, causal pathways need to be elucidated, and outcomes will change slowly because behavior changes slowly. A long-term research agenda would likely improve WIC research.
Moderator: Gail G. Harrison
The open discussion focused primarily on methodological issues. Participants made new or clarifying points that included the following:
The more that dietary intakes vary from day to day, the larger the subsample should be of people providing 2-day diet recalls, and there may be some value to obtaining information on a third day’s intake.
The end of a certification period may provide a useful break in receipt of services.
Qualitative research is helpful in explaining why models fail or why research methods are producing erroneous answers, but, Joyce said, it does not provide information about causality.
Fixed methods control only for time-invariant factors.
In WIC, cluster-randomized trials would involve the random assignment of clinics to treatments. The outcomes are measured at the individual level. Cluster-randomized trials ordinarily require much larger samples than do individually randomized trials. However, it appears that the sample size requirements for cluster-randomized trials of dietary intake are somewhat smaller than the requirements for studies of other outcomes.
A better synthesis is needed of biology, sociology, and research methods. What is implausible to the methodologist may be completely plausible to a biologist and pediatrician, for example.
Presenter: Gail G. Harrison
Harrison highlighted the following key research topics that were identified during the workshop sessions:
Effects of the revised WIC food package on diet and many other behaviors. The uneven rollout of the change affords good research opportunities.
Long-term effects of WIC on both maternal and child health, e.g., WIC plus 5 years, WIC plus 10 years, and the child upon reaching 21 years.
The economic context for families and the implications for food security.
Regarding methods, Harrison emphasized the importance of selecting appropriate outcomes, the need to consider possible biases and timing issues, the potential value of the phased approach described by Whitaker and Taveras, the value of the applying the logic model described by Findley, and the need to rely on multiple studies with multiple methods, as indicated by Gleason and Joyce. Over the course of the workshop, participants called for both programmatic and research partnering, and the National Children’s Study was mentioned many times as a possible resource with which to address some of the long-term research topics.
At Harrison’s invitation, Jay Hirschman made a final statement and once again encouraged all present to send their recommendations for the top three research priorities to the Institute of Medicine website within the next few days so that the recommendations could be considered by the U.S. Department of Agriculture within its tight timeline.
El-Bastawissi, A. Y., R. Peters, K. Sasseen, T. Bell, and R. Manolopoulos. 2007. Effect of the Washington Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) on pregnancy outcomes. Maternal and Child Health Journal 11(6):611–621.
Figlio, D., S. Hamersma, and J. Roth. 2009. Does prenatal WIC participation improve birth outcomes? New evidence from Florida. Journal of Public Economics 93(1-2):235–245.
Foster, E. M., M. Jiang, and C. M. Gibson-Davis. 2010. The effect of the WIC program on the health of newborns. Health Services Research 45(4):1083–1104.
Gueorguieva, R., S. B. Morse, and J. Roth. 2009. Length of prenatal participation in WIC and risk of delivering a small for gestational age infant: Florida, 1996–2004. Maternal and Child Health Journal 13(4):479–488.
Kowaleski-Jones, L., and G. J. Duncan. 2002. Effects of participation in the WIC program on birthweight: Evidence from the national longitudinal survey of youth. Special Supplemental Nutrition Program for Women, Infants, and Children. American Journal of Public Health 92(5):799–804.