This chapter identifies programs designed to facilitate scalability, such as through integration into existing services or efficient use of local resources. Panelists spoke about the impact and effectiveness of large-scale interventions, including characteristics that ensured positive impact. Challenges and issues of implementation were explored, and key criteria for scaling up, such as human capacity, were discussed.
REACHING MORE CHILDREN AND FAMILIES WITH EVIDENCE-BASED PARENTING INTERVENTIONS1
Susan Walker spoke about approaches for improving home environments of children up to age 3 by ensuring they have interactive caregivers and opportunities to learn. She described approaches for reaching more children and families through evidence-based interventions. There are two critical needs, according to Walker: (1) effective evidence-based programs for children that are feasible to implement at scale, and (2) skilled trainers and supervisors who can maintain program quality.
To address the first need, Walker and her colleagues designed a pilot that delivered parenting interventions through routine child health clinical visits in three countries in the Caribbean region. This pilot was designed using a minimalistic approach. Walker explained that they did
_____________
1 This section summarizes information presented by Dr. Susan Walker of the University of the West Indies’ Child Development Research Group.
not want to employ extra staff at the clinics or unduly burden parents by increasing the length or frequency of visits. The program implementers produced a series of nine films featuring Jamaican mothers modeling desirable parenting behaviors, such as talking with their babies, offering praise, and reading books. Community health workers showed the films to mothers and caregivers who brought their children in for routine immunizations and checkups, from the ages of 3 to 18 months. These workers were trained not only to show the videos but also to demonstrate the activities and give mothers a chance to practice while at the clinic. The mothers were also given message cards and play materials to take home.
Walker noted that the pilot demonstrated benefits to children’s cognition and parental knowledge of child development, with effect sizes comparable to some home visit programs, though it was not as effective as the more intensive weekly home visitation program. There were no adverse effects of adding early childhood development activities to existing measures for improving child nutrition and health. However, Walker did point out a few challenges, such as the limited number of visits (five contacts between ages 3 and 18 months). Compared to weekly home visits, these contacts were too few, and the majority of them occurred prior to age 12 months. Walker observed that this program could be integrated into a more comprehensive strategy that would also include programs for children ages 1 to 3 years. An additional challenge she highlighted was difficulty with infrastructure and resources. Many of the clinics were crowded and noisy, and nurses had limited time. Walker asserted that going forward, such a program would need to be properly provisioned so as not to have a negative impact on the existing functions of staff.
One factor contributing to the success of such a program, according to Walker, is the high coverage of, and compliance with, child health visits in much of Latin America and the Caribbean. She cautioned that adapting this program in areas with less coverage might not result in the same success. Finally, Walker observed that the Inter-American Development Bank estimated the annual cost of the intervention at 14 U.S. dollars per child.
Walker also described the work done to expand access to the Jamaica Home Visit Intervention and its Web-based training and support package. The home visit program has a strong evidence base, is feasible in low-resource settings, and has been successfully replicated in a number of countries. Its goal is the delivery of positive parenting skills to develop more responsive, interactive caregivers. To further scale up the program, Walker and her colleagues, in partnership with Grand Challenges Canada, developed a Web package to increase the capacity of staff to implement the program.
The program included a series of films developed in Bangladesh, Jamaica, and Peru, including three 15-minute films that walk through the
steps of a home visit, and 28 shorter films that depict specific concepts and activities. Additional elements will include updating the curriculum and developing training manuals and a guide for adapting the program to specific country needs or context. The training package will be rolled out in four countries to “early users” to identify challenges and areas for modification and for preliminary evaluation. Walker noted that the next step would be to put the training materials online and develop technical support, with further iterations of modification and evaluation as organizations take up the training. All of the materials will be available via a Creative Commons license with the hope that users will share their own adaptations and experiences.
DESIGNING, IMPLEMENTING, AND EVALUATING A SCALABLE HOME VISITING INTERVENTION: IMPACTS AND CHALLENGES2
Marta Rubio-Codina spoke about the evaluation she and her colleagues developed to bring the Jamaican home visiting model to medium scale in Colombia. This evaluation will help determine the intervention’s effectiveness in Colombia and provide some evidence to the government about whether the program can be implemented at a larger scale. The intervention took place over 18 months, Rubio-Codina noted, and included two components: psychosocial stimulation, delivered in 1-hour weekly home visits; and micronutrient supplementation (Attanasio et al., 2014). She explained that home visitation is about changing behaviors and practices regarding parenting in the homes and its impacts have been demonstrated in improving child learning and mental well-being, among other effects (Grantham-McGregor et al., 1991). Rubio-Codina explained that the intervention was delivered within the sampling frame of an existing conditional cash transfer (CCT) program, Familias en Acción, which targets the poorest 20 percent of families in Colombia. The program was rolled out with a randomized controlled trial design, with four groups in multiple areas—those who received the psychosocial stimulation, those who received micronutrient supplementation, those who received both, and a control group that received neither.
As with most CCT programs in Latin America, groups of beneficiaries select a representative to serve as a “mother leader” who speaks for them to the program authorities. Rubio-Codina remarked that “mother leaders” displayed characteristics suitable for home visitors, and these
_____________
2 This section summarizes information presented by Marta Rubio-Codina, Centre for the Evaluation of Development Policies at the Institute for Fiscal Studies and the Inter-American Development Bank.
women were trained and supported by mentors from Bogotá to deliver the program. She also noted that this supported the intention to embed the program within the local community to maximize community ownership and engender fundamental cultural change.
The evaluation showed a reduced effect size compared to the Jamaican home visiting intervention, which Rubio-Codina explained is an example of how scaling up can reduce impact. However, she commented that the effect was still significant and sizeable. The size of the estimated effect is equivalent to closing a third of the cognitive gap between low- and middle-income families in Bogotá reported in Rubio-Codina et al. (2015). Importantly, she stated that parenting practices were improved as measured by the level of play materials and play activities in the homes. The estimated cost of the program was 515 U.S. dollars per child per year, which Rubio-Codina remarked could be reduced by identifying mentors from local communities to provide supervision.
Rubio-Codina described a few challenges she and her colleagues encountered:
- Design—including adapting the intervention to deliver to scale, through the participation of local women with limited education (and adapting training materials accordingly), and incorporating the local context, all while maintaining quality and program fidelity
- Implementation—including selection of home visitors, motivation of home visitors, adequate training and mentorship, use of technology, sustained engagement of all program personnel as well as beneficiaries, and provision of salaries and career development
- Evaluation—including ensuring a representative sample, maintaining enrollment, reaching children at the appropriate age, evaluating both process and outcomes to assess quality of interaction and feasibility of implementation, and providing adequate funding for both the program and the evaluation
EVALUATING LARGE-SCALE INTERVENTIONS ON CHILD HEALTH IN BRAZIL: THE ROLE OF CONTExT AND CHALLENGES3
Mauricio L. Barreto shared three examples of large-scale child health interventions and evaluations related to sanitation and family and child health in Brazil. The first was an expansion of sanitation in the city of
_____________
3 This section summarizes information presented by Mauricio L. Barreto, the Collective Health Institute, Federal University of Bahia.
Salvador in order to reduce childhood diarrhea. In 1995, 25 percent of households were connected to sewer lines, and this intervention increased coverage to 80 percent of households (Barreto et al., 2007). Barreto noted that while it is commonly understood that improved sanitation leads to improved health, there are very few evaluations to show the connections. He and his colleagues designed an evaluation of the intervention, which examined several variables related to the outcome measured—diarrheal prevalence in children (see Figure 4-1).
The evaluation showed a 20 percent reduction in diarrhea prevalence, as well as a 50 percent reduction in other intestinal infections. In a related study using an econometric evaluation, the intervention proved not only to be cost-effective, but also that there was high approval in the population for the improved sewage system (Santos et al., 2011).
Programa Saúde da Família, the Family Health Program in Brazil, is another large-scale program that is based on the principle from the World Health Organization’s Alma-Ata meeting that emphasized the importance
FIGURE 4-1 Conceptual model of a sanitation program evaluation.
SOURCE: Barreto et al., 2007.
of primary health care, according to Barreto. This program quickly grew throughout Brazil, and now covers approximately 66.5 percent of the population. In an evaluation of the impact of the program on child mortality, morbidity, and other outcomes, Barreto and his colleagues found that infant mortality has indeed been reduced (Aquino et al., 2009). This reduction was greater in areas that had higher levels of poverty prior to entry into the programs, suggesting also that the program might help to reduce health inequities.
Finally, Barreto shared an evaluation of Bolsa Familia,4 the world’s largest CCT program in Brazil (Rasella et al., 2013), which is linked to the health system. He and his colleagues assessed the impact of various tiers of Bolsa Familia, controlling for social and economic factors. Bolsa Familia was associated with decreased infant mortality, as well as hospitalization. This effect increased with greater coverage. Barreto also speculated that the Bolsa Familia and Programa Saúde da Família operated synergistically, with the former affecting the social determinants of health, and the latter more directly impacting health.
Barreto closed with the following observations:
- Large-scale interventions are usually implemented by political decision.
- Well-informed and science-based interventions have a great chance of success.
- Well-designed and well-conducted evaluations are an important resource to measure and tease out the effects of different interventions.
- Child health outcomes have multiple causes. It is necessary to develop a clear framework of design and analysis.
- While cost-effectiveness of programs is important, it is also crucial that the public perceives the value and benefits of the programs.
- A key part of evaluation is to understand the context of the intervention, which includes having national information systems that collect and ensure the quality of data on processes and outcomes.
Raquel Bernal of the Universidad de los Andes reflected on the preceding presentations and shared her insights:
_____________
4 For more information about Bolsa Familia, visit http://www.mds.gov.br/bolsafamilia (accessed June 23, 2015).
- Effects of large- or medium-scale programs could be lower than those seen in efficacy trials, but still large enough to make investments worthwhile.
- Scaling could be a key factor in achieving impact; sometimes community-based variables have a greater effect than individual-level factors.
- “Minimalist” program designs can be scalable and effective, especially if integrated within existing services.
- Parenting program services can multiply effects of educational interventions, because home-based behaviors are changing and enhancing early education as well.
- Challenges in scaling up interventions are related to fidelity and quality, which require measuring, evaluating, monitoring, and assessing.
- Scalability might be advanced by using existing community services, particularly the training of embedded program facilitators.
One workshop participant raised the question about how programs address demand within the community, and how community members receive the specific programs proposed. Walker noted that health workers seem accepting of programs, and Rubio-Codina indicated that beneficiaries often describe a greater understanding of the cognitive and emotional needs of their children, as well as expectations for a better future for them. Barreto further noted that while many large programs are implemented because of political decisions, sometimes they transcend political decisions because the demand is large within the population. And the success of some programs can build additional demand within the population for other programs to improve quality of life.
Panelists further examined issues and opportunities in program design. In response to a question about the micronutrient component of the home visitation programs, Rubio-Codina noted that micronutrient supplementation showed little to no effect, and speculated that further exploration of the design and use of micronutrients would be necessary. She noted, for example, that iron deficiency is not the largest cause of anemia among young children in Colombia, so supplementing with iron would not have a substantial effect. In response to a question about the use of discipline, Walker explained that the home visitation program in Jamaica did not specifically address discipline, but rather showed the importance of praise in reinforcing good behavior.
Human resources and training certification issues were also raised, and panelists noted that there is enormous variability in the skills and competencies of community health workers. Walker further explained that in some countries community health workers are not paid, though
they are in Jamaica, and their basic competencies have increased. She also cautioned that the training materials are intended for literate women, though not necessarily with a great deal of education. For supervisors, however, she pointed out that they tend to have experience with research projects, and their skills are critical for scaling up because they play an important role in maintaining program quality. Rubio-Codina cautioned that, in scaling up, consideration should be given to how to expand training without introducing too many layers between supervisors, mentors, and home visitors, so as to avoid information loss. She also remarked that certification schemes could assist in maintaining motivation to improve skills and sustaining engagement and quality standards in the delivery.
Finally, workshop panelists discussed the potential for community empowerment, particularly among women, and its impact on health. Barreto noted that some studies of Bolsa Familia indicated increasing feelings of empowerment and behavior change among women, as well as a shift in local business as disposable income increased. Women trained as community health workers or home visitors can also model healthy behaviors for other family members as well as in their communities, so that health is valued and a demand for healthy programs is created.
Aquino, R., N. F. d. Oliveira, and M. L. Barreto. 2009. Impact of the Family Health Program on infant mortality in Brazilian municipalities. American Journal of Public Health 99(1):87-93.
Attanasio, O. P., C. Fernández, E. O. Fitzsimons, S. M. Grantham-McGregor, C. Meghir, and M. Rubio-Codina. 2014. Using the infrastructure of a conditional cash transfer program to deliver a scalable integrated early child development program in Colombia: Cluster randomized controlled trial. BMJ 349:g5785.
Barreto, M. L., B. Genser, A. Strina, M. G. Teixeira, A. M. O. Assis, R. F. Rego, C. A. Teles, M. S. Prado, S. M. A. Matos, D. N. Santos, L. A. d. Santos, and S. Cairncross. 2007. Effect of city-wide sanitation programme on reduction in rate of childhood diarrhoea in northeast Brazil: Assessment by two cohort studies. Lancet 370:1622-1628.
Grantham-McGregor, S. M., C. A. Powell, S. P. Walker, and J. H. Himes. 1991. Nutritional supplementation, psychosocial stimulation, and mental development of stunted children: The Jamaican Study. Lancet 338(8758):1-5.
Rasella, D., R. Aquino, C. A. Santos, R. Paes-Sousa, and M. L. Barreto. 2013. Effect of a conditional cash transfer programme on childhood mortality: A nationwide analysis of Brazilian municipalities. Lancet 382(9886):57-64.
Rubio-Codina, M., O. P. Attanasio, C. Meghir, N. Varela, and S. M. Grantham-McGregor. 2015. The socio-economic gradient of child development: Cross-sectional evidence from children 6-42 months in Bogotá. Journal of Human Resources 50(2):464-483.
Santos, A. C., J. A. Roberts, M. L. Barreto, and S. Cairncross. 2011. Demand for sanitation in Salvador, Brazil: A hybrid choice approach. Social Science & Medicine 72(8):1325-1332.
