What happens to children during infancy and early childhood has a profound influence on their experiences once they enter school and throughout life. The deficiencies that many children experience from birth to school age—in health care, nutrition, emotional support, and intellectual stimulation, for example—play a major role in academic achievement gaps that persist for years, as well as in behavior and other problems (Karoly, Kilburn, and Cannon, 2005; Kilburn and Karoly, 2008; National Research Council and Institute of Medicine, 2000, 2009). Findings from neuroscience, developmental psychology, and other fields have supported the development of many interventions designed to strengthen families, provide disadvantaged children with the critical elements of healthy development, and prevent adverse experiences that can have lasting negative effects. Early childhood interventions may focus on educational experiences in preschool classrooms, home visits, parenting education, health and wellness support, or some combination of these approaches. They may identify short- and long-term goals, such as reducing health problems, improving cognitive development and school readiness, or preventing negative behaviors like child abuse or juvenile crime.
Do these programs pay off economically? Many studies have documented benefits to children and their families, and many different models are being implemented in the United States. Policy makers are increasingly recognizing the importance of this phase of life and the potential value of early childhood interventions, and they are increasingly willing
to invest public funds in them. In a climate of economic uncertainty and tight budgets, however, hard evidence not only that such interventions provide lasting benefits for children, their families, and society, but also that the benefits translate into savings that outweigh the costs is an extremely important asset in policy discussions. Convincing analysis of benefits and costs would provide a guide to the best ways to spend scarce resources for early childhood programs. Methods for conducting the benefit-cost analysis that can provide this kind of evidence are complex in the context of early childhood, even as researchers are developing new approaches. The purpose of the workshop this report documents was to explore ways to strengthen benefit-cost analysis so it can be used to support effective policy decisions.
With the support of the John D. and Catherine T. MacArthur Foundation, the Board on Children, Youth, and Families held the workshop in March 2009 to examine strategies for strengthening the methodology for evaluating the benefits and costs of early childhood interventions. An ad hoc committee, formed to plan the workshop, was asked to explore the following questions:
What state-of-the art examples of benefit-cost methodology can be drawn from evaluation of diverse early childhood interventions, such as home visitation programs; child care programs; Head Start; the Special Supplemental Nutrition Program for Women, Infants, and Children; Bright Beginnings; Healthy Steps; low-birthweight studies; immunization and vaccine studies; Medicaid and the State Children's Health Insurance Program; and other areas? How are benefits and costs for children identified and assessed in each program area? Are there particularly influential benefit and cost assumptions that seem important and worthy of standardizing in determining the value of selected interventions?
How does the status of benefit-cost methodology in the field of early childhood interventions compare with studies of other vulnerable populations, such as those experiments used in assessing the impact of housing subsidies (such as Moving to Opportunities), income assistance programs (such as Temporary Assistance for Needy Families), and related activities?
What is known about the influences of scaling up early childhood health and educational programs on both costs and benefits?
What has been the experience with assigning a dollar (shadow) value to long-term impacts on nonmonetary outcomes like crime, health, etc.? What assumptions influence this practice and are they sensitive to specific characteristics of the populations served by selected programs?
What lessons can be learned from the experience of other fields, such as environmental economics, to develop other approaches to program evaluation, when true benefits and costs cannot be determined within a reasonable time frame? For example, do methods such as contingent valuation analysis or estimates of “willingness to pay” offer important lessons for the assessment of the value of early childhood interventions?
This report describes the information and analysis that were presented at the workshop and the discussions that ensued. This chapter provides an overview of the nature of benefit-cost analysis and an introduction to the three primary issues: (1) evaluating early childhood interventions, (2) assessing the costs associated with them, and (3) assigning value to the benefits they yield. Chapters 2 through 6 provide a more detailed look at methodological advances and conceptual questions associated with these three basic challenges. The report closes with a discussion of the role of benefit-cost analysis in today’s policy context, including how to communicate results to policy makers. Appendix A provides a glossary of technical terms used in the report, and Appendix B contains the workshop agenda and a list of participants.
AN OVERVIEW OF BENEFIT-COST ANALYSIS
There are two primary purposes for benefit-cost analysis, as committee chair Barbara Wolfe explained in her opening remarks. The first is to identify the programs or interventions that are the most effective—that is, those most likely to improve the well-being and future productivity of young children, given a particular level of expenditure. The second purpose is to guide comparisons of the benefits of investing in early childhood interventions with the benefits of other public expenditures—that is, to quantify the net long-term economic and other benefits of effective early childhood programs.
She noted that this sort of analysis is particularly challenging in the context of early childhood interventions because many of the benefits do not accrue until many years later. Numerous challenges follow from this one, such as how to identify a fair value for benefits expected far in the future, or how to extrapolate the current benefits from interventions that took place many years in the past. Moreover, it is difficult to measure many of the potential impacts of early childhood programs and to monetize their value. Nevertheless, given the Obama administration’s intention to invest $10 billion per year in early childhood interventions that are evidence-based and have high benefit-to-cost ratios, she suggested, this is an ideal time to focus attention on the most up-to-date methodology
and ways to strengthen benefit-cost analysis as applied to early childhood interventions.
Although results-based accountability is a priority in any policy context, decision makers in both the private and the public sectors who are ready to invest in early childhood interventions face many additional challenges. Lynn Karoly explained that policy makers are particularly interested in analysis that can demonstrate that a dollar invested in a particular area will yield multiple dollars in savings and other benefits—that is, cost and outcomes analysis. There are four different approaches, with ascending complexity.
Simplest is a cost analysis, in which a program is evaluated in terms of its full economic cost (not just the direct expenditures required). A step up in complexity is a cost-effectiveness analysis, in which success at producing a particular outcome is part of the analysis. Cost-effectiveness analysis does not entail placing a dollar value on the outcome, however; the result is reported in terms of how much must be invested to achieve a particular outcome, as measured in the natural units for the outcome (e.g., a decrease in the percentage of youth who are held back in school or incarcerated).
The third and fourth approaches, cost-savings and cost-benefit analyses, make it possible to measure and translate multiple outcomes into dollars, which in turn make it possible to aggregate them, providing the potential to compare the total benefits of various possible investments. Cost-benefit analysis, which considers the value of an intervention both to the government (in terms of reducing the need for expensive interventions later in the life cycle, for example, reduced incarcerations for criminal behavior) and to society (in terms of broader social goods that can also have economic value, such as increases in the literacy level of a population), offers the potential for the most complete policy information—and is the most demanding type of analysis.1
Figure 1-1 illustrates the way benefit-cost analysis works. The two ovals on the left represent the program being analyzed and the possible alternatives to it. The analysis begins with evaluation of the resources needed to operate the program and of the outcomes or impact it may yield for children and families. The rectangular box lists some of the methods used to translate the information about resources and impact into monetary values. These methods may include calculating shadow prices for the resources used to provide the program (the implicit or true cost of continuing a program, as distinct from the monetary cost);2 generating shadow prices in order to value any favorable (or unfavorable) effects of the program on child and family outcomes; calculating discount rates (to
Benefit-cost analysis is also sometimes referred to as cost-benefit analysis.
Shadow prices are discussed in greater detail in Chapter 5.
determine the monetary value of future costs or benefits); and considering other factors, such as the impact on costs of scaling up the program. The oval at the right shows the information this analysis can yield, including the bottom line, the benefit-cost ratio.
Benefit-cost analysis might be used to look at a single program and demonstrate how that program produces a net benefit or benefits that accrue to different stakeholders. It might also be used to compare programs with one another or to compare different categories of early childhood interventions, such as programs that focus on home visits or programs that provide some form of preschool education. Similarly, the analysis might compare the effects of intervening at different stages of life, or might compare early childhood interventions with other kinds of policies or programs. As Karoly explained, however, the field is not yet at the point at which benefit-cost analyses for different early childhood interventions are comparable, nor can they be compared with those conducted for other types of social programs.
Some of the challenges of conducting full benefit-cost analyses are particular to the early childhood context and others are more general, but they cluster around the three primary elements of the task: (1) the need for rigorous program evaluation, (2) assessment of costs, and (3) assessment of impact. Much of the workshop focused on these three issues, and Karoly provided an overview of the questions they raise.
Need for Rigorous Program Evaluation
Evaluation is a challenge in many social policy contexts because real-world circumstances make the ideal—a true randomized controlled trial that compares two alternatives—difficult or impossible (for example, when ethical considerations make random group assignments untenable). Although a variety of quasi-experimental methods are available, these approaches are also complicated by basic questions, such as how to define the baseline against which a program or approach is to be compared. For example, if one is examining the long-term results of an early childhood program that was initiated 20 years in the past, the alternative to the intervention at that time would probably have been no program at all. Today, however, the vast majority of young children participate in some sort of preschool program, so the baseline comparison would need to reflect that. Because economic analysis depends on accurate evaluation, analysts must clearly understand the issues that complicate the evaluation.
Assessment of Costs
Assessment of the true cost of a program needs to capture not just the items shown on budget sheets, but other costs as well, such as time spent by unpaid participants or the value of lost opportunities—for example, the potential benefits of employment or other opportunities the unpaid participants could otherwise have pursued. Moreover, if approaches or programs are to be compared, the methodology for assessing the value of the necessary resources must be consistent.
Assessment of Impacts
Many different outcomes may be affected by early childhood interventions. These include changes in the children who receive the intervention (e.g., improvements in behavior or emotional or cognitive development, reduction in antisocial or risky behavior) as well as changes involving the adults in those children’s lives (e.g., improved family functioning, reduction in crime or substance abuse). Most of the desired outcomes are likely to affect both children and their parents, to persist for many years, and to yield a variety of economic benefits. However, not all potential benefits may be included in benefit-cost analyses, in part because some benefits are easier to translate into dollar values than others. For example, the dollar benefits of positive emotional and cognitive development are hard to quantify. In addition, the economic value of early educational benefits doesn’t emerge until later adult employment and earnings. Valuing other benefits that may not be evident until far into the future—such as lack of
involvement in crime—is also tricky. Table 1-1 shows some outcomes that have and have not been captured.
In practice, benefit-cost analysis has been conducted for relatively few early childhood interventions. In a 2005 study (Karoly, Kilburn, and Cannon, 2005), Karoly and her colleagues examined the research on 20 early childhood interventions. Of those, only seven had been the subject of benefit-cost analysis, which severely limits the field’s ability to identify programs that generate returns that exceed their costs and are thus the best investments. Moreover, the analyses for the seven programs revealed that different outcomes were valued in the various studies, frequently with different methodologies. She suggested that standard practices for the conduct of these analyses would make the results much more meaningful to policy makers.
In Karoly’s view, standardization would be easier for some issues, such as selecting a discount rate. Other issues—such as outcomes to be measured, the baseline to which the intervention should be compared, the length of time for follow-up analyses, and the economic values (or shadow prices) to be attached to various outcomes—are less settled. And other issues are likely to complicate standardization as well. For example, some programs begin at birth, whereas others target 3- and 4-year-olds.
TABLE 1-1 Inclusion of Benefits in Benefit-Cost Analysis
Thus, a benefit-cost analysis for these two types of intervention would discount to different points in the life cycle, and the discounted values would not be strictly comparable, unless this difference was properly taken into account. The program design may allow for follow-up with the children who participate, their parents and siblings, or even their own offspring. Analysis that includes projected outcomes for these potential beneficiaries will be constrained by the program design, which would further complicate efforts to standardize.
Thus, the current state of benefit-cost analysis of early childhood interventions might be described as promising but still somewhat unsettled. With this overview in mind, participants considered the three primary elements of benefit-cost analysis and then considered their policy implications.