A Closer Look at the Problem of Valuation
The questions that arise in assessing benefits and costs for early childhood interventions have emerged in other contexts, and the workshop was designed to consider relevant insights and examples. David Weimer provided a detailed look at the development of shadow prices in general. Myrick Freeman, Philip Cook, and Donald Kenkel discussed the ways monetary values are assigned to outcomes in three sectors, respectively: (1) environmental economics, (2) criminal justice, and (3) health.
THE IMPORTANCE OF SHADOW PRICES
Shadow prices are a means of (1) converting projected program impacts into social benefits (which can be measured in terms of society’s willingness to pay for them) and (2) converting program resources into social costs (measured as opportunity costs). Many plausible, but imperfect, shadow prices are available in the early childhood context, mostly based on data from long-term experiments, such as the Perry Preschool Project. These are extremely useful, Weimer suggested, but they cannot be “the answer to all our problems because we’re just never going to have enough resources to do enough of them.” Moreover, he stressed, studies of a single program by definition can answer only a constrained set of questions, from the point of view of decision makers. More “wholesale” experiments are necessary to provide the basis for useful shadow prices—which are key to benefit-cost analysis.
In general, different kinds of information can be used to calculate shadow prices. One is the market price of various resources in the early childhood context, such as wages and benefits for teachers. If the market is distorted, as, for example, when a preschool program does not pay market price for the use of school buildings, an adjustment might be made by calculating opportunity costs. Economists might also use indirect methods to calculate values for which there are no clear market prices (missing markets). For example, they might calculate the statistical value of a life year, or infer the contingent value—the amount people say (usually in response to survey questions) they would be willing to pay for a resource that does not have a market value. In the context of early childhood, that might mean calculating opportunity costs for volunteer time or benefits of improved educational outcomes or a reduction in crime.
Theoretically, the ideal way to conduct a benefit-cost analysis for early childhood interventions would be to use a long-term random assignment experiment, much like the Perry Preschool Project, “except bigger and perhaps more geographically representative,” Weimer explained. These data would make it possible to predict the impact of other similar programs, using shadow prices to estimate earnings changes, quality of life changes, willingness to pay for various benefits, and so forth. However, Weimer suggested that this model is not actually ideal from a public policy perspective. Long-term studies are expensive, so sample sizes tend to be small, and such studies are relatively rare. Researchers often encounter problems with attrition and have difficulty accurately taking into account long-term shifts in the context—such as changes in the sorts of alternatives that are available to the program being studied. And, of course, results are often delayed.
Weimer offered several alternative approaches. First, work could be done to develop better shadow prices for the early childhood context. He pointed out, for example, that the RAND Health Insurance Experiment (Manning et al., 1987) provided a way of developing estimates of the price elasticity of demand for health care. Existing early childhood studies provide observational data that could be used in a similar fashion to link program effects to outcomes, such as school completion, for which shadow prices may be more readily available. More work is also needed in the development of shadow prices for willingness to pay for societal benefits, such as reducing poverty, using contingent valuation techniques.
Another promising approach is to improve strategies for linking observable outcomes to a wider array of social benefits (Weimer and Vining, 2009). Decades ago, Haveman and Wolfe (1984) used a household utility approach to estimate the nonlabor market benefits of schooling (such as reduction in crime, efficiency of consumption). They calculated a monetary value for such outcomes as children’s cognitive develop-
ment, the use of contraceptives, consumption efficiency, and improvements in health. They concluded that the nonlabor market gains were approximately equal to the labor market gains, and their rough estimate suggested that each additional dollar in earnings resulting from an intervention produces about an additional dollar in social benefits. In another study, the same researchers looked at an even wider array of effects that schooling might have, including effects on the schooling of participants’ children, on family members’ health, on participants’ daughters’ fertility, among others (Wolfe and Haveman, 2001). However, few others have attempted this sort of analysis.
A third approach is to improve ways of linking immediate impacts to future benefits. By drawing on other empirical research, Weimer explained, one could link short-term impacts, such as school readiness, with longer term outcomes for which shadow prices are available, such as school completion. The next step (Weimer acknowledged that Katherine Magnuson had suggested a similar analysis) would be to compress the chain of causality to produce a shadow price for the immediate impact that can be used in comparing alternative programs. Another example of this approach is a meta-analysis conducted by Aos, Miller, and Drake (2006), using studies of the criminal justice system.
For Weimer, the bottom line in a policy context is to obtain the best estimates possible to support decision making. Doing that requires less emphasis on whether a program worked in the past and more on mining its results for indications of what will work now and in the future. It is also important to decrease the cost of conducting benefit-cost analyses, because, until that happens, “we’re not going to have enough of it.” The shadow prices are key to efficient, low-cost analyses, but, he noted, “we have to go outside of our discipline sometimes to do that.”
EXAMPLES FROM OTHER SECTORS
Environmental economics, criminal justice, and health economics are three fields that have made considerable progress in the use of benefit-cost analysis, and each offers insights that could be useful in the context of early childhood.
The degradation of environmental resources—such as clean air and water, biodiversity, a healthy ecosystem—was an early impetus for economists to develop ways of assigning monetary value to benefits or resources that are not traded for money. Myrick Freeman described the two primary methods of using nonmarket valuation to assess the effects
of environmental policies in situations in which no market prices are available for analysis. The objective for these methods, he explained, is to estimate the willingness of people affected by a government policy to pay for the benefits it is expected to yield.
One set of methods, the revealed preference methods, examines the choices made by people who are or may be affected by a policy. Specifically, these methods use data about people’s choices to identify the implicit prices that they would pay to achieve a particular outcome. These methods are based on the rational choice economic model1—that is, analysis based on the assumption that when people are rational and have enough information to make an informed choice, their marginal willingness to pay for an environmental improvement indicates the economic value they attach to that improvement. Thus, if the marginal implicit price can be estimated, the marginal willingness to pay can be inferred. One example of how this works would be to analyze avoidance behaviors: People’s willingness to pay to avoid the risk of waterborne disease can be inferred from the prices they pay for water filters or for bottled water. Similarly, one might examine the relationship between the risk of death or injury on the job and wages to identify people’s marginal willingness to pay to reduce these risks. This is done using hedonic wage models; similar models are used to examine housing prices to identify people’s willingness to pay to live near such amenities as a park, a waterfront, or a school.
The other set of methods is the stated preference method, in which people are asked hypothetical questions about their preferences and willingness to pay for various benefits. (Contingent valuation, discussed above, is one form of this approach.) This can be done in various ways. One could provide a reasonably detailed description of the resource or benefit in question, ask people to supply a dollar figure or choose from options, and then calculate the mean response. One could ask whether respondents would or would not be willing to pay a particular amount, perhaps following up with a second amount, depending on the answer. There are also various ways of asking people to rate or rank a set of alternatives and, using discrete choice models (mathematical functions that take into account the attractiveness of various options), to predict the tradeoffs people are willing to make between price and the selected attributes.
However, Freeman explained, the stated preference models are all controversial, particularly in the context of litigation regarding assessments of damage to natural resources. The lawsuit that followed the
Exxon Valdez oil spill in 1989, for example, generated considerable dispute and research to advance the techniques. Observers have questioned the reliability of the responses, which is difficult to assess because true values are not available (i.e., hypothetical choices may not accurately represent the choices people would make if they faced a real-life decision). The possibility that respondents have an incentive to misrepresent their values in some strategic way has also been raised, as have questions about whether respondents can be assumed to have full information about the alternatives (the latter question would be relevant to the revealed preference approach as well).
An additional problem is how to address the likelihood that respondents do not always have well-defined preferences regarding the options they are asked about and may make them up on the spot. Such decisions are likely to be influenced by the information they are presented with. Studies of the issue find a degree of consistency in people’s responses, suggesting that they tend to have formed preferences that guide their responses. While researchers have developed various strategies for addressing these concerns, Freeman was clear that the results are definitely affected by the way questions are framed, and that it is both “easy to do a bad study and very hard to a good study.”
To illustrate the application of some of these methods, Freeman described three analyses of the benefits of reducing childhood exposure to lead, two of which were conducted by the U.S. Environmental Protection Agency. Studies dating back to 1985 have established clear health benefits for both adults (reducing hypertension and risk of heart attack) and children (avoiding cognitive deficit and other health problems) from limiting lead exposure. These health benefits have economic benefits, such as reduced education and medical costs, improved lifetime earnings, and reduction in antisocial behavior. (Freeman cited Schwartz et al., 1985, and U.S. Environmental Protection Agency, 1997.) Similar studies have examined the value of reducing childhood exposure to mercury.
These examples highlight several issues regarding the economic valuation of early childhood interventions. The first is the question of choosing a normative perspective, as reflected in whose willingness to pay should be counted—the child’s, the parents’, or that of a hypothetical child endowed with the financial resources and cognitive abilities of an adult. Different analytic methods, Freeman explained, imply different normative perspectives, so it is important that researchers consider this point and make its implications clear in their analysis.
A second issue is the challenge of capturing third-party effects. A potential crime victim can be presumed to have some willingness to pay for the reduction in crime that could result from reductions in lead exposure to children, for example, but the potential victims cannot be identi-
fied or asked in advance. And third, there might also be societal benefits from early childhood interventions that people may be willing to pay for, even though they will not personally be affected by the intervention. For example, would people in general have a willingness to pay to see a reduction in the prevalence of childhood obesity? While these issues remain somewhat unsettled, researchers and policy makers have made good progress in valuation in environmental economics that could be a useful resource for research and policy work in early childhood.
Attaching value to the impacts of crime is done using similar methods and raises many of the same points, Philip Cook explained. He illustrated valuation for crime-related questions with a simple example. If a residential community is considering hiring a guard in order to reduce crime rates, their decision would be based on whether the value of having the guard is greater than the cost (the guard’s salary). Looking first at property crime, the residents could begin with the property value that might be lost to theft or vandalism without the guard. Even in this simple case, though, there are complications, such as whether or not the residents have property insurance (and whether the rates might be affected by the presence of the guard), how risk averse they happen to be, the possible sentimental value of items (apart from their monetary value), and other negatives associated with crime, such as invasion of privacy or the inconvenience of replacing lost items.
In thinking about violent crime, the residents would again begin with direct costs that could be averted, such as medical care and lost earnings. Indirect costs, such as the fear and disutility the residents anticipate from being a victim, are likely to be significantly higher but difficult to monetize. If all the residents are willing to pay the cost of hiring the guard, then it is clear that the benefit exceeds the cost, but this answer is imprecise because individual residents might be willing to pay very different amounts. A market test of the decision is whether the value of the property in the community goes up in spite of the fee for the guard; this result would be evidence that a larger group of people (beyond the residents) believes the community is more attractive with the guard.
But it is not clear that this evidence of the community’s collective willingness to pay for the guard is the same as the value they attach to the social benefit of reducing crime, Cook explained. Looking more broadly makes other issues apparent. Hiring a guard for one community might simply displace the crime to other communities, so the net benefit to society at large would be zero. Other factors, such as people’s views about the
other effects having a guard might have on the community, would also be factors in the valuation.
Having introduced some of the primary issues, Cook described a study in which he and Jens Ludwig examined people’s views of policies designed to reduce gun violence (Cook and Ludwig, 2000). Using the stated preference model, they asked respondents how they would vote on a policy that was described as having the potential to reduce gun violence by 30 percent. Using randomized samples, they told respondents the cost would be a tax increase of $50, $100, or $200 and used follow-up questions to refine the responses. With these data they were able to trace a demand curve and calculated that the 30 percent reduction in gun violence was worth an average of $240 per household, or approximately $1 million per shooting. In a similar study Cohen and colleagues (2004) found that preventing a burglary was worth $25,000, preventing a robbery was worth $232,000, preventing a rape was worth $237,000, and preventing a murder was worth $9,700,000.
Some have questioned these numbers, but potential benefits are nevertheless very large. Picking up the example of reducing children’s exposure to lead, Cook suggested that if a generation grows up with a lower average criminal propensity because of widely decreased lead exposure, the result will be “not just less crime but an array of outcomes that will be interacting with each other: less crime, lower response costs from the public and private sectors, and so on.” Valuing these system-wide benefits is challenging but nevertheless important.
For Cook, “the bottom line is that there is no very reliable approach in this area—it is tough to get stable numbers that can be reproduced.” Looking forward, he suggested that continued development of analyses of willingness to pay will be important, as will further developing analogies between crime and disease.
In research on the economics of health, Don Kenkel explained, two methods are used for valuation: (1) cost-benefit analysis based on willingness to pay (using shadow prices as described above), and (2) cost-effectiveness analysis. The first, cost-benefit analysis, is done the same way whether the context is health, environmental policy, or criminal justice, but in practice it is less common in the health context, so Kenkel focused on cost-effectiveness analysis.2 The simplest version of cost-effectiveness
analysis is to relate the costs of an intervention to a direct effect, such as the cost per cases of cancer detected using a particular screening method. If one takes the extra step of considering the utility of particular outcomes, or people’s preferences, it is called cost-utility analysis.
In considering utility, economists may use a health-adjusted life year—a way of measuring both the quality and the length of lives saved by a health intervention—as a common unit to represent the value of health. This tool makes it possible to consider not only how many lives were saved, but also whether it was the life of an 80-year-old or a 20-year-old, and whether the remainder of the life was spent in bad health or disability or in good health. (He noted in response to questions that adjusting for age also comes up in cost-benefit analysis based on monetary willingness to pay for health and safety. Making monetary adjustments based on age is complex; not only is there a range of views about whether it makes sense to adjust for age, but also there is no consensus on how best to do it.)
A commonly used example of the health-adjusted life year is known as the quality-adjusted life year (QALY). Stated preferences are used to construct this tool. Here, however, respondents are not asked about their willingness to pay for an outcome but are presented with a “standards gamble” as a way of finding out how they would weigh the risks and benefits of staying in a suboptimal state of health or risking a worse outcome in pursuit of an improvement. A stark example would be to ask respondents whether they would risk an operation that could restore them to perfect health but carries a 10 percent risk of death. Another approach is to ask respondents to report the relative value they would place on, say, 10 years lived in suboptimal health versus 1 year in optimal health. From the responses, researchers can estimate the relative value of different outcomes.
Kenkel described an early example of cost-utility analysis of a medical issue—childhood lead poisoning—conducted by Glotzer, Freedburg, and Bauchner (1995). The researchers examined the cost-effectiveness of several different approaches both to testing children for exposure and to treating those who are exposed, including remedial education to address cognitive disability and chelation to remove the lead (a painful and expensive procedure). They estimated the value of detecting and medically treating lead poisoning at approximately $1,300 per QALY gained. When they factored in the cost of remedial education that would not be needed if the poisoning was prevented, they found that the intervention was cost saving.3 A study of the cost utility of screening for fetal alcohol spectrum
disorder provides another example (Hopkins et al., 2008), in which the researchers found an incremental cost-effectiveness ratio of about $66,000 per QALY.
How were these values for a QALY calculated? In analyzing the effects of lead exposure, Glotzer, Freedburg, and Bauchner (1995) assumed that life with a lead-based disability would be counted as 77 percent as valuable as life without—and thus calculated a QALY weight of 0.77. However, they based that figure on what Kenkel described as “thin evidence,” a survey of 13 pediatricians and pediatric educators at their own institution. The QALY weight calculated for fetal alcohol spectrum disorder was 0.47, in this case based on a survey of 126 children and families about their experiences with moderate to severe dysfunction resulting from the disorder. Kenkel noted that there are many other studies using QALYs that may have stronger evidence to support these sorts of calculations, but that applying the approach to children is not a well-developed procedure.
Although some aspects of the approach are not fully settled, cost-utility analysis is widely used in health and medicine. Kenkel noted a registry housed at Tufts University, the National Institute on Clinical Excellence in Great Britain, and efforts in other countries to collect this kind of evidence of cost-effectiveness for pharmaceuticals and other medical options. The U.S. Office of Management and Budget provides guidance for using this approach in regulatory analysis (Executive Order 12866), and Kenkel suggested that all federal agencies should prepare such an analysis as part of any rulemaking related to public health and safety. The Institute of Medicine, he noted, has also made recommendations for using measures of cost-effectiveness to support federal regulations (Institute of Medicine, 2006).
Nevertheless, Kenkel noted, some are skeptical about benefit-cost analysis for health issues. The analysis is a way of getting around putting an explicit monetary value on health effects, but some question whether estimates of willingness to pay are reliable. Willingness to pay may vary with income, and many are uncomfortable with the idea of connecting the allocation of health care to personal income. Still, the analysis does provide implicit monetary values that are useful. The challenges of calculating willingness to pay in the context of questions related to morbidity and mortality are not conceptually different from the challenges of valuation in other contexts. Some would also argue, Kenkel suggested, that questions about health are special and should not be subject to utility-based analysis. However, cost-effectiveness, or cost-utility analysis solves this problem because its purpose is to identify the optimal way to allocate limited resources—to produce the maximum health benefits for a fixed amount of money.
Kenkel observed that “theoretical purity doesn’t necessarily translate
into effective persuasive policy advice.… It is very clear that estimates of cost savings from interventions have a lot of persuasive appeal.” He closed with a quotation from a 1971 paper called Evaluation of Life and Limb: A Theoretical Approach, “In view of the existing quantomania, one may be forgiven for asserting that there is more to be said for rough estimates of the precise concept than precise estimates of economically irrelevant concepts” (Mishan, 1971).