Considerations in Applying Benefit-Cost Analysis to Preventive Interventions for Children, Youth, and Families Workshop Summary (2014) / Chapter Skim
Currently Skimming:
4 Issues to Consider in Benefit-Cost Analysis
4 Issues to Consider in Benefit-Cost Analysis
Pages 35-52
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 35... ...
In those cases, different experimental designs must be pursued for generating causal knowledge, including regression discontinuity analyses, interrupted time series designs, nonequivalent control group designs, and single case design. Thomas Cook explored the potential of several research designs that can be employed in benefit-cost analyses.
|
From page 36... ...
More important is the one relevant study comparing experimental results to those from a comparative regression discontinuity study that includes pretest values along all the assignment variable and so can serve to index the functional form relating the assignment variable to the outcome without treatment. This one study showed the same results as a randomized controlled trial in all areas away from the cutoff, and not just at it as in the simpler regression discontinuity design with only posttest data (Cook et al., 2008)
|
From page 37... ...
They show close approximation of randomized controlled trial results when the comparison groups are very local, when pretest measures of outcomes are used to match groups, when there is independent knowledge of all or most of the selection process, and when there is multilevel matching. It also works when there is a hybrid matching strategy that combines local matching where it works well with nonlocal focal matching wherever local matching does not work.
|
From page 38... ...
In particular, greater consideration could be given to the special status of comparative regression discontinuity studies and to comparative interrupted time series designs into the clearinghouse. Results also would benefit by an external warrant, said Cook, so they are not the product of an individual researcher's choice or preferences.
|
From page 39... ...
One approach that the research field may consider in order to aid research consumers is to adopt an alternative system that simplifies the process of determining the difference between the good and bad studies by using a high-quality study design bar, which includes fewer classes of research designs. The trade-off for this approach is excluding some good studies that may fall short of the quality bar that is set for research designs.
|
From page 40... ...
One approach could be to include as evidence only the results of randomized controlled trials or regression discontinuity studies. A limitation to this approach, admitted Ludwig, is that such a decision would throw away good information from other kinds of studies.
|
From page 41... ...
The Relevance of the Impact Estimates Even if a study has relevance to a policy or practice problem, there may be issues that affect whether and, if so, how one might apply the findings. For example, does the reference period for the impact estimates match the intervention being considered, whether by a local school board, a state legislature, a parent group, or the federal government?
|
From page 42... ...
It is risky to include those measures in the same category as the outcome measures that were front and center in the original analysis. The Causal Validity of the Impact Estimates and Reporting Even studies that start out as well-executed randomized controlled trials can suffer from attrition (including differential attrition)
|
From page 43... ...
ISSUES WITH RANDOMIZED CONTROLLED TRIAL DESIGNS Several of the speakers discussed randomized controlled trials during the question-and-answer sessions. Basu expressed his concern about calling randomized controlled trials the "gold standard" for benefit-cost analysis.
|
From page 44... ...
Maynard agreed that the term "gold standard" is inflammatory and should not be used. Randomized controlled trials are not necessarily perfect or even better than some alternative designs, she said, because they require much more than just effective randomization.
|
From page 45... ...
rs, as Al of the studi have perf ll ies formed some kind of sensitivity analys sis, Karoly continued. For example, they may ha disaggreg y F , ave gated estimattes of bennefits to show the different sources of b w t benefits, such as the benef h fits
|
From page 46... ...
Decision makers then can adjust their use of the results based on the sensitivity analysis. In other cases, sensitivity analyses look at the effects of basic assumptions in the model, such as the size of the dead weight loss from taxation.
|
From page 47... ...
With regard to uncertainties, in addition to reporting standard errors, the percentage of simulations with positive net benefits could be reported in the base case, a useful and intuitive result for decision makers. Other areas of uncertainty, such as the methods for projecting future benefits, assumptions about the efficacy of scale-up, or distributional weights, could be better addressed with sensitivity analyses.
|
From page 48... ...
Stochastic uncertainty is used to study random variation across individual outcomes, often using Monte Carlo techniques. Parameter uncertainty is used to study variation in expected outcomes through probabilistic approaches, especially second-order Monte 5 This section summarizes information presented by Anirban Basu, Ph.D., University of Washington, Seattle.
|
From page 49... ...
This simple product provides an upper bound on the value of future research and can be used to inform many decisions, such as whether to fund future research, how to prioritize across future research proposals, and how to design future research studies. THE POTENTIAL COMPENSATION TEST AND DISCOUNT RATES6 Richard Zerbe discussed two contentious topics in the area of benefitcost analysis.
|
From page 50... ...
This is true even if some projects do not pass a benefit-cost test or if some gaming of the system occurs. Discount Rates The second topic Zerbe discussed is discount rates.
|
From page 51... ...
One such principle is that ethical considerations and other extraneous considerations, such as environmental goods or the value of life, should be excluded from the discount rate. These factors can be included in values and thus in benefit-cost analyses through willingness to pay tests, but they should not be included in discount rates.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.