used by many private sector actors are important public benefits.
Many of the case studies in the committee’s sample experienced the changes in policy and other changes that occurred in DOE R&D programs outlined in Chapter 1. This fact needs to be taken into account in judging the outcomes observed when the committee applied the framework. For example, the programs to develop a technology for making liquid fuels from coal were not notably successful. However, had oil prices continued to rise, as expected at the time the program was designed, the outcome might have been more favorable. In some cases, these effects are so striking that the committee notes them explicitly. In all cases, the reader should consider the context of the program before arriving at a final judgment about its benefits.
More refined analysis of knowledge benefits would improve the methodology. The committee’s focus on outcomes results in many benefits falling into the knowledge category. In some cases, this is because recently begun research projects have not yet had time to achieve their expected results. In other cases, research that is abandoned before producing a realized or optional technology also produces mainly knowledge benefits. Distinguishing between these two kinds of knowledge benefits may provide useful information that the present version of the methodology does not provide.
The committee’s use of the 5-year rule should not be be interpreted to mean that the only effect of federal R&D is to accelerate the introduction of a technology into the marketplace by 5 years. The committee recognizes that there may be many effects of federal R&D, including the acceleration of a technology into the marketplace by more than 5 years, or other effects such as an increase in the ultimate market penetration of a technology. The committee used the 5-year rule because it needed a uniform, conservative standard for the analysis of these particular case studies.
As noted earlier, quantification of the benefits suffers from inherently difficult methodological problems. The time and resource constraints of this study made it difficult even to apply fully the valuation methods that do exist. Where it has used quantified benefits to support its findings and recommendations, the committee considers it has been conservative in establishing upper and lower bounds for its benefit estimates. In general, the committee believes it is more likely than not that a more thorough analysis would increase the values of the benefits that the committee has assigned to DOE’s programs.
Perhaps the most difficult analytic problem is assigning to DOE a proportion of the overall benefit of an R&D program that properly reflects DOE’s contribution to it. In most of the case studies, DOE, industry, and—sometimes—other federal and nonfederal governmental research organizations contributed to the outcome of the research program. In some cases, as in the development of seismic technology, for example, industry made virtually all of the contribution, but DOE nevertheless made an important one. The committee has found no reliable way to quantify the DOE contribution in most cases, and doing so remains a methodological challenge for the future. For the purpose of this study, the committee has simply attempted to identify in its case study analyses the specific role that DOE played, by looking at the outcome that would not have happened had DOE not acted.
The committee considers that it has used conservative judgment in characterizing the DOE contribution for the purpose of developing findings and recommendations.
Department of Energy (DOE). 2000. Strategic Plan. Strength Through Science: Powering the 21st Century. Washington, D.C.: U.S. Department of Energy. Available online at <http://www.energy.gov/index/indexs.html>.