arching aim or vision; a plan of action for achieving the goals; and measures for indicating when the goals have been achieved. When that concept is applied to a complex subject, such as scientific research, developing suitable goals, implementable action plans, and measures of success becomes similarly complex. Research is often open-ended and serendipitous, and it can be difficult to formulate goals that will not stifle innovation. Even when the goals are clear—for instance, “to cure cancer” or “to develop renewable energy sources”—the road map for achieving them can be less than obvious. Promising research avenues can lead to dead ends, and seemingly trivial research directions sometimes turn out to be vitally important. Identifying measures of success ahead of time can sometimes seem like staring into a crystal ball. But, as difficult as the process is, strategies are required for science; in which resources are limited and there is a need to justify what is spent on the basis of what is achieved.

Ensuring efficient progress, or “performance,” is a key aspect of any research strategy, and selecting useful measures requires a degree of sophistication. In 2002, the Office of Management and Budget designed the Program Assessment Rating Tool (PART) (OMB 2008) in an attempt to evaluate the performance of publicly funded programs, including research and development (R&D) programs. PART does not explicitly address the need for strategies, but it requires agencies to take strategically relevant steps that include defining outcome-based metrics, measuring the efficiency of research programs, and achieving annual efficiency improvements. Applying those steps to scientific research is not easy. The 2008 National Research Council report Evaluating Research Efficiency in the U.S. Environmental Protection Agency concluded that “no agency had found a method of evaluating the efficiency of research based on the ultimate outcomes1 of that research” (p. 10), and indeed the report stated that

BOX 2-1

Elements of a Research Strategy

  • Vision, or statement of purpose.

  • Goals.

  • Evaluation of the existing state of science.

  • Roadmap.

  • Evaluation.

  • Review.

  • Resources.

  • Mechanisms.

  • Accountability.


Ultimate outcomes include such results as lives saved or clean air and cannot be predicted or known in advance, may occur long after research is completed, and usually depend on action taken by others (NRC 2008).

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement