Cover Image

PAPERBACK
$35.00



View/Hide Left Panel

EVALUATING RESEARCH AND DEVELOPMENT

Research is difficult to evaluate by any mechanism. Useful evaluation requires substantial elapsed time because research on a given scientific question may span 3-5 years from initiation of laboratory or field experiments to analysis and publication of results. Substantial time may also be required for training of EPA staff or the scientific community in scientific and technical advancements prior to the conduct of the research. Considerably more time may elapse before the broader impacts of published research are apparent (NRC 2003).

Although the committee was asked specifically to render advice on how EPA could best comply with the efficiency questions of PART, it concluded that more general suggestions on the evaluation of research would also have value for research-intensive agencies, for the Office of Management and Budget (OMB) and the Office of Science and Technology Policy (OSTP), and for Congress. It therefore examined the particular details of PART (Chapters 2-4), proposed principles that can be used to evaluate the results of research in any federal agency, and provided recommendations for EPA that other agencies also may find useful (Chapter 5).

THE PROGRAM ASSESSMENT RATING TOOL AND EFFICIENCY

Efficiency is a common enough concept, as illustrated by familiar dictionary definitions: “effective operation as measured by a comparison of production with cost (as in energy, time, and money)” and “the ratio of the useful energy delivered by a dynamic system to the energy supplied to it.”2 The PART approach to efficiency is explained this way by OMB (OMB 2007a, p. 9):

Efficiency measures reflect the economical and effective acquisition, utilization, and management of resources to achieve program outcomes or produce program outputs. Efficiency measures may also reflect ingenuity in the improved design, creation, and delivery of goods and services to the public, customers, or beneficiaries by capturing the effect of intended changes made to outputs aimed to reduce costs and/or improve productivity, such as the improved targeting of beneficiaries, redesign of goods or services for simplified customer processing, manufacturability, or delivery.

APPLYING EFFICIENCY TO INPUTS, OUTPUTS, AND OUTCOMES

Any definition of efficiency depends on the process to which it is applied. Of relevance to this report is its application to the processes of research and development, which, as described by OMB, are complex and variable and involve

2

Merriam Webster Online, http://www.m-w.com/.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement