search-intensive agencies¹ and industries, the committee evaluated how EPA and other agencies were attempting to comply with PART. The committee focused its deliberations on several fundamental issues posed by the charge questions, including

1. How—and why—should research be evaluated in terms of efficiency?

2. What is a “sufficient” measure of efficiency?

3. What measures of efficiency are “outcome-based,” and should they be?

In its discussion the committee uses the terms inputs, outputs, and outcomes as defined by OMB, except as modified and discussed below:

• Inputs are agency resources—such as funding, facilities, and human capital—that support research.

• Outputs are activities or accomplishments delivered by research programs, such as research findings, papers published, exposure methods developed and validated, and research facilities built or upgraded.

• Outcomes are the benefits resulting from a research program, which can be short-term, such as an improved body of knowledge or a comprehensive science assessment, or long-term, such as lives saved or enhancement of air quality, that may be based on research activities or informed by research but that require additional activities by many others. The committee distinguishes these two types of outcomes using the terms, intermediate outcomes and ultimate or end outcomes.²

With respect to the question, “How—and why—should research be evaluated in terms of efficiency?”, the committee suggests that some of the frustration expressed by federal research-intensive agencies in complying with PART derives from confusion over the concept of “efficiency.” From its review of the OMB PART guidance and efficiency measures used by EPA and other federal agencies, the committee concludes that two conceptually different kinds of efficiency are integral to the execution and evaluation of R&D programs. The committee distinguished between investment efficiency and process efficiency.

Investment efficiency focuses on portfolio management, including the need to identify the most promising lines of research for achieving desired outcomes.