tum. The office wanted another opportunity to focus on defining success and on the design and implementation of programs. OMB has decided that efficiency should be measured because R&D programs need to maintain a set of high-priority, multi-year objectives with annual performance outputs and milestones that show how one or more outcomes will be reached despite limited resources.
PART is used for several purposes. The most basic is to evaluate the success of programs. The second is to monitor the annual improvement plans required of each program.
Evaluation of the first two PART criteria, quality and relevance, primarily by expert review, has caused few problems. Application of the performance criterion—especially the measures of efficiency—has proved to be a challenge. As a result, OMB has approached implementation of that third criterion as a learning process.
The two relevant PART questions concerning the efficiency of R&D are questions 3.4 and 4.3. Question 3.4 asks, “Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies in program execution?” A way to measure efficiency is required for a “yes” response. Question 4.3 asks, “Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?” Answering question 4.3 is predicated on a “yes” response to question 3.4 and improved efficiencies or cost effectiveness in achieving goals should be described in terms of dollars when possible.
For the President’s Management Agenda (PMA), the agency is given a “yellow” score when at least 50% of agency programs rated by PART have at least one efficiency measure and a “green” score when all agency programs rated by PART have at least one efficiency measure.
The meaning of efficiency, as OMB has applied PART, includes both outcomes or outputs for a given amount of inputs and inputs for a given amount of outcomes or outputs. Outcome efficiency might be measured in the economic terms of benefit-cost ratio, cost-benefit ratio, or cost effectiveness. Output efficiency might be measured in terms of productivity (input/output) or unit cost (output/input) or with respect to a standard or benchmark.
For outcomes, attribution of success or failure is inexact and may be based on indicators as diverse as improved targeting of beneficiaries or customers, a radically different mode of intervention, productivity improvements, or cost reductions. For outputs, efficiency might be described in relation to a program’s resources, such as the use of labor or material, improved capability, or procurement. PART also requires that measures of outcome efficiency “consider the benefit to the customer and serve as an indicator of the program’s operational performance.”
Output efficiencies have various potential criteria. They must reflect efficient use of resources, measure changes over time that should correspond to a decrease or increase in related costs, and include an assessment of the comparability of the kinds of outputs produced.