National Academies Press: OpenBook

Evaluating Research Efficiency in the U.S. Environmental Protection Agency (2008)

Chapter: Appendix I: PART Guidance on Efficiency Measures

« Previous: Appendix H: Charge to the BOSC Subcommittee on Safe Pesticides/Safe Products Research
Suggested Citation:"Appendix I: PART Guidance on Efficiency Measures." National Research Council. 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: The National Academies Press. doi: 10.17226/12150.
×
Page 128
Suggested Citation:"Appendix I: PART Guidance on Efficiency Measures." National Research Council. 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: The National Academies Press. doi: 10.17226/12150.
×
Page 129
Suggested Citation:"Appendix I: PART Guidance on Efficiency Measures." National Research Council. 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: The National Academies Press. doi: 10.17226/12150.
×
Page 130
Suggested Citation:"Appendix I: PART Guidance on Efficiency Measures." National Research Council. 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: The National Academies Press. doi: 10.17226/12150.
×
Page 131
Suggested Citation:"Appendix I: PART Guidance on Efficiency Measures." National Research Council. 2008. Evaluating Research Efficiency in the U.S. Environmental Protection Agency. Washington, DC: The National Academies Press. doi: 10.17226/12150.
×
Page 132

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix I PART Guidance on Efficiency Measures1 DESCRIPTION OF EFFICIENCY MEASURES FOR PART Efficiency Measures While outcome measures provide valuable insight into program achieve- ment, more of an outcome can be achieved with the same resources if an effec- tive program increases its efficiency. The President’s Management Agenda (PMA) Budget and Performance Integration (BPI) Initiative encourages agen- cies to develop efficiency measures. Sound efficiency measures capture skillful- ness in executing programs, implementing activities, and achieving results, while avoiding wasted resources, effort, time, and/or money. Simply put, effi- ciency is the ratio of the outcome or output to the input of any program. Because they relate to costs, efficiency measures are likely to be annual measures. • Outcome efficiency measures: The best efficiency measures capture improvements in program outcomes for a given level of resource use. Outcome efficiency measures are generally considered the best type of efficiency measure for assessing the program overall. For example, a program that has an outcome goal of “reduced energy consumption” may have an efficiency measure that shows the value of energy saved in relation to program costs. • Output efficiency measures: It may be difficult to express efficiency measures in terms of outcomes. In such cases, acceptable efficiency measures could focus on how to produce a given output level with fewer resources. How- ever, this approach should not shift incentives toward quick, low-quality meth- ods that could hurt program effectiveness and desired outcomes. 1 OMB 2006. 128

Appendix I 129 Meaningful efficiency measures consider the benefit to the customer and serve as indicators of how well the program performs. For example, reducing processing time means little if error rates increase. A balanced approach is re- quired to enhance the performance of both variables in pursuit of excellence to customers. In these instances, one measure (e.g., increase in customer satisfac- tion) may be used in conjunction with another complementary measure (e.g., reduction in processing time). In all cases, efficiency measures must be useful, relevant to program pur- pose, and help improve program performance. An efficiency measure for a Fed- eral program tracks the ratio of total outputs or outcomes to total inputs (Federal plus non-Federal). Leveraging program resources can be a rational policy deci- sion, as it leads to risk or cost sharing; however, it is not an acceptable effi- ciency measure, because the leveraging ratio of non-Federal to Federal dollars represents only inputs. Although increasing the amount leveraging in a program may stretch Federal program dollars, this does not measure improvements in the management of total program resources, systems, or outcomes. 3.4: Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution? Purpose: To determine whether the program has effective management procedures and measures in place to ensure the most efficient use of each dollar spent on program execution. Elements of Yes: A Yes answer needs to clearly explain and provide evi- dence of each of the following [see Box I-1]: • The program has regular procedures in place to achieve efficiencies and cost effectiveness. • The program has at least one efficiency measure with baseline and targets. BOX I-1 Measures and PARTWeb To receive a Yes answer, the program must include at least one effi- ciency measure, baseline data/estimates, and targets in the Measures screen in PARTWeb. Only measures that meet the standards for a Yes should be entered in PARTWeb. Please ensure that the proper characterization of measures is se- lected in PARTWeb (that is “efficiency”). Make sure to indicate the term of the measure in PARTWeb too (that is, long-term, annual, or long- term/annual).

130 Evaluating Research Efficiency in EPA There are several ways to demonstrate that a program has established pro- cedures for so improving efficiency. For example, a program that regularly uses competitive sourcing to determine the best value for the taxpayer, invests in IT with clear goals of improving efficiency, etc., could receive a Yes. A de-layered management structure that empowers front line managers and that has under- gone competitive sourcing (if necessary) would also contribute to a Yes answer. For mandatory programs, a Yes could require the program to seek policies (e.g., through review of proposals from States) that would reduce unit costs. Also con- sider if, where possible, there is cross-program and inter-agency coordination on IT issues to avoid redundancies. The program is not required to employ all these strategies to earn a Yes. Rather, it should demonstrate that efforts improving efficiency are an established, regular part of program management. An efficiency measure can be the per-unit cost of outcomes or outputs, a timing target, and other indicator of efficient and productive processes germane to the program. Efficiency measures are likely to be annual measures since they relate to cost. The answer to this question should describe how measures are used to evaluate the program’s success if achieving efficiency and cost effectiveness improvements. Elements of No: A No must be given if the agency and OMB have not reached agreement on efficiency measures that meet PART guidance. Not Applicable: Not Applicable is not an option for this question. For more detailed discussion on defining acceptable efficiency measures please see the section called “4. Select Performance Measure” of this document or visit OMB’s PART website.2 Evidence/Data: Evidence can include efficiency measures, competitive- sourcing plans, IT improvement plans designed to produce tangible productivity and efficiency gains, or IT business cases that document how particular projects improve efficiency. 4.3: Does the program demonstrate improved efficiencies or cost effective- ness in achieving program goals each year? Purpose: To determine whether management practices have resulted in ef- ficiency gains over the past year. Elements of Yes: A Yes answer needs to clearly explain and provide evi- dence of each of the following [see Box I-2]: • The program demonstrated improved efficiency or cost effectiveness over the prior year. When possible, the explanation should include specific in- formation about the program’s annual savings over the prior year as well as what the program did to achieve the savings. 2 http://www.omb.gov/part/.

Appendix I 131 BOX I-2 Question Linkages If a program received a No in Question 3.4, the program must receive a No answer to this question. Efficiency improvements should generally be measured in terms of dollars or time. For example, programs that complete an A-76 competition—an indica- tor of cost-efficient processes—would contribute to a Yes answer, provided that the competition resulted in savings. Not Applicable: Not Applicable is not an option for this question. Evidence/Data: Evidence can include meeting performance targets to re- duce per unit costs or time, meeting production and schedule targets; or meeting other targets that result in tangible productivity or efficiency gains. Efficiency measures may also be considered in Questions 4.1 and 4.2. REFERENCES OMB (Office of Management and Budget). 2006. Program Assessment Rating Tool Guidance 2006-02. Office of Management and Budget. March 2006 [online]. Available: http://www.whitehouse.gov/omb/part/fy2006/2006_ guidance_final.pdf [accessed Dec. 17, 2007].

Evaluating Research Efficiency in the U.S. Environmental Protection Agency Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

A new book from the National Research Council recommends changes in how the federal government evaluates the efficiency of research at EPA and other agencies. Assessing efficiency should be considered only one part of gauging a program's quality, relevance, and effectiveness. The efficiency of research processes and that of investments should be evaluated using different approaches. Investment efficiency should examine whether an agency's R&D portfolio, including the budget, is relevant, of high quality, matches the agency's strategic plan. These evaluations require panels of experts. In contrast, process efficiency should focus on "inputs" (the people, funds, and facilities dedicated to research) and "outputs" (the services, grants, publications, monitoring, and new techniques produced by research), as well as their timelines and should be evaluated using quantitative measures. The committee recommends that the efficiency of EPA's research programs be evaluated according to the same standards used at other agencies. To ensure this, OMB should train and oversee its budget examiners so that the PART questionnaire is implemented consistently and equitably across agencies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!