National Academies Press: OpenBook
« Previous: References
Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

Appendixes

Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

This page intentionally left blank.

Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

A
PART Assessment Questions

1.
PROGRAM PURPOSE AND DESIGN

1.1 Is the program purpose clear?


1.2 Does the program address a specific and existing problem, interest or need?


1.3 Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?


1.4 Is the program design free of major flaws that would limit the program’s effectiveness or efficiency?


1.5 Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program’s purpose directly?

2.
STRATEGIC PLANNING

2.1 Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?


2.2 Does the program have ambitious targets and timeframes for its long-term measures?


2.3 Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program’s long-term goals?


2.4 Does the program have baselines and ambitious targets for its annual measures?


2.5 Do all partners (including grantees, subgrantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or longterm goals of the program?


2.6 Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?


2.7 Are budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program’s budget?


2.8 Has the program taken meaningful steps to correct its strategic planning deficiencies?


2RD1 If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts that have similar goals?


2RD2 Does the program use a prioritization process to guide budget requests and funding decisions?

3.
PROGRAM MANAGEMENT

3.1 Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?


3.2 Are federal managers and program partners (including grantees, subgrantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?


3.3 Are funds (federal and partners’) obligated in a timely manner and spent for the intended purpose?

NOTE: This appendix is based on Department of Energy PART Assessments, available at <http://www.whitehouse.gov/omb/budget/fy2006/pma/energy.pdf>.

Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×

3.4 Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?


3.5 Does the program collaborate and coordinate effectively with related programs?


3.6 Does the program use strong financial management practices?


3.7 Has the program taken meaningful steps to address its management deficiencies?


3RD1 For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

4.
PROGRAM RESULTS/ACCOUNTABILITY

4.1 Has the program demonstrated adequate progress in achieving its long-term performance goals?


4.2 Does the program (including program partners) achieve its annual performance goals?


4.3 Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?


4.4 Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?


4.5 Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 67
Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 68
Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 69
Suggested Citation:"Appendix A PART Assessment Questions." National Research Council. 2007. Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two). Washington, DC: The National Academies Press. doi: 10.17226/11806.
×
Page 70
Next: Appendix B Committee Biographies »
Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two) Get This Book
×
Buy Paperback | $69.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Since its inception in 1977 from an amalgam of federal authorities, the U.S. Department of Energy (DOE) has administered numerous programs aimed at developing applied energy technologies. In recent years, federal oversight of public expenditures has emphasized the integration of performance and budgeting. Notably, the Government Performance and Results Act (GPRA) was passed in 1993 in response to questions about the value and effectiveness of federal programs. GPRA and other mandates have led agencies to develop indicators of program performance and program outcomes. The development of indicators has been watched with keen interest by Congress, which has requested of the National Research Council (NRC) a series of reports using quantitative indicators to evaluate the effectiveness of applied energy research and development (R&D).

The first such report took a retrospective view of the first 3 years of DOE R&D programs on fossil energy and energy efficiency. The report found that DOE-sponsored research had netted large commercial successes, such as advanced refrigerator compressors, electronic lighting ballasts, and emission control technology for flue gas desulfurization. However, some programs were judged to be costly failures in which large R&D expenditures did not result in a commercial energy technology. A follow-up NRC committee was assigned the task of adapting the methodology to the assessment of the future payoff of continuing programs.

Evaluating the outcome of R&D expenditures requires an analysis of program costs and benefits. Doing so is not a trivial matter. First, the analysis of costs and benefits must reflect the full range of public benefits that are envisioned, accounting for environmental and energy security impacts as well as economic effects. Second, the analysis must consider how likely the research is to succeed and how valuable the research will be if successful. Finally, the analysis must consider what might happen if the government did not support the project: Would some non-DOE entity undertake it or an equivalent activity that would produce some or all of the benefits of government involvement?

This second report continues to investigate the development and use of R&D outcome indicators and applies the benefits evaluation methodology to six DOE R&D activities. It provides further definition for the development of indicators for environmental and security benefits and refines the evaluation process based on its experience with the six DOE R&D case studies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!