National Academies Press: OpenBook

An Assessment of ARPA-E (2017)

Chapter: 1 Introduction

« Previous: Summary
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×

1

Introduction

When the U.S. Congress authorized creation of the Advanced Research Projects Agency-Energy (ARPA-E) within the U.S. Department of Energy (DOE), it mandated that the secretary of energy ask the National Academy of Sciences to assess how well the agency was achieving its statutory mission and goals. ARPA-E’s authorizing statute states that its mission is “to overcome long-term and high-risk technology barriers in the development of energy technologies,”1 and its goals are

  1. to enhance the economic and energy security of the United States through the development of energy technologies that result in—
    1. reductions of imports of energy from foreign sources;
    2. reductions of energy-related emissions, including greenhouse gases; and
    3. improvement in the energy efficiency of all economic sectors; and
  2. to ensure that the United States maintains a technological lead in developing and deploying advanced energy technologies.”2

ARPA-E began operating in 2009, and as requested by the Congress, the National Academy of Sciences convened an ad hoc study committee to undertake this assessment starting in 2015, after 6 years of agency operations. The purpose of this study was to provide the information requested by the Congress as input to its consideration of the future of ARPA-E, to summarize

___________________

1 42 U.S.C. § 16538(b). This subsection is labeled “ESTABLISHMENT,” but its second clause appears to state Congress’s intended mission for ARPA-E.

2 42 U.S.C. § 16538(c).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×

lessons learned that may apply to the operations of other DOE programs,3 and to provide ARPA-E leadership with information it can utilize as the agency continues to evolve and seeks to improve its efficacy and impact.

STATEMENT OF TASK

ARPA-E was originally conceived in a recommendation from a previous study of the National Academies (NAS et al., 2007). The statutory language mandating this assessment requested a review of the agency’s methods, its success in fostering transformational innovation, and the adequacy of its resources. Those topics are reflected in the statement of task for this study (see Box 1-1).

___________________

3 42 U.S.C. § 165381(l).

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×

STUDY SCOPE

To operationalize the statement of task for this study, the ad hoc committee posed three broad questions that this report is intended to answer:

  • What qualitative and quantitative standards are appropriate for assessing ARPA-E’s performance?
  • To what extent are ARPA-E’s organization, operational methods, and personnel practices positioning it to accomplish its mission and goals as established by the Congress?
  • What impacts has ARPA-E had on its awardees and on the nation’s energy technology innovation system?

In attempting to answer these questions, the committee had to take several caveats into account:

  • ARPA-E’s performance has been affected by external factors, such as the length of time that elapsed while awaiting Senate confirmation of both its first and second directors.
  • The 6 years of ARPA-E operations provides insufficient data to enable a full assessment of potentially transformational research whose impact likely will not be fully observable until years or decades in the future.
  • A number of important terms in the statement of task—such as “cutting-edge inventions,” “revolutionary advances,” “qualified key personnel,” and “transformational technological advances”—are intrinsically subjective, difficult to define, and even more difficult to measure.

STUDY APPROACH

ARPA-E’s authorizing legislation is not specific as to how or to what standards the agency’s operations and performance were to be measured. Accordingly, the committee elected to apply metrics traditionally used in the study of innovation in attempting to measure intermediate outcomes of the work completed under projects funded by ARPA-E.

The committee approached its task by utilizing both quantitative and qualitative methods. To carry out the assessment, it formed four subgroups with a number of committee members and National Academies staff participating in more than one subgroup. Some data gathering and processing was performed by independent consultants. The Internal Operations Qualitative Data Team collected and analyzed qualitative data, including information gathered during interviews with current and former ARPA-E personnel and DOE officials and discussions with stakeholders at agency events, such as program kickoffs and

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×

interim meetings. The Case Study Team conducted three types of case studies (a focused program, a portfolio of energy storage projects, and individual projects). The Internal Operations Quantitative Data Team accessed internal, proprietary ARPA-E data and reported aggregated findings. Finally, the External Quantitative Data Team obtained data from external sources, such as the U.S. Patent and Trademark Office’s (USPTO) patent database and publications. (See Appendix C for more details on the data and methods used in this report.) Using the statement of task as the point of departure, the subgroups considered the evidence that became available through their research activities. Their deliberations on this evidence led to provisional findings that were summarized and presented to the full committee, which discussed, debated, and compared them. This process went through a number of iterations, leading to final findings and recommendations based on those findings.

The first ARPA-E awards were made in October 2009 for projects of 3 years’ duration, meaning that any external impacts from these first-generation projects after ARPA-E funding had ended would have begun to manifest only during the 3 years preceding the start of this study. By contrast, the development of new energy technologies usually requires decades of effort, and decades also must usually elapse before it becomes evident that innovations are truly transformative. It is likely, therefore, that any external impact of more recently initiated projects, which represent the bulk of ARPA-E’s portfolio, will not be observable for some years. Consequently, the committee’s findings are best regarded as early assessments based on the evidence available over a relatively short timeframe. While more data and more time for analysis may result in more understanding, the committee is confident that the findings and recommendations presented in this report are more than sufficient for use by the Congress in determining ARPA-E’s future, and will be of value to ARPA-E as it continues to evolve and improve its operations.

ORGANIZATION OF THE REPORT

Chapter 2 of this report presents an overview of ARPA-E’s genesis, structure, and operations; its congressional mandate and funding; and the results of previous agency audits and oversight initiatives. Chapter 3 provides details of the committee’s operational assessment of ARPA-E’s performance as an organization, its operational methods, and its personnel practices. Chapter 4 presents the committee’s technical assessment of the extent to which the projects ARPA-E has funded have contributed to the agency’s achievement of its statutory mission and goals, to the extent this can be ascertained from the relatively brief experience base accumulated since the agency began its operations. The report also includes six appendixes: Appendix A provides biographical information on the committee members; Appendix B provides a disclosure of conflict of interest of a committee member; Appendix C

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×

summarizes the various methodologies used in this assessment; Appendix D contains the full text of the 10 case studies developed as input for this assessment; Appendix E provides a glossary of acronyms and abbreviations used in this report; Appendix F contains the committee’s formal request for data made to ARPA-E and the agency’s official response; and Appendix G includes the white papers prepared by external consultants detailing their data analyses.

Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Page 15
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Page 16
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Page 17
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Page 18
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Page 19
Suggested Citation:"1 Introduction." National Academies of Sciences, Engineering, and Medicine. 2017. An Assessment of ARPA-E. Washington, DC: The National Academies Press. doi: 10.17226/24778.
×
Page 20
Next: 2 Overview of ARPA-E »
An Assessment of ARPA-E Get This Book
×
Buy Paperback | $64.00 Buy Ebook | $49.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In 2005, the National Research Council report Rising Above the Gathering Storm recommended a new way for the federal government to spur technological breakthroughs in the energy sector. It recommended the creation of a new agency, the Advanced Research Projects Agency-Energy, or ARPA-E, as an adaptation of the Defense Advanced Research Projects Agency (DARPA) model—widely considered a successful experiment that has funded out-of-the-box, transformative research and engineering that made possible the Internet, GPS, and stealth aircraft. This new agency was envisioned as a means of tackling the nation’s energy challenges in a way that could translate basic research into technological breakthroughs while also addressing economic, environmental, and security issues.

Congress authorized ARPA-E in the 2007 America COMPETES Act and requested an early assessment following 6 years of operation to examine the agency’s progress toward achieving its statutory mission and goals. This report documents the results of that assessment. It includes both an operational assessment of the agency’s funding programs and a technical assessment of its awards, to the extent possible.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!