National Academies Press: OpenBook

An Enabling Foundation for NASA's Earth and Space Science Missions (2010)

Chapter: Appendix D: Benchmarking High-Risk/High-Payoff Research

« Previous: Appendix C: Traceability of Mission-Enabling Activities from Strategic Goals
Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×

D
Benchmarking High-Risk/High-Payoff Research

DEPARTMENT OF DEFENSE SERVICE LABORATORIES

While there is no precise organizational analogy to NASA’s Science Mission Directorate (SMD), examining how other aerospace-related agencies allocate resources for high-risk/high-payoff endeavors can be instructive. The three service science and technology (S&T) enterprises (Army, Navy, Air Force) under the Department of Defense (DOD) provide one useful data point for addressing high-risk/high-payoff resource allocation.

To provide a context for discussing how the military services pursue high-risk/high-payoff research endeavors, it is important to begin with an understanding of the budget architecture for DOD science and technology as a whole. The budgetary architecture for science and technology (S&T, or the “R” in R&D) within the DOD consists of three budget categories:

  • 6.1: Basic research, typically associated with technology readiness level (TRL) 1 and 2, in which new scientific phenomena are sought in an effort to discover and advance fundamental knowledge in fields important to national defense. Such research is generally broad in nature, and because of its low TRL, can be considered inherently “high risk.”

  • 6.2: Applied research (also called exploratory development), typically associated with TRL 3 and 4, in which technology is developed based on a newly discovered scientific phenomena, or by the application of scientific phenomena in a totally different manner than currently applied.

  • 6.3: Advanced technology development, typically associated with TRL 5 and 6, in which multiple technologies (often from cross disciplines) are integrated and demonstrated to enable the development of a new military capability to satisfy a military need.

Budget categories 6.1, 6.2, and 6.3, which collectively comprise S&T, can be envisioned as mission enabling and represent the feedstock for the “D” in R&D (D is sometimes called engineering development; it is the engineering of advanced technology into an end item that can eventually be tested, manufactured, and delivered to the military war fighter). However, within the S&T spectrum, the 6.1 category (basic research) can most readily be labeled high risk/high payoff. That is because in examining basic scientific phenomena, there are limited expectations that such research will always yield usable results; however, when such discoveries are made, the accompanying expectation is that at least some will yield exceptionally high results, eventually enabling revolutionary

Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×

military capabilities. A National Research Council report, Assessment of Department of Defense Basic Research,1 discusses the basic research (6.1) element of S&T in terms of “unfettered exploration” (p. 14) and describes it as “farsighted, high-payoff research that provides the basis for technological progress” (p. 44). Past examples of game-changing military capabilities enabled by discoveries from basic research projects include night vision; stealth technologies; near real-time delivery of battlefield information; navigation, communication, and weather satellites; and precise munitions. As noted in the study referenced in the follow-on benchmarking discussion, investments in basic research have led to revolutionary civilian capabilities as well. The Internet, communications and weather satellites, global positioning technology, and even the search technologies used by Google all had their origins in DOD basic research.

A 6.1 portfolio typically is comprised of numerous projects that are relatively small in terms of budget when compared to projects in the 6.2 budget category (fewer projects, bigger in scope and cost as compared to the average 6.1 project) and the 6.3 budget category (even fewer projects, bigger in scope and cost as compared to the average 6.2 project budget). A large number of small 6.1 projects across the full spectrum of scientific disciplines maximize the opportunity of discovering scientific phenomena that can be game changing in nature if successfully implemented. As previously stated, if only a few such projects result in the discovery of phenomena that can subsequently be translated to technology in the 6.2 and 6.3 budget categories, that 6.1 investment portfolio would be considered as having achieved its purpose. The challenge: it can sometimes take years following discovery of a new phenomenon to know whether or not the technology it spawns has really enabled a useful new capability to the sponsoring agency.

Within the service laboratories, overall 6.1 budgets are usually established by upper-management, such as by the particular service secretary or the senior-most executive responsible for the Service’s overall R&D component. For the Army and Air Force, the 6.1 budget has varied between 15 to 20 percent of the overall S&T portion of the president’s budget request between FY 1998 and FY 2008. For the Navy, the 6.1 budget has varied between 25 to 30 percent of the overall S&T portion of the president’s budget request for the same 10-year period. While not every dollar of these basic research budgets can be definitively associated with high-risk research, the percentages represent an upper bound, and the committee would surmise in its collective experience that a significant portion of such budgets are aligned with high-risk, high-payoff research endeavors. To the committee’s knowledge, there is no documented analytical basis for these specific 6.1 budget percentages. Rather, this allocation has developed over the years for each respective service, has waxed and waned, and has ultimately been determined to be a reasonable level of investment for research of a broad and unfettered nature when traded off against budget needs for more specific S&T outcomes as pursued under the 6.2 and 6.3 budget categories. It should be noted that these 6.1 percentage allocations typically decrease when looking at the actual budget appropriated by Congress, since Congress often adds resources to the president’s budget request, predominantly in the 6.2 and 6.3 program elements in support of specific projects or capabilities.

Debates often rage during annual budget formulations as to whether a specific top-down mandated 6.1 budget is over- or underfunded. Natural tensions tend to want to decrease 6.1 budgets to provide more resources for 6.2 and 6.3 (i.e., exploratory and advanced technology development), respectively, to meet more specific, shorter-term needs or desired capabilities. Yet, history has shown that fencing and protecting a basic research budget as a means of providing “seed corn” to the technology developers and encouraging the pursuit of high-risk, very-high-payoff endeavors is critical to unimagined future capabilities. The protective fencing of an overall 6.1 budget for high-risk, high-payoff research does not imply that the portfolio of individual projects should not or cannot be actively managed. On the contrary, there will be more ideas for basic research projects than there are resources for pursuing them, and actively managing the basic research portfolio is necessary to ensure that the quality and relevance of the assigned projects is maximized. A basic research portfolio plants the seeds for future technologies, cultivates successful emerging capabilities, and eliminates less successful efforts.

The services typically manage their basic research budgets through separate organizational entities vis-à-vis the organizations that manage the remainder (and much larger portion) of the S&T budget. For instance, the Air

1

National Research Council, Assessment of Department of Defense Basic Research, The National Academies Press, Washington, D.C., 2005.

Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×

Force Research Laboratory (AFRL) has nine technology directorates, each of which manages the 6.2 and 6.3 budgets for its respective technology discipline (e.g., sensor technology, directed energy technology, information technology, propulsion technology, etc). But the 6.1 budget is managed separately by the AFRL’s Air Force Office of Scientific Research (AFOSR).

As the sole 6.1 budget manager for the Air Force, AFOSR is organized along scientific disciplines (e.g., chemistry and life sciences; aerospace and materials; physics and electronics; and math and geosciences), and its project managers seek discovery of new phenomena in these disciplines that could enable potential game changers when translated to a technology that can, in turn, provide a significantly improved military capability. AFOSR establishes the strategic objectives and investment strategy for the Air Force 6.1 budget, and AFOSR project managers then allocate that budget to universities (who are awarded the majority of the basic research portfolio), the AFRL technology directorates, or industry for execution. One rationale for a separate agency to manage this investment is the realization that managing basic research projects, with their unfettered and open approach where failure can occur as often as success, is different from managing technology development projects that often have performance and schedule milestones typically associated with more definitive and specific projects. Another rationale is that mixing the basic research budget with the technology budget in the same organization could lead to migration of the basic research budget into technology development activities, especially when technology projects (which have more definitive milestones because of their higher TRLs) encounter technical difficulties that lead to cost and schedule overruns.

Rising Above the Gathering Storm Report Recommendations

Another source regarding the amount that R&D-related government agencies should invest in high-risk, high-payoff endeavors can be found in the seminal report Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future.2 The study’s rationale was based on a widespread consensus among scientific and technological leaders throughout the United States that the nation’s science and innovation enterprise was either stalled or in decline, with the consequence of jeopardizing future U.S. prosperity. Accordingly, the study committee was asked to identify the top 10 actions that federal policy makers could take to enhance the science and technology enterprise so that the United States can successfully compete, prosper, and be secure in the global community of the 21st century. Significantly, one of those actions concerned basic research, and the need not only for a higher level of investment but also for additional emphasis on high-risk research endeavors. Specifically:

  • Recommendation B of Gathering Storm says the following: “Sustain and strengthen the nation’s traditional commitment to long-term basic research that has the potential to be transformational [emphasis added] to maintain the flow of new ideas that fuel the economy, provide security, and enhance the quality of life” (p. 7). The phrase “that has the potential to be transformational” can be equated to “high risk, high payoff” and thus tends to validate the premise from the earlier benchmarking discussion that long-term basic research is indeed the appropriate budget element to sponsor high-risk research.

  • Of perhaps greater significance in the context of this particular benchmarking discussion, Action B-4, entitled “High-Risk Research,” states the following: “At least 8 percent of the budgets of federal research agencies should be set aside for discretionary funding managed by technical program managers in those agencies to catalyze high-risk, high-payoff research” (p. 149). The discussion accompanying this recommended action points out that an important subset of basic research is the high-risk or transformative research involving new theories, methods, or tools that are often developed by new investigators—the group most likely to generate radical discoveries or new technologies.

The Gathering Storm report highlights two additional points relevant to an R&D entity’s resource allocation for high-risk, high-payoff research:

2

National Academy of Sciences-National Academy of Engineering-National Research Council, Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future, The National Academies Press, Washington, D.C., 2007.

Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×
  1. It is not necessarily a matter of providing additional resources for high-risk research, but rather providing incentives for program managers to fund high-risk research out of a discretionary portion of the existing budget. Lack of incentives for (or barriers to) performing high-risk research include (1) a peer review system that tends to favor established investigators using well-known methods; (2) pressure from customers and management for short-term results; and (3) risk adverseness, since high-risk projects are prone to failing and increased government and public scrutiny make “projects that fail” increasingly untenable. The study points out that partly because of these barriers, the percentage of efforts allocated to high-risk research is often quite small—1 to 3 percent being common. Individual committee members, based on their long-standing expertise in this area, believed that additional discretionary funding within existing budgets between 5 and 10 percent should be devoted to high-risk research. Thus, 8 percent seemed a reasonable compromise and is reflected in the committee’s recommended Action B-4 discussed earlier. Significantly, the committee also pointed out that the degree to which a high-risk research program will be successful depends heavily on the quality of the program staff.

  2. The study indicates the importance of government involvement in high-risk basic research. Specifically, while federal government spending in R&D has declined over the past several years, corporate R&D spending has dramatically increased and, in fact, has become the linchpin of the U.S. R&D enterprise. However, corporate R&D funds relatively little basic research, since basic research does not typically offer immediate benefits to its sponsor. The study states that basic research, by its nature, is almost by definition risky (emphasis added), and shareholder pressure for short-term results discourages long-term, speculative investment by industry.

To summarize, the consensus of the Gathering Storm committee is as follows:

  • Approximately 8 percent of a government entity’s S&T budget should be allocated to high-risk, high-payoff research.

  • Government agencies must take the lead in funding such research, since industry R&D is predominantly short-term in nature.

  • Additional government budget is probably not needed; rather, it is a matter of ensuring that 8 percent of the existing S&T budget is discretionary and fenced for high-risk research, and that the barriers which discourage government program managers to pursue high-risk research are eliminated for this portion of the S&T investment portfolio.

Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×
Page 56
Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×
Page 57
Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×
Page 58
Suggested Citation:"Appendix D: Benchmarking High-Risk/High-Payoff Research." National Research Council. 2010. An Enabling Foundation for NASA's Earth and Space Science Missions. Washington, DC: The National Academies Press. doi: 10.17226/12822.
×
Page 59
Next: Appendix E: Committee and Staff Biographical Information »
An Enabling Foundation for NASA's Earth and Space Science Missions Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

NASA's space and Earth science program is composed of two principal components: spaceflight projects and mission-enabling activities. Most of the budget of NASA's Science Mission Directorate (SMD) is applied to spaceflight missions, but NASA identifies nearly one quarter of the SMD budget as "mission enabling." The principal mission-enabling activities, which traditionally encompass much of NASA's research and analysis (R&A) programs, include support for basic research, theory, modeling, and data analysis; suborbital payloads and flights and complementary ground-based programs; advanced technology development; and advanced mission and instrumentation concept studies.

While the R&A program is essential to the development and support of NASA's diverse set of space and Earth science missions, defining and articulating an appropriate scale for mission-enabling activities have posed a challenge throughout NASA's history. This volume identifies the appropriate roles for mission-enabling activities and metrics for assessing their effectiveness. Furthermore, the book evaluates how, from a strategic perspective, decisions should be made about balance between mission-related and mission-enabling elements of the overall program as well as balance between various elements within the mission-enabling component. Collectively, these efforts will help SMD to make a good program even better.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!