National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×

2
Assessment Methodology

FUNDAMENTAL SCIENCE AND ENGINEERING RESEARCH

For this study it is important to have a clear definition and understanding of “fundamental science and engineering research.” The committee chose to adopt the definitions used by the Office of Management and Budget (OMB) in tracking and reporting federal spending for research and development (R&D).1 The OMB requires all federal agencies to report R&D spending in four categories:

  • Basic research. Systematic study directed toward a fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications toward processes or products in mind.

  • Applied research. Systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need can be met.

  • Development. Systematic application of knowledge or understanding directed toward the production of useful materials, devices, and systems or methods, including design, development, and improvements of prototypes and new processes to meet specific requirements.

  • Facilities and equipment.

    • R&D equipment. Such equipment includes the acquisition or design and production of movable equipment, such as spectrometers, research satellites, detectors, and other instruments.

    • R&D facilities. Such facilities include the acquisition, design, and construction of, or major repairs or alterations to, all physical facilities for use in R&D activities. Facilities include land, buildings, and fixed capital equipment, regardless of whether the facilities are to be used by the government or by a private organization, and regardless of where title to the property may rest. This category includes such fixed facilities as reactors, wind tunnels, and particle accelerators.

The OMB has used these or similar categories in the collection of R&D data since 1949. The use of the OMB categories and the financial data reported allowed the committee to determine trends in expenditures for fundamental science and engineering research and for the facilities and equipment that support that research.

NASA uses the terminology of technology readiness levels (TRL levels 1 through 9)2 to define the technological maturity of a hardware or software system under development. TRL 1 refers to scientific knowledge generated as an underpinning of hardware or software concepts and applications. TRL 2 applies when a practical application has been identified but is speculative and without experimental proof or detailed analysis to support the conjecture. TRL 3 applies when analytical studies place the technology in an appropriate context and laboratory demonstrations, modeling, simulation, and nonintegrated software components have validated the analytical prediction. The definitions of the TRL system are provided in Appendix B.

1

Office of Management and Budget Circular A-11, Section 84, 2008.

2

NASA Research and Technology Program and Project Management Requirements—NASA Procedural Requirements 7120.8, Appendix J. Effective date February 5, 2008.

Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×

The committee decided that the NASA TRL 1 through 3 (TRL 1-3) categories best matched the study requirement to “focus on an appraisal of equipment, facilities and support services used for fundamental science and engineering research” and directed NASA to limit presentations on facilities, equipment, support services, and science/engineer interviews to these categories.

STUDY PROCESS AND LIMITATIONS

The committee invited NASA Headquarters personnel and representatives of each NASA center that sponsored fundamental research to make a series of presentations at the first committee meeting held at the National Academy of Sciences building in Washington, D.C., on September 8 and 9, 2009. Before the meeting, the committee had requested that NASA Headquarters and each NASA center provide the following information at this first meeting and limit all presentations and material to activities encompassing TRLs 1-3:

  • Expenditures by year for new laboratory equipment over the past 5 years;

  • Expenditures for new facilities and major upgrades to existing facilities over the past 5 years;

  • Planned expenditures for laboratory equipment and facility improvements over the next 3 years;

  • The age distribution of existing laboratory equipment and the maintenance, repair, and upgrades of older equipment;

  • Facility maintenance budgets by year over the past 5 years; and

  • The uniqueness and importance of specific facilities to the NASA scientific and technology missions.

Also, before that first meeting, the committee decided that the expertise of its members matched the two main disciplines into which fundamental science and engineering research at NASA—namely, aeronautics research and space/Earth science research—were classified. Accordingly, two subcommittees were formed; their members are listed in Appendix C.

At the first meeting on September 8, 2009, the committee received presentations from NASA Headquarters representatives of the Aeronautics Research Mission Directorate (ARMD), the Science Mission Directorate (SMD), and the Exploration Systems Mission Directorate (ESMD) an overview of how NASA assesses facility conditions and details of the NASA budgets over the past 5 years. Presentations from the Ames Research Center (ARC), Jet Propulsion Laboratory (JPL), Marshall Space Flight Center (MSFC), Glenn Research Center (GRC), and Johnson Space Center (JSC) were made on the second day of the meeting.

It is important to mention the limitations on this study that resulted from the narrowly focused statement of task (SOT), the constrained study and travel budgets, the limited time available to committee members, and the inability to command as much time as they would like from NASA center personnel. Another limitation was the reliance on NASA to let the committee know which NASA facilities are or were engaged in TRL 1-3 work. Not only was the committee unable to verify independently that the list of facilities presented by NASA was comprehensive, but it also became apparent to the committee during site visits that each center had defined the committee’s requests somewhat differently—that is, some centers only presented facilities that are currently engaged in low-TRL work, while others included facilities that have been or could have been so engaged.

The focus on fundamental science and engineering research eliminated several NASA centers that do not conduct a significant amount of, or any, TRL 1-3 research, such as JSC, the Stennis Space Center (SSC), and the Kennedy Space Center (KSC). JSC does conduct a small amount of TRL 1-3 research, but the committee decided, based on the material presented at the first meeting, that it did not warrant a visit.

Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×

Although the NASA Authorization Act of 2005, Section 507, designated the International Space Station (ISS) as a national laboratory, the committee chose not to include that site in this study because of its difficulty of access, its uniqueness, and the potential decommissioning in 2015. The committee also chose not to include fundamental medical research conducted at NASA in support of the manned flight program because several reports had been issued on human space research, some by the NRC,3 over the past few years. Following a query to NASA Headquarters, it was determined that supercomputers used for the study of computational fluid dynamics, among other areas of application, would not be examined by this committee.

The SOT (Appendix A) asked for an appraisal of equipment, facilities, and support services used for fundamental science and engineering research, as well as on their adequacy for supporting NASA goals. The SOT did not call for a detailed study of the individual programs that constitute the NASA fundamental research enterprise or an assessment of the efficacy and quality of the research performed, or the adequacy of those programs to support the NASA goals. As required by the SOT, the committee focused on assessing the state of laboratory capabilities and on whether they are equipped and maintained to support the NASA research activities. Further, the SOT required the committee to rely on the experience and expertise of its members to make comparisons of NASA’s laboratory capabilities with other entities.

In 2004, a task force of the NASA Advisory Council, the NASA Federal Laboratory Review (NFLR) Task Force, was formed to provide an independent evaluation of NASA’s R&D. The NFLR defined “laboratory” as all the activities and facilities at a center and subordinate organizational units that perform or support the performance of R&D. JPL, although a federally funded research and development center operated by the California Institute of Technology, was included as a NASA center since NASA is the sponsor and owns the property and equipment. The committee adopted that definition of a laboratory for this study.

PREPARATIONS FOR CENTER VISITS

Before visiting a NASA center, the committee presented the center management with a list of questions. The questions ranged from specifics on the types of research funding used by the center to how their acquisition, maintenance, and upgrading of equipment and support services were managed. The list of questions sent to the NASA centers appears in Appendix G. The committee requested that a significant portion of the time for a visit be devoted to touring research laboratories and interacting with research scientists and engineers. Center management responded with proposed agendas that aimed to satisfy committee requirements. The agendas were then mutually adjusted to optimize committee goals and needs during the visit.

For those centers that focus on aeronautics or space and Earth science research, the committee members who visited were experts in that particular discipline. For example, the aeronautics group identified earlier and by name in Appendix C, visited GRC, Langley Research Center (LaRC), and ARC, while the space and Earth science group visited GSFC and JPL. Both groups visited GSFC in connection with the first committee meeting in nearby Washington, D.C. A subgroup of the space and Earth science group joined with the aeronautics group to visit GRC to assess some of the space science activities at that center. Likewise, a subgroup of the space and Earth science group visited ARC but at a different time from the aeronautics group because of schedule conflicts, and a separate subgroup visited MSFC. At each visit, subgroups were assigned to collect the committee inputs and prepare a draft report on the visit.

3

The following National Research Council reports—A Constrained Space Exploration Technology Program: A Review of NASA’s Exploration Technology Development Program (2008); Managing Space Radiation Risk in the New Era of Space Exploration (2008); and A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA’s Bioastronautics Roadmap (2006)—were published by The National Academies Press, Washington, D.C.

Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×

A typical center visit consisted of initial presentations by center senior management to acquaint the committee with the operations of the center and to address earlier questions and information requirements. Walking tours of R&D facilities and laboratories followed. During these tours committee members had adequate time to interact with the research scientists and engineers and to ask questions. After the tours the committee members had the opportunity to meet and interact with working research scientists and engineers. Typically, around 15 people attended these hour-long sessions, and the committee developed a keen sense of the issues facing the research scientists. Before departure the committee members met to record their initial findings.

At the second committee meeting, in Irvine, California, on November 11 and 12, 2009, the committee was briefed on the fundamental research, equipment, and facilities at Dryden Flight Research Center (DFRC). This briefing and the resulting discussions form the basis of the assessment of DFRC in Chapter 4.

Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×
Page 6
Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×
Page 7
Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×
Page 8
Suggested Citation:"2 Assessment Methodology." National Research Council. 2010. Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research. Washington, DC: The National Academies Press. doi: 10.17226/12903.
×
Page 9
Next: 3 Support for Fundamental Research at NASA »
Capabilities for the Future: An Assessment of NASA Laboratories for Basic Research Get This Book
×
Buy Paperback | $41.00 Buy Ebook | $32.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Over the past 5 years or more, there has been a steady and significant decrease in NASA's laboratory capabilities, including equipment, maintenance, and facility upgrades. This adversely affects the support of NASA's scientists, who rely on these capabilities, as well as NASA's ability to make the basic scientific and technical contributions that others depend on for programs of national importance. The fundamental research community at NASA has been severely impacted by the budget reductions that are responsible for this decrease in laboratory capabilities, and as a result NASA's ability to support even NASA's future goals is in serious jeopardy.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!