National Academies Press: OpenBook

Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report (2000)

Chapter: Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs

« Previous: Appendix C: Statement of Task
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

APPENDIX D

NASA SHORT PAPER: OFFICE OF SPACE SCIENCE'S WRITTEN RESPONSE TO THE 1998 SPACE STUDIES BOARD REPORT SUPPORTINGRESEARCH AND DATA ANALYSIS IN NASA'S SCIENCE PROGRAMS

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

From: Executive Summary of “Supporting Research and Data Analysis in NASA's Science Programs”, prepared by the Space Studies Board, 1998

The Recommendations are shown in 12-pt BOLD, like this.

NASA/OSS Response is showm indented, like this.

FINDINGS AND RECOMMENDATIONS

Principles for Strategic Planning

Finding: The task group finds that R&DA is not always thoroughly and explicitly integrated into the NASA enterprise strategic plans and that not all decisions aboutthe direction of R&DA are made with a view toward achieving the goals of the strategies. The taskgroup examined the trend and balance of R&DA budgets and found alarming results (Chapter 4, Sections 5.1 and5.3); it questions whether these results are what NASA intends.

Recommendation 1: The task group recommends that each science programoffice at NASA do the following:

1a) Regularly evaluate the impact of R&DA on progress toward the goals of the strategic plans.

1b) Link NASA research announcements (NRAs) to addressing key scientificquestions that can be related to the goals of these strategic plans.

For the “R” portion of R&DA: The Office of Space Science (OSS) has used strategic goals and objectives in making manyresearch program decisions. For example, the need for biomarkersfor Astrobiology research determined the selection of two of themembers of NASA's Astrobiology Institute. OSS has also initiateda process of triannual evaluations of the Research (SR&T) programsin relation to the OSS Strategic Plan. This process will includeindependent outside review of the SR&T programs, assessing the performanceof various SR&T programs towards the goals of the OSS strategic plan,beginning in mid-2001. Furthermore, beginning with the late CY2000SR&T NASA Research Announcements (NRAs), all future NRAs will containexplicit prioritization criteria with respect to the goals and objectivesin the OSS Strategic Plan.

When OSS shapes new space science initiatives to address strategicscience questions, the resulting requests include not only missiondevelopment, but also supporting research (“R”) and mission data analysis(“DA”) programs. An example is the “Living With a Star” initiative, which explicitly includes development, research, anddata analysis components.

For “DA”, data analysis for operating missions: For major missions in theirprime phase, the OSS Strategic Planning process provides the balanceand relevance test with respect to the OSS strategic goals and objectives. In Astrophysics and Space Physics,OSS already has an established process for regular assessment ofDA programs in relation to the goals and priorities of the OSS StrategicPlan. A similar review process of planetary missions began in July2000; again, relevance to the OSS Strategic Plan is the highest-prioritycriterion.

1c) Regularly evaluate the balance between the funding allocationsfor flight programs and the R&DA required to support those programs (e.g., assess whether the currentprogram can support R&DA for the International Space Station).

First of all, we assume that the “International Space Station” comment does not apply to OSS.

This recommendation calls for balance evaluation among three—traditionally separate—elements of the OSS program: development (“DEV”) of flight programs, “R” and “DA”.

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

We have routinely assessed the balance between DEV and DA for individualmissions as well as for mission “lines” like the Discovery and Explorerprograms. Where appropriate, funds are moved from DEV to DA, e.g.to extend worthy operating missions. At other times it is more appropriateto move DA funds to DEV. We are continuing to make such assessmentsand actions accordingly. In some cases transitions between DEV andDA are made within the same program, sometimes across different programs.

1d) Regularly evaluate the balance among various subelements of theR&DA program (e.g.,theoretical investigations; new instrument development; exploratoryor supporting ground-based and suborbital research; interpretation of data from individual ormultiple space missions; management of data; support of U.S. investigators who participatein international missions; and education, outreach, and public information).

This balance assessment has already begun within several of the 9SR&T science cluster areas, and has already been practiced for anumber of years in all DA programs.

1e) Use broadly based, independent scientific peer review panelsto define suitable metrics and review the agency's internal evaluations of balance. (Ref: National Research Council(NRC), Space Studies Board, “On NASA Field Center Science and Scientists,” letter to NASA Chief Scientist France Cordova, March 29, 1995; NRC, Space Studies Board and theCommittee on Space Biology and Medicine, “On Peer Review in NASA Life Sciences Programs,” letter to Dr. Joan Vernikos, director of NASA's Life Sciences Division, July 26, 1995; NRC, Space Studies Board,“On the Establishment of Science Institutes,” letter to NASA Chief Scientist France Cordova, August 11, 1995.)

It is not clear whether the emphasis of this recommendation refersto OSS or not. OSS has, for many years, used the peer review processfor close to 100% of R&DA award selection. Internal and externalscience efforts are reviewed side-by-side, with the same evaluationcriteria.

We would welcome the Space Studies Board's definition of suitable metrics for the peer review process, andany recommendations for improvements in our peer review process.

OSS currently has almost 20 IPAs, Resident Management Associates,Presidential Management Interns, and detailees in OSS. They bringrecent, specialized expertise in science and technology to the civilservant staff at OSS. Furthermore, as directed by the NASA Administrator,OSS will expand the use of IPAs and other “rotators” in order to bring additionalhighly specialized expertise to OSS, e.g. for Astrobiology, interferometry,and helioseismology. At least five additional IPA positions havealready approved, and the first of these “additional” IPAs will join OSS inmid-August 2000.

Innovation and Infrastructure

Finding: Although there are sporadic funding opportunities for researchinfrastructure, there is no systematic assessment of the state of the research infrastructure,nor are there coherent programs to address weaknesses in the infrastructure base (Section 5.2).

Recommendation 2: The task group recommends that NASA take the followingactions on research infrastructure:

General Comments to Recommendation 2:

On one hand, NASA is not chartered to assure the general health ofthe university community (both personnel and infrastructure). Onthe other hand, it can be argued that for certain space science disciplineswhich require space-based observations, NASA is the only part ofthe U.S.

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

government which might provide support, and therefore may have someobligation to provide infrastructure support.

NASA does, of course, provide infrastructure support for researchwhich is required for its missions, and which has been selected viaopen, competitive review.

Among Space Science disciplines, long-term (not sporadic) infrastructuresupport for R&DA is provided for the planetary community via thePlanetary Instrument Definition and Design Program (PIDDP) and thePlanetary Instrument Upgrade Program (PIUP). For the Astrophysicscommunity, support is provided via the Laboratory Astrophysics Program.All of these programs use fully open, peer-review-based selectionprocesses, regardless of whether applicants are based at NASA centers,at universities, in industry, or elsewhere in the U.S.

Detailed Recommendations and Response:

2a) Conduct an initial assessment of the need and potential for acquiringand sustaining infrastructure in universities and field centers.

At this time, OSS does not see a general need for additional infrastructure across all areas of space science SR&T. However, OSS has identified specific needs for additional infrastructure in the solar system sample return area,and is seeking advice on whether, or not, there is a need for laboratoryresearch infrastructure in Astrobiology. For both of these specific-needcases, OSS intends to select recipients via fully open, peer-reviewedselection methods. It should be pointed out, however, that thereare advocates for assigning these two infrastructures to specificlocations without any open competition.

2b) Determine options for minimizing duplication of expensive researchfacilities.

It is difficult to imagine having enough funds, in space science,to devote to “expensive research facilities” that duplication would be a concern. Might this commentapply to other Enterprises?

2c) Evaluate the level of support for infrastructure in the contextof the overall direction and plans for R&DA activities.

The philosophy expressed in response to item 2a) is that OSS willsupport infrastructure only where a specific need exists. “SpecificNeeds” could include major facilities like the National ScientificBalloon Facility, Data Systems, Sample Curation and Analysis; inthis case it is our intent to select new or continued major facilitiesvia truly open, competitive selection.

The writers of this report may also have intended to include corecapability for instrument development in this “infrastructure” recommendation. Here,again, OSS will provide support for specific areas of emphasis; therequest for a budget augmentation for “Astrobiology Science and InstrumentTechnology”, currently under review by Congress, is an excellentexample.

If there is a general “level of support for infrastructure”, as opposed to specific areas of need, then we would like to hearfrom the Space Studies Board on specifics and rationale.

2d) Maximize the use of infrastructure by supporting partnering betweenuniversities and field centers.

Several universities use NASA's supercomputing infrastructure; OSS will aim to do the same in reversefor university-centered infrastructure.

2e) Explore approaches for providing peer review and oversight ofinfrastructure investments, which should include regular evaluation of a facility's role and contribution as a national academic

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

resource, its degree of scientific and technical excellence, andits contribution to NASA's long-term technology planning and development.

Every one of the OSS research infrastructure investments has users' committees in place which meet the intent of this recommendation.Furthermore, we intend to tie periodic evaluations more closely tothe OSS strategic goals and objectives.

2f) Institute periodic assessment of the research infrastructurein university and NASA field centers to ensure that the infrastructure is appropriate for current programs.

We intend to obtain these assessments through the triennial SR&T Senior Review process. If that should be inadequate from the pointof view of assessment detail, or too infrequent, then we will createa separate avenue for this assessment.

Management of the Research and Data Analysis Programs

Finding: The median of NASA research grants to universities decreasedin constant FY 1995 dollars from $64,000 per year in FY 1986 to $59,000 in FY 1995 for the Officeof Space Science disciplines, remained relatively flat at $79,000 for Earth science disciplines,and grew from $69,000 to $100,000 for life and microgravity science disciplines during the period from1986 to 1995 (Section 4.4, Figure 4.3). (These award sizes compare to a median of $85,000 at the NationalAgency.) It is well known that a single researcher cannot support a salary and a graduate studentat grant levels of $50,000 and that such researchers must seek additional grants to maintain a viable

Recommendation 3: NASA should routinely examine the size and numberof grants awarded to individual investigators to ensure that grant sizes are adequateto achieve the proposed research and that their number is consistent with the time commitments of eachinvestigator. The differences in award sizes for the Offices of Space Science, Earth Science, andLife and Microgravity Science and Applications should be reconciled with program objectives, especiallythose for space sciences, which often are funded at levels of less than $50,000 to $60,000. Wherewarranted, actions should be taken to address the deficiencies.

Response:

We remind the reader the “R&DA” combines the research awards of the SR&T programs and the data analysis awards from mission observing programs.The former are overwhelmingly 3-year awards, and have a stable fundinglevel (on average near $60-70k/year). The latter are mostly 1-yearawards, for the analysis of single or small numbers of observations,and have lower average grant sizes; the exception for DA are theHubble Space Telescope awards which are adequately funded, rise withinflation, and have durations from 1 to 3 years.

For “R” awards, it is the intention of OSS to stabilize grant sizes andto establish – except in rare cases, when appropriate – minimum award sizes in the range of $60-70k per year.We intend to address these policies in the context of the first “Senior Review” of the SR&T program in mid-2001.

For “DA” awards, the flat or decreasing award sizes are driven by the factthat missions beyond their nominal prime phases must continue toshow high science productivity while reducing both operations anddata analysis costs. As a result, some missions which have been extendedwell beyond their nominal lifetime issue DA awards for most GuestInvestigations; they cover primarily travel and publication costs,and perhaps part of a student's salary. This pattern was chosen deliberatelyover the alternative of providing larger grants to a small numberof DA investigations, and none at all to most others. We would welcomefeedback on this issue from the SSB.

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

Participation in the Research and Data Analysis Programs

Finding: The task group recognizes that university-based instrumentdevelopment projects led by principal investigators (PIs) can provide important training andversatility for graduate students in NASA-funded sciences. Often, innovative instrument prototypes canbe developed at a fraction of the cost of facility instruments, and the analysis of instrument dataand preparation of high-quality scientific results are closely coupled with Understanding of andexperience in the design of scientific instrumentation. However, although the university arena frequentlytraining advantages, that the economies of scale for some facility development projects are high,

Recommendation 4: NASA should preserve a mix of PI-university awardsand nonuniversity funding for the development of technologies, instruments, and facilities.NASA should make these decisions within the agency's overall plan for R&DA activities (Recommendation 1), with sensitivity to the advantages of the academic environment but guided by peer reviewof scientific and technical merit.

Response:

OSS has a strong tradition of basing SR&T selections on open, side-by-side peer review, regardless of whetherthe research is conducted at universities, NASA centers, industry,or other locations. As stated in the response to Recommendation 1d),nearly 100% of the SR&T program, and 100% of DA programs are peerreviewed for scientific and technical merit.

Unless this recommendation calls for set-asides for universities,it is not clear what this recommendation calls for that is not alreadyroutine practice in OSS.

Creation of Intellectual Capital

Finding: NASA's principal graduate student fellowship programs are all tied tostudent research interests or concentrations.

Recommendation 5: NASA should explore using training grants likethose of the National Institutes of Health or the National Science Foundation for first-year graduatestudents as a possible alternative to supporting these students as research assistants orNASA fellows. These training grants should be designed to ensure breadth in graduate education

Response:

NASA is a mission-oriented agency; therefore, all NASA-provided supportmust be directed towards research in support of NASA's missions.Consequently, the graduate fellowship program is deliberately focussedtowards supporting excellence in the research areas relevant to NASA's strategic and near-term goals.

On the other hand, NASA is committed to supporting education efforts,and to assuring that adequate intellectual capital exists to serveNASA's needs in the future. Furthermore, NASA's Office of Human Resources and Education has been using traininggrants for several years, and has begun to introduce this mechanismto other offices, including the Office of Space Science. OSS intendsto pursue this recommendation for the “research” portion of R&DA programs.

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×

Accounting as a Management Tool in Research and Data Analysis Programs

Finding: NASA does not use the extended records of its budgets andexpenditures as management tools to monitor the health of its R&A and DA programs. Moreover, the fragmented budget structure for R&DA makes it difficult for the scientific community to understandthe content of the program and for NASA to explain the content to federal budget decisionmakers.

Recommendation 6: NASA's science offices should establish a uniform procedure for tracking budgets and expenditures by the class of activities and the typesof organizations (including intramural and extramural laboratories, industry, and nonprofit entities)that are actually performing the work. These data should be gathered and reported annuallyand used to inform regular evaluations of R&DA activities (Recommendations 1 and 2). One approach would be to itemize the following elements in the budget: theoretical investigations;new instrument development; exploratory or supporting ground-based and suborbital research; interpretationof data from space missions; management of data; support of U.S. investigators who participatein international missions; and education, outreach, and public information. In addition,these data should be made publicly available and reported annually to the Office of Managementand Budget and to Congress.

Response

This recommendation concerns management practices which NASA Headquartersvalued highly and had routinely employed when we had contractors;they traditionally compiled information and provided analysis alongthe lines recommended above. During the past 5 years, when most contractorshave been eliminated, and staff at NASA Headquarters has been trimmedby roughly 50%, we could not track such information. However, therecently signed contract with a single Headquarters-wide contractorfor peer reviews should be able to provide the necessary supportto create this useful management tool.

Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 15
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 16
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 17
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 18
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 19
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 20
Suggested Citation:"Appendix D: NASA Short Paper: Office of Space Science's Written Response to the 1998 Space Studies Board Report Supporting Research and Data Analysis in NASA's Science Programs." National Research Council. 2000. Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/9980.
×
Page 21
Next: Appendix E: NASA Short Paper: “Reviewing NASA's Space Science Supporting Research and Technology Programs,” NASA, Office of Space Science »
Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!