Interim Assessment of Research and Data Analysis in NASA's Office of Space Science: Letter Report (2000)

Advisers to the Nation on Science, Engineering, and Medicine

National Academy of Sciences

National Academy of Engineering

Institute of Medicine

National Research Council

September 22, 2000

Dear Dr. Weiler:

As you requested in your letter of June 16, 2000 ( Appendix A ), the Space Studies Board (the Board; Appendix B ) has conducted a brief review of actions taken by the Office ofSpace Science (OSS) that are relevant to recommendations in the Board's 1998 report Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis. ¹ The statement of task for this review is provided in Appendix C .

The Board conducted this assessment on an ambitious schedule in accordancewith your request for feedback by September 2000. The Board was providedwith relatively little written documentation of NASA's plans forimproving the OSS R&DA program.

The review was based, in part, on inputs received from relevant standingcommittees of the Board—the Committee on Solar and Space Physics,the Committee on Planetary and Lunar Exploration, and the Committeeon Astronomy and Astrophysics. A major source of information forthe review was a pair of short papers provided to the Board on July25, 2000, by Dr. Guenter Riegler, director of the OSS Research ProgramManagement Division ( Appendix D and Appendix E ). Dr. Riegler then briefed the Board's executive committee and standing committee chairs at a meetingon August 16 at the National Academies' study center in Woods Hole,Massachusetts. At that meeting, members of the Board reviewed anddiscussed the information from NASA and

2101 Constitution Avenue, NW, Washington, DC 20418 Telephone (202) 334 3477 Fax (202) 334 3701 ssb@nas.edu

Office Address: Milton Harris Building, Room 584, 2001 Wisconsin Avenue, NW, Washington,DC 20007

the Board's discipline committees' responses and assembled this consensus assessment. ² The Board concluded that the proposals that Dr. Riegler describedfor responding to the 1998 report are appropriate; however, a finalassessment awaits action guided by a concrete implementation plan.

The 1998 Space Studies Board report analyzed the roles and contributionsof R&DA grants in the research programs of NASA's three science offices, and it presented a set of strategicand programmatic recommendations to enhance the R&DA programs. TheBoard reaffirms the conclusions of the 1998 report: research anddata analysis activities are critical elements of a viable spacescience program. ³ The Board is aware of a number of actions within OSS that are underway or planned that will strengthen the R&DA programs and that willbe entirely consistent with the recommendations of the 1998 report.For example, Dr. Riegler described plans to reallocate current budgetsand to seek funds for new projects that will provide selected increasesin data analysis funding at an overall rate of 8% per year. He alsoreported on the OSS intent to provide explicitly for data analysisfunding in all new projects when they are initially proposed. Further,Dr. Riegler described a regular process of “senior reviews” of the research grantsprogram that would complement the senior reviews of operating spacecraftmission programs and provide a mechanism to accomplish a number ofactions recommended by the Board in the 1998 report.

While the Board supports the steps noted above, there are still twoconcerns to be addressed. First, many of the OSS responses to the1998 report's recommendations are planned rather than ongoing activities,and so any assessment of their effectiveness must await their implementation.Second, there are areas where the plans appear to be incomplete orwhere the attention being given may be inadequate. In the remainderof this report, the Board provides additional comments on those areasby addressing each of the six major recommendations in the 1998 reportin order.

The first recommendation of the 1998 report addressed a number ofaspects of managing R&DA programs strategically. To be able to doso requires, of course, a strategic plan for the program as a wholeand an approach that integrates attention to R&DA into that plan.In its May 2000 review of the OSS draft 2000 strategic plan, theBoard indicated that while many aspects of the draft were solidlygrounded, the document still lacked several important aspects ofa strategic plan, as follows:

Although the draft document is called “The Space Science Enterprise Strategic Plan,” it lacks, in fact, some key characteristics of a strategic plan.For example, the document does not explicitly discuss how choiceswere or are made in setting priorities, and it does not identifypriorities for missions or other program elements that are presentedin the plan. . . . ⁴

Regarding the integration of R&DA into that strategic plan, the Board's May 2000 report said:

The OSS draft plan should reflect a clearer sense of the prioritiesfor R&DA, the linkages between R&DA and other parts of the OSS program, and the overall importanceof R&DA in the space science enterprise. Finally, also needed isa more explicit discussion of the OSS strategy for achieving balancebetween flight mission development, supporting ground and suborbitalresearch, theory and modeling, and data analysis. . . . ⁵

The Board is aware of OSS's plans to institute a new senior review process for evaluating theresearch grants program ( Appendix D ), probably on a triennial basis, to complement the senior reviewsfor operating satellites. Together these two reviews will go a longway toward responding to regular evaluations of balance as recommendedin the 1998 report. What is apparently missing, however, is a processto integrate these decisions and to look across the whole programstrategically. This integrating function is particularly importantfor handling cases in which senior reviews of operating missionsand of the grants program might arrive at different conclusions.The NASA Space Science Advisory Committee may be a possible venuefor integrating the senior reviews and evaluating balance acrossOSS.

The second recommendation addressed the need to examine strategicallythe requirements, priorities, and health of research infrastructuresat universities and NASA field centers. This issue was also addressedin the Board's review of the OSS draft strategic plan:

The OSS draft document says little about what responsibility OSSassumes for universities. It notes the intention to “maintain essentialtechnical capabilities at the NASA centers,” and although it recognizesthe role of scientists at universities in research and planning,and in developing the next generation of space research professionals,it is silent about intentions of OSS to maintain essential capabilitiesat universities. . . . Furthermore, a long-standing question withinNASA has concerned the extent to which universities should be consideredto be vendors, sources of members of the technical workforce, integralpartners, or some mix of those roles. The OSS plan could be strengthenedby more clearly recognizing that the universities are elements ofthe fabric of space science and that their capabilities also needto be nurtured. ⁶

Dr. Riegler called the Board's attention to plans within the executive branch to strengthen government-universitypartnerships, based on the “Principles of the Federal Partnershipwith Universities in Research” laid out in the National Science andTechnology Council's report Renewing the Federal Government-University Research Partnership forthe 21^st Century. ⁷ He cited several proposed NASA initiatives to increase universityinvolvement in developing space hardware and infrastructure. Theseplans, if implemented, will enhance the research infrastructure insome areas. However, based on the information provided by OSS, theBoard concluded that a more systematic assessment of research infrastructurealong the lines recommended in the 1998 report is still needed.

The third recommendation focused on the need to assess the distributionof grant sizes in each of NASA's science program areas. NASA presenteddata regarding grant sizes in different areas of the OSS researchprogram as well as a description of the logic and history of thedifferences in sizes among those research areas. However, there doesnot appear to have been any systematic assessment across the program.In addition, the Board recognizes that a response to Recommendation6 of the 1998 report is required in order to conduct such an assessment.Finally, the planned senior review of the research grants programdescribed by NASA could be an appropriate vehicle for carrying outthis systematic review.

The fourth recommendation emphasized the value in preserving a mixof university and non-university participation in technology, instrument,and facility development. OSS did not provide the Board with anyinformation indicating that OSS has conducted or plans to conducta systematic evaluation of the mix of university principal investigatorawards and non-university funding for technology, instrument, andfacility development. The Board notes that in assessing the mix ofinstitutions involved in technology development, NASA should alsopromote university-industry-field center partnerships.

The fifth recommendation addressed the use of training grants asa way to ensure breadth in graduate education. NASA indicated anintent to increase the number of (or introduce a new element into)training grants in the university program; however, no actions hadbeen undertaken at the time of this review. The Board is interestedin seeing an implementation plan for this initiative.

The sixth recommendation addressed the need to establish a uniformprocedure for collecting data on R&DA funding and funding trendsfor use as a management tool. This issue was also raised in the Board's reports on technology development in OSS ⁸ and in the report Federal Funding of Astronomical Research. ⁹NASA presented plans for acquiring the types of data recommendedin the 1998 report, and the Board views this plan as a positive response.These plans would involve using a single contractor to administerthe proposal review process as a means for collecting the data. Ifappropriate data are collected (e.g., on trends with respect to discipline,class of activity, and type of performing institution), they willprovide a useful management tool for assessing the balance amongelements and participants in the R&DA program. However, these dataon R&DA funding will be incomplete until NASA implements full-costaccounting at the NASA field centers. ¹⁰ In addition, these data will be required before OSS can respond appropriatelyto Recommendation 3 of the 1998 report.

The Board believes that OSS's proposals for responding to the recommendations of the 1998 reportare moving in the right direction. It cannot, however, be confidentthat these recommendations will be met until an explicit implementationplan is available. The Board is prepared to assist OSS in any wayit can.