6
Findings And Recommendations

6.1 PRINCIPLES FOR STRATEGIC PLANNING

Having explored the role of R&DA in NASA's strategic planning in Section 3.2, the task group identifies five principal elements that figure prominently in the strategic planning process: (1) a foundation built on basic scientific goals, objectives, and key questions; (2) clear linkage between the science questions, supporting research and technology, advanced technology development, flight missions, research and data analysis, and the strategic plans; (3) use of the peer review process to determine the merit of the scientific goals, objectives, and key questions; (4) use of independent advisory bodies to regularly review the progress and future directions of the strategic plans; and (5) room for flexibility, innovation, and evolution within the strategic plans.

Some of NASA's science offices, for example the Office of Space Science, have been thorough in their involvement of the scientific community in the strategic planning process and in their use of independent peer review to evaluate strategic plans and their underlying science questions.1 The extent to which NASA has devoted explicit attention to strategic and operational linkages between basic scientific goals, R&DA activities, and space flight missions is much less clear.

Finding: The task group finds that R&DA is not always thoroughly and explicitly integrated into the NASA enterprise strategic plans and that not all decisions about the direction of R&DA are made with a view toward achieving the goals of the strategies. The task group examined the trend and balance of R&DA budgets and found alarming results (Chapter 4, Sections 4.1 and 4.3); it questions whether these results are what NASA intends.

1  

 For example, the NASA Space Science Advisory Committee held a workshop in May 1997 in Breckenridge, Colorado, to integrate science and technology "roadmaps" into a strategic plan for the Office of Space Science; see also letter sent by Space Studies Board Chair Claude Canizares to NASA Associate Administrator for Space Science Wesley Huntress on NASA's Office of Space Science draft strategic plan, August 27, 1997.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 63
Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis 6 Findings And Recommendations 6.1 PRINCIPLES FOR STRATEGIC PLANNING Having explored the role of R&DA in NASA's strategic planning in Section 3.2, the task group identifies five principal elements that figure prominently in the strategic planning process: (1) a foundation built on basic scientific goals, objectives, and key questions; (2) clear linkage between the science questions, supporting research and technology, advanced technology development, flight missions, research and data analysis, and the strategic plans; (3) use of the peer review process to determine the merit of the scientific goals, objectives, and key questions; (4) use of independent advisory bodies to regularly review the progress and future directions of the strategic plans; and (5) room for flexibility, innovation, and evolution within the strategic plans. Some of NASA's science offices, for example the Office of Space Science, have been thorough in their involvement of the scientific community in the strategic planning process and in their use of independent peer review to evaluate strategic plans and their underlying science questions.1 The extent to which NASA has devoted explicit attention to strategic and operational linkages between basic scientific goals, R&DA activities, and space flight missions is much less clear. Finding: The task group finds that R&DA is not always thoroughly and explicitly integrated into the NASA enterprise strategic plans and that not all decisions about the direction of R&DA are made with a view toward achieving the goals of the strategies. The task group examined the trend and balance of R&DA budgets and found alarming results (Chapter 4, Sections 4.1 and 4.3); it questions whether these results are what NASA intends. 1    For example, the NASA Space Science Advisory Committee held a workshop in May 1997 in Breckenridge, Colorado, to integrate science and technology "roadmaps" into a strategic plan for the Office of Space Science; see also letter sent by Space Studies Board Chair Claude Canizares to NASA Associate Administrator for Space Science Wesley Huntress on NASA's Office of Space Science draft strategic plan, August 27, 1997.

OCR for page 63
Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis Recommendation 1: The task group recommends that each science program office at NASA do the following: Regularly evaluate the impact of R&DA on progress toward the goals of the strategic plans. Link NASA research announcements (NRAs) to addressing key scientific questions that can be related to the goals of these strategic plans. Regularly evaluate the balance between the funding allocations for flight programs and the R&DA required to support those programs (e.g., assess whether the current program can support R&DA for the International Space Station). Regularly evaluate the balance among various subelements of the R&DA program (e.g., theoretical investigations; new instrument development; exploratory or supporting ground-based and suborbital research; interpretation of data from individual or multiple space missions; management of data; support of U.S. investigators who participate in international missions; and education, outreach, and public information). Use broadly based, independent scientific peer review panels to define suitable metrics and review the agency's internal evaluations of balance.2 Examine ways to maximize familiarity with contemporary advances and directions in science and technology in the process of managing R&DA, for example, via the appropriate use of rotators.3 6.2 INNOVATION AND INFRASTRUCTURE Innovations often require state-of-the-art facilities. The task group found evidence of few mechanisms to provide this essential research infrastructure. Section 3.4 notes that the difficulties experienced by universities, in particular, in acquiring and maintaining infrastructure are exacerbated in the smaller, faster, cheaper program environment where certified facilities and state-of-the-art laboratories might remain the expected norm, but there is neither the time nor the money to develop them that there was in the era of large, longer-duration flight projects. Finding: Although There Are Sporadic Funding Opportunities For Research Infrastructure, There Is No Systematic Assessment Of The State Of The Research Infrastructure, Nor Are There Coherent Programs To Address Weaknesses In The Infrastructure Base (Section 5.2). Recommendation 2: The task group recommends that NASA take the following actions on research infrastructure: Conduct an initial assessment of the need and potential for acquiring and sustaining infrastructure in universities and field centers. 2    National Research Council, Space Studies Board, "On NASA Field Center Science and Scientists," letter to NASA Chief Scientist France Cordova, March 29, 1995; National Research Council, Space Studies Board and the Committee on Space Biology and Medicine, "On Peer Review in NASA Life Sciences Programs," letter to Dr. Joan Vernikos, director of NASA's Life Sciences Division, July 26, 1995; National Research Council, Space Studies Board, "On the Establishment of Science Institutes," letter to NASA Chief Scientist France Cordova, August 11, 1995. 3    Federal agencies have used rotators—scientists from outside the federal government—for 1 to 2 years to participate in management of research programs. NASA has used interagency personnel appointments—visiting scientists administered by the Universities Space Research Association and JPL—as rotators to circulate new ideas and new individuals, on temporary appointments, into the agency system.

OCR for page 63
Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis Determine options for minimizing duplication of expensive research facilities. Evaluate the level of support for infrastructure in the context of the overall direction and plans for R&DA activities. Maximize the use of infrastructure by supporting partnering between universities and field centers. Explore approaches for providing peer review and oversight of infrastructure investments, which should include regular evaluation of a facilities role and contribution as a national academic resource, its degree of scientific and technical excellence, and its contribution to NASA's long-term technology planning and development. Institute periodic assessment of the research infrastructure in university and NASA field centers to ensure that the infrastructure is appropriate for current programs. 6.3 MANAGEMENT OF THE RESEARCH AND DATA ANALYSIS PROGRAMS University investigators report increasing competition for fewer grants, a lower success rate for each competition, lower dollar awards for successful grants, and hence the need to secure a larger number of grants to maintain a viable research program. The effect has been to increase the number of proposals that each investigator prepares and the number of grants that each investigator manages. Inefficiencies associated with competing for, evaluating, and administering a larger number of smaller grants appear not only as increases in the time devoted to writing and reviewing proposals and reports but also as an unproductive fragmentation of the efforts of investigators. Most scientists view this fragmentation as a churning that may produce grants and papers but that compromises the focus essential for discovery or innovation. The current system encourages investigators to assume more work than they can reasonably complete well. Because of the relatively low success rate of proposals, the need for several concurrent grants to maintain a viable research program, and the dire consequences for academic scientists of finding themselves without funds to support their students, most investigators propose more work than they could possibly undertake if all proposals were successful. Many scientists conservatively underestimate their eventual success, with the result that their graduate students' research may be inadequately supervised or similar work may be performed under more than one grant. Although neither outcome is unethical, neither is likely to produce seminal discoveries. Finding: The median size of NASA research grants to universities decreased in constant FY 1995 dollars from $64,000 per year in FY 1986 to $59,000 in FY 1995 for the Office of Space Science disciplines, remained relatively flat at $79,000 for Earth science disciplines, and grew from $69,000 to $100,000 for life and microgravity science disciplines during the period from 1986 to 1995 (Section 4.4, Figure 4.3). (These award sizes compare to a median of $85,000 at the National Science Foundation and a mean of between $110,000 and $120,000 at the Environmental Protection Agency.) It is well known that a single researcher cannot support a salary and a graduate student at grant levels of $50,000 and that such researchers must seek additional grants to maintain a viable research program. Recommendation 3: NASA should routinely examine the size and number of grants awarded to individual investigators to ensure that grant sizes are adequate to achieve the proposed research and that their number is consistent with the time commitments of each investigator. The differences in award sizes for the Offices of Space Science, Earth Science, and Life and Microgravity Science and Applications should be reconciled with program objectives, especially those for space sciences, which often are

OCR for page 63
Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis funded at levels of less than $50,000 to $60,000. Where warranted, actions should be taken to address the deficiencies. 6.4 PARTICIPATION IN THE RESEARCH AND DATA ANALYSIS PROGRAMS The competition between academic research and research at NASA field centers is both intense and mutually supportive. In defining their appropriate roles, the task group fully endorses the following excerpt from the report of the Space Studies Board's Committee on the Future of Space Science: NASA requires in-house scientists for its space research, exploration, and technology programs. These scientists coordinate science and operations on larger missions, guide development and utilization of unique research facilities, assist outside scientists and technologists to effectively use NASA facilities or flight opportunities, and enable NASA to act as a "smart buyer." The number of in-house scientists should be determined by the extent of these support functions and not by a desire to exploit perceived flight opportunities. Space science leadership and the generation and testing of new ideas should be the domain of the broader scientific community, of which the NASA scientists are only a part. As noted earlier, the committee believes that scientific research should, for the most part, be conducted outside the agency.4 Within the context of R&DA programs, the task group wants to emphasize the value of PI-led instrument development projects within academia. These projects are often the incubators of the next generation of flight projects and play an essential role in preparing the next generation of investigators for the responsibility of leading costly spaceflight projects. The issues of relative funding allocations and responsibilities are addressed in Section 4.2 and in Sections 5.1 and 5.4, respectively. Finding: The task group recognizes that university-based instrument development projects led by principal investigators (PIs) can provide important training and versatility for graduate students in NASA-funded sciences. Often, innovative instrument prototypes can be developed at a fraction of the cost of facility instruments, and the analysis of instrument data and the preparation of high-quality scientific results are closely coupled with understanding of and experience in the design of scientific instrumentation. However, although the university arena frequently offers these opportunities, the task group also recognizes that some research facilities do not offer training advantages, that the economies of scale for some facility development projects are high, and that support of nonuniversity, multiuser facilities is sometimes necessary. Recommendation 4: NASA should preserve a mix of PI-university awards and nonuniversity funding for the development of technologies, instruments, and facilities. NASA should make these decisions within the agency's overall plan for R&DA activities (Recommendation 1), with sensitivity to the advantages of the academic environment but guided by peer review of scientific and technical merit. 4    National Research Council, Space Studies Board, Managing the Space Sciences, National Academy Press, Washington, D.C., 1995, pp. 43-44.

OCR for page 63
Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis 6.5 CREATION OF INTELLECTUAL CAPITAL Given the limited opportunities for growth in the space-related sciences, colleges and universities must develop the next generation of science and engineering leadership without training a large cohort of investigators who expect to find research positions that will not materialize. The many solutions being discussed in the literature range from "birth control" of Ph.D.s to laissez faire (i.e., students carrying total responsibility for assessing their chances of meaningful employment). The task group is most comfortable with the middle position of insisting on a sufficient breadth in graduate education so that students are prepared to follow jobs beyond the narrow disciplines of their thesis research. Supporting first-year graduate students on research assistantships presents faculty advisers and their students with significant incentives to focus narrowly on a research problem from the very beginning of graduate education. The task group thinks that all of NASA's current graduate student support that falls under training grants is structured to encourage a relatively narrow focus on research. An alternative mode, the NIH and NSF training grant, emphasizes the quality and breadth of an academic plan and breadth of laboratory experiences. 5 Finding: NASA's principal graduate student fellowship programs are all tied to student research interests or concentrations. Recommendation 5: NASA should explore using training grants like those of the National Institutes of Health and the National Science Foundation for first-year graduate students as a possible alternative to supporting these students as research assistants or NASA fellows. These training grants should be designed to ensure breadth in graduate education and thereby may expand students' opportunities for employment within or beyond NASA-funded sciences. 6.6 ACCOUNTING AS A MANAGEMENT TOOL IN THE RESEARCH AND DATA ANALYSIS PROGRAMS As noted in Chapter 4, the task group's review of NASA's R&DA programs was challenged by the limited systematic budgetary and expenditure data available about these programs. NASA is taking important first steps in this direction with full-cost accounting at its field centers, but much more than this is needed (see Section 4.5). There is a need for documentation and mappings between old account items and new items when change is necessary, for long-term tracking of classes of program support (e.g., instrument development, infrastructure, data analysis), for long-term tracking of allocations within each class for types of participants (e.g., field centers, universities, industry), and for openly reporting these budget and expenditure indicators each year. 5    The Committee on Science, Engineering, and Public Policy recommended in its report Reshaping the Graduate Education of Scientists and Engineers (National Academy Press, Washington, D.C., 1995): "To produce scientists and engineers who are versatile, graduate programs should provide options that allow students to gain a wider variety of academic and other career skills," and "To foster versatility, government and other agents of financial assistance for graduate students should adjust their support mechanisms to include new 'education/training grants' that resemble the training grants now available in some federal agencies" (pp. 78-79). On undergraduate education, see Shaping the Future: New Expectations for Undergraduate Education in Science, Mathematics, Engineering, and Technology, a report on its Review of Undergraduate Education by the Advisory Committee to the National Science Foundation Directorate for Education and Human Resources, M.D. George, Chair of the Review Subcommittee, NSF 96-139, National Science Foundation, Washington, D.C., May 1996.

OCR for page 63
Supporting Research and Data Analysis in NASA's Science Programs: Engines for Innovation and Synthesis Care should be taken to implement this recommendation in its intended spirit—that is, to provide a management tool. The effort could be easily subverted to mask actual practices if demonstrating a desired outcome became the de facto norm. For example, science support by university or industry contractors performed on or near a field center might be classified as research in universities or in industry rather than by field centers, or a pass-through contract from a university to industry might be classified as university instrument building rather than industry instrument building. Finding: NASA does not use the extended records of its budgets and expenditures as management tools to monitor the health of its R&A and DA programs. Moreover, the fragmented budget structure for R&DA makes it difficult for the scientific community to understand the content of the program and for NASA to explain the content to federal budget decision makers. Recommendation 6: NASA's science offices should establish a uniform procedure for tracking budgets and expenditures by the class of activities and the types of organizations (including intramural and extramural laboratories, industry, and nonprofit entities) that are actually performing the work. These data should be gathered and reported annually and used to inform regular evaluations of R&DA activities (Recommendations 1 and 2). One approach would be to itemize the following elements in the budget: theoretical investigations; new instrument development; exploratory or supporting ground-based and suborbital research; interpretation of data from individual or multiple space missions; management of data; support of U.S. investigators who participate in international missions; and education, outreach, and public information. In addition, these data should be made publicly available and reported annually to the Office of Management and Budget and to Congress.