National Academies Press: OpenBook
« Previous: 5 The NSF Geospace Section Portfolio Recommended by the ICCGS
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×

6

Implementation Planning

The assessment committee was tasked with considering whether the ICCGS report1 provides “a clear set of recommendations on how GS should implement the [2013 solar and space physics decadal] survey’s[2] priorities within the context of the NSF/Geosciences strategic planning process.” The committee pointed out in Section 3.1 that the NSF AGS and GS do not currently have a strategic plan or a visible strategic planning process and recommended that a strategic framework be developed within which to assess and prioritize the GS portfolio and related programs and initiatives in future years.

The ICCGS report detailed more than 100 recommendations that together address survey priorities. In this chapter, the assessment committee briefly addresses the clarity and completeness of the ICCGS recommendations and the planning and resources needed for their implementation.

6.1 CLARITY AND COMPLETENESS OF ICCGS RECOMMENDATIONS

As part of its statement of task, the assessment committee was asked to discuss the general readability and clarity of the PRC’s report and in particular its recommendations (see Section 1.2). The ICCGS report is logically organized, comprehensive, and readable, with one notable exception: the large number of findings (121) and recommendations (113) reduced the clarity of the report. Recommendations did not always follow from prior findings, and many recommendations were themselves underpinned by several other recommendations. Some recommendations can be distilled into a single recommendation (e.g., ICCGS Recs. 4.1-4.4). Other recommendations are aspirational rather than actionable (e.g., ICCGS Rec. 4.6, discussed in Section 5.4). The assessment committee recognizes that the large number of recommendations reflects, to some extent, a high degree of granularity of the GS portfolio. The assessment committee is less concerned with the readability of the ICCGS report than with the actionability of the recommendations made. The number of recommendations is comprehensive but perhaps daunting when GS staff resources are limited.

___________________

1 National Science Foundation (NSF), 2016, Investments in Critical Capabilities for Geospace Science 2016 to 2025, Geospace Section of the Division of Atmospheric and Geospace Science, February 5, https://www.nsf.gov/geo/adgeo/geospace-review/geospace-portfolio-reviewfinal-rpt-2016.pdf.

[2] National Research Council (NRC), 2013, Solar and Space Physics: A Science for a Technological Society, The National Academies Press, Washington, D.C.

Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×

The large number of ICCGS findings and recommendations notwithstanding, ICCGS Chapter 9 distills many of the findings and recommendations detailed in ICCGS Chapters 4, 6, 7, and 8 into 17 specific recommendations for the GS portfolio (ICCGS Recs. 9.1-9.17). Collectively, the recommendations made in ICCGS Chapter 9 and the broad priorities given therein constitute the implementation plan for GS facilities and grants programs. The balance of GS investments in the ICCGS-recommended portfolio roughly maintains the current budget percentages among core grants, strategic grants, and facilities (33%, 28%, and 38%, respectively, in FY2015), but shifts 2 percent from facilities funding into strategic grants programs to address survey DRIVE priorities.3

The ICCGS prioritizes the recommended program in terms of first, second, and third priority, in the event that the GS budget is more pessimistic than the flat budget the PRC assumed. These are as follows:

  • Priority 1: The AER, MAG, and STR core grants programs; the CEDAR, GEM, SHINE, and SWM strategic grants programs; Class 1 and 2 facilities; the EISCAT partnership; the DASI Facilities Program; and the Data Systems Program.
  • Priority 2: Grand Challenge Projects and the Facilities I&V Program.
  • Priority 3: CubeSat program and FDSS.

A Midscale Projects Program is considered by the ICCGS to be out of budget and is therefore not prioritized.

The assessment committee regards ICCGS Recs. 9.1-9.15 as actionable under the priorities given. Taken together with recommendations on workforce development and diversity in ICCGS Chapter 4 and partnerships in ICCGS Chapter 8, ICCGS presents a complete, balanced, and actionable program.

The ICCGS recommendations regarding investment in a new Midscale Projects Program (ICCGS Recs. 9.16 and 9.17) are contingent on additional changes to the geospace facilities landscape (such as the closure of Arecibo Observatory) or future increases in the GS budget and are not actionable at this time. As discussed in Section 5.2.1, the assessment committee concludes that the creation of a Midscale Projects Program lies outside the means or ability of GS to implement. Funds outside GS are required to develop a Midscale Projects Program within AGS for which the geospace sciences community is eligible to apply.

In further considering the completeness of the ICCGS report, the assessment committee was tasked with addressing whether the recommended program adequately takes into account “the integration of technology development with the NSF-GS science program.”4 The charge to the PRC does not discuss the integration of technology development with the GS science program, although it is part of the survey’s DRIVE initiative, as detailed in ICCGS Table 6.1. Technology development is briefly discussed in ICCGS Rec. 6.3 (that “AER/MAG/STR grants research also should continue to serve as a technology incubator”) and in connection with CubeSats (ICCGS Rec. 6.25: that “NSF GS should provide an annual, tabulated set of detailed metrics, documenting the accomplishments and challenges of the program in terms of training, research, technology development, and contributions to geospace science and/or space weather forecasting”). Appropriate technology development could be included as a component in future announcements of opportunity for the DASI Facilities Program or the Facilities I&V Program.

6.2 PORTFOLIO EVOLUTION AND RENEWAL

There is broad agreement that GS needs to renew and rebalance its portfolio continuously by investing in new instrument development; ensuring the continuing scientific productivity of existing facilities; tuning its core, targeted, and strategic grants programs to address community science imperatives and modeling needs; supporting workforce development and diversity; and being alert to partnerships that leverage resources to enhance GS activities. A GS and AGS strategic plan is needed to provide the framework within which GS establishes priorities (Section 3.3). Periodic reviews are needed to address portfolio balance and evolution within the strategic framework.

___________________

3 NSF, 2016, Investments in Critical Capabilities for Geospace Science 2016 to 2025, p. 104.

4 See the Portfolio Review Committee’s charge, reprinted in Appendix A.

Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×

6.2.1 Senior Reviews

The ICCGS recognizes the need for a regular review process of the GS portfolio. ICCGS Rec. 9.8 (see also ICCGS Section 6.7 and ICCGS Recs. 6.4-6.6 and 6.29-6.30) recommends a senior review of the GS Core and Strategic Grants Programs. The objectives of the grants programs senior review are twofold: (1) to “review and balance of investments in core and strategic grants programs in light of the budget and research environment at the time of the review, [as well as] evaluate the programs’ effectiveness in achieving Section and Decadal Survey science goals and, . . . recommend adjustments in the direction and balance of the grants programs . . .”; and (2) to “facilitate transparency in GS investments in its grants programs by evaluating progress of the Section in implementing recommendations of the decadal portfolio review. . . .” (ICCGS, p. 71).

ICCGS Rec. 9.15 (see also ICCGS Section 7.8 and ICCGS Recs. 7.29-7.32 and 7.35-7.36) recommends a senior review of all GS facilities. The objectives of the facilities senior review are to (1) “reconcile the GS facilities budget with the costs required to provide adequate M&O for all GS facilities and to maintain the state-of-the-art in facilities instrumentation and capabilities” and (2) “to review and rank each facility’s capabilities (i) to enable, as a standalone instrument or system, transformative scientific discoveries, and (ii) to contribute to integrative scientific understanding as a complementary element in NSF’s distributed capabilities for observing geospace as a system” (ICCGS, p. 89). The ICCGS recognizes that such a review “may require closure or divestment of some facilities to accommodate the innovative capabilities provided by new facilities or augmented facilities.”

In agreement with ICCGS, the assessment committee endorses the use of senior reviews as a recognized means of prioritizing existing investments to allow investment in new opportunities and to rebalance and renew the portfolio. The ICCGS makes a number of suggestions on how the senior reviews could be most effective based on their experience with the GS portfolio review and suggests criteria for reviewing major scientific initiatives (ICCGS Section 7.8). The ICCGS concludes with Rec. 7.36, which recommends that “NSF GS should develop a common set of annual metrics from each facility, which can be collected year-on-year to provide the underpinning of the next Senior Review.” As concluded in Section 4.1, the assessment committee agrees with the ICCGS’s recommendation but cautions that the administration of uniform data and metrics may require additional resources.

Two concerns have arisen regarding the senior review process proposed by the ICCGS. First, the assessment committee questions the need for two senior reviews, one for the Core and Strategic Grants Programs and another for GS facilities. Separate senior reviews may lead to “myopic reviews . . . rather than a holistic portfolio review,” as recognized in ICCGS Section 6.7. The administration of two reviews may add considerable burden to an already over-extended GS staff. The ICCGS favors two reviews because each senior review has a different scope and metrics, and the members of the review committees could be optimized for each review. The assessment committee suggests that a possible approach for a combined review may be to form subcommittees optimized to review each of the major components of the GS portfolio (e.g., grants, facilities, workforce and diversity, partnerships) and then reconcile subcommittee recommendations jointly.

A second concern of the assessment committee is the frequency with which senior reviews occur. The ICCGS recommends semi-decadal reviews. Again, this may place administrative burden on the GS staff, especially if GS elects to conduct separate senior reviews for grants and facilities. GS will have to consider the number of senior reviews that it can realistically perform with current resources and what additional resources would be needed to perform more frequent senior reviews.

Conclusion: The assessment committee endorses ICCGS Recommendations 9.8 and 9.15 to conduct periodic senior reviews of the NSF Geospace Section’s grants programs and facilities.

Conclusion: The assessment committee questions the need for two separate senior reviews, one for the Core and Strategic Grants Programs and another for GS facilities. The committee is also concerned about the burden placed upon GS administration by two separate semi-decadal reviews and concluded that a single unified review is preferable.

Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×

6.2.2 Management Processes

ICCGS Section 9.6 discusses organization and management issues internal to GS. In particular, the report recommends reforms to organizational processes that would improve the transparency and efficiency of future reviews (e.g., portfolio reviews and senior reviews) based on ICCGS experiences with the portfolio review process. The suggested improvements would also facilitate transitions in GS program management by inevitable turnover of personnel. The ICCGS makes recommendations in ICCGS Section 9.6 but does not enumerate them as such. The bulleted list of recommendations may be summarized as follows: GS should (1) develop “accurate, complete, and transparently understandable data metrics . . . for all grants in the various programs of the Section,” including both “historical data and data for the current fiscal year” in order to assist with understanding trends and vitality of these programs; (2) consider making a set of these metrics publicly available on a GS webpage; (3) separate fully funded GS research proposals from special categories of awards (conference awards, MRI and co-funded projects, AGS postdocs, NSF/DOE awards); (4) “track, maintain, and publish data on workforce issues” for awards in each program; (5) track M&O costs in grants and facility awards; (6) “establish guidelines for determining the impacts of encumbering program resources M&O of new facilities before committing future budget to them”; and (7) communicate the “outcomes, rationale, and impacts of decisions” made in response to senior reviews and other advisory bodies. The assessment committee endorses each of these suggestions. In addition, the committee reiterates its strong support for ICCGS Rec. 7.36 for GS to “develop a common set of annual metrics from each facility.” The assessment committee is aware that to develop, collect, maintain, and exploit data and metrics from GS facilities, grants, and other programs requires resources in informatics that GS may not currently possess.

As noted in Section 6.1, the assessment committee recognizes that the large number of ICCGS recommendations to some extent reflects the high degree of granularity of the GS portfolio. To the suggestions made in ICCGS Rec. 9.6, the committee would add the suggestion that GS consider reducing administrative burden through a consolidation and reduction of program elements where it makes sense to do so.

Conclusion: The suggestions to NSF GS regarding management processes are excellent and will underpin future senior reviews and allow greater transparency into the decision-making process.

To conclude this assessment, the PRC met its charge in the face of challenging constraints. The ICCGS report presents a comprehensive program for the GS portfolio that is aligned with survey priorities. The exception is funding for a midscale projects program which, for reasons discussed in Section 5.2.1, cannot be implemented within the budget guidance provided to the PRC by NSF. The recommended program nevertheless enables investments in new programs, instrument development, and facilities and defines a framework for ensuring program renewal and balance going forward through a more uniform set of metrics for evaluating existing programs and facilities and through periodic senior reviews. The ICCGS also provides a number of suggestions regarding GS processes that will support and streamline GS management function. The responsibility now passes to NSF AGS and GEO to implement the GS portfolio recommended in the ICCGS and to engage with the community in developing a strategic vision and plan that identifies and builds on the strengths of AGS and GS within the broader solar and space physics enterprise, identifies partnerships within the NSF and external to NSF, and leverages opportunities from the NSWS5 and SWAP6 initiatives.

__________________

5 National Science and Technology Council (NSTC), 2015, National Space Weather Strategy, Office of Science and Technology Policy, Washington, D.C., October.

6 NSTC, 2015, National Space Weather Action Plan, Office of Science and Technology Policy, Washington, D.C., October.

Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×

This page intentionally left blank.

Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×

This page intentionally left blank.

Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 40
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 41
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 42
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 43
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 44
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 45
Suggested Citation:"6 Implementation Planning." National Academies of Sciences, Engineering, and Medicine. 2017. Assessment of the National Science Foundation's 2015 Geospace Portfolio Review. Washington, DC: The National Academies Press. doi: 10.17226/24666.
×
Page 46
Next: Appendix A Charge to the Portfolio Review Committee »
Assessment of the National Science Foundation's 2015 Geospace Portfolio Review Get This Book
×
Buy Paperback | $65.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

At the request of the Advisory Committee for Geosciences of the National Science Foundation (NSF), a review of the Geospace Section of the NSF Division of Atmospheric and Geospace Sciences was undertaken in 2015. The Portfolio Review Committee was charged with reviewing the portfolio of facilities, research programs, and activities funded by Geospace Section and to recommend critical capabilities and the balance of investments needed to enable the science program articulated in the 2013 NRC decadal survey Solar and Space Physics: A Science for a Technological Society. The Portfolio Review Committee's report Investments in Critical Capabilities for Geospace Science 2016 to 2025 (ICCGS) was accepted by the Advisory Committee for Geosciences in April 2016.

Assessment of the National Science Foundation's 2015 Geospace Portfolio Review provides an independent assessment of the ICCGS report. This publication assesses how well the ICCGS provides a clear set of findings, conclusions, and recommendations for Geospace Section that align with the science priorities of the NRC decadal survey, and adequately take into account issues such as the current budget outlook and the science needs of the community. Additionally, this study makes recommendations focused on options and considerations for NSF's implementation of the ICCGS recommendations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!