6
Implementation Planning
The assessment committee was tasked with considering whether the ICCGS report1 provides “a clear set of recommendations on how GS should implement the [2013 solar and space physics decadal] survey’s[2] priorities within the context of the NSF/Geosciences strategic planning process.” The committee pointed out in Section 3.1 that the NSF AGS and GS do not currently have a strategic plan or a visible strategic planning process and recommended that a strategic framework be developed within which to assess and prioritize the GS portfolio and related programs and initiatives in future years.
The ICCGS report detailed more than 100 recommendations that together address survey priorities. In this chapter, the assessment committee briefly addresses the clarity and completeness of the ICCGS recommendations and the planning and resources needed for their implementation.
6.1 CLARITY AND COMPLETENESS OF ICCGS RECOMMENDATIONS
As part of its statement of task, the assessment committee was asked to discuss the general readability and clarity of the PRC’s report and in particular its recommendations (see Section 1.2). The ICCGS report is logically organized, comprehensive, and readable, with one notable exception: the large number of findings (121) and recommendations (113) reduced the clarity of the report. Recommendations did not always follow from prior findings, and many recommendations were themselves underpinned by several other recommendations. Some recommendations can be distilled into a single recommendation (e.g., ICCGS Recs. 4.1-4.4). Other recommendations are aspirational rather than actionable (e.g., ICCGS Rec. 4.6, discussed in Section 5.4). The assessment committee recognizes that the large number of recommendations reflects, to some extent, a high degree of granularity of the GS portfolio. The assessment committee is less concerned with the readability of the ICCGS report than with the actionability of the recommendations made. The number of recommendations is comprehensive but perhaps daunting when GS staff resources are limited.
___________________
1 National Science Foundation (NSF), 2016, Investments in Critical Capabilities for Geospace Science 2016 to 2025, Geospace Section of the Division of Atmospheric and Geospace Science, February 5, https://www.nsf.gov/geo/adgeo/geospace-review/geospace-portfolio-reviewfinal-rpt-2016.pdf.
[2] National Research Council (NRC), 2013, Solar and Space Physics: A Science for a Technological Society, The National Academies Press, Washington, D.C.
The large number of ICCGS findings and recommendations notwithstanding, ICCGS Chapter 9 distills many of the findings and recommendations detailed in ICCGS Chapters 4, 6, 7, and 8 into 17 specific recommendations for the GS portfolio (ICCGS Recs. 9.1-9.17). Collectively, the recommendations made in ICCGS Chapter 9 and the broad priorities given therein constitute the implementation plan for GS facilities and grants programs. The balance of GS investments in the ICCGS-recommended portfolio roughly maintains the current budget percentages among core grants, strategic grants, and facilities (33%, 28%, and 38%, respectively, in FY2015), but shifts 2 percent from facilities funding into strategic grants programs to address survey DRIVE priorities.3
The ICCGS prioritizes the recommended program in terms of first, second, and third priority, in the event that the GS budget is more pessimistic than the flat budget the PRC assumed. These are as follows:
- Priority 1: The AER, MAG, and STR core grants programs; the CEDAR, GEM, SHINE, and SWM strategic grants programs; Class 1 and 2 facilities; the EISCAT partnership; the DASI Facilities Program; and the Data Systems Program.
- Priority 2: Grand Challenge Projects and the Facilities I&V Program.
- Priority 3: CubeSat program and FDSS.
A Midscale Projects Program is considered by the ICCGS to be out of budget and is therefore not prioritized.
The assessment committee regards ICCGS Recs. 9.1-9.15 as actionable under the priorities given. Taken together with recommendations on workforce development and diversity in ICCGS Chapter 4 and partnerships in ICCGS Chapter 8, ICCGS presents a complete, balanced, and actionable program.
The ICCGS recommendations regarding investment in a new Midscale Projects Program (ICCGS Recs. 9.16 and 9.17) are contingent on additional changes to the geospace facilities landscape (such as the closure of Arecibo Observatory) or future increases in the GS budget and are not actionable at this time. As discussed in Section 5.2.1, the assessment committee concludes that the creation of a Midscale Projects Program lies outside the means or ability of GS to implement. Funds outside GS are required to develop a Midscale Projects Program within AGS for which the geospace sciences community is eligible to apply.
In further considering the completeness of the ICCGS report, the assessment committee was tasked with addressing whether the recommended program adequately takes into account “the integration of technology development with the NSF-GS science program.”4 The charge to the PRC does not discuss the integration of technology development with the GS science program, although it is part of the survey’s DRIVE initiative, as detailed in ICCGS Table 6.1. Technology development is briefly discussed in ICCGS Rec. 6.3 (that “AER/MAG/STR grants research also should continue to serve as a technology incubator”) and in connection with CubeSats (ICCGS Rec. 6.25: that “NSF GS should provide an annual, tabulated set of detailed metrics, documenting the accomplishments and challenges of the program in terms of training, research, technology development, and contributions to geospace science and/or space weather forecasting”). Appropriate technology development could be included as a component in future announcements of opportunity for the DASI Facilities Program or the Facilities I&V Program.
6.2 PORTFOLIO EVOLUTION AND RENEWAL
There is broad agreement that GS needs to renew and rebalance its portfolio continuously by investing in new instrument development; ensuring the continuing scientific productivity of existing facilities; tuning its core, targeted, and strategic grants programs to address community science imperatives and modeling needs; supporting workforce development and diversity; and being alert to partnerships that leverage resources to enhance GS activities. A GS and AGS strategic plan is needed to provide the framework within which GS establishes priorities (Section 3.3). Periodic reviews are needed to address portfolio balance and evolution within the strategic framework.
___________________
3 NSF, 2016, Investments in Critical Capabilities for Geospace Science 2016 to 2025, p. 104.
4 See the Portfolio Review Committee’s charge, reprinted in Appendix A.
6.2.1 Senior Reviews
The ICCGS recognizes the need for a regular review process of the GS portfolio. ICCGS Rec. 9.8 (see also ICCGS Section 6.7 and ICCGS Recs. 6.4-6.6 and 6.29-6.30) recommends a senior review of the GS Core and Strategic Grants Programs. The objectives of the grants programs senior review are twofold: (1) to “review and balance of investments in core and strategic grants programs in light of the budget and research environment at the time of the review, [as well as] evaluate the programs’ effectiveness in achieving Section and Decadal Survey science goals and, . . . recommend adjustments in the direction and balance of the grants programs . . .”; and (2) to “facilitate transparency in GS investments in its grants programs by evaluating progress of the Section in implementing recommendations of the decadal portfolio review. . . .” (ICCGS, p. 71).
ICCGS Rec. 9.15 (see also ICCGS Section 7.8 and ICCGS Recs. 7.29-7.32 and 7.35-7.36) recommends a senior review of all GS facilities. The objectives of the facilities senior review are to (1) “reconcile the GS facilities budget with the costs required to provide adequate M&O for all GS facilities and to maintain the state-of-the-art in facilities instrumentation and capabilities” and (2) “to review and rank each facility’s capabilities (i) to enable, as a standalone instrument or system, transformative scientific discoveries, and (ii) to contribute to integrative scientific understanding as a complementary element in NSF’s distributed capabilities for observing geospace as a system” (ICCGS, p. 89). The ICCGS recognizes that such a review “may require closure or divestment of some facilities to accommodate the innovative capabilities provided by new facilities or augmented facilities.”
In agreement with ICCGS, the assessment committee endorses the use of senior reviews as a recognized means of prioritizing existing investments to allow investment in new opportunities and to rebalance and renew the portfolio. The ICCGS makes a number of suggestions on how the senior reviews could be most effective based on their experience with the GS portfolio review and suggests criteria for reviewing major scientific initiatives (ICCGS Section 7.8). The ICCGS concludes with Rec. 7.36, which recommends that “NSF GS should develop a common set of annual metrics from each facility, which can be collected year-on-year to provide the underpinning of the next Senior Review.” As concluded in Section 4.1, the assessment committee agrees with the ICCGS’s recommendation but cautions that the administration of uniform data and metrics may require additional resources.
Two concerns have arisen regarding the senior review process proposed by the ICCGS. First, the assessment committee questions the need for two senior reviews, one for the Core and Strategic Grants Programs and another for GS facilities. Separate senior reviews may lead to “myopic reviews . . . rather than a holistic portfolio review,” as recognized in ICCGS Section 6.7. The administration of two reviews may add considerable burden to an already over-extended GS staff. The ICCGS favors two reviews because each senior review has a different scope and metrics, and the members of the review committees could be optimized for each review. The assessment committee suggests that a possible approach for a combined review may be to form subcommittees optimized to review each of the major components of the GS portfolio (e.g., grants, facilities, workforce and diversity, partnerships) and then reconcile subcommittee recommendations jointly.
A second concern of the assessment committee is the frequency with which senior reviews occur. The ICCGS recommends semi-decadal reviews. Again, this may place administrative burden on the GS staff, especially if GS elects to conduct separate senior reviews for grants and facilities. GS will have to consider the number of senior reviews that it can realistically perform with current resources and what additional resources would be needed to perform more frequent senior reviews.
Conclusion: The assessment committee endorses ICCGS Recommendations 9.8 and 9.15 to conduct periodic senior reviews of the NSF Geospace Section’s grants programs and facilities.
Conclusion: The assessment committee questions the need for two separate senior reviews, one for the Core and Strategic Grants Programs and another for GS facilities. The committee is also concerned about the burden placed upon GS administration by two separate semi-decadal reviews and concluded that a single unified review is preferable.
6.2.2 Management Processes
ICCGS Section 9.6 discusses organization and management issues internal to GS. In particular, the report recommends reforms to organizational processes that would improve the transparency and efficiency of future reviews (e.g., portfolio reviews and senior reviews) based on ICCGS experiences with the portfolio review process. The suggested improvements would also facilitate transitions in GS program management by inevitable turnover of personnel. The ICCGS makes recommendations in ICCGS Section 9.6 but does not enumerate them as such. The bulleted list of recommendations may be summarized as follows: GS should (1) develop “accurate, complete, and transparently understandable data metrics . . . for all grants in the various programs of the Section,” including both “historical data and data for the current fiscal year” in order to assist with understanding trends and vitality of these programs; (2) consider making a set of these metrics publicly available on a GS webpage; (3) separate fully funded GS research proposals from special categories of awards (conference awards, MRI and co-funded projects, AGS postdocs, NSF/DOE awards); (4) “track, maintain, and publish data on workforce issues” for awards in each program; (5) track M&O costs in grants and facility awards; (6) “establish guidelines for determining the impacts of encumbering program resources M&O of new facilities before committing future budget to them”; and (7) communicate the “outcomes, rationale, and impacts of decisions” made in response to senior reviews and other advisory bodies. The assessment committee endorses each of these suggestions. In addition, the committee reiterates its strong support for ICCGS Rec. 7.36 for GS to “develop a common set of annual metrics from each facility.” The assessment committee is aware that to develop, collect, maintain, and exploit data and metrics from GS facilities, grants, and other programs requires resources in informatics that GS may not currently possess.
As noted in Section 6.1, the assessment committee recognizes that the large number of ICCGS recommendations to some extent reflects the high degree of granularity of the GS portfolio. To the suggestions made in ICCGS Rec. 9.6, the committee would add the suggestion that GS consider reducing administrative burden through a consolidation and reduction of program elements where it makes sense to do so.
Conclusion: The suggestions to NSF GS regarding management processes are excellent and will underpin future senior reviews and allow greater transparency into the decision-making process.
To conclude this assessment, the PRC met its charge in the face of challenging constraints. The ICCGS report presents a comprehensive program for the GS portfolio that is aligned with survey priorities. The exception is funding for a midscale projects program which, for reasons discussed in Section 5.2.1, cannot be implemented within the budget guidance provided to the PRC by NSF. The recommended program nevertheless enables investments in new programs, instrument development, and facilities and defines a framework for ensuring program renewal and balance going forward through a more uniform set of metrics for evaluating existing programs and facilities and through periodic senior reviews. The ICCGS also provides a number of suggestions regarding GS processes that will support and streamline GS management function. The responsibility now passes to NSF AGS and GEO to implement the GS portfolio recommended in the ICCGS and to engage with the community in developing a strategic vision and plan that identifies and builds on the strengths of AGS and GS within the broader solar and space physics enterprise, identifies partnerships within the NSF and external to NSF, and leverages opportunities from the NSWS5 and SWAP6 initiatives.
__________________
5 National Science and Technology Council (NSTC), 2015, National Space Weather Strategy, Office of Science and Technology Policy, Washington, D.C., October.
6 NSTC, 2015, National Space Weather Action Plan, Office of Science and Technology Policy, Washington, D.C., October.
This page intentionally left blank.
This page intentionally left blank.
