A decadal survey, regardless of the scientific discipline, is a monumental undertaking. At the end of the day, only one or two committee members are ultimately responsible for overseeing the entire enterprise over the course of 2-year study. For the first panel session, four chairs of the most recent decadal surveys1 discussed their experience with their respective decadal surveys and highlighted the similarities and differences between them. Each of the former chairs gave brief introductory remarks, followed by a moderated discussion, including audience questions.
Charles F. Kennel, University of California, San Diego, and Chair, Space Studies Board; Co-Chair, Workshop Planning Committee
Berrien Moore III, Dean, College of Atmospheric and Geographic Sciences; Director, National Weather Center; Vice President for Weather and Climate Programs, University of Oklahoma; 2007 Earth Science and Applications from Space Decadal Survey
Roger Blandford, Luke Blossom Professor in the School of Humanities and Sciences; Director, Kavli Institute of Particle Astrophysics and Cosmology, Stanford University; 2010 Astronomy and Astrophysics Decadal Survey
Steven Squyres, Goldwin Smith Professor of Astronomy, Cornell University; 2011 Planetary Science Decadal Survey
Daniel Baker, Professor, Astrophysical and Planetary Sciences, and Physics; Director, Laboratory for Atmospheric and Space Physics, University of Colorado, Boulder; 2013 Solar and Space Physics Decadal Survey
Berrien Moore, co-chair of the 2007 decadal survey committee that authored Earth Science and Applications from Space: National Imperatives for the Next Decade and Beyond,2 began by saying that the survey committee and panels set out to produce a decadal that had community support, integrity, and honesty, as well as create political momentum for the Earth sciences program. This decadal was the first of its kind for the Earth sciences, which, by the nature of the field, produced many unique challenges that the other disciplines do not have to deal with—namely, how to create an integrated and holistic set of observations and missions for Earth system science. Moore expressed concern for the current state of the Earth sciences program and said that while the survey achieved its goal of community support and an honest attempt at designing a robust program of Earth system science, they failed to maintain the survey’s integrity. Subsequently, the program became stove-piped as various elements were assigned to NASA centers without a coherent implementation plan. In hindsight, Moore identified different mechanisms that
1 The 2007 Earth science and applications from space decadal survey had two co-chairs, Berrien Moore and Richard Anthes. Dr. Moore represented the decadal leadership on this workshop panel, while Dr. Anthes spoke on the program formulation panel later in the workshop.
2 National Research Council (NRC), Earth Science and Applications from Space: National Imperatives for the Next Decade and Beyond, The National Academies Press, Washington, D.C., 2007.
he believes should have been incorporated into the 2007 decadal survey in Earth science and applications from space: aggressive use of cost caps; more engagement with political leaders during the survey process and dissemination; and clearer decision rules for navigating unforeseen budgetary and programmatic challenges—for instance, the “unraveling” of the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program. Although he believes that the 2007 decadal survey had a positive impact on the Earth sciences community and the field in general, there is now a growing sense of frustration across the discipline. Moore concluded by recommending that NASA return to the strategic planning apparatus that was in place in the 1980s to improve budget and programmatic alignment.
Roger Blandford noted that the steering committee and supporting panels established to undertake the 2010 astronomy and astrophysics decadal survey set out to conduct a science-led, cost- and schedule-risk validated program. Unfortunately, many factors out of the survey’s control have impeded implementation of the recommended program—most significantly, cost overruns on the James Webb Space Telescope. These factors were not accounted for during the survey period because they did not materialize until after the 2010 survey report was released. Blandford explained how the 2010 report was the first survey to use the cost and technical evaluation (CATE) process, which would go on to dominate much of the discussion at the workshop. Despite the challenges in implementing the recommended program, he believes that it was executed successfully and that it, surprisingly, uncovered more issues with the ground-based facility proposals than the space-based proposals from the community. Blandford also described efforts made by National Research Council (NRC) staff and committee members as the survey was being set up to engage with international partners and solicit their input.
Steven Squyres, chair of the 2011 planetary science decadal survey committee that authored Vision and Voyages for Planetary Science in the Decade 2013-2022,3 remarked that predicting the future is the most difficult aspect of crafting a decadal survey. One can never truly know what the budget environment will be like or what discoveries might be made that could drastically alter a discipline’s landscape. He then highlighted what he considers two of the most important elements for a successful decadal survey: the statement of task (SOT) and decision rules. The report must be responsive to the SOT, and the SOT for the 2011 planetary science survey was clear on several points: the survey was to consider missions, cost them, and not treat Mars or the Moon any different from the other planets. The SOT reflects the programmatic reality at the time it is written. Squyres explained that the greatest challenge in writing a SOT is crafting it to stand the test of time and maintain relevancy throughout the entire decade. The SOT must also be strategic, not tactical, and take a long-term historic view. Concerning the decision rules, Squyres recommended in the future that—recognizing the limitations of knowing what the future holds—NASA and other sponsor agencies should be more forthcoming at the very beginning of the survey with budget realities and potential future budget environments. Because of this inherent uncertainty, it is important to provide budget decision rules to decision-makers at the agencies as tools to mitigate challenges to implementation of the recommended program while maintaining the strategic integrity of the survey. Although other surveys have used some type of decision-rule process in the past, those were more on the order of guidelines. The 2011 planetary science decadal survey was the first to use very explicit decision rules, but Squyres cautioned that it is too early to assess their utility, and he warned that future surveys should be careful not to make the decision rules overly prescriptive—it is not the role of the survey to tell NASA how to do its job.
Daniel Baker explained that the 2013 solar and space physics survey had a unique advantage over the other three decadal surveys: the survey committee and panels saw the myriad challenges that the other three surveys faced when the time came for them to implement the recommended programs in Earth science and applications from space, astronomy, and planetary science. Baker explained that the solar and space physics survey committee and panels tried to focus on what would be most achievable and what type of smaller-scale activities could be undertaken to accomplish the goals of the survey and move the discipline forward over the coming decade. To this end, he highlighted DRIVE (Diversify, Realize,
3 NRC, Vision and Voyages for Planetary Science in the Decade 2013-2022, The National Academies Press, Washington, D.C., 2011.
Integrate, Venture, Educate), an initiative recommended in the 2013 solar and space physics decadal that takes maximum advantage of present resources for advanced technology development.
All of the chairs complimented their respective discipline community’s involvement in their surveys, without which the surveys would not have been possible or particularly successful. The chairs also framed the context for future discussions to be held at the workshop, raising the issues surrounding the crafting of the SOT, balancing science and missions, the importance of—and process for—CATE, and the utility of decision rules for programmatic execution.
During their discussion, the moderator and panelists talked about their experience with and opinions on:
• Decadal survey decision rules;
• Science and mission prioritization among the surveys;
• Survey steering committee and panel interactions;
• Governance and interagency considerations;
• Workforce issues and research-to-operations transition;
• Thoughts on future decadal surveys; and,
• International Traffic in Arms Regulations (ITAR) reform.
The moderator opened the panel discussion by asking the panelists to elaborate on their prior remarks concerning decision rules for decadal survey implementation. The panelists said that it is too early to say how exactly they will be followed, especially those for the planetary science decadal survey, which had the most detailed and explicit decision rules out of the four most recent surveys. In order to be successful, one of the panelists explained, the decision rules must represent the views of the community accurately, but also give NASA the flexibility to manage their programs. At the same time, it was clear that the four decadal surveys handled the concept of decision rules very differently. While the 2011 planetary science decadal survey had the most explicit decisions rules of the four, the 2010 astronomy and astrophysics survey had similar guidelines but did not call them decision rules, nor were they quite as explicit as with planetary science survey. The 2013 solar and space physics survey did include decision rules that accounted for the possibility of reductions in budget and program as well as for possible augmentation thereof, should the budget allow. One of the panelists noted, however, that the augmentation rules have not received much attention.
Another difference between the decadal surveys that emerged from the discussion was how each decadal evaluated missions and science objectives for prioritization. Unlike the other surveys, the 2013 solar and space physics decadal survey was tasked with focusing on the science that needs to be pursued in the coming decade, as opposed to the other surveys that looked at both missions and science. The challenge for the solar and space physics survey committee revolved around the difficulty of costing the science objectives; there is no easy way to do this, as opposed to generating an approximate cost figure for a notional mission via the CATE process. The question of how to balance prioritizing missions and science in the surveys was discussed at great length throughout the entire workshop and was discussed in great detail in Sessions 3 and 4 (Chapters 5 and 6 of this report, respectively). The Earth science,
astronomy, and planetary decadal surveys all had some kind of cost estimation process for notional missions, but the astronomy and planetary surveys utilized the more rigorous CATE process, which had not yet been developed when the Earth science survey was underway. Squyres noted that the mission studies used to inform the surveys require adequate resources and commended NASA for supplying the mission studies to the planetary science decadal survey. Blandford noted that without consideration of the missions, the astronomy and astrophysics decadal survey would have been far less worthwhile.
Charles Kennel then asked the panelists to focus on the issue of costing “ideas” and how to attach a mission idea to a potential science objective. Blandford replied that this is precisely what happened in the 2010 astronomy and astrophysics decadal survey, where the survey committee was confronted with multiple scientific objectives that could be accomplished with a shared suite of instruments on a single spacecraft. The mission that resulted from these deliberations, the Wide-Field Infrared Survey Telescope (WFIRST), was developed by one of the panels of the astronomy and astrophysics survey and presented to the survey committee. Blandford admitted that it was a risky choice, but ultimately the right choice. In addition, the WFIRST mission can be further refined beyond what the committee did to reduce costs, and that is precisely what NASA has been doing since the decadal was released. Baker remarked that the 2013 solar and space physics decadal survey did not have the same resources—financially or temporal—as the other decadal surveys. In particular, it did not have the resources for the survey committee and its panels to work with NASA centers on mission studies like the planetary science decadal did. However, Baker continued, the panels worked closely with the Aerospace Corporation to develop notional missions, and he is proud of that effort. His experience with the 2013 solar and space physics survey—and having seen the initial fits and starts of implementing the other three surveys—emphasized for him how important it is to adequately invest in pre-Phase A efforts to fully understand a mission. Moore seconded this statement and said that an extended Phase A allows for the maturation of missions and is important for building community buy-in.
Another issue that arose during the panel discussion was the relationship between the survey panels and their steering committees. Specifically, whether the panels should come up with (or synthesize) their own mission concepts that are not explicitly submitted via community white papers, as the astronomy and astrophysics decadal survey panel did with the WFIRST mission. Squyres remarked that decadal survey chairs have very little input into the composition of the panels, which, in his opinion, is a good thing. At the same time, he explained, this places a burden on the National Research Council (NRC) to assemble a properly constituted group. Squyres also noted that the steering committees in general contain more experienced individuals than the panels4 and are able to compare ideas that come out of the panels to one another. In his opinion, this worked out very well for the planetary science decadal survey, and he commended the NRC for finding the right balance between what the panels are allowed to do and how the steering committee makes its final decisions. The other panelists agreed and said that their decadal surveys were similarly set up, although Blandford recommended that future surveys have fewer members than the 23 who served on the 2010 astronomy and astrophysics decadal steering committee. He also noted that more mid-career professionals should be involved. Finally, Baker noted that to avoid stovepiping of the panels and steering committee, his decadal survey had five working groups that worked on issues across all of the panels and steering committee, and he recommended a systems-engineering approach like this for future surveys.
4 Although many steering committee members also served on sub-panels.
The challenge of governance for Earth and space sciences was also discussed by the panelists. Earth science and solar and space physics in particular touch upon larger governance issues within the U.S. government and international engagement. Such issues that arise in these disciplines include which agency is responsible for observations for which subdiscipline, who is responsible for research and analysis, and who is responsible for maintaining continuity of measurements. When asked if the decadal survey committees for these two communities were able to interact with their appropriate national and foreign counterparts, Moore said that he thinks the Earth science decadal survey engaged the right people but that still did not prevent the Earth science program from unraveling to the degree it did. He also said that the Earth science decadal survey recommended that the U.S. government undertake an effort to create a remote-sensing strategy, but after only a few months the effort stalled. Baker commented that climate science is heavily politicized, while space weather is more bipartisan in nature. To him, this shows how difficult it is for agencies to work together at senior levels; in fact, as a former co-chair of an NRC study that looked at the challenges of interagency cooperation, the take-away message from the report was “don’t do it.”
Although a great deal of the discussion at the workshop—and, indeed, coverage of the decadal surveys when they are released—focused on programmatic and mission recommendations, the next discussion focused on addressing issues of workforce and research to operations. Kennel noted that it is difficult to draw a clear line between research and operations, citing Earth science as an example where if a research mission is successful and useful then there is an immediate call for continuation of said mission. Moore responded that this was a very important issue that the 2007 Earth science and applications from space decadal survey tackled—the survey committee recognized that weather satellites have historically been used for research, even though they are primarily operational satellites. He went on to say that once NASA has done a mission once, and it is successful, there is a tendency to make it “operational” even if there is no true demand for operational data. This often results in pushing mission management off to NOAA, which can create tension between the two agencies. Baker remarked that the United States now has developed a space weather capability on the back of a scientific program, but then asked, What does having an operational space weather capability really mean? A resulting recommendation in the 2013 solar and space physics decadal survey from this question is a call to establish a space weather program at a higher level in the administration. Finally, regarding the nation’s space sciences and engineering workforce, Blandford noted that the 2010 astronomy and astrophysics survey paid a great deal of attention to soliciting demographic information to more accurately determine where graduate students and post-doctoral scholars go after graduation and how successful they are. Incorporating this into the larger survey of the community is an important element in understanding how the field is evolving. Blandford concluded by noting the power that a successful space program has in motivating children to pursue science, technology, engineering, and/or math (STEM) careers.
Charles Kennel concluded the panelist interaction session by asking the panelists to pretend they were going to chair the next decadal survey for their discipline, asking, “Based on past experience and current knowledge, how would they incorporate international collaboration into future surveys?” Squyres responded that for planetary science, there really are no domestic agencies to partner with, so if there is to be collaboration, then it must be with international counterparts. Unfortunately, U.S. and foreign space science strategic planning timelines are not aligned, and the processes are not necessarily analogous.
When the 2011 planetary science decadal survey report was released and presented to European counterparts, it was met with confusion because the Europeans were unfamiliar with the decadal process. On the other hand, the Japanese thought that they should try to emulate the U.S. model. The 2011 planetary science decadal survey had international representation on almost every panel and on the steering committee and regularly consulted with NASA on matters concerning international partnerships. Consulting with international partners before crafting a survey’s SOT would also benefit the survey and allow NRC staff and sponsor agencies to negotiate a more forward-looking and strategic SOT. Moore underscored the differences in planning horizons between the United States and its main space science collaborators. He said that a more honest dialog with international partners is necessary and that decadal surveys should not shy away from explicitly recommending missions associated with foreign missions. Finally, Blandford noted that the rest of the world is getting more heavily involved in space-related activities, but he wondered whether the United States can be a reliable long-term partner in international space ventures.
Closing out the issue of international collaboration, Squyres lamented the challenges to international cooperation posed by ITAR. Although there was talk of ITAR reform, it had not yet occurred.5 Squyres also noted that this is an issue that has also been discussed a great deal by the NASA Advisory Council, which believes NASA should be “deeply engaged” with the White House in the reform process. Moore concluded by saying that Earth observations are strategic by definition, and the United States benefits from international Earth observations as much as the world benefits from U.S. observations. Therefore, any actions the U.S. government can take to allow federal agencies to have honest discussions with international counterparts are a step in the right direction.
Workshop participants made comments and posed questions to the panelists, as described below. Topics discussed included:
• Augustine Committee6 and the decadal surveys,
• Survey panel interactions,
• Committee formation process,
• Scope of the decadal survey,
• Public engagement,
• Survey roll-out,
• Decadal survey stewardship, and
• Mission selection.
5 On January 3, 2013—after the workshop took place—President Obama signed into law H.R. 4310, the National Defense Authorization Act for Fiscal Year 2013. Specifically, Sec. 1261 of the bill gives the President the authority to remove satellites from the United States Munitions List (USML) which would remove them from ITAR purview. Following signing of this law, both the U.S. Department of Commerce and the U.S. Department of State have proposed multiple rule changes that are open to public comments and have not yet been enacted.
6 Office of Science and Technology Policy, Review of U.S. Human Spaceflight Plans Committee, Seeking A Human Spaceflight Program Worthy of a Great Nation, Washington, D.C., October 2009. This review was carried out under the chairmanship of Norman Augustine and is referred to by panelists in this discussion as the “Augustine Committee.”
In response to a question about how the Augustine Committee affected the decadal survey, Kennel, a member of the Augustine Committee, explained that the purpose of the committee—initiated by the President in 2009—was to evaluate NASA’s Constellation program and the retirement of the space shuttle. It had very little or nothing to do with Earth and space science. On the other hand, Moore said that, in his opinion, the Augustine Committee had a negative impact on the decadal surveys because it failed to make the argument for increased budget levels that a human spaceflight program would require. Without increased funding for human spaceflight efforts, the Earth and space sciences would be at risk of having funds shifted to human spaceflight in a flat or declining budget environment. In response, Kennel explained that the scope of the Augustine Committee was constrained to consider options for human spaceflight in a level budget scenario.
Another audience member criticized the 2011 planetary science decadal survey for a lack of cross-talk and hardly any interaction at all among the various panels of the decadal survey. In addition, he said that the survey also lacked transparency at the community level. Squyres responded by explaining that the integration of panel input occurred at the steering committee level, where representatives from the panels were in constant communication with one another and the steering committee to share information. Regarding transparency, Squyres explained that the survey did its best to webcast all meetings and maintain archives of such proceedings, some of which are still online.
An audience member criticized how the 2001 astronomy and astrophysics decadal survey7 formed its steering committee and panels, saying that the NRC had minimal input into the committee formation and was overly reliant on recommendations from the community. Citing a need to comprise committees and panels with members who are good at consensus building, the audience member asked if the NRC process has changed to address his criticisms. Moore agreed with the questioner and said he encountered the same problem when seeking membership for the 2007 decadal survey on Earth science and applications from space. Kennel commented that this has not been the case with the surveys following the 2007 Earth science and applications from space decadal, and that the NRC has taken a much firmer stand on certifying the integrity and balance of its committees. Hence, the burden of responsibility in this matter falls on NRC staff instead of the survey chair(s), allowing them to focus appropriately on the process and structure of the survey itself.
An audience member asked if the decadal survey can become counterproductive by attempting to address too much at a strategic level. Moore responded by acknowledging that there were things the 2007 Earth science decadal survey could have done differently. In hindsight, he would have scaled back the number of recommendations. The survey tried to formulate a strategic Earth science program and, hopefully, initiate positive change in the program resulting in a larger budget. In hindsight, though, it appears that NASA would have been more successful implementing a recommended program that asked
7 National Research Council, Astronomy and Astrophysics in the New Millennium, National Academy Press, Washington, D.C., 2001.
for less and prioritized more. Still, he said, the strategic position taken by the survey to increase the budget was a worthy one, albeit one that preceded the economic woes that began in late 2008. In this same vein, Baker said that the 2013 solar and space physics decadal survey is the ultimate result of the “triumph of diminished expectations.”
The panelists were then asked by an audience member whether the perception that the surveys have been largely unsuccessful in their subsequent implementation is due to inadequate public engagement. The audience member proffered that future surveys should consider focusing on a single theme that the public can understand and rally behind. The panelists said their decadal surveys all took public engagement and perception into account when designing the surveys. Blandford said the 2010 astronomy and astrophysics survey crafted top-level descriptions of themes with the public in mind, while Baker said that the public’s interest in space weather as it relates to the 2013 solar and space physics decadal was something of which the steering committee was well aware. Squyres said the steering committee of the 2011 planetary science survey gave serious consideration to what the public would be most interested in; it is not a coincidence that the highest-priority recommendations were for Mars sample return and a mission to Europa, he explained, since life in the solar system is a deeply engaging topic for the public.
Another audience member followed up by asking the panelists to comment on how the surveys engaged their respective communities, and if they think their surveys had adequate time to carry out their tasks. Although, at the time the survey was occurring, the questioner felt like it was too short of a time period, he came to the conclusion that the amount of time was appropriate and that a longer time period would not necessarily translate to an improved product. The panelists, by and large, said they felt the same way as the audience member and that, at the time of their surveys, they were concerned that they did not have enough time. Despite this, none of the panelists said that the surveys needed more time to accomplish their goals.
Kennel asked the panelists how they would roll out and disseminate their surveys if they had another opportunity to do so. Moore said that the 2007 Earth science survey report was too long and should have been more concise. In addition, that survey could have used more multimedia to aid in dissemination, which the other panelists agreed with. Baker said that he had to fight to use more animations and graphics to help tell the story, and that should not have been the case. The panelists also said the short-form public versions of the surveys were also very useful and should continue to be developed for future surveys.
Citing mission and programmatic failures, administration transitions, and rising costs, an audience member asked the panelists to comment on what type of stewardship apparatus should be in place for the decadal surveys after their release. The panelists recommended that the NRC standing committees responsible for decadal survey stewardship stay actively engaged with the implementers and program managers at NASA. Although NASA leadership often cites the decadal surveys as a “bible” of sorts, they still need help interpreting the surveys, which is an appropriate and necessary activity for the standing committees whose membership is drawn up from the ranks of decadal survey membership.
The session concluded with a question about how much notional mission concept analysis during the decadal surveys is appropriate and if there had been an even more rigorous assessment of community-and panel-derived mission concepts, would the surveys have picked better reference designs to recommend. Squyres acknowledged that more time and resources could have helped, but all of the panelists said that the community and the membership of the surveys are deserving of the trust placed in them to come up with mission concepts, and, overall, the surveys struck a balance in terms of time and resources available. If anything, Moore remarked, what is necessary is an extended Phase A for mission studies after a survey is released.