Proceedings of a Workshop

Earth System Predictability Research and Development

Proceedings of a Workshop—in Brief

SUMMARY

Predictions of weather, air pollution, sea ice, soil moisture, ecosystem functioning, and many other components of the Earth system are increasingly critical for decision making across a wide range of sectors and timescales. Further improvements in these predictions will need to be guided by a clear understanding of what aspects of the Earth system are predictable and of the limits to that predictability. On June 4-5, 2020, the National Academies of Sciences, Engineering, and Medicine hosted a workshop on Earth system predictability research and development. An earlier community roundtable discussion informed the themes of the workshop, which was designed to explore opportunities for key research and development activities that would be most valuable with regard to understanding fundamental, theoretical limits of Earth system predictability. The purpose of the two convening activities was to solicit feedback on the direction that the Federal government should take to advance understanding and application of Earth system predictability.

INTRODUCTION AND WORKSHOP OBJECTIVES

Understanding to what degree different features of the Earth system are predictable across its physical, biogeochemical, ecological, and human-system components—from individual thunderstorms to regional or continental-scale droughts and floods, from local air pollution episodes to ocean carbon uptake, from wildfires to fishery and crop yields—has great practical value to society. Past research into Earth system predictability has led to profound insights into the Earth system and has facilitated improved predictions, said James Hurrell, Colorado State University. However, he continued, accelerating progress in providing practicable predictions across a broader set of phenomena will require deep and sustained interactions with user communities, understanding the theoretical limits of predictability and the sources of predictability, improvements in modeling, targeted observations, and infrastructure such as computing power and supporting workforce focused specifically on the science and applications of Earth system predictability research.

In his opening remarks, Kelvin Droegemeier, Director of the Office of Science and Technology Policy, explained that understanding the extent to which phenomena are predictable across multiple time and space scales is vitally important for improving understanding of what is being predicted, for assessing the value of a prediction, for informing investments in research and operations, and for developing approaches to improve predictions. Droegemeier noted that prediction and predictability are often confused; they are not the same but are closely related (see Box 1). A prediction is an estimate of the future; foundational theory of predictability is not needed to make a prediction, but it can help to assess value of a prediction. He explained that improved understanding of the limits of predictability, which may in some cases be intrinsic to a dynamical system or particular states therein, can help focus and coordinate a national research and development strategy aimed at predicting the future of Earth’s integrated human and natural systems.

In the memorandum “Fiscal Year 2021 Administration Research and Development Budget Priorities,” Departments and Agencies are directed to prioritize R&D in the area of Earth system predictability, with an awareness of its

importance to society.¹ This workshop was designed to serve as one mechanism to solicit feedback on the direction that the Federal government should take to improve understanding of these limits. The specific workshop themes were informed by an earlier community roundtable discussion² and designed to explore opportunities for key research and development activities that would be most valuable with regard to understanding fundamental, theoretical limits of Earth system predictability.

The Workshop on Earth System Predictability Research and Development was held on June 4-5, 2020, by the National Academies of Sciences, Engineering, and Medicine. The workshop included exploration of six themes over a two-day period. Each session was chaired by a member of the workshop’s planning committee and included a plenary speaker, followed by a moderated panel discussion on the topic.

PURPOSE-DRIVEN PRACTICABLE PREDICTABILITY

Addressing “practicable predictability” requires understanding the information needs of decision makers and incorporating that understanding into Earth system predictability research, said Brad Colman, The Climate Corporation, who chaired this session. Discussions in this session focused on the demand for predictions, explored the barriers to expanding their utility, and emphasized the importance of understanding predictability for the development of co-designed prediction systems.

Earth system prediction offers major benefits for improving risk assessment and management in multiple sectors. Keynote speaker Sarah Jones, Deutscher Wetterdienst, began the session with an example from the electricity sector in Germany, where renewable energy sources accounted for 42% of electricity production in 2019. Weather forecasts are indispensable for planning the electricity supply. Jones described how work with stakeholders identified which parameters are most useful to predict (e.g., fog, snow cover, small-scale clouds, and mineral dust for solar photovoltaic systems), leading to research and development targeted at improving the model representation of relevant processes (e.g., changes in boundary layer turbulence) and adding new processes (e.g., transport of Saharan dust). Achieving these benefits involved a multi-year process to build collaborations among operational and academic research organizations, transmission system operators, and private companies.

Jones pointed to other ways that improved predictions are yielding benefits. For example, the Polar Prediction Project brought together international research teams to improve weather and environmental prediction services for the polar regions on timescales from hourly to seasonal. After engaging with users, the need for information on the location of the ice edge, in addition to the ice coverage, was identified, leading to novel observations and coupled models that have improved these predictions.³ Jones also highlighted the Subseasonal-to-Seasonal (S2S) Prediction Project, organized by the World Weather Research Program and the World Climate Research Program. A key activity of the Project has been to develop a database of S2S model forecasts, where the database serves as an important research tool, intended to help inform disaster risk reduction.⁴

__________________

Jones concluded by making the case that advancing estimates of seamless Earth system predictability from minutes to centuries to meet societal needs can be done more effectively through a value cycle approach that focuses on users’ needs (Figure 1). In this framework, information generation is critical but not sufficient. The cycle allows for information to be provided in increasing layers of customization and to cycle insights from stakeholder interaction back into an information pipeline.

The need to understand and interact with stakeholders to ensure that Earth system science output is usable and useful was also highlighted by panelist Olga Wilhelmi, National Center for Atmospheric Research (NCAR). She discussed how engaging decision makers about their information needs and capabilities can help increase usability of model products (e.g., relevant scale, interpretation, format, access). Furthermore, including social and behavior scientists in the co-design of prediction systems and identification of research needs is important for ensuring usability and usefulness of predictions.

Andrew Robertson, Columbia University, explained the International Research Institute for Climate and Society’s four pillars for providing beneficial prediction products to the stakeholder. Predictions must be (1) generated, (2) translated into information relevant to the stakeholder and then (3) transferred in appropriate formats so that they can be (4) used to make decisions. Robertson stressed the importance of identifying the information necessary for developing a useable forecast, from both the developer and the user perspective, and translating the confidence of that forecast into numerical terms for stakeholders.

Speaking from the perspective of developing predictions for natural resource management, panelist Nathan Mantua, National Oceanographic and Atmospheric Administration (NOAA), noted that it is important to account for a range of capacities for using prediction tools. Larger, well-funded organizations often have the technical expertise on staff to use sophisticated tools, whereas some of the smaller organizations do not. Enabling access and ease-of-use of prediction tools for a wider audience may lead to surprising and valuable applications in many sectors that are currently hard to predict.

Wilhelmi highlighted the importance of sharing information with users about what can be skillfully predicted and what are the limits to predictability. Limited predictability and model and observation uncertainties (not only in the physical system, but in the biological and human systems) mean that it is important to characterize the uncertainties and communicate them in a usable way for a variety of stakeholders. Mantua similarly emphasized the importance of communicating with users what systems have extremely limited predictability and to develop decision-making approaches that are robust to these sorts of situations (e.g., a resilient “portfolio” approach to salmon management). Robertson noted the significant increase in accessibility of forecast data across society over the past decade and the opportunity this presents to assess the limits of practical predictability.

Panelist Roger Pulwarty, NOAA, discussed the need to build prediction systems in ways that address systemic, integrated risks and build resilience across multiple sectors. Traditional risk assessment and management strategies are increasingly challenged by systemic risks that connect local conditions to broader global systems and that comingle risks from systems that are often considered or managed independently. Examples include risks that cross transboundary watersheds, fall at the water-energy-food nexus, or involve the intersection of pandemics and other threats to national security. These systemic risks are unconstrained and include the potential for thresholds and surprises, along with the need to account for evolving impacts of climate extremes, variability, and change across time and space. These risks can be globally interconnected, driving local imbalances and reducing anticipated benefits of information use. Addressing such complex risks requires analytical, technical, and deliberative capacity, as well as consideration of equity and broader participation to consider implications beyond a single project or decision context. Pulwarty thus called for development of a multidisciplinary research and applications agenda to systematically link Earth system predictability science and decision making to inform management, adaptive learning, and innovation in approaches to finance.

During the discussion, the speaker and panelists underscored the importance of working in partnership: internationally, multidisciplinary, across agencies and organizations, and through private-public engagement. Pulwarty highlighted the opportunity to draw upon and use lessons from existing prototypes and long-standing efforts such as NOAA’s Regional Integrated Sciences and Assessment programs and National Integrated Drought Information System programs, the National Weather Service’s Weather-Ready Nation, the Climate Corporation, StormCenter Communications, the United States Department of Agriculture’s Regional Hubs, and the Department of Interior’s Climate Science Centers, among others. Jones suggested that it might be worth considering an effort similar to the Global Atmospheric Research Program (GARP)⁵ that the World Meteorological Organization initiated in 1967—a GARP for the 21st Century.

THEORETICAL LIMITS ON EARTH SYSTEM PREDICTABILITY

This session focused on the theoretical foundations of Earth system predictability. Session chair James Hurrell, Colorado State University, noted that current understanding of predictability limits is based on imperfect models and incomplete understanding and representation of critical processes, such as those linking the atmosphere to the ocean or land surface, which evolve more slowly. The panelists addressed the targeted research required to improve the understanding of Earth system predictability limits.

Prashant Sardeshmukh, NOAA/CIRES and University of Colorado at Boulder, opened the session by harkening back to the way Edward Lorenz first characterized predictability in chaotic systems. Lorenz highlighted the existence of two types of predictability in the Earth system: predictability of the first kind is associated with predictable evolution from known initial conditions, and predictability of the second kind is associated with the predictable response to slowly varying forcing. In both cases, predictability is determined by the signal-to-noise ratio,⁶ Sardeshmukh added.

Sardeshmukh noted that the noise in most ensemble forecasting systems is underestimated, which leads to an overestimation of predictability. One way to more accurately estimate predictability is to introduce additional stochastic terms into a model’s equations. This has the effect of broadening the variability in the predicted quantity by crudely accounting for chaotic atmospheric physics in models. Adding stochasticity can also lead to better predictions at weather timescales and for subseasonal and longer timescales.

Panelist Marika Holland, NCAR, highlighted significant advances in the understanding of initial-value sea ice predictability over the last decade and emphasized that predictability varies regionally and seasonally and has climate-state dependence. Comparisons of perfect model studies of the kind that Sardeshmukh discussed with observations suggest that sea ice models are “too predictable”—they have a stronger signal-to-noise ratio than the real world, making it appear that the models are predicting sea ice better than they actually do. The possible underestimation of variability (or noise) in modeled sea ice is potentially due to the fact that the models lack finer-scale processes (e.g., wave-sea ice interactions) or need to include more stochasticity in the representation of processes already included in the models. Holland suggested using theoretical understanding of sources of predictability, combined with a focus on predicting quantities relevant to stakeholders, to inform the design of forecast systems. Better predictions of sea ice could also lead to improved predictions of the Earth systems coupled with the sea ice, such as the biological systems, and regional climate systems.

A commonly used strategy to improve predictability is to use large, well-calibrated forecast ensembles. Panelist Dale Durran, University of Washington, posited that new computationally efficient approaches to numerical weather prediction—e.g., using machine learning to replace dynamical modeling steps—could enable much larger ensembles, thereby yielding benefits for predictability. He also noted that a machine learning context could also allow reconsideration of which variables are predicted, with the possibility of replacing current outputs with more societally relevant outputs (e.g., the electric sector needs solar radiance at the surface).

__________________

Errors in the model representation of processes can lead to incorrect estimation of forecast signals. For example, tropical influences are an important source of extratropical predictability but are not accurately captured in models. Sardeshmukh shared an example of how errors in the representation of sea surface temperatures in the tropical Indo-Pacific warm pool region are major concerns for both estimating and attaining Earth system predictability globally from subseasonal to climate change scales.⁷

For predictions that are limited by predictability of the first kind, reducing errors in the initial conditions can also improve forecast skill. Sardeshmukh showed results from a study in press that indicate that weather prediction skill could be improved by approximately 1 day—in other words, a weather prediction for 7 days out could have as much skill as current weather predictions for 6 days out—simply by eliminating initial errors.

Panelist Nikki Lovenduski, University of Colorado at Boulder, noted that several recent papers point to the theoretical potential to improve predictability of ocean and terrestrial biogeochemical variables (e.g., pH of surface ocean waters,⁸ phytoplankton,⁹ land carbon uptake¹⁰), which could be exploited for various applications, including management of fisheries. These and other recent studies are pointing to quantities that vary more slowly and therefore contribute to predictability over longer timescales. Such slowly varying quantities are said to have longer “memory” of preceding conditions that influence current or future conditions. For example, observations in the ocean of inorganic nutrients, which are related to rates of upwelling, could improve predictability of primary production and ecosystem dynamics.¹¹ Likewise, said Lovenduski, soil moisture reflects water storage in the landscape from previous precipitation and can indicate how much moisture could be returned to the atmosphere via evaporation or how vulnerable an area might be to future rates of ecosystem respiration. However, the sparsity of observations of biogeochemical quantities makes it difficult to initialize simulations and to assess the realism of predictions. Using information about what variables influence simulated quantities could help design observing strategies to prioritize those quantities that have greatest potential to improve predictions.

The unrealized predictive potential in some societally relevant systems was highlighted by panelist Emanuele Di Lorenzo, Georgia Institute of Technology. In particular, regional coastal systems hold tremendous potential for realizing predictability and could have significant value for managing coastal flooding and ecosystems. Di Lorenzo also noted that choosing the prediction targets is important for shaping such research. For example, if the goal is to improve longer lead-time predictions of precipitation, then a research focus on surface hydrology or ground water, which have longer memory, could allow more improvements in predictability.

EXPLORING PREDICTABILITY THROUGH NEW METHODOLOGIES AND TECHNOLOGIES

This session focused on technological advances and other new methodologies and approaches—from machine learning to coupled data assimilation—that can accelerate progress on theoretical understanding of predictability and inform the development of models that more accurately represent the coupled Earth system, as noted by session chair, Jeanine Jones, California Department of Water Resources.

Elizabeth Barnes, Colorado State University, opened the session by discussing “forecasts of opportunity”, the concept that certain environmental conditions lead to more predictable behavior than others. Identifying these sorts of conditions can help focus efforts to understand predictability across Earth system disciplines. Machine-learning methods are well suited to pattern classification tasks and are now at a stage of maturity where they can be used to discover and explore these forecasts of opportunity.

Furthermore, Barnes explained that one criticism of artificial neural networks—that they are “black boxes” and it is difficult to reconcile their choices with physical understanding of systems—is increasingly being addressed with tools that help users visualize what factors influenced their outputs. Layerwise Relevance Propagation is an approach routinely applied in other pattern recognition applications to develop heat maps of the parts of a pattern that had greatest influence on machine learning outputs. This means that when machine learning methods are capable of making an accurate prediction, it is possible to explore why, and this provides information about sources of predictability that could be further exploited.

__________________

Barnes discussed three frontiers in using machine learning to advance understanding of Earth system predictability:

1. transfer learning can be used to train neural networks by leveraging existing (and plentiful) Earth system model simulations, which are imperfect but can capture many of the important dynamics;¹²
2. physics-guided machine learning can leverage existing knowledge to improve transparency and trustworthiness of artificial intelligence tools;¹³ and
3. interdisciplinary collaboration—particularly between Earth system scientists and computer and data scientists—can enable the creation of new tools and methods tailored to specific science questions.

Barnes also pointed out that many artificial intelligence and machine learning tools have been developed for the private sector, and may not be useful “right out of the box” for science applications. Thus there is a need to develop new tools that are specifically tailored to scientific applications, and this will require close collaboration between machine learning experts and scientists with knowledge of the science domain.

Panelist Jeff Anderson, NCAR, spoke about the need for new data assimilation tools and techniques. Data assimilation is an essential piece of numerical weather prediction and can be extended to other Earth system prediction problems when data are available. Anderson noted that data assimilation can be particularly useful for identifying quantities that might be predictable, much in line with the “forecast of opportunity” concept that Barnes presented. Another promising application is to use data assimilation to conduct a prediction system simulation experiment, which takes a synthetic truth and generates the synthetic observations that lead to that truth. Prediction system simulation experiments can be useful for fabricating long observational records, which can be used as analogs to advance understanding of systematic error and overall system predictability.

Delineating the limits of predictability of the Earth system is fundamentally a question of uncertainty quantification, said panelist Tapio Schneider, California Institute of Technology. He suggested that research and development efforts capitalize on the availability of large data sets and new machine learning tools to improve uncertainty quantification. He noted in particular that weather data have not yet been used as extensively as they could be to improve Earth systems models or to quantify uncertainties.

Schneider also discussed how the evolution in computing capabilities could affect Earth system predictions. With improvements in processor clock speed tapering off, computational platforms now have accelerators, combine graphic processing units and central processing units, and increasingly are hosted in cloud computing environments. These advancements in computing capabilities present an opportunity to reimagine the architecture for Earth system models to better leverage these new platforms, and also better tailor models to specific decision contexts. For example, some processes that are computationally intensive (e.g., river or urban flooding) might be resolved in high resolution in limited areas but not necessarily globally. Schneider challenged the workshop participants to adapt new ways of thinking and consider taking the leap to redesign and rewrite models.

Panelist Gudrun Magnusdottir, University of California, Irvine, described research from her team that focuses on understanding what drives regime shifts in California precipitation in winter, which is the rainy season. In combination with more traditional approaches, she gave an example of using hindcasts for sensitivity studies in which the model is nudged to observations in certain parts of the domain to represent processes that are poorly represented in the free-running model. Especially insightful are experiments where interactions between different processes are examined. Magnusdottir emphasized that new tools and methodologies are only part of the effort and that observations and process studies are needed in key areas of sensitivity, along with continued development of Earth system models, to advance understanding of Earth system predictability.

Panelist Michael Dietze, Boston University, works with land surface models, carbon cycle, and paleoecological hindcasts. He said that this is an exciting time in his field as there are many new opportunities to bring land surface data into a data assimilation framework as a way to improve Earth system prediction. Dietze emphasized the need to support open and scalable cyberinfrastructure to reduce redundant efforts, improve research quality, and improve accessibility of system models to a larger fraction of the community.

To understand the predictability of the Earth system, Dietze also emphasized the need to understand which uncertainties dominate predictions at different timescales. Within the biosphere in particular, there are key uncertainties that need to be propagated that have not traditionally been part of the coupled Earth system model or the data assimilation algorithms. Assumptions made about the dominant uncertainties at the weather and climate timescales need to be revisited at these intermediate timescales, and additional uncertainties need to be accounted for in these predictions. Dietze added that methodological advances in community cyberinfrastructure, hierarchical Bayesian calibration, and new data assimilation algorithms that relax assumptions are providing a means to accommodate these additional uncertainties, partition their impact, and increase our understanding of the predictability of the Earth system.

__________________

As accentuated by each speaker in this session, capacity building in terms of workforce development and infrastructure development is critical for advancing understanding of Earth system predictability. Training the next generation in emerging technologies, and having the resources in place to foster collaboration in research is key.

OPTIMIZING OBSERVATIONS TO EXPLORE PREDICTABILITY

This session identified opportunities to take a much more deliberate approach to observations in the context of Earth system predictability research and development, said session chair Gabriele Pfister, NCAR. The session speaker and panelists were asked to discuss the observations and experimental strategies needed to advance theoretical understanding of predictability and to improve the modeling of those processes.

Sarah Gille, University of California, San Diego, opened the session by using ocean processes and observations to illuminate broader themes in understanding predictability that extend across the full Earth system. Gille said that many challenges in understanding the Earth system lie at the interfaces. A prime example is global mean air-sea heat flux, a quantity for which model estimates widely vary. While the current Argo¹⁴ ocean profiling float fleet is sufficient to support decadal predictability, a more nuanced and local understanding of the upper ocean mixed layer is needed for advancing weather and climate predictability on 10-day to 3-year timescales. The recent OceanObs’19 activity brought together scientists to develop recommendations for observation systems that could address these challenges related to air-sea fluxes.¹⁵

Characterizing the ocean mixed layer requires understanding of surface fluxes (e.g., heat, freshwater, momentum, gas), fluxes through the base of the mixed layer, and physical processes responsible for turbulent mixing and fluxes (see Figure 2). In designing an optimal observing system to improve predictability, Gille noted that it is important to determine what needs to be measured, how much can be parameterized or simulated stochastically, what instruments are needed to measure these processes, and what statistical and modeling tools could be useful. In the case of the ocean mixed layer, Gille argued that studies to evaluate episodic, turbulent processes for which parameterizations have not yet been developed would be particularly useful for improving predictability. In addition, she said that combining satellite-based observations with in situ observations is also a promising avenue for improving estimates of heat, freshwater, and momentum exchanges.¹⁶

Gille concluded her presentation with a vision to optimize observations to advance predictability. First, she said that in situ and satellite observing systems need to be expanded to provide boundary conditions and model verification, and they should be configured to resolve key variability. Second, she called for process studies focused on improving the representation of physical processes that are not well represented in the models. Finally, she emphasized that observations and model output need to be FAIR: findable, accessible, interoperable, and reusable.

__________________

Panelist Klaus Keller, Pennsylvania State University, noted that optimizing observations requires identifying which objectives to optimize, or figuring out how to “do the right science.” Possible candidate objectives, which would likely have different optimal observing strategies, could include reducing short-term verification error, increasing the power to differentiate between different hypotheses, or improving a decision. Keller pointed out that designing an optimal observing system to better inform decision making is particularly difficult. It requires sustained and careful engagements with decision makers, innovative training for a new generation of researchers and practitioners, and boundary-spanning organizations. While trade-offs across objectives are likely, sustained and careful engagements with decision makers can help design observation systems that provide actionable information across a wide range of applications and divergent stakeholder needs.

A number of opportunities to leverage existing resources and new technologies to explore predictability were highlighted in this session. Panelist Joellen Russell, University of Arizona, discussed opportunities that exist from the explosion in autonomous observing and communications technologies (e.g., the Argo floats), new satellite data (e.g., small satellite constellations), nontraditional observations (e.g., cell phone data), and a robust and growing commercial ecosystem of providers. These resources can be leveraged with exascale computing and support for optimization of observational systems, including simulation experiments and process studies to escalate Earth system prediction efforts.

In addition to leveraging existing resources, critical gaps in observations were also discussed. For example, panelist James Randerson, University of California, Irvine, discussed how gaps in satellite data limit practicable predictions of wildland fires, in part because the higher-resolution satellite observations from the Visible Infrared Imaging Radiometer Suite are only available twice per day, at 1:30 pm and 1:30 am local solar time, but fires are most active in the later afternoon. Increased spatial and temporal coverage of winds, relative humidity, and vapor pressure deficit would improve predictions that inform fire response activities. At the subseasonal-to-seasonal timescale, incorporating observations of subsurface soil moisture, as well as subsurface and sea surface conditions (important for predicting winds), is key to improving predictability.¹⁷ Randerson also noted opportunities to merge statistical and machine learning approaches with predictions from the National Multi Model Ensemble to improve predictability on these longer timescales.¹⁸

Peter Neilley, The Weather Company, discussed the need for a holistic optimized observational network that goes beyond geophysical observations to optimize use of predictions for society. Neilley explained that predictions of the Earth system are increasingly middleware toward the end goal of predictions of impacts on society. For example, today’s predictions have moved beyond forecasts of temperature, precipitation, or sea state, and are now forecasting decision-relevant quantities such as energy production, transportation disruptions, agricultural efficacy, and supply chain impacts. In order to create the tools and models needed for these kinds of derivative forecasts, Neilley argued that the observation network should likewise expand beyond just the physical measurement of the Earth system to include measurements of the impacts of weather and climate on society. He further pointed out that the optimal use of forecasts requires understanding complex decision contexts and how decision makers interpret and use forecast products. This means that an optimal observation network should also measure societal decision-making processes, along with the context of extenuating factors that influence decisions.

As observations systems expand into domains beyond geophysical data, Neilley noted some new challenges that arise. Most societal impact and decision information is hidden behind the privacy walls of individuals, companies, and in some cases governments. For personal, competitive, and political reasons, there is a general reluctance to share such information. Neilley said that new public private frameworks and paradigms are needed to collect, secure, and anonymize societal data in order to build the trust necessary to encourage widespread sharing of such information.

A HOLISTIC EARTH SYSTEM MODELING FRAMEWORK

The current understanding of predictability limits is based on imperfect models and incomplete understanding and representation of critical processes that are key to making skillful predictions, said session chair Scott Doney, The University of Virginia. As such, the upper bounds of Earth system predictability are difficult to quantify. This session focused on the necessity to better integrate predictability research with Earth system model development and application. The speaker and panelists in this session underscored that nurturing and greatly expanding the Earth system research community would benefit the emergence of predictability studies, with better, more routine links between S2S forecasting and Earth system modeling efforts.

Jean-Francois Lamarque, NCAR, provided an overview of the current Earth system modeling framework. Earth system models have moved toward more complexity (more components of the Earth system are represented), larger ensemble size (to capture internal variability of the system), and higher resolution (especially useful for capturing key processes in atmosphere and ocean fluid dynamics and in areas with variable topography). Lamarque posed the

__________________

question of what is driving predictability in these broader representations of the Earth system (See Figure 3), pointing to different attempts to address the question on the subseasonal to decadal timescales (looking at predictability sources from soil moisture to the El Niño-Southern Oscillation to the deep ocean). The advent of models that couple multiple components allows for a deeper exploration of where the memory resides in the system and whether models are accurately representing that memory.

Panelist Ruby Leung, Department of Energy Pacific Northwest National Laboratory, emphasized that model biases are limiting understanding of Earth system predictability and the ability to make predictions. Lamarque agreed, pointing out that several of these biases have been present in climate models for decades, but the reasons for them are not well understood. Improving understanding of how to resolve these biases has not been a focus of climate model intercomparison efforts.

Increasing model resolution could be a game changer in how persistent model biases and improving predictability are addressed, said Leung, because it would allow simulation of subgrid-scale processes that are currently parameterized. Furthermore, fully representing subgrid moist convection processes could help address the lack of variability or chaotic behavior in the models that results in misrepresentation of the signal-to-noise ratio used to estimate predictability. Similarly, fully simulating mesoscale eddies could improve modeling of air-sea interactions that contribute to subseasonal-to-interannual predictability.

Multi-disciplinary teams of observationalists, modelers, software engineers, computational scientists, and data analysts are needed to make progress in Earth system modeling, said Lamarque. Panelist Cecilia Bitz, University of Washington, used sea ice as an example of how bringing multidisciplinary teams together could help with that progress. Bitz explained that improved representation of the fine-scale exchanges between sea ice and the ocean could lead to improved estimates of predictability. Developing these parameterizations requires detailed observations of the processes, assimilation of observations into models using coupled and multivariate methods to produce more complete data sets based on dynamical and statistical relationships, and application of machine learning approaches to develop simulation approaches that are less computationally expensive.

Lamarque highlighted the need to improve the interoperability of data and models. Lots of information needs to flow between components of prediction systems, he added. Several panelists noted that it is becoming increasingly impossible to build model components separately and then couple them; the coupling is so intrinsic to model performance that it must be incorporated and considered throughout the development process. Likewise, it is important to consider the synergy and research needs of coupling observational systems, data assimilation of model initialization, and predictability studies with Earth system models.

Developing a hierarchy of globally integrated models could be promising as a way to advance predictability and address specific stakeholder needs, said Lamarque. Individual models are built for particular purposes, and an integrated hierarchy approach could exploit the strengths of each.¹⁹ Such a unified framework would require codesign, he added.

__________________

Panelist Charles Stock, NOAA Geophysical Fluid Dynamics Laboratory, discussed ways that the fisheries community is beginning to integrate models into decision making. Coastal areas are poorly observed and modeled and the processes are poorly understood, so significant work will be needed to bring them to a point of being practicably predictable. He recommended the development of ensembles and purpose-driven models that balance the desire to explore predictability with the need for meaningful, skillful, societally relevant predictions.

Biogeochemical, biological, and human processes provide important feedbacks and should be included in models to improve predictability, said Leung. For example, aerosols such as dust and black carbon are an integral part of many monsoon systems; the monsoons affect the aerosol distributions, which in turn affect monsoon circulation and precipitation. The seasonal greening and browning of vegetation and human activities such as urban processes, irrigation, and water management may also provide sources of Earth system predictability through land-atmosphere interactions. To the extent that these biogeochemical, biological, and human processes provide important feedbacks in the Earth systems, Leung argued that they should be realistically represented in models to tap their contributions to predictability.

Incorporating additional processes or quantities into models also can ensure models are meeting stakeholder needs, said panelist Natalie Mahowald, Cornell University, and this should be done in consultation with stakeholders. She shared an example of working with the microinsurance industry in east Africa to develop projections related to pasture usage. After consulting with stakeholders, they focused on model projections of leaf area index, a quantity more relevant for the pastoralists. Model projections of leaf area index showed less spread than precipitation, potentially indicating that this measure is also more predictable.²⁰

The session concluded with a rich discussion of Earth system model development. Panelists highlighted the need to integrate better predictability research into Earth system modeling through emphasis on subseasonal-to-seasonal forecasting, model-observational comparisons, data assimilation for initial conditions, and focus on bias reduction. Leung noted opportunities for different modeling communities to learn from each other; for example, numerical weather prediction could benefit from efforts to incorporate aerosols into Earth system models, and Earth system models could better apply the rigor used in numerical weather prediction. Bitz commented on the need for a balance between large research centers and smaller programs, which can be more nimble, explore more radical ideas, and sometimes integrate work across larger centers.

A NEW RESEARCH FRAMEWORK FOR PRACTICABLE EARTH SYSTEM PREDICTABILITY

Development of a national approach and strategy to knit together predictability-focused theoretical work with observational, modeling, and technology research is an imperative for advancing practicable prediction, said session chair Jenni Evans, The Pennsylvania State University. This session explored opportunities to break down compartmentalization of communities. By making convergent research the new normal, and developing and sustaining a creative workforce, a new foundation on the science and applications of Earth system predictability research can be created.

Duane Waliser, NASA Jet Propulsion Laboratory, started off the session by suggesting the application of a more formal systems engineering approach (see Box 2), to break down the complexity of Earth system predictability into a coordinated and collaborative outcome-driven program. The need for a systems engineering approach stems from the sheer complexity of the questions and objectives being considered: Earth system science is complex, the technology and tools (including models and observations) are rapidly evolving, and the programmatic aspects of the enterprise (including civil, commercial and social) are challenging to optimally coordinate. Waliser argued that a system of systems (SoS) approach could be a way to judiciously integrate and evolve the underlying components to maximize value and societal impacts.

Waliser explained that the Earth system prediction enterprise could be roughly equated to a “collaborative” SoS (Box 2), one that has developed over the last 50 years on a somewhat ad-hoc basis. While this type of SoS tends to rely on a voluntary approach to coordination, it has yielded significant environmental forecast capabilities and decision support guidance. However, given the critical importance of Earth system prediction to the security and resilience of society, there may be reasons to consider moving to an SoS approach that would entail a more formal design and management process, in order to achieve future advances. Waliser posed the questions: “Are there means to help optimize the (science, technology, and enterprise) components? Would a top-down SoS design and development approach help advance Earth system predictability? Are there aspects of a systems engineering approach that would help to achieve an overall vision for Earth system prediction and the decision-support guidance it enables? Is there a need for a coordinating office or body that could direct effort and resources, one that takes into account the strengths and complementary elements of the various agencies and commercial enterprises that have a role and stake in contributing to this critical national capability?” To answer these questions, Waliser suggested assembling a team of systems engineering and Earth system prediction experts to assess the value of more formally engaging an SoS perspective to help guide the nation’s Earth systems predictability roadmap and prioritizations.

Panelist Paula Bontempi, NASA, highlighted the need for having a structure in place that integrates communities and avoids compartmentalization. Bontempi urged agencies to create opportunities that encourage disciplines,

__________________

as well as scientists and managers, to work together towards common objectives. She said that one solution is to craft solicitations and competitions for federal research and development funding in ways that inspire the next generation to be creative in proposing ideas that break down compartmentalization.

Panelist Waleed Abdalati, NOAA/CIRES and University of Colorado at Boulder, reinforced the need to employ systems-level thinking. Abdalati spoke of the importance of a shared focus, shared vision, and shared strategy to empower agencies to prioritize a collective effort and move away from the sum of the parts approach for Earth systems predictability research. Abdalati said that agencies need to be liberated to do more than just play in the sandbox together; they need to build the sandbox together.

Panelist Chris Bretherton, University of Washington, reiterated the need for a coordinated interagency research agenda and identified other challenges to avoiding compartmentalization. To foster an environment of interdisciplinary research, it is important to have open, accessible, well-documented and publicized community models and data sets. An investment in software engineering is needed to make existing data and models as useful for interdisciplinary research as possible by lowering barriers to access. Furthermore, Bretherton advised clearly defining shared goals that naturally bring communities together.

Several panelists emphasized that achieving a new research framework to progress understanding of Earth system predictability requires an inspired next generation of scientists and engineers. Bretherton explained that students need to be educated on Earth system predictability as interdisciplinary research. According to Abdalati, to attract a talented workforce, a perception needs to prevail that this research is of utmost importance and is recognized and supported from leaders of all sectors of society.

CONCLUDING THOUGHTS

The potential to make significant leaps forward in improving estimates of Earth system predictability was a recurring theme of the workshop discussions. The convergence of advances in computing capability, access to new observations, incorporation of more components in Earth system models, and the application of machine learning and other data analytic techniques all point to the potential to extend predictability to longer timescale and to a much broader range of decision contexts.

Workshop participants discussed a number of cross-cutting challenges related to realizing those benefits. First, sustained dialogue between forecast producers, translators, and decision-making groups could build trust, educate all parties involved, and shape the development of custom-tailored information. Second, attracting, training, and retaining the next generation of scientists to conduct this complex, interdisciplinary research is challenging with the limited resources currently available to educational institutions. The Earth system prediction workforce spans physical science, biogeochemistry and ecology, social and behavioral sciences, and computational and data sciences; and researchers working across traditional disciplinary boundaries would help combat the challenges of compartmentalization and help progress understanding of Earth system predictability. Finally, many participants pointed to the need for improved coordination of resources, models, data, and research priority setting.

DISCLAIMER: This Proceedings of a Workshop—in Brief was prepared by Kelly Oskvig and Amanda Staudt as a factual summary of what occurred at the meeting. The statements made are those of the rapporteur(s) or individual meeting participants and do not necessarily represent the views of all meeting participants; the planning committee; or the National Academies of Sciences, Engineering, and Medicine.

REVIEWERS: To ensure that it meets institutional standards for quality and objectivity, this Proceedings of a Workshop—in Brief was reviewed in draft form by Cecilia Bitz, University of Washington, Scott C. Doney, University of Virginia, and Marshall Shepherd, University of Georgia, to ensure that it meets institutional standards for quality and objectivity. The review comments and draft manuscript remain confidential to protect the integrity of the process.

Workshop on Earth System Predictability Research and Development Planning Committee:

James Hurrell (Chair), Colorado State University, Scott C. Doney, University of Virginia, Jenni-Louise Evans, Pennsylvania State University, Jeanine A. Jones, California Department of Water Resources, Brad R. Colman, The Climate Corporation, Gabriele Pfister, National Center for Atmospheric Research.

SPONSORS: The workshop was supported by the US Global Change Research Program, funded through the National Aeronautics and Space Administration.

For additional information regarding the workshop, including the workshop recordings, agenda, and information on the speakers and panelists, please visit https://www.nationalacademies.org/event/06-04-2020/workshop-on-earth-system-predictability-research-and-development.

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2020. Earth System Predictability Research and Development: Proceedings of a Workshop—in Brief. Washington, DC: The National Academies Press. https://doi.org/10.17226/25861.

Division on Earth and Life Studies

Copyright 2020 by the National Academy of Sciences. All rights reserved.