CHAPTER 2

The Motivation for Integrating Social and Behavioral Sciences Within the Weather Enterprise

Building appreciation of how social and behavioral sciences (SBS) can contribute to the goals of the weather enterprise means appreciating both the diversity of SBS disciplines and research methodologies that exist, and the diversity of issues that such expertise can help address—reaching well beyond the task of refining how weather forecasts and warnings are communicated. This chapter provides an overview of these many potential contributions (see Section 2.1) and examples of recent contributions (see Box 2.1); it discusses how the weather enterprise is currently evolving, and why such developments raise new needs for SBS research (see Section 2.2); and it discusses why it can be challenging to define success and assign value to the outcomes of such research (see Section 2.3).

2.1 WHY THE WEATHER ENTERPRISE NEEDS SOCIAL AND BEHAVIORAL SCIENCE INSIGHTS

The motivation for integrating social and behavioral sciences within the weather enterprise is tied directly to the official mission of the National Weather Service (NWS), and, arguably by extension, the mission of the weather enterprise at large: Provide weather, water, and climate data, forecasts, and warnings for the protection of life and property and the enhancement of the national economy. Success in achieving this mission requires the capability to effectively inform decision making and to help people prepare for and respond to many types of hazardous weather—not just among individuals, but among numerous institutional actors. Improving scientific understanding of the factors that affect decision making and behavior has the potential to fundamentally strengthen the performance of the weather enterprise.

In the recent decades, there have been numerous advances in meteorological and hydrological sciences that have led to substantial improvements in technical forecasting and warning capabilities (NASEM, 2016a; NWS, 2017a). Yet, this progress often has not translated into similar advances in protecting lives from hazardous weather. The high death toll from a historic tornado outbreak across numerous states in 2011 is

cited by many National Oceanic and Atmospheric Administration (NOAA) officials as the “wake up call” that led to the creation of the Weather-Ready Nation initiative (NWS, 2011), discussed later in this section.

Other recent examples: In 2012, Superstorm Sandy affected millions and is reported to have killed 117 people along the U.S. East Coast. In this event, accurate forecasts and widespread warnings failed to trigger, for many people, the protective behaviors that experts in the weather enterprise recommended and were hoping to see (CDC, 2013; Terrell, 2016). In 2013, during a deadly tornado outbreak in Oklahoma, thousands of people fled their homes in cars despite years of messaging about the dangers of encountering tornados in a vehicle. Analysis found that several factors, including people’s recent experiences with another large tornado event and conflicting advice from different local broadcast officials, contributed to this outcome (NWS, 2014).

Most recently, in autumn 2017, Hurricane Harvey brought record-breaking rainfall to the Texas Gulf Coast, leading to dozens of deaths and many thousands of water rescues; Hurricane Irma affected the entire state of Florida with power outages, storm surges, widespread damage, and fatalities; and Hurricane Maria caused widespread devastation across Puerto Rico. These events illustrated the weather enterprise’s many successes, with forecasts and public safety communications that enabled millions of people to take recommended precautionary actions. Yet, these events also illustrated many of the huge challenges we still face, for instance, in effectively communicating about weather hazards with inherent uncertainties (e.g., rapidly changing information about storm tracks), and informing decisions about protecting and evacuating dense population centers, developing more resilient urban infrastructure and more sustainable flood insurance policies, and providing response and recovery support that ultimately increases weather readiness for all.

As a result of such experiences, there is increasing recognition that SBS insights provide a critical underpinning of the weather enterprise. The breadth of possible contributions becomes clear when looking at the goals that the NWS has articulated for its signature initiative, “Weather Ready Nation” (WRN). These goals are

• to provide forecast information in a way that better supports emergency managers, first responders, government officials, businesses, and the public to make fast, smart decisions that save lives and property and enhance livelihoods;
• to move new science and technology into NWS operations that will improve forecasts and ultimately increase weather-readiness; and
• to foster dialog with local communities that will ultimately reduce the risk of being adversely impacted by extreme weather and water events and increase community resilience for future extreme events.

A centerpiece of the WRN strategy is the concept of Impact-based Decision Support Services (IDSS), defined as “provision of relevant information and interpretative services to enable core partners’ decisions when weather, water, or climate has a direct impact on the protection of lives and livelihoods” (NWS, 2013).

The paradigm of WRN and IDSS has far-reaching implications for how the weather enterprise operates. It requires NWS to work with a wide array of partners in government agencies that address emergency management, transportation management, and public health, as well as researchers, the media, civic institutions, the insurance industry, and private-sector weather companies. These partners and the myriad relationships and interactions among them raise a host of new social and behavioral science issues to understand and navigate effectively. Figure 2.1 illustrates the stages involved in weather-related communication and decision support that must be addressed under the WRN paradigm—with a few examples of how SBS research can provide critically needed insights and understanding in each of these stages. Figure 2.2 presents a temporal perspective on these weather enterprise activities. It illustrates the broad potential impact of SBS research by showing how decision making and actions at different scales within the timeframe of a hazardous weather event depend on, and can influence, social and physical contexts.

The growing emphasis on IDSS also points to the need to focus beyond just forecast and warning products toward services that support decisions for “end to end” integrated planning and for building resiliency throughout the full cycle of preparedness and mitigation; monitoring, assessment, and forecasting; dissemination of warnings and recommended actions; response efforts of institutions and individuals; and post-event assessment and recovery efforts. Below we briefly discuss how SBS research is critical to each of these different stages. This builds on a framework defined by Quarantelli (1990), as well as a variety of other activities that require organizational coordination and decision making (Bostrom et al., 2016; Morss et al., 2015).

• Preparedness and mitigation (that is, weather readiness) efforts are motivated by the fact that preventing adverse impacts from hazardous weather requires understanding many contextual factors that can turn a potential weather hazard into an actual disaster—ranging from land use, built environment, and geographic vulnerabilities to the demographic, cultural, and economic characteristics of the specific populations at risk. It also requires understanding (well before a specific weather hazard ever arises) the highly context-specific factors that affect what “protective actions” are actually feasible and effective for any given community, household, or individual to follow. SBS research is needed to inform all of these considerations.

• Assessment and monitoring of weather (e.g., at specialized national forecast centers and at local forecast offices) involves designing or selecting monitoring infrastructure and interfaces, developing or interpreting standards for categorizing weather conditions, and conducting monitoring activities individually and in teams. There are multiple points at which social, behavioral, and human factors research can inform improvements, such as design and use of visualizations and other tools showing dynamic results from sensors, expert consultation and judgment to interpret model outputs, design and selection of thresholds for weather risks that will trigger particular response actions, decisions regarding whether such thresholds have been met, and translation of information from one domain to another (e.g., visualizations into stylized text). Weather monitoring involves diverse data from multiple sources, ranging from national satellites to regional radar to local spotters. This raises communication and decision-making challenges about how to weigh these diverse sources of input. Decisions about where and when to issue watches and warnings often entail both inter- and intra-organizational coordination.
• Dissemination of weather information can include formal products, channels, and standardized operating procedures, as well as informal or ad hoc local procedures. SBS research can improve the design, implementation, and evaluation of each of these, as well as how they function collectively. For example, hurricane-related communication products may be developed and issued by multiple organizations (e.g., National Hurricane Center, local Weather Forecast Office, various media outlets). Dissemination may include intra-agency hotline calls and automatic feeds to emergency managers, as well as standard operating procedures for moving to 24-7 broadcasting at major media outlets (Bostrom et al., 2016). SBS can inform the design, operation, and evaluation of such systems and products, and can inform goal-setting by examining these systems comparatively and experimentally. Moreover, dissemination is not solely a top-down process originating from weather organizations; it also involves social networks and processes that interact with (or can even supersede) formal dissemination processes. Research on these dynamics is essential.
• Response is a function of both selective perception and social confirmation, and it is driven by vulnerability, social and physical context, awareness and beliefs, and numerous other social and behavioral factors, in addition to the actual content of warning messages (Quarantelli, 1990; Sorensen, 2000). Examining responses to weather warnings is a rich field of research, but much remains to be learned, especially in light of the rapid evolution of communications technologies and practices. SBS can inform the evaluation of current responses, design of new response options, and the development of feedback

• mechanisms to improve the entire system. It can also inform organizational coordination and the prediction of responses. This encompasses studies to understand the information needs and dynamics not just of the public at large, but likewise for the many different response professionals (e.g., emergency managers, road maintenance engineers, school superintendents) who must make critical decisions well in advance of a pending weather event.

• Recovery efforts taking place in the days to years after a major weather event occurs present another opportunity where SBS research can provide numerous critical insights. This includes, for instance, studies of how and why injuries and deaths occurred and what sort of changes are needed to prevent recurrence of such outcomes in the future, as well as studies of how households, communities, and regions can rebuild in ways that reduce vulnerability to similar future events.

2.2 SOME CURRENT DEVELOPMENTS THAT MAGNIFY THE NEED FOR SBS RESEARCH

The weather enterprise is a highly dynamic system that evolves continuously in terms of new technologies and forecasting capabilities, new means of collecting and sharing observations, and ever-growing public demand for new types of weather services. Many of these changes raise important new questions that require SBS expertise and methodologies to address. A few examples discussed below help to illustrate that SBS research is not only critical to the weather enterprise today, but also may become even more important in the future.

Proliferation of Weather Information Sources

While the NWS is the original source of most U.S. weather observations, forecasts, and watches/warnings, it is no longer the direct source of weather information for most Americans. Rather, NWS information is filtered through a complex, ever-evolving communication chain of secondary sources. Institutional actors such as private firms, large event and venue managers, and state Departments of Transportation (DOTs) often get customized information from “value-added meteorology” companies. A 2009 study (Lazo et al., 2009) found that the most common source of forecast information for the general public is local television stations; cable television and radio are the next most popular sources, followed by web pages and newspapers. In recent years, there has been a huge growth in various types of weather apps for mobile phones, as well as growth in the communication of weather information via social media channels such

as Twitter and Facebook, and crowdsource-driven platforms where anyone can report weather conditions in their area—similar to how some popular mobile phone apps gather information from drivers about traffic congestion.

Role of SBS. This ever more diverse and complex communication chain presents a tremendous challenge that underscores the need for SBS research to better understand how different target populations (e.g., in different age groups, different geographic locations) receive and process weather information in different contexts, and how people are affected by differing, sometimes conflicting information coming from these diverse sources.

FACETS and Warn-on-Forecast

Forecasting a Continuum of Environmental Threats (FACETs) is a project of the NOAA National Severe Storms Laboratory that proposes to modernize the high-impact weather forecasting and communication processes by addressing seven interrelated functions of the watch and warning process: the nature of hazardous weather; observations and guidance; forecaster decisions; forecast generation tools; useable output; effective response; and verification. Currently NWS follows a “warn-on-detection” strategy, in which warnings for local severe weather are not issued until there is an early signal on radar, or the weather hazard is physically spotted, which in some cases does not provide the public with enough lead time to move to safety. An important development being advanced within FACETs to address such concerns is an alternative “Warn-on-Forecast” strategy, which aims to create computer-model-generated probabilistic maps of storm-scale hazards (tornadoes, large hail, extreme rainfall), allowing forecasters to issue warnings up to an hour before they strike (NSSL, 2015). Researchers use high-resolution surface, satellite, and radar data with an ensemble of forecasts from convection-resolving numerical weather models to produce the probabilistic information. Threat levels are updated in real time based on current weather observations, including data from rapidly scanning radars. Warn-on-Forecast methods are being evaluated in the NOAA Hazardous Weather Testbed, as a precursor for transitioning these technologies into forecasting operations.

Role of SBS. The Warn-on-Forecast capabilities, and probabilistic-based forecasting strategies more generally, raise a host of questions that require SBS research, such as: How do longer hazard warning lead times affect the ways that people heed and react to warnings? Can they result in unintended and potentially worse societal response (e.g., if people stop sheltering before the storm arrives or try to drive away through crowded urban areas)? How should probabilistic information be displayed to achieve

the most accurate interpretation by the public and emergency managers? How, and how frequently, are warnings best updated in time for different forecasting contexts?

GOES-R Satellite Weather Information

Geostationary Operational Environmental Satellites (GOES) have been providing imagery and data on atmospheric conditions and space weather since the mid-1970s. In November 2016, the first of three satellites being built to replace the aging U.S. weather satellite system, the GOES-R satellite series, was launched. The GOES-R satellites have numerous advanced features and capabilities that were not available in the previous generation of satellites. For example, they will be able to capture weather details for the entire United States in the same time it took older GOES satellites to image one small stormy region. With these improved detailed pictures and measurements, the forecast models will have more precise data. This development will help the routine process, and it will help track severe storms and create better hurricane forecast tracks. This near-real-time imagery will help meteorologists see which storms are strengthening or weakening, possibly improving lead times for warnings of severe storms, particularly in areas devoid of radar. Storm initiation, which often is seen on satellite images before it appears on radar, will be known much more quickly, allowing forecasters to pinpoint the most likely regions of interest for upcoming severe weather.

Role of SBS. Forecasters already have a large amount of incoming data from multiple sources. How will this large new influx of information affect their decision-making processes? Social science research can examine how forecasters access, interpret, and use the newly available information; how they integrate the data into their forecast process; and what is a useful and usable mix of displays to aid the forecaster in various decision making contexts, given the even more frequent input of data coming from the GOES-R satellites.

National Water Model and Hydrometeorological-Forecasting Advances

Forecasting of flooding hazards has the potential to take a significant step forward with the development and recent launch of the National Water Model (NWM). Prior to the NWM, forecasts involved significant forecaster input (e.g., manual modifications to model forcings, states, and parameters) and were limited to deterministic values at ~4,000 locations, with probabilistic forecasts based only on climatology. The NWM provides hourly high-resolution national streamflow guidance, including at locations that

are currently underserved. The system will provide real-time flood forecast inundation mapping and, eventually, daily national “situational awareness” products, including visualizations of how flooding hazards are likely to develop in a given location. Related innovations are also advancing at the state level; for instance, new inundation mapping tools and watershed-scale flood mitigation activities are being developed at the Iowa Flood Center.¹

Role of SBS. The NWM opens an entirely new set of water resource information for the public and for emergency managers. Both the spatial coverage and temporal resolution will be fundamentally different, and as such, the NWM promises to provide major improvements in predicting and tracking flooding events. Social science can inform how to best use output from the NWM to create useful products for decision makers, for short-term events and longer-term water resource management, and for planning efforts to manage future potential hazards. Social science may also be useful in determining the optimal interactions and task sharing among disparate River Forecast Centers and the National Water Center Operations Center.

Automated and Connected Vehicles

Widespread options already exist for drivers to choose “connected vehicles” that are equipped with features such as internet-enabled navigation and safety alerts, including hazardous weather information. An even more revolutionary development on the horizon is the advent of automated vehicles—sometimes referred to as “self-driving” cars—motivated by the desire to eliminate accidents related to human error and to enable more efficient use of roadways. A number of trial vehicles and systems are already being field tested in several cities around the United States. Automation will allow vehicles to receive and react to real-time information about traffic dynamics and to adjust accordingly. Such vehicles could presumably also react to real-time geotargeted information about weather hazards and resulting road conditions.

___________________

¹ See http://iowafloodcenter.org/ifc-projects for more information.

that determine how automated vehicles respond to weather. As noted by Kyriakidis and colleagues (2017), some key research challenges are to understand the synergy between the humans and automation, potential changes in driving behavior due to automation, and the type of information that the drivers will receive from the automated driving system.

Role of SBS. These technological developments raise a host of questions about human-vehicle interactions that require careful study. Connected vehicles raise new questions about the right balance between providing useful real-time alerts to drivers and encouraging drivers to focus on the road instead of information being provided on a screen. Other complex research questions arise with the development of “partial autonomy” technologies that allow the drivers to take the wheel in certain conditions (including severe weather), and with the fact that humans create the algorithms

Climate Change and Extreme Weather Risks

In addition to these many changes in technologies and processes related to weather information, changes in the weather itself must be considered. The report Attribution of Extreme Weather Events in the Context of Climate Change (NASEM, 2016a) reviews observed trends in different types of extreme weather events and assesses current scientific understanding of the degree to which those trends can be attributed to long-term climate change. This study also reviews the latest scientific thinking about the changes in extreme weather risks that can be expected for the coming decades, briefly summarized below²:

• It is expected that cold events should become less frequent and less severe as the climate warms, but it is possible for them to increase in frequency or intensity for periods of time due to increases in the intensity of cold air advection from polar to lower-latitude regions.
• It is very likely that heat waves (spells of days with temperature above a threshold determined from historical climatology) will occur with a higher frequency and longer duration.
• It is very likely that the frequency and intensity of heavy precipitation events over land will increase on average; however, this trend will not be apparent in all regions because of natural variability and possible influences of anthropogenic aerosols.
• It is suggested by theory that for the coldest climates, the occurrence of extreme snowfalls should increase with warming due to increasing atmospheric water vapor, while for warmer climates it should decrease due to decreased frequency of subfreezing temperatures.
• It is expected that tropical cyclones will become more intense and will have greater precipitation as the climate warms, but the global frequency of tropical cyclone formation is projected to decrease.

___________________

² Details, including the primary references for all of these conclusions, can be found in NASEM (2016a, Chapter 4).

• It is expected that coastal flood risk due to storm surge will increase due to both sea level rise and tropical cyclone intensity change.
• It is unclear how extratropical cyclones are affected by climate change because there are competing factors that could either weaken or strengthen them, and storm track positions could change location in the future.
• It is also unclear how severe convective activity, including tornados and hailstorms over the U.S. plains will be affected. Convective instability increases in a warming climate, but wind shear decreases, and changes in storms will depend on which of these dominates the other.

Role of SBS. A critical factor underlying how people react to hazardous weather warnings is their past experiences with such events. This dynamic poses challenges even in the context of normal temporal variability of weather hazards. But when historical weather patterns change such that a region receives more intense or more frequent extreme weather than in the past, the need for effective communication is escalated so that people are in a position to make response decisions based on accurate, up-to-date information and appropriate assumptions about the levels of risk they face.

2.3 THE CHALLENGES OF DEFINING SUCCESS AND QUANTIFYING THE VALUE OF SBS RESEARCH

Making the case for greater support of SBS-weather research would be much easier if it were possible to demonstrate a clear, simple “return on investment.” Although many examples of successful investments in SBS research can be found in the weather enterprise (see examples in Box 2.1), as well as in other realms (see examples in Box 2.2), it is often challenging to draw a direct line from specific investments in SBS studies that yield new insights to specific benefits and outcomes that are measurable and even quantifiable in economic terms.

As noted earlier, the stated NWS mission is to provide weather, water, and climate data, forecasts, and warnings for the protection of life and property and enhancement of the national economy. While clear in many respects, this mission involves some fundamental ambiguities, especially regarding how to evaluate progress in meeting this mission. For instance, it is often difficult to unambiguously measure lives lost due to hazardous weather events, since fatalities can result from a complex collection of individuals’ vulnerabilities, decisions and actions, and other factors (see Combs et al., 1999). It is likewise difficult to measure how well an individual or a population is “prepared” for a weather hazard or how preparedness relates to actual outcomes in terms of taking recommended actions and reducing risk. Furthermore, it is hard to scientifically prove

a negative—i.e., to prove that deaths and damage were avoided specifically because of preparedness efforts and timely warnings.

Weather forecasters and broadcasters have long debated whether their role is simply to provide information about weather risks or to persuade people to take specific actions in response to those risks. In contrast, this question is unambiguous in other parts of the weather enterprise, including FEMA, which directly aims to both inform sound preparedness and response actions and actively persuade people to take such actions. These differing views within the weather enterprise have both ethical implications and practical implications for how one defines and measures progress over time. SBS expertise can help with framing and working through these sorts of considerations. (See Box 2.3 for further discussion.)

Assessing and determining the value of improvements in a domain such as hazardous weather communication require consideration of the end-to-end system—from the

natural hazard itself, to the people, organizations, and technology providing support to those impacted. In each phase of a natural hazard (preparedness, observations, warnings, response, recovery, mitigation), the information that is collected and analyzed will differ. All the organizations involved are embedded in unique social and technical systems, and the individuals within these systems operate under different constraints with respect to time, resources, and knowledge. The people impacted by a weather phenomenon are operating in response to numerous considerations related to their own needs, abilities, and resources, as well as the needs of those they support. To measure and value the performance of such a complex system, one must consider the many technological, organizational, and human elements, and the interdependencies among all these elements.

Consider, for example, how one might evaluate and assign value to the outcomes of a hazardous weather warning. What may seem like a relatively simple measure of performance—compliance by taking protective action—poses a challenge in that it is often impossible to know why people fail to comply with recommended actions (e.g., Were they unaware of the recommendations? Did they willfully choose to ignore them? Were they aware but unable to take the recommended action? Did they have reason to believe that the recommendations did not apply to them or their situation?). People take a wide range of actions in response to a warning, and outcomes are often dependent on people’s ability to recognize and respond to the warning, as well as how they seek out information and utilize cues, which in turn impacts response. Also, from a scientific perspective, we typically lack natural benchmarks or control conditions that are necessary to put measurements in their proper perspective and context. These difficulties are a sharp contrast to the context of physical weather forecasts, which have direct, easy-to-measure metrics, such as absolute accuracy of temperature or precipitation forecasts, duration of extreme events, or warning window of approaching events. It is thus much easier to quantify forecast improvements relative to costs.

A recent analysis of the weather information “value chain” (Lazo, 2016) suggests several key points:

• that many research and operational programs justify themselves as providing benefits to society without actually measuring or even characterizing that value, or how the new products and services will be created, communicated, understood, or used;
• that valuation ultimately depends on the specific outcomes being evaluated (e.g., mortality/morbidity, reduced costs, reduced damages, increased profits, improved welfare); and

• that the value of weather information is ultimately a function of the ability of decision makers to receive, understand, and act on that information.

Economics research can help address some challenges of quantifying the value of weather information. In particular, much can be learned from economic evaluations undertaken in other realms that weigh the costs and benefits of investments in public safety. The Department of Transportation, for instance, regularly issues guidance on methodologies for valuing the reduction of fatalities and injuries due to safety regulations and investments (DOT, 2016). Yet, overly simplistic efforts to define metrics for assessing and quantifying the value of research outcomes might lead to picking the easiest quantities to measure, rather than the factors most relevant to societal well-being. Defining appropriate performance metrics is a crucial part of measuring the value of a research or operational program.

Similar challenges arise when attempting to identify “successful” research. Integration of social and behavioral science with the weather enterprise is a multifaceted process, and thus what constitutes success is also multifaceted. Some examples of different types of success that should be recognized and encouraged include the following:

• Success can be exposing people to new ideas through conferences, workshops, and other interactions. These venues provide opportunities for a dynamic, interactive exchange that can spur new ideas, problem framings, and collaborations. The agenda-setting, community program and capacity-building, and communication and information-sharing activities discussed in Section 3.1 are examples of efforts that have been successful in this regard. For instance, the Weather and Society * Integrated Studies (WAS*IS) workshops provided a mechanism for exposing people to new ideas and building a community of people interested in this interdisciplinary space. Another example comes from the Integrated Warning Team workshops that bring together forecasters, emergency managers, broadcast meteorologists, social scientists, and other core forecast information users (e.g., transportation managers).
• Success can be defining a new research question to pursue, applying a concept for studying in a new context, or developing a new methodological approach to a research problem—especially at the interface among disciplines. For example, a research team with expertise in geography, anthropology, and physics developed an interactive web-based simulation to evaluate how people dynamically seek and interpret information about hurricane risks (Meyer et al., 2013). A research team that included scientists with training in meteorology, policy analysis, risk communication, economics, and anthropology adapted a mental models research approach developed and used in risk

• communication studies (on a variety of topics ranging from radon to sexually transmitted diseases) to study flash flooding perceptions, understanding, decision making, and responses, and compare these among and between weather forecasters, broadcasters, emergency managers, and laypeople (Lazrus et al., 2016; Morss et al., 2015).

• Success can be development of new understanding about human cognition, behavior, and culture (at individual, group, organizational, or other levels) pertaining to weather. Numerous past studies have examined how people receive and respond to warnings (e.g., Mileti and Sorensen, 1990; Sorensen and Mileti, 2017a,b,c; other references in Section 3.1a). Such efforts for instance, have debunked the myth that people panic in hazardous weather situations, have revealed that people seek confirmation of warnings, and have shown the range of weather information sources that people have used over the years. Much recent research has also has focused on how people interpret, perceive, and respond to specific weather forecast and warning messages (e.g., Ash et al., 2014; Drost et al., 2015; Morss et al., 2016a,b; Perreault et al., 2014; Rickard et al., 2017; Sherman-Morris et al., 2015), as well as other factors such as the effect of people’s past weather experiences (Demuth et al., 2016), folk knowledge (Klockow et al., 2014), and people’s perceptions and attitudes (e.g., Weinstein et al., 2000). Because science is incremental and cumulative, such successes provide an important foundation; other researchers and practitioners can leverage this knowledge to develop new applications.
• Success can be development of a new product, display, tool, algorithm, or approach that is developed and transitioned for use in the operational environment. Some examples include the development and testing of perceptions and preferences for storm surge visualization products (Morrow et al., 2015) (see Box 2.1) and the development of improved ways to simply and visually communicate the type and timing of hazardous weather threats on the NWS point-and-click webpage (Demuth et al., 2013). The NWS’s Hazard Simplification Project is a current research-based effort to modify the types of and ways that hazardous weather information is communicated (NWS, 2017a).