2

Prioritize Core Capabilities

The entire weather, water, and climate enterprise is built on a foundation anchored by the core capabilities of the National Weather Service (NWS). These core capabilities include foundational datasets, essential functions, and operationally related research. Prioritization will allow the NWS to focus time and resources on those things it does best and are most needed. This chapter provides details and sub-recommendations in support of Recommendation I and as guidance to the NWS as it attempts to prioritize its core capabilities. Above all, the Committee feels that clear performance metrics and evaluation of the impact of scientific and technological advances on those performance metrics will best guide the NWS in prioritizing its investments.

Recommendation I: Prioritize Core Capabilities

The National Weather Service (NWS) should

1. Evaluate all aspects of its work that contribute to its foundational datasets, with the explicit goal of ensuring that those foundational datasets are of the highest quality and that improvements are driven by user needs and scientific advances. As part of this initial and ongoing evaluation effort, clear quality and performance metrics should be established. Such metrics would address the technical components of NWS operations, as well as the efficiency and effectiveness of the flow of weather information to end users.

2. Ensure that a similarly high priority is given to (a) product generation and dissemination, (b) the brokering and provision of data services, and (c) development and enhancement of analysis tools for maintaining a common operating picture (COP).

3. Engage the entire enterprise to develop and implement a national strategy for a systematic approach to research-to-operations and operations-to-research.

FOUNDATIONAL DATASETS

Foundational datasets include collected and integrated observations, advanced analyses either from modern data assimilation or other objective methods, and predictions obtained from deterministic and probabilistic models. As indicated in the first bullet of Recommendation I, evaluation of the NWS’s foundational datasets with the goal of improving their quality will necessitate unambiguous quality and performance metrics. These would include existing metrics such as probability of detection and lead time for severe weather (e.g., tornadoes, flash floods), hurricane track and intensity forecast skill, quantitative precipitation forecast skill, and hydrologic prediction forecast skill. For numerical weather prediction, such metrics include the traditional comparison of model performance at the 500 hPa level. Evaluation of these models will need to expand to include surface-level performance as well. False alarm ratios are currently included for most severe weather events, but the strict definition of this metric will need to be reevaluated to expand its usefulness. Finally, as the NWS moves away from deterministic forecasts toward probabilistic forecasts, metrics will need to move toward more appropriate measures of performance, such as reliability, calibration,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 17
2 Prioritize Core Capabilities T he entire weather, water, and climate enterprise (b) the brokering and provision of data services, and is built on a foundation anchored by the core (c) development and enhancement of analysis tools capabilities of the National Weather Service for maintaining a common operating picture (COP). (NWS). These core capabilities include foundational 3. Engage the entire enterprise to develop and datasets, essential functions, and operationally related implement a national strategy for a systematic approach research. Prioritization will allow the NWS to focus to research-to-operations and operations-to-research. time and resources on those things it does best and are most needed. This chapter provides details and sub- FOUNDATIONAL DATASETS recommendations in support of Recommendation I and as guidance to the NWS as it attempts to pri- Foundational datasets include collected and inte- oritize its core capabilities. Above all, the Committee grated observations, advanced analyses either from feels that clear performance metrics and evaluation of modern data assimilation or other objective methods, the impact of scientific and technological advances on and predictions obtained from deterministic and those performance metrics will best guide the NWS in probabilistic models. As indicated in the first bullet of prioritizing its investments. Recommendation I, evaluation of the NWS's founda- tional datasets with the goal of improving their quality Recommendation I: Prioritize Core Capabilities will necessitate unambiguous quality and performance metrics. These would include existing metrics such The National Weather Service (NWS) should as probability of detection and lead time for severe weather (e.g., tornadoes, flash floods), hurricane track 1. Evaluate all aspects of its work that contribute and intensity forecast skill, quantitative precipitation to its foundational datasets, with the explicit goal of forecast skill, and hydrologic prediction forecast skill. ensuring that those foundational datasets are of the For numerical weather prediction, such metrics include highest quality and that improvements are driven the traditional comparison of model performance at by user needs and scientific advances. As part of this the 500 hPa level. Evaluation of these models will initial and ongoing evaluation effort, clear quality need to expand to include surface-level performance and performance metrics should be established. Such as well. False alarm ratios are currently included for metrics would address the technical components of most severe weather events, but the strict definition NWS operations, as well as the efficiency and effec- of this metric will need to be reevaluated to expand tiveness of the flow of weather information to end its usefulness. Finally, as the NWS moves away from users. deterministic forecasts toward probabilistic forecasts, 2. Ensure that a similarly high priority is metrics will need to move toward more appropriate given to (a) product generation and dissemination, measures of performance, such as reliability, calibration, 17

OCR for page 17
18 WEATHER SERVICES FOR THE NATION and sharpness. The NWS will also need to develop a example of success in the area of computing technol- systematic method for evaluating the effectiveness of ogy has been the leasing of research and operational communicating warnings to the public. high-performance computing assets. Leasing allows for more rapid updating of computing infrastructure and Maintaining Infrastructure Through Technology prevents the procurement of out-of-date technology. Infusion The NWS is only one part of NOAA that feeds requirements to, and utilizes data from, the NOAA Keeping up with advances in technology requires satellite systems. Therefore, processes for technol- a continual NWS program of technology infusion. ogy infusion related to the satellite programs need to Areas to be covered by such a program include major involve NOAA management, NESDIS, and other communications and processing systems, satellites and branches, along with the NWS. It is also important to ground processing, and ground-based radar, sensor, note that a "satellite system" typically consists of mul- and gauge networks. tiple segments, or elements, including space vehicle; In each of these areas, a well-organized program instruments; launch vehicle; command, control, and of technology infusion would provide a means for communications (C3); and data processing hardware the NWS to avoid becoming obsolete and requiring and software. Costs for each of these elements can be another massive overhaul like the MAR. This is con- substantial, as can costs for development of the science sistent with Lesson 1 of NRC (2012a). Lesson 2 of that versions of the data processing algorithms and program report found that such a technology program requires management. (These last two elements can include established systems engineering processes, including a large portion of government personnel.) Each of setting system-level requirements and performance the system elements alone can be quite complex and metrics for evaluating progress toward meeting those therefore costly. These disparate elements of the sys- requirements. The capability for development and tem need to be specified, acquired, built, and activated testing, and a process for rapid field-testing of proto- synergistically as they work together to provide data type systems, would also be an important part of the and products. Exercising a well-established systems program and is discussed in further detail later in this engineering process is therefore critical both for exist- chapter. ing data observation and product continuity and for In the computing area, five-year-old hardware is planned technology infusion. obsolete in today's world of rapidly advancing tech- NOAA has a relatively well-established approach nology. Thus, planning and budgeting for replace- to the acquisition of its satellite system elements that ment needs to begin as soon as a new generation has typically includes a process for developing instrument been deployed, if not sooner. For example, during the requirements, study and risk reduction phases involving MAR, the information technology (IT) systems for preliminary designs, and possibly early component- both the Advanced Weather Interactive Processing builds from competitors.1 These phases of the program System (AWIPS) and Next Generation Weather Radar include both space and ground segment elements that (NEXRAD) were upgraded, in some cases before the apply to spacecraft, instruments, ground algorithms, systems were fully commissioned. These upgrades and processing. NOAA also typically requests detailed were developed through prototyping and involved interaction with the research community and the 1 This assessment applies primarily to NOAA's Geostationary contractor. The AWIPS-II program is an example Operational Environmental Satellite (GOES) and Polar Operational Environmental Satellite (POES) series. As noted in the Committee's of a computing upgrade that addresses the need for first report, the National Polar-orbiting Operational Environmental systems refreshment. However, AWIPS-II is also an Satellite System (NPOESS) program, while following a similar example of a failure to draw upon what was learned procurement process, was not managed directly by NOAA but by a tri-agency Integrated Program Office, of which NOAA was during the development and deployment of the major a member (GAO, 1995). NOAA, the NWS, the other agencies systems of the MAR. Rather than making continual involved, and the enterprise as a whole have been affected by the upgrades, all upgrades were stopped almost ten years issues encountered by the NPOESS procurement approach. The full ago in anticipation of the large AWIPS-II upgrade. An range of lessons to be learned from that approach is beyond the scope of this report.

OCR for page 17
PRIORITIZE CORE CAPABILITIES 19 life-cycle cost estimates for elements of various system products and research mission and products. Both architectures. With these standard procurement pro- aspects are critical to NOAA and to the nation. But cesses, the government can obtain a "rack and stack" of the distinction needs to be made very clearly. An an assortment of sizes and designs of spacecraft, instru- operational mission and the observations required to ment designs, and data processing approaches--with accomplish that mission are presumably thus identi- the associated cost, risk, performance, and schedule fied because of their very criticality to the nation and metrics. With respect to data products, NOAA has, its more immediate security and well-being. A well- on various programs, endeavored to group the products planned research program and the associated require- into categories that indicate a prioritization. These ments and products serve a different purpose; indeed, techniques all show recognition of the importance of it is the cornerstone of technology improvement and the systems engineering process, including top-level infusion. Operations and research are synergistic, requirements development. but the goals and metrics of each need to be clearly Infusion of new technology into the NOAA satel- stated and differentiated. NOAA's current approach lite programs requires an even more rigorous approach is to include a select group of lower-priority "product to developing a clearly defined and recognized process. improvement" requirements along with the higher- At the least, because the satellite system elements can priority product requirements. Although this allows be so costly and time-consuming, each planned infu- for some research infusion, perhaps a clearer distinc- sion of new technology requires metrics that illustrate tion needs to be made, if possible, with a separate, the potential benefit. NOAA has at times included well-recognized national approach to research satellite in the requirement set "pre-planned improvements" missions that feed specifically into NOAA operational grouped in a lower-priority category. This is a good way missions. In this way, there is a synergistic process yet to encourage competitors to strive for new approaches a clear distinction between missions. in design while keeping the primary focus on higher- This has been a part of the process in prior years: priority requirements. an example is the Operational Satellite Improvement It would serve NOAA to make the connection Program (OSIP), a NOAA-National Aeronautics and between specific products and metrics that illustrate Space Administration (NASA) agreement in effect the importance of these products even more clear. The from 1973 to 1981, in which the results from the products are linked to observations and thus instru- NASA research missions were fed into requirements ment and other element designs as well as processing development for the NOAA operational missions algorithms. Although the trade-offs needed to develop (GAO, 1997). It should also be noted that part of the some of these metrics can be difficult and costly, they original intent of the National Polar-orbiting Opera- can be useful for directing resources most efficiently and tional Environmental Satellite System (NPOESS) Pre- potentially eliminating some less-critical observations. paratory Project (NPP) was to facilitate risk reduction As an example, methods for determining impacts of for NPOESS in areas including sensors, algorithms, various measurements on numerical weather predic- and ground processing. NPP was not intended to be tion are described later in this chapter. NOAA pro- an operational mission itself. NOAA, along with the vided a good visual example of impact metrics when other members of the tri-agency NPOESS Integrated it compared the "Snowmageddon" weather forecast Program Office, embraced this approach. with and without the polar afternoon orbit satellite In the area of radar observations, the just-being- data ( Sullivan, 2011). In this case, however, no distinc- completed NEXRAD polarimetric upgrade is bringing tion was made as to which instruments on the polar new and valuable capabilities to the field. At the same platform made the most critical difference. A further time, the basic NEXRAD design is nearly 25 years old, level of detail would be needed to adequately prioritize and the other major apparatus (apart from data and among instruments or instrument designs. signal processing equipment) is more than 15 years old. Another very important distinction to be made in Given the long development, design, procurement, and technology infusion (and indeed in general product deployment cycle for such a large system, as well as the prioritization) is between the operational mission and need for multiagency requirements development and

OCR for page 17
20 WEATHER SERVICES FOR THE NATION approval, the time is already here to plan for the next age is insufficient, such as urban areas or mountainous generation of radar surveillance equipment. regions. Various avenues are open. The simplest and per- In addition to hardware platforms, the NWS will haps least costly option (at least in the short run) also need to address technique and algorithm develop- would be a Service Life Extension Program (SLEP). ment, which will be critical to maximizing the use of Assuming regular replacement and upgrading of signal radar data. In the past, the development and evolution and data processing facilities and data display equip- of data processing algorithms has been done in col- ment, the major NEXRAD components that would laboration with the National Severe Storms Laboratory need SLEP attention are electromechanical--such (NSSL), the Department of Defense (DOD), and the things as motors, gears, and bearings. A second, more FAA. In order to avoid a large and expensive upgrade, expensive, option would involve wholesale replace- as noted in Lesson 1 of the Committee's first report, ment of the NEXRAD radars with new equipment of incremental, collaborative advances in data algorithms similar design. A third option is to collaborate with the will need to continue. Federal Aviation Administration (FAA) (and perhaps For surface observations, the NWS currently relies other agencies) in the development and deployment on the joint NWS/FAA Automated Surface Observ- of a multifunction surveillance radar system. Work ing System (ASOS) network. As noted in this Com- toward that goal is ongoing, with emphasis on phased mittee's first report, the ASOS network was designed array radar (PAR) technology. At this point, however, primarily to support aviation needs. Because of issues the ability of a PAR system to provide NEXRAD- with ASOS sensor performance many scientists have like polarimetric capabilities has not been established. developed their own networks for surface observing Moreover, the likely procurement and life-cycle costs needs (NRC, 2012a). It has become commonplace that of such a system are not yet known. Another potential surface observations from private or public mesoscale obstacle lies in the frequency allocations required for networks (mesonets) are utilized in conjunction with, such a PAR system. According to current design con- or instead of, data from ASOS. Mesonets have proven cepts, the bandwidth required is substantially greater to be far more agile in terms of evolving sensor suites than that of the NEXRAD system, but there is already than has the ASOS network. The sensor suite and the great pressure to dedicate more of the S-band spectrum spatial and temporal resolution of ASOS data have seen to communications applications. little improvement since the end of the MAR. Despite Any of the preceding options leaves unresolved the relative maturity of surface observing technology, the issue of comprehensive low-level coverage. Good critical gaps remain. Land-surface properties important low-level coverage is important for many severe-storm to numerical weather prediction, particularly soil mois- warning and precipitation-estimation applications as ture, lack the necessary spatial resolution (NRC, 2009). well as in complex-terrain situations. Dealing with such Urban areas, mountains, and coastal zones all present issues would require radars that are more closely spaced, unique observing and forecasting challenges. A recent particularly in regions of mountainous terrain, which NRC report addressed the unique needs of the urban would have to be more numerous than the NEXRAD environment (NRC, 2012b). As the NWS expands the radars but could be smaller and less expensive (because range of users it serves under the Weather-Ready Nation of the overlapping coverage). Here the Collaborative paradigm, it will be necessary to consider whether the Adaptive Sensing of the Atmosphere (CASA) expe- surface observation data collected by the ASOS network rience demonstrates much of the needed capability is sufficient to meet user needs or whether other surface (McLaughlin et al., 2009). Here again, however, the observing systems would provide the needed agility. procurement, deployment, and life-cycle costs of a Similar challenges exist with respect to hydrologic practical system are not yet known. It may well be observation and forecasting activities. Presently, the that benefit-cost considerations would favor a hybrid observational infrastructure for hydrologic forecasting system, including a mix of NEXRAD-like surveillance is fragmented across such different agencies as the U.S. capabilities with smaller local networks of CASA-like Geological Survey (USGS), the Natural Resources radars to deal with areas where the NEXRAD cover- Conservation Service, NASA, and the NWS, in addi-

OCR for page 17
PRIORITIZE CORE CAPABILITIES 21 tion to various state and regional entities. The lack model4; the global Real Time Ocean Forecast System of standards between these networks combined with (RTOFS)5 (based on the Hybrid Coordinate Ocean nonuniform acquisition quality control standards for Model [HYCOM]6); hurricane models (Hurricane merging these datasets translates into significant prob- Weather Research and Forecasting [HWRF]7, Geo- ability that use of existing observations is suboptimal. physical Fluid Dynamics Laboratory [GFDL]8); air Such new technologies as multi-scale estimates of soil quality applications and models (Hybrid Single Particle moisture or skin temperature or satellite altimetry of Lagrangian Integrated Trajectory [HYSPLIT]9); avia- river and reservoir levels are not easily or readily being tion applications; and many specialized products. The integrated into the existing data assimilation and numerical model used by the medium-range ensemble forecasting workflows. Finally, a preliminary plan was system is the GFS. Additionally, the GFS code and presented to the Committee on how to address these development are closely linked with the Climate Fore- issues, though these topics were raised in discussion cast System (CFS). It follows that the performance of related to the development of the new National Water the GFS has a crucial impact on many downstream Center being constructed in Tuscaloosa, Alabama. A models and products. thorough, integrated assessment of observational, data Performance in numerical weather prediction at assimilation, and data management needs for hydro- the National Centers for Environmental Prediction logic prediction activities would provide much-needed (NCEP), including medium-range modeling, has been guidance to address current challenges in developing steadily improving and NCEP is widely considered one and adopting the hydrologic prediction components of the world leaders in weather and climate prediction. of the NWS (these challenges are elaborated on in However, global forecast models run by several other Chapter 3). national centers consistently outperform NCEP by established metrics of numerical modeling skill. The Recommendation I.a performance of the GFS was partially addressed in the Committee's first report (Finding 4-4) and is discussed The National Weather Service (NWS) should con- in more detail in this section. tinue technology infusion programs that have been One method to assess the performance of the NWS effective subsequent to the Modernization and global medium-range forecast model is to compare Associated Restructuring. Parallel support from its accuracy to that of model-based forecasts made the National Environmental Satellite, Data, and by other operational weather centers of the state of Information Service (NESDIS) is needed to provide the atmosphere at approximately 18,000 ft (500 hPa). continuing upgrade of satellite capabilities. Such This metric is often considered the gold standard for infusion programs should include both hardware and medium-range prediction models because the 18,000-ft software development. level is a dynamically significant region near the vertical midpoint of the troposphere that contains signatures Numerical Weather Prediction of weather systems from the surface up through the jet stream aloft. Thus, this integrative metric is widely Numerical weather prediction guidance affects used for evaluation of the overall model performance. nearly every facet of the NWS's mission. The NOAA Many centers have a long history of this metric that Global Forecast System (GFS)2 is one of the center- tracks the model performance and improvements over pieces of the NWS modeling enterprise. Not only is the the years, and this is one of the WMO standard statis- GFS a key model used for short- and medium-range tics for model inter-comparisons. Weather prediction forecasting, it is also used to initialize or provide bound- ary conditions for many downstream models, applica- 4 http://polar.ncep.noaa.gov/waves/index2.shtml tions, and products. Those include the North American 5 http://polar.ncep.noaa.gov/ofs/ Mesoscale Model (NAM)3; the Wave Watch III wave 6 http://hycom.org/ 7 http://www.emc.ncep.noaa.gov/index.php?branch=HWRF 2 http://www.srh.noaa.gov/ssd/nwpmodel/html/gfs.htm 8 http://www.gfdl.noaa.gov/hurricane-portal 3 http://www.srh.noaa.gov/ssd/nwpmodel/html/nam.htm 9 http://www.arl.noaa.gov/HYSPLIT_info.php

OCR for page 17
22 WEATHER SERVICES FOR THE NATION centers evaluate many other metrics as well (such as precipitation, low-level winds, tropical cyclone tracks and intensity, and surface pressure). Figure 2.1 compares the 5-day forecast per- formance of several operational centers, includ- ing the NCEP GFS, for the time period 1996 through the present, averaged over the Northern and Southern Hemispheres. The leading models, including the GFS, have exhibited steadily increasing skill over the past 15 years. However, the European Centre for Medium- range Weather Forecasts (ECMWF) and more recently the UK Meteorological Office (UKMO) consistently outperform the GFS (and all other operational global medium-range forecast models). Additionally, it is apparent that the GFS has not closed the gap in terms of forecast skill with ECMWF over the past decade or more. Environment Canada and Japan have also shown significantly positive trends in predictive skill over the past decade. Systematic comparison using metrics other than overall accuracy at 500 hPa, particularly at the surface, is more limited. However, the conclusions are similar. Wedam et al. (2009) compared surface forecasts of sea level pressure along the East and West Coasts of the United States during the winters of 2005 through 2008. On average, the NCEP errors were 26 percent greater than those of the ECMWF. Froude et al. (2007) and Froude (2010) compared the performance of the NCEP and ECMWF ensemble forecasts in forecast- ing extratropical cyclones in the Northern Hemisphere. Again, the ECMWF consistently produced better forecasts than did NCEP. The Committee believes that a dedicated, com- FIGURE 2.1Five-day forecast 500 hPa anomaly correla- munity R2O effort, based on lessons from other cen- tion (top panels) for different forecast models (NCEP's Global Figure 2-1 ters, would aid the NWS in improving its numerical Forecast System [GFS], European Center for Medium-range Weather Forecasting Bitmapped [ECM in figure legend], UK Meteorological weather prediction skill (for example, the efforts of Office [UKM in figure legend], Fleet Numerical Meteorology and the Navy as described later in this chapter). Addition- Oceanography Center [FNO], the baseline Coordinated Data ally, the UKMO and ECMWF systems have more Analysis System [CDAS], and Canadian Meteorological Centre advanced data assimilation systems than the current [CMC]) from 1996 to 2012 for the Northern (top) and Southern NWS GFS has, which may partially explain some of (bottom) Hemispheres. A higher anomaly correlation indicates better model forecast performance. The differences with respect the differences in the performance of the systems. For to the GFS system are shown along the bottom panels, with example, both the UKMO and the ECMWF make positive values indicative of higher forecast skill than the GFS. use of an advanced data assimilation method, namely, SOURCE: National Centers for Environmental Prediction. four-dimensional variational (4DVar) assimilation. And both the UKMO and the ECMWF have made various improvements to their 4DVar systems, includ- ing recent advances to achieve a hybrid assimilation

OCR for page 17
PRIORITIZE CORE CAPABILITIES 23 capability using synergistic variational and ensemble and software infrastructure efforts that are needed to approaches. Recently, NCEP has also pursued a hybrid maintain multiple numerical modeling systems. A new data assimilation approach and implemented a major generation of numerical techniques and dynamical upgrade in May 2012 to their three-dimensional cores that are sufficiently flexible to address the mul- variational (3DVar) system using a hybrid approach, tiple needs of the NWS ranging from high-resolution which is intended to improve the overall performance limited areas to global weather and climate applications (Lapenta, W., NWS Environmental Modeling Center, are rapidly maturing and becoming available, suggest- personal communication to member of the Commit- ing the time is right to consider such a unified model tee). However, NCEP can further improve the GFS approach. The use of cloud computing technology through implementing existing data assimilation could help the NWS acquire computational flexibility technology, such as 4DVar, that is already being used to address changing priorities, and possibly reduce at other leading centers. computing costs. Given the central role of the NCEP GFS, improve- New community efforts are now emerging that ments made to its skill translate to improved products will attempt to unify research and operational predic- and performance in downstream models. An example tion systems and capabilities across a number of U.S. of the relationship between the GFS performance and agencies. An inclusive and holistic approach (uniting a downstream model is seen in the performance of nations, agencies, and disciplines) to Earth system the HWRF modeling system. It is estimated that the science and numerical prediction is essential to realize forthcoming improved hybrid (3DVar/ensemble) data much-needed major advances in observations, analy- assimilation system in GFS alone will lead to a signifi- sis, data assimilation, and prediction of high-impact cant improvement in the track prediction skill of the weather and climate (Brunet et al., 2010; Shapiro et al., HWRF, based on a large sample of cases from the past 2010; Shukla et al., 2009, 2010). Some of the challenges several years (Tallapragada, V., NWS Environmental include infrastructure issues related to national com- Modeling Center, personal communication to member putational resources, especially operational computing of the Committee). capacity, required for high-resolution weather and Another major difference between the leading climate forecasting; the importance of collaboration operational center modeling systems and the GFS is between the weather and climate research communi- the horizontal resolution in the models. The ECMWF ties for advances in seamless prediction; and improved uses a horizontal resolution of 17 km, while the GFS end-user products produced by forecast systems. The uses 27 km. Increases in resolution need to remain a Earth System Prediction Capability (ESPC) represents high priority not only for the GFS but for downstream the national response to this need and many challenges. models as well. The International Council for Science (ICSU) and Trade-offs might be considered between the num- international partners, including the Belmont Forum ber of numerical models being currently run by NCEP, of funding agencies (the National Science Foundation the frequency with which these models are run, and is a lead member), have developed a new interna- the horizontal resolution in the models. Fewer models tional initiative--Future Earth: Research for Global (either through consolidation or replacement with Sustainability--and this demonstrates the importance unified models that can address multiple scales and of linking improved predictions of weather and climate applications) being executed less frequently (within with a broad range of societal issues such as food, water, requirement limits) would allow for higher resolu- energy, health, and human security. tion to be achieved, as well as more sophisticated data The Committee notes that improving NWS's assimilation and physical parameterization approaches numerical weather prediction capabilities will require a to be used. Furthermore, within the NWS the concept systematic approach to transitioning research to opera- of a unified modeling system--a single model for local, tions. In designing such an approach, useful lessons can regional, global, and climate scales, which has been be drawn from the success of the R2O systems of the embraced by the UKMO--needs to be considered. Navy and ECMWF programs. Such a system minimizes duplicative developmental

OCR for page 17
24 WEATHER SERVICES FOR THE NATION Recommendation I.b Research Laboratory, and Gelaro, R., NASA, personal communication to member of the Committee; Errico, The National Weather Service (NWS) global and 2007; Gelaro et al., 2007). These methods can pro- regional numerical weather prediction systems vide detailed information about the impact of various should be of the highest quality and accuracy, with observations and can help to identify problems with improvements driven by user needs and scientific any given observation or the way that observation is advances. To achieve this goal, the NWS should give assimilated. One caveat is that the estimate is subject priority to upgrading its data assimilation system to assumptions and limitations inherent in the use and increasing the resolution of its deterministic and of adjoint models (i.e., the so-called tangent linear ensemble modeling systems. The product develop- assumption). The adjoint observation impact method ment process can be improved by developing a sys- can quantify information related to the observation tematic approach for research-to-operations through type and location. Similar observation impact informa- collaboration with users and partners in the entire tion can be obtained from ensemble-based approaches weather, water, and climate enterprise, both in the as well. United States and around the world. In the last decade, adjoints for the modeling and data assimilation systems have been developed and Current and Future Observing Needs for applied at many centers around the world, including Numerical Weather Prediction Models NASA, the U.S. Navy, ECMWF, UKMO, Meteo- France, and Environment Canada. NCEP has closely To better anticipate future observing needs and collaborated with NASA to develop an adjoint for the requirements, it is beneficial to consider the impact data assimilation systems because they share similar of the various current generation observation types on data assimilation methodology. However, it would be numerical weather forecasts. Furthermore, analysis of useful for the NWS to develop an adjoint for the pre- the relative value of observation systems can be use- diction systems as well, such as the GFS, to enhance ful for prioritizing investments. A plethora of surface, the monitoring and guidance of the impact of obser- aircraft, radiosonde, satellite, and radar observations vations on their own Numerical Weather Prediction are used to initialize numerical weather prediction (NWP) systems (and downstream forecast systems). models. The impacts of these observations on numeri- The observation impact tool can be especially valuable cal weather forecasts have been quantified traditionally for the evaluation of hyper-spectral satellite sounders through observing system experiments (OSEs), in (with thousands of channels) since the methodology which observations are removed from (or added to) a provides quantitative guidance on the selection of data assimilation system and the resulting forecasts are which channels to assimilate (given one only has suf- compared to a control set of forecasts. Such OSEs can ficient resources to assimilate a fraction of them). provide an indication of a gross impact of observations An example of the daily average observation on forecasts. However, OSEs are expensive because of impacts for January 2007 is shown in Figure 2.2. The the number of experiments required to test the large results compare the observation impact on analysis and number of observation types, including such things as prediction systems from the NASA (GEOS-5), U.S. multiple individual sensing channels on satellite sound- Navy (NOGAPS), and the ECMWF (EC-MSGFS) ing instruments. operational systems. The Advanced Microwave Sound- New approaches have recently been developed ing Unit (AMSU-A), radiosondes, satellite winds, and based on adjoint10 sensitivities through the linking aircraft observations all have the largest impact in all of the adjoint for the data assimilation and model- of the systems for 24-hour forecasts using a synoptic- ing systems (Langland, R.H., and N.L. Baker, Naval scale metric. One can also compare the impact per observation 10 The adjoint--the transpose of the forecast model's forward that reduces the forecast error. On a per observa- tangent propagator--provides a particular forecast output's tion basis, some of the surface observations such as sensitivity to initial state changes in a mathematically rigorous and ships and land surface data become more important. computationally feasible manner.

OCR for page 17
PRIORITIZE CORE CAPABILITIES 25 FIGURE 2.2 Daily average impacts of various observation types on 24-hour forecasts from January 2007 in U.S. Navy NOGAPS (top left), NASA GEOS-5 (top right), and ECMWF GDPS (bottom left). The results are presented as an energy-based measure of forecast error, and the units are J kg-1. Larger negative numbers imply that the observation types are reducing the forecast error. The observation types include ships and buoys (Ship), geostationary satellite winds (Satwind), radiosondes and dropsondes (Raob), QuikSCAT winds (QSCAT), MODIS satellite winds (MODIS), land surface-pressure observations (Land), commercial aircraft (Aircraft), and AMSU-A radiances (AMSU-A). Results are also shown for SSM/I wind speeds (SSMIspd) in NOGAPS and GEOS-5 as opposed to profiler winds (Profiler) in GDPS, but both those observations types have the smaller overall impact in the respective forecast systems. In all systems, the AMSU-A, radiosondes, satellite winds, and aircraft observations have the largest impact on the forecast. SOURCE: Gelaro et al. (2010). American Meteorological Society. Reprinted with permission. The impact of individual satellite channels can be Radiosondes provide direct measurements of tem- assessed as well. perature, moisture, and winds, in contrast to satellites Several key aspects are emerging from these studies that typically provide vertically layered radiance infor- that are important for the future directions of observing mation that is often more difficult to assimilate. NWP systems used for NWP models. The radiosondes are models likely suffer from a lack of accurate wind data, still important, and degradation to the global radio- particularly in the troposphere. Cloud track winds are sonde network could be detrimental to synoptic-scale typically one of the most important satellite observa- and mesoscale forecasts. There are large differences tions for NWP models; however, their positive impact in NWP analyses where models assimilate radiances may be mitigated by inaccurate estimates of cloud from satellite sensors; the differences are small where height assignments. Radiances are now available from they assimilate radiosonde data (Langland, R.H., and sensors on several different satellites (AMSU-A, IASI, N.L. Baker, Naval Research Laboratory, personal AIRS) and the data may currently be somewhat redun- communication to member of the Committee). While dant from a modeling perspective (but may not be in radiosondes are essential and necessary to anchor the the future as some key satellites expire). Although NWP satellite observing systems, the number of daily radio- models are thought to be especially sensitive to moisture sonde observations has decreased during the same observations, these observations are often more limited time period that the skill of global forecast models and very difficult to use. For example, water vapor chan- has increased. Further analysis is needed to determine nels from satellite infrared sounders are quite challeng- how many radiosondes are needed and how they can ing to use in current data assimilation systems. be optimally distributed. The results of such an analysis One major caveat is that most of these data impact could differ based on whether the goal is improving the studies have been made with medium-range prediction skill of regional or global forecasts. models, and the results may not necessarily translate

OCR for page 17
26 WEATHER SERVICES FOR THE NATION to the mesoscale and high-impact severe weather. allowing for expansive data coverage and a large payload Moisture and boundary layer wind field observations for meteorological instruments. The NASA Hurricane are likely to be even more important on the mesoscale. and Severe Storm Sentinel (HS3) is a five-year mission Accurate tropospheric wind measurement is con- that will use two NASA Global Hawks to investigate sidered one of the highest observation priorities to processes that lead to hurricane formation and intensity improve weather prediction and climate (Hays et al., change in the Atlantic Ocean basin. One Global Hawk 2005). One critical future observing system is satellite will have an instrument suite designed to observe the LIDAR (LIght Detection and Ranging); LIDAR- environment around the hurricane, and the other will derived winds would fulfill a vital national need for have instruments aimed at measuring the inner-core high-resolution global tropospheric wind observation structure and processes. Field measurements will take and will lead to improved long-range weather forecast- place for one month each during the hurricane seasons ing and more accurate hurricane landfall prediction of 2012 to 2014 and will be an excellent opportunity to (e.g., Baker et al., 1995; Hays et al., 2005). Currently assess the capabilities of such systems. available radiosonde wind measurements are sparse. The Atmospheric Dynamics Mission (ADM-Aeolus) Recommendation I.c of the European Space Agency (Stoffelen et al., 2005), set to be launched in 2013, will provide global observa- To increase the capability of its numerical weather tions of vertical wind profiles with the aim of demon- prediction systems to keep up with technological strating improvement in atmospheric wind analyses for advances and prioritize investments in data assimila- NWP and climate studies. tion and observations systems, the National Weather Adaptive or targeted observations have been Service (NWS) should develop and advance soft- explored for applications such as tropical cyclones ware tools to monitor the impact of observations (Harnisch and Weissmann, 2010; Weissmann et al., on numerical weather prediction and downstream 2010, 2012) and winter storms (Szunyogh et al., 2000). forecast systems. Targeted observations refer to data collected in spe- cific areas at specific times with the aim of improving Probabilistic Forecasting the quality of pre-selected NWP forecast aspects or metrics. The NOAA Winter Storm Reconnaissance There is now a wide consensus that assessing and (WSR) program (Szunyogh et al., 2000) goal is to communicating uncertainty needs to be considered reduce forecast errors for significant winter weather as an essential part of weather and water forecasting. events over the contiguous United States and Alaska in This has been expressed in a range of reports from the the one- to four-day forecast lead time through the use National Research Council and the American Meteo- of adaptive observations over the data-sparse northeast rological Society (AMS, 2008; Hirschberg et al., 2011; Pacific. Targeting by satellite observations (e.g., using NRC, 2006, 2010). rapid scan capabilities selectively) can be very effective Probabilistic weather and water forecasts ben- and might be considered by the NWS as an important efit many areas of society. Dutton (2002) estimated priority. that more than $3 trillion in annual private industry Additionally, the emergence of Unmanned Aircraft activities in the United States is subject to weather- Systems (UAS) will provide an unprecedented future related risk. Probabilistic forecasting allows for opti- opportunity for targeted observations and an observing mal decision making in a wide range of applications and monitoring network that is adaptive to meet NWS (Krzysztofowicz, 2001; Palmer, 2002; Zhu et al., 2002). needs (based on weather or end-user requirements), For example, extreme low temperatures, extreme pre- further providing agility. These UAS have a variety of cipitation, or high winds can force the transportation capabilities that are amenable to a multitude of meteo- industry to cancel flights or reroute ships and can cause rological situations and applications, including collec- authorities to salt roads or clear snow. In mountainous tion of vital offshore data. A potential UAS platform of regions, sudden heavy localized precipitation can lead interest is the Global Hawk, which has a long duration to flash floods. Locally accurate forecasts are also an

OCR for page 17
PRIORITIZE CORE CAPABILITIES 27 important tool in agricultural planning, such as to take the case for the surface weather quantities that are most measures to avoid damage to crops when there is a risk important for practical forecasting. In their 2011 report of low temperatures, or to avoid unnecessary irrigation on forecast uncertainty, the AMS Board on Enterprise when sufficient precipitation is anticipated (Katz and Communication summarized the situation this way: Murphy, 1997; Stern and Coe, 1982). "Current-generation ensemble prediction systems pro- The most widespread information about forecast duce uncertainty forecasts that are biased and under- uncertainty takes the form of probabilistic weather estimate the forecast uncertainty (i.e., underdispersion forecasts. These forecasts provide probabilities of future of the ensemble members collectively). This is partly weather events (such as probability of precipitation) because of the low resolution of the forecast models, and probability distributions of future weather quanti- partly because of improper initial conditions, and ties (such as temperature or amount of precipitation). partly because the ensemble prediction systems do not Probabilistic weather forecasts have been shown to include effective treatments for the error introduced by improve weather-related decisions, to increase trust model deficiencies" (Hirschberg et al., 2011). Possibly in the forecast, and to reduce the effects of forecast as a result, the NWS and most other national weather error ( Joslyn and LeClerc, 2012). They are particularly agencies do not yet routinely issue public probabilistic important for forecasting in very-high-impact weather forecasts of most meteorological and hydrologic vari- situations (NRC, 2010). ables, 20 years after ensemble forecasts of such variables The NWS has regularly issued probability of pre- started being produced on a regular basis. cipitation forecasts for about 40 years, much longer than Over the past decade or so, there has been intense most other national weather agencies. There is some research on the development of methods for produc- evidence that, perhaps as a result, the American public ing calibrated probabilistic forecasts based on forecast is used to probabilities in the forecast and reacts better ensembles. These use various kinds of statistical post- to such information than does the public in some other processing, and vary according to the quantity being countries (Gigerenzer et al., 2005). However, the NWS forecast. The research on these methods is now fairly does not routinely issue probabilistic forecasts of other mature, but they have not yet been widely integrated weather quantities of public interest, such as surface into operational forecasting. temperature, amount of precipitation, or wind speeds. The most straightforward variable to forecast The dominant approach to probabilistic forecasting probabilistically is temperature, for which probabilistic uses ensemble forecasts. Ensemble forecasts consist of forecasting methods include the rank histogram adjust- multiple runs of one or more numerical weather pre- ment method (Hamill and Colucci, 1997), the best diction models, varying the initial conditions and/or member dressing method (Roulston and Smith, 2003), model physics (Palmer, 2002). The NWS has produced Bayesian model averaging (BMA; Raftery et al., 2005), ensemble numerical weather forecasts since December and ensemble model output statistics, also called non- 1992 (Toth and Kalnay, 1993). Ensemble forecasts homogeneous regression (Gneiting et al., 2005). often exhibit a spread-error correlation, in which the Probabilistic forecasting of quantitative precipita- spread of the ensemble is correlated with the absolute tion is more difficult, because precipitation has a high error of the forecast compared to the verifying observa- probability of being zero, and because the observed tion (Buizza et al., 2005). distribution of the amount of precipitation is typically Nevertheless, most ensemble forecasts under highly skewed. The probabilities of exceeding specific estimate the size of the forecast errors and hence are thresholds can be found by model output statistics not calibrated (Buizza et al., 2005).11 This is especially (MOS; Glahn and Lowry, 1972). More recent research has shown that logistic regression gives better results for quantitative precipitation than do the linear regres- 11 A probabilistic forecast is said to be calibrated when events sion methods on which MOS is based (Applequist forecast to happen with probability x% actually happen x% of the time on average. For example, a probabilistic forecasting system that et al., 2002; Hamill et al., 2004). A full predictive issues forecasts of the probability of freezing is calibrated if, of the probability distribution of the amount of precipitation times when it forecasts freezing with probability 20%, it actually can be found by rank histogram adjustment (Hamill freezes on average about 20% of the time.

OCR for page 17
28 WEATHER SERVICES FOR THE NATION and Colucci, 1998), BMA (Sloughter et al., 2007), or forecasts of temperature and precipitation for the extended logistic regression (Wilks, 2009). Pacific Northwest was designed using results from cog- For wind speed, quantiles can be found by quantile nitive research (Mass et al., 2009). An earlier version of regression (Bremnes, 2004). One major reason for the the website was cited as an example of how probabilistic probabilistic forecasting of wind speeds is the manage- forecasting could be done (NRC, 2006). ment of wind energy systems, and for this, as for other Overall, the research of the past decade on proba- applications calling for a cost-loss analysis, a full predic- bilistic forecasting has three implications. One is that tive probability distribution is needed. Such a distribu- the NWS needs to employ statistical methods to post- tion can be found by BMA (Sloughter et al., 2010) or process its ensemble forecasts so as to obtain calibrated ensemble model output statistics (Thorarinsdottir and probabilistic forecasts--these are now within reach. Gneiting, 2010). Other national agencies have not yet started issuing Generally, these probabilistic forecasting methods probabilistic forecasts of the main weather and water have been found to be calibrated and to give forecast elements, such as temperature, precipitation, and wind intervals that are narrow enough to be useful in a variety speed, on a regular basis, so this is an area in which of experiments. The different methods perform simi- the NWS has an opportunity to take the lead globally. larly, but all clearly outperform probabilistic forecasts Statistical post-processing may be best developed based on the raw ensemble with no post-processing. through collaboration with the broader weather, water, They work for different types of ensembles, including and climate enterprise. ensembles whose members are all distinct, ensembles The second implication is that increasing the with subsets of exchangeable members, and multi- size of the ensembles used, which is quite expensive model ensembles (Fraley et al., 2010). They have the computationally, will not by itself yield calibrated advantage of typically requiring only short data training probabilistic forecasts. Increasing the resolution of periods. They may be improved by being used in con- the ensemble members and statistical post-processing junction with reforecasts (Hamill et al., 2004), although are more likely to yield sharper and better calibrated these can be computationally expensive to produce. probabilistic forecasts than increasing ensemble size, An important challenge is effective communication and are thus a better investment in the context of of probabilistic forecasts. Recent research in this area limited resources. is encouraging, suggesting that it is possible to com- The third implication is that probabilistic forecasts municate uncertainty in the forecast in a way the public can be effectively communicated to users, but it is can understand. The communication format has to be important that the communication format be carefully carefully designed, however, and cognitive research is designed using cognitive research. When calibrated and proving useful in showing how to do this. well communicated, probabilistic forecasts can increase Research has shown that decision making by non- trust in the forecast and lead to better decision making experts can be better when they are given uncertainty in the face of uncertainty. It is possible that the design information than when they are given traditional deter- of formats for communication of probabilistic forecasts ministic forecasts only (Nadav-Greenberg and Joslyn, may be better done by a partner organization. 2009). The format is important. For example, box plots and uncertainty charts enhance reading accuracy and Recommendation I.d awareness of the degree of uncertainty, while showing a visualization of the worst-case scenario can cause bias The National Weather Service (NWS) should take (Nadav-Greenberg et al., 2008). Uncertainty informa- the lead in a community effort to provide products tion can make people less reluctant to act in situations that effectively communicate forecast uncertainty in which precautionary action is required at low prob- information. The format for communicating proba- abilities, which is often the case with rare events ( Joslyn bilistic forecasts requires careful design using cog- and LeClerc, 2012). A website12 showing probabilistic nitive research. Calibrated probabilistic forecasts would be produced by statistical post-processing of 12 www.probcast.com forecast ensembles, and improvement efforts should

OCR for page 17
PRIORITIZE CORE CAPABILITIES 29 focus on increasing the resolution and accuracy of the of the success of the Warning Coordination Meteo- ensemble forecasts. rologist (WCM) position at WFOs in coordinating with external partners and customers. Perhaps more Hydrologic Prediction Services importantly, the hydrologic services program presently desires to have a hydrologic-centric MAR, especially to The MAR had a substantial impact on NWS address current staffing profiles (NRC, 2012a). hydrologic prediction services operations. Hydrologic National, mutually agreed upon, quantitative prediction services were not addressed directly by the hydrologic forecast performance metrics provide a MAR, however, and the benefits to hydrologic services clear, consistent, and objective way to track predic- were co-benefits of improvements designed to benefit tion service skill over time. Comparison of hydrologic meteorological services. These co-benefits included the forecasts from different sites or watersheds can be greatly improved observation of precipitation through complicated, but this difficulty need not inhibit the the deployment of the NEXRAD network; the greatly development of a consistent framework for hydro- increased density of surface observation with the logic prediction skill assessment. Because any single Automated Surface Observing System (ASOS); and metric is often deficient in quantifying forecast service increased coordination of Weather Forecast Offices performance, it is common place to use a select set of (WFOs) with River Forecast Centers (RFCs), thus metrics to characterize many aspects of forecast qual- allowing the NWS to expand its hydrology mission and ity over time. This is particularly true for hydrologic services (NRC, 2012a). Although the Committee notes forecasts where different hydrologic behaviors such as the need for more integrated planning and technology peak flow amount, period of inundation, duration of improvement in the meteorological and hydrologic low flow conditions, and location of overbank flows services, hydrologic services are treated separately in can each have specific impacts that present threats this report. This is because improvements in hydrologic to society. Flash flood forecast performance based services have been an afterthought in the past and the on flash flood warnings issued by NWS WFOs have Committee feels they need special attention. NWS been monitored since the late 1980s and show a hydrologic prediction services were partially addressed steady increase in forecast lead time and reasonable in the Committee's first report (Finding 4-7a) and are values of probability of detection of flooding events. discussed in more detail in this section. River flow forecasts issued from RFCs have only been Coincident with the MAR, the Advanced Hydro- monitored since 2008 under a relatively new program logic Prediction System (AHPS) was developed and entitled the Point-based Flood Warning Verification implemented, which also aimed to improve and expand Program. Forecast verification statistics made available hydrologic forecasts and services. Although AHPS to the Committee suggest that on a national basis there wasn't funded until midway through the MAR, it have been increases in river flood forecast lead times was essential for enabling the RFCs to capitalize on but the skill of the forecasts, in terms of probability MAR advancements. Hydrologic model development, of detection and false alarm ratios, has remained rela- calibration, and forecast verification are important tively constant while there has been an increase in the functions of the RFCs. However, the MAR did not absolute timing error (error in the forecast of the time provide the RFCs with the full complement of infor- of flood onset). mation processing tools, through AWIPS or other While still early, the Committee sees this effort as tools, required to fulfill those functions. a positive step toward tracking forecast performance in As a whole, the MAR did have a positive impact on NWS products and services. The Committee suggests hydrologic forecasts and services as evidenced, in part, that this effort be sustained and potentially expanded by a significant improvement in flash flood forecast to include additional river flow levels (also known as lead time during the period of 1994 to 1998 when the probabilistic "exceedance" thresholds), which would NEXRAD radar system was completed (NRC, 2012a). allow for more complete assessment of river flow fore- The recent addition of Service Coordination Hydrolo- casts in addition to flood flows. The existing and addi- gists (SCHs) at the RFCs was based on their evaluation tional metrics will provide a clear history of the benefit

OCR for page 17
30 WEATHER SERVICES FOR THE NATION of past forecast innovations and provide a continuous and include product generation (e.g., general weather platform from which experimental methodologies may forecasts, watches, warnings, advisories, and guidance) be easily compared. and dissemination; brokering and provision of weather NWS hydrology is also moving toward more and water data; international responsibilities to WMO m odern distributed, physics-based Earth system programs; and the creation of critical analysis tools that models for hydrologic prediction (Cline, 2012). If enable NWS staff to execute its functions. implemented, such models would gradually replace the spatially lumped conceptual hydrologic models13 that Data Management have been used operationally for the past 20 to 30 years in NWS hydrologic prediction services. While there As outlined in Observing Weather and Climate from are numerous reasons for undertaking this modeling the Ground Up: A Nationwide Network of Networks, there transformation, many of which are described below, has been a proliferation of networks designed to acquire it is important that such model development efforts meteorological and hydrologic data for various applica- be informed and guided by clear model assessment tions (NRC, 2009). From agriculture and transporta- activities. Presently, it is not clear to the Committee tion interests to recreation and severe weather and flood that there is a formal, objective vetting process for dif- forecasting, it is commonplace that surface observations ferent modeling systems at either the national scale or from private or public mesoscale networks (mesonets) at the individual RFC level, though the aforementioned are utilized in conjunction with, or instead of, data forecast evaluation efforts could be part of such an from the joint NWS/FAA Automated Surface Observ- effort. The Distributed Model Intercomparison Pro- ing System (ASOS) network. Mesonets have become gram (DMIP) has provided some significant degree of widespread and popular for a variety of reasons, includ- model assessment but it is only a periodic, voluntary, ing cost-effective and customized instrumentation and and largely unfunded research activity and could be readily available communications infrastructure. These improved to include a more formal and continuous private and public networks have also proven to be far framework for model assessment. In summary, the more agile in terms of evolving sensor suites than the Committee views these forecast metrics and evalua- ASOS network has. The ASOS network continues tion activities as key elements in conducting continu- its singular focus on aviation interests. Very little in ous assessment value determination of new modeling the way of modification has occurred with the ASOS innovations. network in terms of the sensor suite or enhancements in spatial and temporal resolution. Recommendation I.e As in situ sensors are being deployed to measure conditions near the Earth's surface at mesoscale time The National Weather Service (NWS) hydrologic and space scales, a similar revolution is taking place with prediction services should coordinate with other remote sensing platforms. The dual polarization upgrade entities in the hydrologic prediction community to the NWS NEXRAD radars is under way, and research to continue and expand a set of common, objective continues at NSSL on multi-function phased array radar model metrics from which operational and experi- (MPAR) capabilities. However, as is the case in the sur- mental models may be inter-compared and continu- face observation realm, a myriad of non-NWS weather ally assessed. radars are operating with the goal of protecting life and property. Examples include the CASA radar project ESSENTIAL FUNCTIONS and radars operated by private industry, primarily the broadcasting sector. As with the case of surface mesonets, Essential functions constitute those activities and these radar networks are operated independently with services that are mandated by the NWS mission dissimilar sensing protocols, spatial and temporal resolu- tion, and quality assurance protocols. 13 A spatially lumped hydrologic model represents a drainage As water quantity and quality become a more basin as a single entity and simulates state variables and fluxes into critical national resource and national security issue, and out of the basin as a whole (NRC, 2010).

OCR for page 17
PRIORITIZE CORE CAPABILITIES 31 the next decade will likely result in the increase Recommendation: The centralized authority should of new mesonet measurements focused on water, and require metadata of every component in an integrated, new radar and satellite algorithms focused on pre- multi-use observing system (NRC, 2009). cipitation estimation. New methods of determining In addition to having an infrastructure to address precipitation rates, precipitation type, evaporation, soil issues of data standards, the NWS will also need to moisture, surface moisture fluxes, and groundwater consider an infrastructure to address issues of data availability will result in another round of mesonet, security to ensure the integrity of information and radar, and satellite enhancements that the NWS is not data throughput to delivery in times of disaster. well prepared to integrate into its operational mission. It is expected that this trend will continue with regard OPERATIONALLY RELATED RESEARCH to observations throughout the troposphere. In addition, the most recent decade has seen an Operationally related research aims to infuse NWS increase in the use of nontraditional data as proxies core capabilities with improved techniques from the for meteorological and hydrologic information. These research community (e.g., research-to-operations, or datasets include such vehicular information as wind- R2O). Such activities also include research priorities shield wiper speeds and the use of headlights, fog lights, based on issues that inhibit analysis and prediction skill and antilock brakes to provide information regarding or the production of forecast and warning products (e.g., visibility and rainfall rates. There is also potential to operations-to-research, or O2R). Achieving effective gather useful information from the tens of thousands R2O processes has challenged most technology-based of networked video cameras spread throughout the organizations, both public and private. The reasons for country. this are many, and the potential solutions are diverse. The NWS is better suited than entities in the pri- Prior NRC reports have addressed this on multiple vate or academic sectors to play a critical leading role occasions (NRC, 2000, 2003b, 2010). The issue is of with respect to observation data quality requirements sufficient importance to be a major element of Recom- and standards. System of systems architecture analyses mendation I. In particular, the Committee includes in can be strengthened to prioritize data requirements and, its recommendation the new approach of reaching out therefore, the key design parameters of the observing to the broader weather, water, and climate enterprise instruments and platforms. In view of the increasing to identify improved R2O techniques and to assist the complexity, cost, and quantity of data, it is critical to NWS with making such transitions more successful. approach this up-front activity in a rigorous fashion The Committee believes that a community-based, and is clearly a priority government role. Therefore, it systematic approach to R2O--developed by profes- is critical that the NWS carefully review the recom- sionals from all sectors of the enterprise and inspired mendations made in Observing Weather and Climate by a dedicated team of experts--is likely to have wide- from the Ground Up: A Nationwide Network of Networks reaching consequences. As stated in the third bullet of (NRC, 2009) and apply them to weather, water, and Recommendation I, it is especially important that the climate data in all of their forms. In particular, the development and implementation of a national strategy Committee endorses the following recommendations for a systematic approach to R2O/O2R involve the from that report: entire enterprise, and that the NWS lead such an effort. Recommendation: Stakeholders, including all levels Several system and research and development of government, various private-sector interests, and (R&D) activities are being conducted that contribute academia, should collectively develop and implement to the NWS infrastructure and service program. Each a plan for achieving and sustaining a mesoscale observ- offers an example of the type of research partner- ing system to meet multiple national needs. ships that would be useful in developing a national R2O/O2R system as called for in the third bullet of Recommendation: To ensure progress, a centralized authority should be identified to provide or to enable Recommendation I, and each offers lessons that could essential core services for the network of networks. be used as guidance when developing such a strategy. In addition to the specific programs and partnerships

OCR for page 17
32 WEATHER SERVICES FOR THE NATION like the examples provided here, an overall improved and storm surge forecast guidance. The goals of the partnership between the NWS and OAR will be critical HFIP are to reduce the average errors of hurricane track to strengthening NWS core capabilities through R2O and intensity forecasts by 20 percent within five years and O2R. These include the following. and 50 percent in ten years, with a forecast period out to seven days. Among HFIP's goals is to provide research High-capacity R&D Computing Capability and "experimental operational" products in real time to the NHC forecasters for their use and evaluation. Access to computing capacity for NOAA research HFIP is an example of a community-based approach has significantly increased with dedicated systems that has brought together the research and operational located at or near research institutions and available communities to address NWS science and service goals. for projects directly applicable to NWS requirements. Additionally, the Developmental Testbed Center The computer systems include GAEA (Oak Ridge, (DTC) assists in the evaluation of the real-time and Tennessee), T-Jet (Boulder, Colorado), S4 (Madison, retrospective HFIP forecasts. These collaborations with Wisconsin), and Zeus (Fairmont, West Virginia). academia and government agencies within and outside of NOAA (e.g., the Navy) have demonstrated that Global Data Assimilation increased model resolution and improved data assimi- lation through the application of the EnKF for both Researchers from NOAA/NCEP, NASA, NOAA/ global and regional models, enhanced model physics, Earth System Research Laboratory (ESRL), and data assimilation of observations around the hurricane the University of Oklahoma have jointly undertaken core in regional models, and new post-processing tech- the development and testing of an advanced 3DVar/ niques, have the potential for improvements in both ensemble Kalman filter (EnKF) ensemble hybrid data hurricane track and intensity prediction. The global assimilation system using the NOAA R&D computing EnKF ensembles indicate a 20 percent improvement resource. Initial testing of the system obtained positive in track guidance, while the intensity forecasts have results in terms of standard NWP metrics, and the sys- not seen improvement over the past four years.14 In the tem was transitioned to operations in May 2012. This meantime, intensity improvements were demonstrated initial step is part of a longer NWS NWP development in a research model (the NCAR Advanced Research objective and illustrates the power of utilizing a collab- WRF) that continues to outperform both NWS hur- orative enterprise approach to major R&D objectives. ricane models (GFDL and HWRF). In spite of the short-term progress forged through the HFIP effort, Hurricane Forecast Improvement Program improvements in hurricane intensity forecasts from dynamical models continue to be a major challenge. Established in 2008, the Hurricane Forecast Hurricane track forecast skill has steadily improved Improvement Program (HFIP) has aligned the inter- over the last 20 years, but the intensity forecasts have agency and scientific hurricane community at large, improved little if at all (NRC, 2012a). For example, in including a consortium of researchers from universities, 2011 the NOAA high-resolution operational dynami- NCAR, NOAA/ESRL, NOAA/Atlantic Oceano- cal models (GFDL and HWRF) had larger intensity graphic and Meteorological Laboratory (AOML), errors than those of the statistical models, which is NOAA/NCEP/National Hurricane Center (NHC), a basic metric of skill. Clearly a longer-term sus- NOAA/NCEP/EMC, DOD/NRL, as well as a hur- tained and focused effort is needed to improve hur- ricane specialist from NHC, to address the major ricane intensity forecasts. The progress of the NCAR challenge of improving hurricane forecasts. HFIP has Advanced Research WRF model in reducing hurricane assembled a computational infrastructure and imple- intensity forecast error, while the operational models mented a focused set of cross-organizational R&D have failed to show similar improvement, highlights the activities addressing global models, regional models, need for improving the transition of R2O. To meet its surge models, hybrid data assimilation, and statistical post processing to improve hurricane track, intensity, 14 http://www.nhc.noaa.gov/verification/verify5.shtml

OCR for page 17
PRIORITIZE CORE CAPABILITIES 33 goals, HFIP will need to develop a systematic approach entities of the weather, water, and climate enterprise to R2O. together for mutual collaboration. Over the past six It should be noted that the Joint Hurricane Testbed years, NWS scientists, engineers, researchers, trainers, ( JHT), under the auspices of the USWRP, predates and operational forecasters from the NOAA Radar the HFIP and has also shown success in improving Operations Center (ROC), NOAA National Severe technologies for forecasting in operational centers and Storms Laboratory (NSSL), NOAA Warning Decision improving modeling and data assimilation capabilities Training Branch (WDTB), NOAA Storm Predic- at the EMC. The success of both the DTC and the tion Center (SPC), and the NWS Norman Weather JHT suggest that testbeds are an effective approach to Forecast Office (OUN) have been collocated with facilitating R2O. the NOAA-OU Cooperative Institute for Mesoscale Meteorological Studies (CIMMS), the University NOAA Center for Weather and Climate Prediction of Oklahoma School of Meteorology and College of (NCWCP) Atmospheric and Geographic Sciences, the Center for the Analysis and Prediction of Storms (CAPS), the The construction of this new facility located near Atmospheric Radar Research Center (ARRC), and NASA Goddard and the University of Maryland will the Oklahoma Climatological Survey, a state agency. provide the infrastructure and location required to These agencies and programs are located on the OU facilitate the R2O necessary to carry out a major part Research Campus with private-sector partners and of the vision for the NWS outlined in this report. OU support entities, including the Office of the Vice Included in the transition is the collocation of five of President for Research and the OU Office of Technol- the NCEP centers, major elements of NESDIS opera- ogy Development. tions and research, and OAR's Air Resources Labora- The NWC has resulted in significant opportuni- tory. To facilitate collaboration between NOAA and ties for collaboration, creating an environment where the external science community, forty spaces in the each of the sectors of the enterprise can learn from, and new facility have been dedicated for visiting scientists. with, the other sectors. Within the NWC, NOAA has There is currently some funding allocated to support created a Hazardous Weather Testbed (HWT) that visiting scientists from the University Corporation serves as the proving ground for new technologies, for Atmospheric Research (UCAR), and the NWS is forecast techniques, training regimens, and datasets. working with other tenants of the NCWCP to develop This has resulted in the real-time transfer of cutting- a coordinated approach to supporting an expanded vis- edge research into forecast operations to protect life iting scientist program (Uccellini, L., NWS National and property. Regular programs within the HWT con- Centers for Environmental Prediction, personal com- sistently transition new meteorological understanding munication to member of the Committee). Such a into advances in warnings for hazardous weather events program would benefit from engaging scientists from nationwide. The ability to team researchers with fore- outside the NOAA labs or other federal agencies that casters and students provides for a unique environment the NWS already works with, and increasing partner- whereby existing practitioners continually develop their ships with all parts of the weather, water, and climate professional skills while working alongside the next enterprise and internationally. In developing such a generation of meteorologists. As the NWS seriously program, the NWS could draw valuable lessons from considers optimal future collocation of its facilities with existing visitor programs at other national centers such stakeholders, the NWC could be viewed as a possible as ECMWF. model for how to design such a collaborative facility. Severe Storm Prediction NRL/FNMOC Interactions The National Weather Center (NWC) in N orman, The Navy's meteorology and oceanography enter- Oklahoma, is a multi-agency facility built in 2006 prise differs in scope from the NWS (e.g., the Navy designed to bring government, academic, and private is focused on different applications and areas of

OCR for page 17
34 WEATHER SERVICES FOR THE NATION responsibility) and the overall mission is quite differ- cooperative agreement from the National Science ent in its nature. Nevertheless, some relevant parallels Foundation (NSF) with the mission of conducting between the two enterprises are illustrative. To develop foundational atmospheric and related science research leading, operationally viable technology (including and serving the broader academic atmospheric and models and applications) in key areas for the Navy, related sciences research communities. In addition the Naval Research Laboratory (NRL) leverages basic to managing several state-of-the-art observational and applied research programs that are among the and computational research facilities, NCAR also world leaders. This is accomplished through a close codevelops and supports the development of a host of relationship with national and international research community-based atmospheric modeling systems. This partners, as well as various end users within the Navy's suite of models includes the Weather Research and meteorology enterprise. The environmental prediction Forecasting model (WRF) and the Community Earth systems and application technologies are developed System Model. These two models, in particular, are by NRL (Marine Meteorology Division in Monterey, widely used throughout both the United States and the California, and Oceanography Division in Stennis, international academic research and operational fore- Mississippi) and transitioned to the Fleet Numerical casting communities. By design, NCAR has deep roots Meteorology and Oceanography Center (FNMOC) in the atmospheric research community and maintains and the Naval Oceanography Center (NAVO) for participatory observational system and modeling sys- operational implementation and routine dissemination. tem development pathways. Research conducted in Despite fewer resources than other weather centers partnership with NCAR has led to significant techno- (e.g., personnel, financial, computational), the Navy's logical advances in many areas, including polarimetric prediction systems are leading in some aspects or remain radar, lidar, boundary layer and surface flux measure- competitive with other efforts in many respects. One ments, regional numerical weather prediction, climate possible important reason for this success has been the system modeling, paleoclimate modeling, solar physics, collocation of the Navy's primary meteorology research aircraft measurements, and high-performance comput- entity (NRL-Monterey) with its operational partner ing applications. (FNMOC). This has fostered a close working relation- Recently, NCAR has partnered with NCEP and ship and a particularly effective pathway for efficient others within NOAA and the academic community in transitions of research technology to operations. (A the formation of the DTC whose mission is to evalu- similarly close research-operations synergy is present on ate new innovations in operational NWP modeling the oceanography side.) NRL has assumed the role of and model verification techniques. Through existing a developmental testbed center for the Navy in order to partnerships with academia and through structures like solely develop or harvest research and technology rel- the DTC, there exist significant opportunities for the evant for the Navy mission through focused interactions NWS to improve connectivity to the research com- with the research community, and subsequent opportu- munity and to accelerate R2O activities. nities for transition to operations. This close relation- ship between research and operations has enabled the Hydrologic Prediction Navy to mitigate the so-called "Valley of Death" of R2O. Close alignment and coordination of the research Presently, there is an enormous amount of active and operational enterprise within the Navy has provided research in both the land-atmosphere-vegetation focus and efficiency that likely would not be achievable modeling communities and the catchment hydrology if these research and operational entities operated more modeling communities. Funding for such activities independently with less administrative oversight. has increasingly fallen outside of NOAA, being sup- ported instead by NSF, NASA, and the Department National Center for Atmospheric Research of Energy (DOE) in recent years. This has presented barriers to transferring research into NOAA/NWS The National Center for Atmospheric Research operations. The level of sophistication and representa- (NCAR) in Boulder, Colorado, is operated under a tiveness of real-world processes, as well as character-

OCR for page 17
PRIORITIZE CORE CAPABILITIES 35 izations of uncertainties, in those non-NWS research The Indispensable Capability for Development and and operational communities outpace those used in Testing NWS-hydrology operations. NWS hydrologic pre- diction models are simplified, often lack real physical Previous sections of this chapter have detailed meaning, and are limited in terms of ensemble and data the many ways in which the science and technology assimilation capabilities. Alternatively, NWS hydro- underlying the core capabilities of the NWS continue logic prediction models are fairly parsimonious from to advance and require integration into the opera- a parameter perspective, which affords convenience in tions and products of the NWS. New types and new terms of model calibration and the simplified structure amounts of data need to be assimilated; the radars and of NWS hydrologic prediction models translates into other observation technologies need to be upgraded significant computational efficiency. to provide more localized observations for feeding The current suite of NWS hydrologic models ever more detailed and end-user-useful forecasts; requires numerous years of data for calibration to numerical weather forecasting techniques need to be obtain reasonable fidelity. The broader hydrologic sci- enhanced so as to be state of the art; and the work ence community has developed numerous ways to add stations and workstation software need to be upgraded value to hydrologic forecasts through the use of more so as to allow the forecaster to be able to use all of advanced models, through data assimilation, or through these enhancements. the employment of more sophisticated ensemble tech- The existing powerful NWS capability has evolved niques, but there appear to be institutional and resource to its present state by diverse paths, in many stages, barriers to efficiently infusing that science and technol- and it is very complex--much more complex than if ogy into NWS hydrologic prediction service operations. it had all been designed at once using sophisticated Evolution of NWS hydrologic prediction services in a systems design capability. Systems design processes are science-based manner requires new pathways for col- important, but making enhancements is not a matter of laboration. The NWS Office of Hydrological Devel- simply writing detailed specifications and then funding opment (OHD) is nominally charged with developing a vendor to create the required hardware or software. new forecasting methodologies and transferring those The entire experience of the information technol- to the RFCs where operational forecasting duties are ogy (IT) industry to date teaches a powerful lesson: executed. Increasingly, however, it seems R2O tran- "You can't get it right the first time." Incremental sitions have languished, resulting in disconnects or improvements and additions need to be tested, while divergence between the forecasting approaches used in under prototype development, by being made available RFCs and those developed by OHD. The exact reasons to selected NWS forecasters. for this are somewhat unclear but are likely to be due in This capability to test during design goes by many part to limited staffing and technological resources. The names: development and testing; rapid prototyping; OHD has and continues to develop new technology build a little, test a little, field a little; iterative design; and techniques and has some deep, long-standing ties and others. Whatever the name, the process and the with the broader academic community, but migration capability are indispensable. This is especially true in of technology from OHD to RFCs remains lacking. an environment where delay in upgraded functionality To accelerate the transition of new observational means cost overruns. Note that Lesson 3 of the Com- and prediction technologies into NWS hydrologic mittee's first report pointed to the crucial role that prediction activities, a clearer set of protocols and path- rapid prototyping played in the success of the MAR ways for infusing research into operations is needed. and the successful deployment of the AWIPS work This could be achieved through an improved OHD stations in particular. This capability was a result of the capability or through such other means as a hydrologic Program for Regional Observing and Forecasting Ser- prediction testbed. Such an activity would encompass vice (PROFS) program and the Denver AWIPS Risk and become an instrumental component of the multi Reduction and Requirements Evaluation (DAR3E) agency coordination efforts being developed at the new effort, which had support from NWS leadership and National Water Center in Tuscaloosa, Alabama. management and allowed for operational testing of

OCR for page 17
36 WEATHER SERVICES FOR THE NATION many of the modernization concepts by prospective (NSAB) to make it the source of technical advice for users. the NWS. Moreover, this development and testing capability 2.Create a separate Federal Advisory Committee is an essential link in the R2O and O2R processes con- Act (FACA) committee solely devoted to providing necting the NOAA labs and the academic meteorologi- technical advice to the NWS. cal communities. Although the development process 3.Create an independent, standing committee could occur anywhere (within NOAA, the academic charged exclusively with providing ongoing scientific or research community, the private sector), operational and technical advice to the NWS. testing needs to involve NWS forecasters, and the prototypes need to be modified on the basis of what The advantage of the first option is that it is prob- is learned from the testing. Therefore, the Committee ably the simplest to set up. The "overhead" of creating makes the following recommendation. a FACA committee has already been incurred, and the EISWG is already functioning. A possible drawback is Recommendation I.f that the EISWG subset of the NSAB cannot, because of its rather small membership, represent all of the areas or As an absolutely necessary condition for success, sectors in which advice and outside perspective would be although insufficient by itself, the National Weather of use to the NWS. The marginal cost of this option is Service (NWS) should have an ongoing capability for likely only that associated with the meetings of EISWG development and testing of its incremental technical that are distinct from those of the NSAB itself. upgrades. This will allow the NWS to advance its One advantage of the second option is that it could capabilities to respond to new scientific and techno- be structured to include a wide range of subject exper- logical possibilities and to enhance its service to the tise and a wide range of representation of the various nation. stakeholder groups in the weather, water, and climate enterprise. A second possible advantage might be that ADVISORY GROUPS FOR it would interact directly with NWS management. TECHNOLOGICAL IMPROVEMENTS Possible drawbacks might include the substantial over- head and delay associated with creating a new FACA Lesson 6 of the Committee's first report was that committee. The cost in dollars and staff time would the various scientific and technical reports from inde- be borne directly by the NWS, rather than by NOAA. pendent advisory bodies were of substantial help to the The third option probably has less overhead, NWS during the MAR. While the Committee was because setting up an independent committee is a unable to be specific about how much external advice simpler process than setting up a FACA committee. was appropriate, they nevertheless concluded that it It has the advantage, compared to the first option, of a was an important input to NWS and NOAA manage- broader range of expertise and stakeholder representa- ment. The Committee notes that creation of an NWS tion, but it is, of course, not under the control of NWS Advisory Committee was the first recommendation of management regarding committee makeup. The costs the Fair Weather report (NRC, 2003a) and that sub- are likely to be determined primarily by the size of the sequent reports have repeated that recommendation. committee and the expected frequency of meetings, The Committee has considered several ways in which might be as few as one or two per year. The costs which ongoing independent technical advice might be would be borne directly by the NWS. provided to the NWS, and this section presents them No matter which option for technical advice the along with what the Committee judges to be the pros NWS pursues, the advisory body would need exper- and cons associated with each. tise in both the physical and social science aspects of weather, water, and climate services, as well as expertise 1.Institutionalize the current Environmental in systems engineering and infrastructure evolution. Information Services Working Group (EISWG) This latter expertise would partially address Lesson 2 subcommittee of the NOAA Science Advisory Board from the Committee's first report by helping the NWS

OCR for page 17
PRIORITIZE CORE CAPABILITIES 37 clearly define system-level requirements (NRC, 2012a). of the key challenges facing the NWS is keeping pace The advisory body would also help the NWS establish with rapid changes in technology, user needs, and the clear metrics for evaluating improvement in forecasts overall enterprise context. For organizations such as and warnings, consistent with Lesson 3 from the government agencies, which lack the employee turn- Committee's first report and Recommendation I of over common in the private sector, advisory groups this report. Consistent with Lesson 5, representation can be an important mechanism for keeping pace and from all sectors and known stakeholders would also be infusing new perspectives in these areas. Among other important. things, the Committee thinks it is important to take With any advisory group, advice needs to be given advantage of both majority and dissenting perspectives in a constructive manner and the recipient needs to from advisory groups so as to understand the full range be open to the message. As noted in Chapter 1, one of issues and opportunities.

OCR for page 17