4


Impact of the Modernization
and Associated Restructuring

This chapter focuses on the effects of the Modernization and Associated Restructuring (MAR) of the National Weather Service (NWS) on the provision of weather services to the Nation after 2000. The actual impact of the MAR is compared to the promised benefits presented in Chapter 2, and summarized in specific findings about the major aspects of the MAR. The MAR brought such major changes in the capabilities and operation of the NWS that its effects took, in many cases, several years after 2000 to be realized. This is particularly true about the skill of atmospheric analysis and forecasting, which has improved steadily since the end of the MAR, as well as the relationship of the NWS with both the private sector and the emergency management communities. Therefore, this chapter contains several rather extensive discussions of MAR impacts extending up to the present day.

MANAGEMENT AND PLANNING

In addition to the planned system improvements that were the objective of the MAR, execution of the project itself left a legacy of institutional and cultural changes at NWS, largely for the betterment of the organization. Critical to understanding this legacy is differentiating between the near-term impacts during the MAR (influenced by the challenges of dealing with change) and the longer-term impacts (after the changes had been institutionalized).

Management Context and Constraints

The capability of NWS to function within the greater context of issues discussed in Chapter 3 was considerably improved as a result of the MAR. The staff perception now is that NWS is widely seen as more authoritative, it is doing its job better,1 it manages relationships more effectively, and it is more focused on customers and understanding their needs (committee member WFO site visits, see Appendix C for list of WFOs visited). That is not to say that contextual issues have disappeared. Technology is still evolving more rapidly than the NWS can respond, particularly in the area of communications (e.g., social media) and applications. Infrastructure put in place during the MAR is now as much as two decades old, and could present a cost liability as it requires replacement. There is also an increasing need to leverage partnerships; these partnerships bring new challenges, such as quality and standards arising in the case of data partnerships (e.g., weather observing networks from a variety of sources; NRC, 2003b).

Budget and Schedule

NOAA and NWS’s experience with the budget and schedule challenges of the MAR could have resulted

__________________

1Employee comments from one WFO included “the reorganization allowed us to focus better, and the modernization allowed us to do a better job within that focus.”



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 45
4 Impact of the Modernization and Associated Restructuring T Management Context and Constraints his chapter focuses on the effects of the Modernization and Associated Restructur - The capability of NWS to function within the ing (MAR) of the National Weather Service greater context of issues discussed in Chapter 3 was (NWS) on the provision of weather services to the considerably improved as a result of the MAR. The Nation after 2000. The actual impact of the MAR is staff perception now is that NWS is widely seen as compared to the promised benefits presented in Chap- more authoritative, it is doing its job better,1 it manages ter 2, and summarized in specific findings about the relationships more effectively, and it is more focused major aspects of the MAR. The MAR brought such on customers and understanding their needs (commit- major changes in the capabilities and operation of the tee member WFO site visits, see Appendix C for list NWS that its effects took, in many cases, several years of WFOs visited). That is not to say that contextual after 2000 to be realized. This is particularly true about issues have disappeared. Technology is still evolving the skill of atmospheric analysis and forecasting, which more rapidly than the NWS can respond, particularly has improved steadily since the end of the MAR, as in the area of communications (e.g., social media) and well as the relationship of the NWS with both the applications. Infrastructure put in place during the private sector and the emergency management com- MAR is now as much as two decades old, and could munities. Therefore, this chapter contains several rather present a cost liability as it requires replacement. There extensive discussions of MAR impacts extending up to is also an increasing need to leverage partnerships; these the present day. partnerships bring new challenges, such as quality and standards arising in the case of data partnerships (e.g., MANAGEMENT AND PLANNING weather observing networks from a variety of sources; NRC, 2003b). In addition to the planned system improvements that were the objective of the MAR, execution of the Budget and Schedule project itself left a legacy of institutional and cultural changes at NWS, largely for the betterment of the NOAA and NWS’s experience with the budget and organization. Critical to understanding this legacy is schedule challenges of the MAR could have resulted differentiating between the near-term impacts during the MAR (influenced by the challenges of dealing with 1 Employee comments from one WFO included “the reorganiza- change) and the longer-term impacts (after the changes tion allowed us to focus better, and the modernization allowed us had been institutionalized). to do a better job within that focus.” 45

OCR for page 45
46 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT in an improved capability to manage large, complex restructured several of the core technological activities procurements, but it is not clear whether that was of the NWS presumably had a streamlining effect on achieved. Issues with upgrading the NEXRAD system the management of NWS activities. to dual-polarization radars and the implementation of AWIPS-II, suggest that either many of the MAR Processes lessons were not internalized within the agency or that they are not relevant in the current environment. There Many of the processes introduced to execute the have also been issues with the National Polar-orbiting MAR have been retained in one form or another. One Operational Satellite System (NPOESS) program. key process, the research-to-operations transition, con- Although it was managed under an Integrated Pro- tinues to be a major issue (MacDonald, 2011) and has gram Office and not according to the typical NOAA been the subject of numerous NRC reports (e.g., NRC, program approach, it appears to also not have benefited 2000, 2003b, 2010). Partner relationships, (Congress, substantially from lessons learned from the MAR. private sector, the National Weather Service Employee Organization, media, emergency managers) have been substantially improved in most cases (Friday, 2011; Organization and Staff Hirn, 2011; Myers, 2011). Overall NWS processes One of the most important results of the MAR was are now more flexible and responsive to evolving the organizational transition of the meteorological staff. context, though there is considerable room for further improvement.2 The ratio of technicians to professional meteorologists evolved from 2:1 to 1:2 and the number of staff was Among the more important legacies is the capa- bility to assess performance,3 instituted in the early reduced by about 10 percent (Sokich, 2011). Based on committee member visits to WFOs (see Appendix C 1980s and used during the MAR in part to satisfy the for list of WFOs visited), the transition is now viewed Congressional mandate for no degradation of service. positively by employees, but a wide range of issues per- However, it is often difficult to obtain performance sta- sist, presenting opportunity for further improvement. tistics outside the NWS. The government procurement Employees appear to have learned and retained the process, within which NOAA and NWS have limited value for ongoing innovation and change, recognizing flexibility to configure procurement to their particular that it is essential to ongoing organizational survival needs, continues to be a major constraint. There is evi- and improvement. The NWS focus on extensive staff dence of this in the upgrade of the NEXRAD system training has been retained as well, but much of the to dual-polarization radars and the implementation of training is now done online or on the job due to budget AWIPS-II. limitations (Spangler, 2011). A key issue that arose dur- A notable issue with the processes involved in ing the MAR, the balance between standardization of implementing the MAR was the lack of a systems office size and structure and local flexibility, remains a architecture (GAO, 1994). Necessary elements include central tension within the organization. Activities and developing a system-of-systems architecture and con- process development to better achieve the correct bal- cept of operations based on defining achievable, quan- ance are an ongoing focus. For example, there is some tifiable mission goals and prioritizing user needs. Such concern that requirements of some staff positions vary an architecture would have tied the top-level goals and from office to office, and that these variations are not objectives for each individual system with the specific adequately reflected in job descriptions and staffing user needs for each individual system, and synthesized levels. Field office location continues to be a concern these into a set of system-of-systems goals that are in some cases, particularly where the WFO is not sited c lose to the primary community within its area of 2 An example is the ASOS system, which logs data every minute responsibility. The MAR had clear and lasting impacts but only reports hourly summaries. To access minute-scale data, each WFO must connect with each ASOS site using dial-up access. at the field level that are discussed in more detail later. 3 Performance and performance degradation can involve subtle It is less clear whether the MAR improved the overall issues, as with changes in WFO performance when an office is organizational efficiency at the executive level. How- moved, or changes are made in the storm reporting system (Smith, ever, the realignment of the National Centers that 1999).

OCR for page 45
47 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING by the National Weather Service (NWS). Most of functions of the separate systems all working together. these “institutional byproducts” have been as valu- In the case of the MAR, the individual systems were able as the MAR improvements themselves and will ASOS, NEXRAD, GOES-Next, the National Centers help the NWS to continue to modernize. However, computer systems, and AWIPS. One possible goal from viewing more recent projects, implementation of this system-of-systems could have been a specific, of a rigorous systems engineering process to facilitate measurable improvement in Probability of Detection more effective management of the procurement and for various types of severe weather or even an overall development of large, complex systems appears not to scorecard that encompasses multiple metrics that are have been institutionalized within the National Oce- deemed important to the NWS and made available to anic and Atmospheric Administration. The systems the public. Exercise of this rigorous process would next engineering process needs to start at the beginning of have led to development of an optimal set of top-level the program, in the agency’s program office. requirements for the respective individual systems’ contractors. Tying the mission goals and key performance met- MODERNIZATION OF TECHNOLOGY rics to specific user needs via top-level requirements analysis and documentation is essential to enable the Although the technologies improvements of the contractor to develop a design against a set of require- MAR fell behind schedule and had larger than antici- ments that meets both the mission goals and the user pated costs, they contributed to the capability of the needs. Providing a clear and concise set of documents NWS to provide improved weather services to the to the contractor early in the process is crucial so that nation. This improvement is particularly evident in they can execute efficiently and be held accountable the forecasting and detection of severe weather such as for meeting budget, schedule, and performance goals. tornadoes and flash floods, and will be discussed at the Lack of such a systems architecture introduces consid- end of this section. erable risk to a program the size and complexity of the MAR. The larger and more complex a program effort Automated Surface Observing System is, the more important it is to utilize effective systems e ngineering processes. Without this the program The replacement of human observers with the manager (whether government managing contractor Automated Surface Observing System (ASOS) was or contractor managing subcontractor) loses a major quite controversial at the time of the MAR, and con- management tool. Setting mission performance metrics tinues to be controversial. Through the years, a number also allows for a quantitative assessment of the success of conflicting reports from a variety of sources have of a program upon completion. An illustration of the both lauded and criticized ASOS and its implemen- lack of adequate application of the system engineering tation. From the outset, the ASOS implementation process is the AWIPS program. AWIPS requirements was designed to provide a more robust, hourly, and were based on user needs, but they were apparently automated surface observation capability at over 1,000 not tied to mission-based goals (GAO, 1996c). Also, airports (Figure 4.1). The manual observations being the large, complex set of over 20,000 requirements collected at the time of the modernization were at 250 indicates a lack of prioritization of user needs. The airports, and the observations were only taken dur- contractor failed to develop a design that met the needs ing the hours that each airport was open. Although of the primary users, but with a lack of prioritization automation was seen as both a cost cutting measure and overwhelming number of requirements, this is and an opportunity to collect more data, the numerous understandable. stakeholder groups that were destined to use the data questioned the quality of the data collected. In addition, Finding 4-1 each of the primary users of ASOS, the NWS, FAA, Many of the institutional changes (management and DOD, had a different set of requirements for the structure, culture, processes, partner relationships) data and clear, cohesive metrics to evaluate the success introduced to implement the Modernization and of ASOS were never determined. Each user group had Associated Restructuring (MAR) have been retained a different set of metrics, and therefore judged the suc-

OCR for page 45
48 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT FIGURE 4.1 Locations of Automated Surface Observing System (ASOS) sites in the United States. The 315 ASOS sites managed by the National Weather Service are indicated by red diamonds. Blue triangles, blue circles, and green triangles indicate the 571 Federal Aviation Administration, 75 Navy, and 48 Air Force ASOS sites, respectively. SOURCE: National Weather Service. cess of ASOS through its own lens. Another key issue that was realized from the deployment of ASOS was in the implementation of ASOS was the lack of field not as dramatic. Because ASOS was designed primarily testing. This lack of preliminary reliability testing led to support airport aviation needs, and because of well to problems with the sensor suite remaining undiscov- documented issues with sensor performance as they ered until after ASOS was deployed (GAO, 1995h). pertain to weather and climate studies, many scientists Furthermore, ASOS algorithm development likely turned to developing their own networks for surface could have benefited from the type of prototyping that observing. These regional and state networks, called occurred for AWIPS through PROFS. mesonets, were typically operated by state entities and At the end of the MAR era, there were still inter- agencies. Examples include the Oklahoma Mesonet nal reports being commissioned by both the FAA and (commissioned in 1994) and the Florida Automated NWS to examine ASOS. For example, a 1999 FAA Weather Network (FAWN; commissioned in 1997). document claims, “. . . after years of development, Data from these mesonets have become important ASOS correlates quite closely with human observations resources for the NWS severe weather warning opera- most of the time” (AOPA, 1999). No references are tions as well as research. Because the mesonets are state listed, no studies are cited, and it is only an anecdotal initiatives the coverage is not even across the nation, or statement. sometimes even across a region. Such uneven coverage NEXRAD resulted in the ability for NWS fore- needs to be addressed as the weather enterprise further casters to observe weather phenomena at higher resolu- develops. A 2009 NRC report provided recommenda- tions than its pre-MAR technological predecessors, but tions for creating a “network of networks” with even the advancement for weather and climate forecasting coverage across the nation (NRC, 2009).

OCR for page 45
49 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING By 2003, the NRC Fair Weather report outlined The National Weather Service should develop a continuing comprehensive training and education nine major examples of ASOS failures where climate program so that the skills of the Next Generation studies are concerned. The report states, “[i]n the Weather Radar maintenance and operational staffs, as ideal, the ASOS observations would be error-free well as the meteorologists and hydrologists, reflect the and representative of actual conditions. Therefore ever-changing state of the art (NRC, 1991). the interim climate summary, daily climate summary, preliminary climate data, and final official climate The intensive on-site training provided at the out- record would all agree with each other and all reflect set of the MAR has been gradually reduced in scope; the best possible estimate of conditions. As the nine the Warning Decision Training Branch does provide a representative cases make clear, this ideal situation is comprehensive program, but the number of people who not always met” (NRC, 2003a). In addition, Horel et can take advantage of it is limited. A series of COMET al. (2002) state that the widespread use of ASOS will modules provides some online training, but these lack continue to impede efforts to monitor Earth’s climate the hands-on element provided by the on-site experi- and study its variability. The impact of ASOS on the ence and are not regarded as comparable. This is an c limate record is discussed later in this chapter and item of special concern as the polarization upgrade to in Appendix E. the NEXRAD system comes online. Perhaps the main remaining radar coverage issue had to do with the difficult problems encountered Next Generation Weather Radar in complex terrain. A mountaintop site provides a The 1-degree beamwidth and Doppler capabil- long-range view, but cannot see down into many of ity of the NEXRAD radars provided forecasters with the valleys where most people would live (a problem enhanced ability to identify weather features of concern. exacerbated by the NEXRAD restriction to a minimum The NEXRAD network is largely responsible for the elevation angle of 0.5 degree). A valley site may address improvement in the NWS capability to detect severe that problem for one or a few valleys but cannot pro- weather such as tornadoes, as discussed below. The vide broad area coverage. The NEXRAD site selection broad national coverage of the NEXRAD radars was generally opted for the mountaintop; for some purposes also a distinct improvement over that of the predecessor such sites provide adequate support of forecasting and WSR-57, WSR-74, and Air Weather Service FPS-77 warning functions (e.g., NRC, 2005), but for others systems (Figure 4.2). Maddox et al. (2002) provide a they are less satisfactory (e.g., Reynolds, 2011; Westrick more recent analysis of NEXRAD coverage. et al., 1999). The NEXRAD Product Improvement Program NEXRAD radars are owned and operated by the has continued to capitalize on continuing advances in USAF and the FAA, in addition to the NWS. Missions technology and science underlying the processing and of those agencies differ from those of the NWS, and use of the radar data. An “R&D” NEXRAD at NSSL this occasionally led to some nonuniformity of opera- provides a testbed for evaluating proposed system tions. For example, availability of the USAF radars was improvements (Zahrai and Zmic, 1993). an issue in the early days. Archiving of the data on a Implementation of the recommendations from routine basis is of interest to the NWS, while the other NRC (1991) in the NEXRAD program benefitted the two agencies are concerned mainly with data related nation through an organized research-to-operations to some event such as an aircraft incident; this has led program leading to a series of substantial upgrades to to gaps in the archival record. In a similar vein, distri- the basic NEXRAD system. Examples range from the bution of wideband data or products to neighboring conversion from the initial proprietary computational installations is more important to the NWS functions system to an open architecture to the forthcoming than to the USAF or FAA operations; the latter tend to polarimetric upgrade. focus on specific airfields. Again, this has led to uneven- Another recommendation related to the need for ness in the export of data from different NEXRAD ongoing training programs for NWS personnel was as sites. At the same time, it must be said that the FAA well not implemented: has pushed for development of capabilities (such as a

OCR for page 45
50 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT FIGURE 4.2 (a) Composite pre-NEXRAD coverage at 10,000 feet above site level for CONUS is indicated by white circles. Radar locations are indicated by diamonds (WSR-57 and WSR-74S) and circles (WSR-74C). The pink shading indicates areas that have no radar coverage below 10,000 feet above site level. (b) Composite NEXRAD coverage at 10,000 feet above site level for CONUS is indicated by white circles. WSR-88D radar locations are indicated by + (National Weather Service radars) and × (Department of Defense radars). The pink shading indicates areas that have no radar coverage below 10,000 feet above site level. The striped blue shading indicates areas where coverage at the 10,000 feet level is reduced compared with the pre-NEXRAD network. SOURCE: U.S. Department of Commerce. ible imaging, higher resolution infrared (IR) imagery, gust-front algorithm) that would aid the aviation mis- improved Earth location capabilities, and a separate sion but are of less interest to the NWS. sounder. Despite the difficulties in program design and execution, GOES-Next introduced substantial data and Satellite Upgrades product improvements. On earlier geostationary satel- lites, the imager and sounder could not simultaneously NOAA’s objectives for GOES-Next were continu- collect data because they used the same telescopic view- ous Earth-viewing with retention of the existing vis-

OCR for page 45
51 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING TABLE 4.1 Comparison of Measured Imager Performance ing apparatus, and the spin-stabilized satellite rotated on for GOES-7 (Pre-MAR Generation Satellite) and its axis viewing Earth only six percent of the time on each GOES-8 (GOES-Next Generation). 360-degree rotation (GAO, 1997c). Although the initial development of a three-axis, body-stabilized spacecraft IGFOVa at nadir SSRb design for GOES was problematic, it ultimately resulted Wavelength (km [E/W × (km [E/W × in successful establishment of a valuable approach. These (µm) N/S]) N/S]) Noise improvements together enabled continuous, simultane- GOES-7 Characteristics ous, independent imaging and sounding. Each instru- 0.55-0.75 0.75 3 0.86 0.75 3 0.86 6 bit data 1 2 ment had flexible scan control, allowing for coverage counts (3 sigma) of small areas, hemispheric, and full disk global scenes. 3.84-4.06 13.8 3 13.8 3.0 3 13.8 6.0 K @ 230 K Meteorologists were able to access close-up, continuous 0.25 K @ 300 K observations of dynamic, short-lived weather phenom- 6.40-7.08 13.8 3 13.8 3.0 3 13.8 1.0 K @ 230 K ena, such as local severe storms and tropical cyclones, as 10.4-11.3 6.9 3 6.9 3.0 3 6.9 0.2 K @ 230 K 0.10 K @ 300 K well as obtain data on the atmospheric temperature and water vapor structure. 12.5-12.8 13.8 3 13.8 3.0 3 13.8 0.8 K @ 230 K 0.40 K @ 300 K The implementation of GOES-Next resulted in GOES-8 Characteristics substantial improvements to the frequency, spatial 0.52-0.72 1.0 3 1.0 0.57 3 1.0 10 bit data 1 8.1 resolution, data quality, and spectral resolution of NWS counts (3 sigma) geostationary satellite data. Specific impact areas include 3.78-4.03 4.0 3 4.0 2.3 3 4.0 4.0 K @ 230 K 0.16 K @ 300 K • Imagery. Due to the Earth-pointing capability of 6.47-7.02 8.0 3 8.0 2.3 3 8.0 0.27 K @ 230 K the GOES-Next satellite, the five-channel imager could 10.2-11.2 4.0 3 4.0 2.3 3 4.0 0.40 K @ 230 K produce imagery every 5 to 10 minutes for local-scale 0.12 K @ 300 K severe weather events and every 15 minutes for CONUS 11.5-12.5 4.0 3 4.0 2.3 3 4.0 0.40 K @ 230 K coverage, and scan the full disk northern hemisphere 0.20 K @ 300 K in less than 30 minutes (with images provided every 3 Instantaneous Geometric Field of View: The detector IGFOV (or foot- a print) is the size of a pixel on Earth’s surface that a single detector, in the hours). The continuous viewing capability is critical for array of detectors associated with a specific wavelength, is able to “view” monitoring severe storms (GAO, 1991b). Improvements when looking directly below the spacecraft (the sub-satellite point). b Sampled Subpoint Resolution (SSR): Because the combination of the were made in the spectral resolution and signal-to-noise imager’s scan rate and detector sample rate exceeds the pixel E/W IGFOV, performance, as shown in Table 4.1. New uses of imager the viewed scene is oversampled. An IGFOV of 4 km oversampled by a data were developed. For example, the data were com- factor of 1.75 provides an effective resolution, or SSR, of 2.3 km. SOURCE: Purdom (1996). bined with the NEXRAD radar data to enhance winter snowstorm forecasting, nighttime fog detection was enabled using two IR channels, and the higher resolu- tion IR imagery was useful in predicting and monitoring - fog, water, and ice cloud detection both day severe thunderstorms. Additional results include and night using continuous 3.9 µm imagery with other channels; - best 6.7 µm (IR water vapor channel) imagery - identification of super-cooled cloud; ever; an order of magnitude improvement enables - monitoring of snow and ice cover and the identification of mesoscale disturbances embedded detection of cloud over snow; within synoptic scale features; - improved detection of forest fires and biomass - better wind data inferred from cloud drift with burning; 4 km image resolution for better edge detection and - useful imagery well beyond the satellite’s improved target selection; 60-degree zenith angle making possible the detec- - improved wind data inferred from water vapor tion and tracking of sea ice and polar lows; imagery in clear regions with 8 km spatial resolu- - improved low light imaging capability with tion and better signal-to-noise at 6.7 µm; 10-Bit visible-channel data;

OCR for page 45
52 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT - enhanced land and sea surface temperature algorithms. New computational capacity was required monitoring capability using 30-minute interval to assimilate the new observations available as part multispectral IR capability (Purdom, 1996). of the MAR, particularly the satellite retrievals and later radiances, into the various global and mesoscale • Soundings. With the launch of GOES-8 in numerical weather prediction models. 1 994, continuous geostationary sounder data was Public Law 100-685 called for “detailed plans available for the first time. The new, independent and funding for meteorological research to be accom- sounder produced 18 channels of IR data in addition plished under this title to assure that new techniques to one in the visible, yielding improved vertical resolu- in forecasting will be developed to utilize the new tion. Soundings retrieved from the GOES-Next data technologies being implemented in the modernization” proved to be useful aids in qualitative interpretation. (U.S. Congress, 1988). The Strategic Plan stated that They provided timely information about changes in “[f ]undamental model improvements are necessary atmospheric moisture and stability and gradients were to satisfy these requirements and provide guidance better delineated. In 1997, measurements of precipi- products of sufficient quality and frequency to support table water from the sounder were included for the the warning and forecast operations at each office” first time in the input to numerical weather prediction (NWS, 1989). The Development Division within the (NWP) models (GAO, 1997c). National Meteorological Center (NMC), together • Systematic Impacts . The Advanced Weather with the research in numerical modeling being under- Interactive Processing System (AWIPS) included the taken at the Geophysical Fluid Dynamics Laboratory capability to display GOES-Next satellite data on the (GFDL; Princeton, NJ) were both continually involved Weather Forecast Office (WFO) workstations and to in model development. But, so far as the committee combine this imagery data with other data to aid the can ascertain, the MAR planning did not explicitly forecaster. The satellite improvements were critical to include benchmarks, or a timeline, for the very exten- WFOs along the west coast, improving their capability sive software development effort involved in dramatic to analyze approaching weather over the data-sparse improvements in modeling and data assimilation. It Pacific Ocean as well as moisture surges and tropical seems that it was assumed these developments would disturbances from Mexico, the Gulf of Mexico, and occur, without specific planning as a component of the the Caribbean. Many recommendations were made MAR. However, by the end of the MAR, there had to NOAA by the GAO and others regarding the been substantial improvements in atmospheric model- approach to satellite system procurement. NOAA and ing and data assimilation, as well as the development NASA apparently took these recommendations into of an evolutionary capacity of computing technology consideration when planning the follow-on series of within NWS. geostationary satellites, after GOES-Next. Approaches considered included procuring “clones” of prior satel- Advanced Weather Interactive Processing System lites and/or instruments via sole source contracts. NOAA weighed the potential benefits of significant Development of an advanced computer and com- technological advances against the schedule and budget munications system to help forecasters in field offices risks involved. NASA was positioned to once again integrate all sources of weather data, to assist them in act on behalf of NESDIS and manage the instrument analyzing fast-breaking storms, and to aid in the timely contracts directly (GAO, 1996a). preparation of warnings and forecasts was a major accomplishment of the MAR. The system provides a communications network that interconnects each National Centers Advanced Computer Systems WFO and includes the capability for distribution of The strategic and operational planning for the centrally collected data and centrally produced analy- MAR emphasized the need for dramatic upgrades sis and guidance products, as well as satellite data and in the computing capabilities of the NWS. The cited imagery. Together this system is termed the Advanced rationale included the capability to run ever more com- Weather Interactive Processing System (AWIPS) and plex general circulation models and data assimilation NOAAPort (NWS, 1989).

OCR for page 45
53 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING By the end of the MAR, AWIPS was, and contin- the hydrologist-user needs were unable to be as well ues to be, used to addressed due to time and budget constraints, among other issues. The lesson here is that, if all users are • provide computational and display functions at equal, all user-needs need to be equally addressed from operational NWS sites; program initiation throughout all processes, and this • provide open access, via NOAAPort, to exten- effort needs to be reflected in the planned schedule and sive NOAA datasets that are centrally collected and/ budget. In addition, an important component, GFE or produced (e.g., NCEP NWP products, products (Graphical Forecast Editor), was not integrated into f rom other centers such as NHC and SPC, and the AWIPS core software package. Short term warn- international centers producing global analyses and ings use AWIPS; long term forecasts and hydrology predictions); use GFE. However, because AWIPS uses the open • acquire and process data from an array of meteo- source Linux operating system, additional software rological sensors including ASOS, NEXRAD, GOES development and integration is facilitated. As a result instruments, local sources (e.g., mesoscale networks, of the MAR, forecaster workstations and some servers river flow gauges, atmospheric sounders) and other were also upgraded. Prior to the MAR, offices as part sources (e.g., sensor data from commercial aircraft via of AFOS had a few unlinked workstations connected the Aircraft Communications Addressing and Report- via “store and forward” regional communications loops. ing System [ACARS]); WFOs now have half a dozen workstations linked by a • provide an interactive communications system high speed national data network. to interconnect NWS operations sites and to broadcast data to these sites; and Performance of Post-Modernization • assist forecasters in preparation and dissemina- Forecasts and Warnings tion of warnings and forecasts in a rapid, highly reliable manner. The performance of post-MAR forecast and warning operations of the NWS were dramatically With the implementation of AWIPS, forecasters improved by the MAR. The following review is limited are now able to sit down at one workstation and view a to tornado and flash flood warnings; numerical weather large, complex set of weather data in as many as twelve prediction and its application to general weather fore- windows. The total spectrum of weather information casts; and hurricane and extratropical storm forecasts, can be overlaid and integrated on a single map to get a as these are the types of weather of most interest to the unified picture of what’s happening and aid in forecast- public (winter weather forecast performance data from ing. According to one forecaster, before the MAR are not available). AWIPS has greatly improved [forecasters’] ability Tornado and Flash Flood Warnings to quickly ingest, manipulate, and analyze immense amounts of data. One of the most important capa- As proposed in the Strategic Plan, one major goal bilities introduced with AWIPS has been to combine of the MAR was to provide more reliable detection graphical data (e.g., geopotential height analyses) with imagery (including satellite imagery), then view these and prediction of severe weather and flooding. Per- data on a loop with easy zoom and pan capability. This haps the most striking result of the MAR has been the has been an important function of WFO workstation improvement in the probability of detecting and issuing technology given the rapid increase in available nu- warnings for severe weather events (e.g., Figure 4.3a,b). merical model data ( Jackson, 2011). The probability of detection (POD)4 and warning for Although AWIPS met the meteorological fore- caster needs, it did not adequately address hydrologic 4 From the AMS Glossary of Meteorology: “POD and FAR are useful evaluators for binary, yes/no kinds of forecasts, and detec- applications. This inadequacy reflects issues in both tion techniques. For example, if A is the number of forecasts that the requirements development and AWIPS build and rain would occur when it subsequently did occur (forecast 5 yes, test processes. The forecaster-user was very well inte- observation 5 yes), B is the number of forecasts of no rain when grated into the entire development and build cycle; rain occurred (no, yes), and C is the number of forecasts of rain

OCR for page 45
54 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT FIGURE 4.3 Probability of detection (POD)4 and False Alarm Ratio (FAR)4 for (a) tornado warnings and (b) flash flood warnings. Lead times5 for (c) tornado warnings and (d) flash flood warnings. The POD and warning lead times for both tornadoes and flash floods increased steadily over the course of the MAR, while the FAR for tornadoes and flash floods remained relatively constant. SOURCE: Based on data provided by the National Weather Service. tornadoes and flash floods increased steadily from the caused by Earth’s curvature. In any event, the warn- beginning of the MAR until it was completed in 2000. ing system continues to issue many warnings that At the same time, the average lead times5 of tornado are not reported and/or realized. Increased scientific warnings issued on the basis of observations increased understanding of these severe weather processes and from under five minutes to over 12 minutes (Figure integration of such understanding into operations, as 4.3c) and flash flood lead times increased from about well as further improvements in technology—especially ten minutes to over 40 minutes (Figure 4.3d). The radar and radar coverage—could help improve the false failure of the accompanying false alarm ratios (FAR)4 alarm ratio. to decrease at the same time as the POD has increased The NWS severe weather warning system con- has been a disappointment. This problem could have tinues to be impacted by problems associated with the several causes including the unreported occurrence/ dissemination of the warnings to the population at confirmation of predicted tornadoes or the occurrence risk. Again in the severe weather occurrences (tornado of funnel clouds that did not reach the ground, or the outbreaks) in 2011, timely warnings were issued but common problems of atmospheric sampling caused the loss of electrical power due to earlier severe weather by the limitations of the void under the radar beam left many households in the path of the storms without adequate means to hear the warnings and take neces- when rain did not occur (yes, no), then POD = A/(A1B) and FAR 5 C/(A1C). For perfect forecasting or detection, POD 5 1.0 (or sary lifesaving actions. 100 percent) and FAR 5 0.0 (or 0 percent).” 5 L ead time is the time from when the warning is issued until the time the event is reported within the warned area.

OCR for page 45
55 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING MOS were applied to most surface weather variables. Numerical Weather Prediction and its Dallavalle and Dagostaro (2004) examined the appli- Application to General Weather Forecasts cation and value of MOS alone compared to the local In addition to improved severe storm and flash forecasts produced by NWS field forecasters utilizing flood detection resulting from the MAR technologies their professional judgment as well as the MOS guid- and service restructuring, one of the promised benefits ance for the period 1966 to 2004. The study analyzed of the MAR was to improve NWS forecasts and warn- forecasts for 80 stations distributed across the CONUS. ings, making them as accurate and timely as possible. Recently updated results, with data through 2010, for Forecasts result from a complicated process that starts 36- and 60-hour forecasts of minimum temperature are with obtaining all possible observations, such as direct shown in Figure 4.4, and for 24- and 48-hour forecasts measurements of surface and upper-air properties and of probability of precipitation (PoP) are shown in Fig- remote measurements by satellites and radar. These ure 4.5. The study reported by Dallavalle and Dago- observations are assimilated into the initialization staro (2004) included other parameters (e.g., maximum process for numerical models, and the model output is temperature) and other forecast lead times and for the post-processed using Model Output Statistics (MOS) cold season as well as the warm season, with similar procedures to develop guidance which is used, along results as shown in Figures 4.4 and 4.5. The results with real-time observations and the model output itself, clearly show the improvement in the quality of both by field office forecasters to make forecasts. the local forecasts and the guidance—largely reflecting The NWS has performed numerical prediction the model improvement. The skill between the MOS operations at NCEP6 beginning in the mid-1950s and and the locally generated product converges later in the continuing to today. The four-times per day execution period, showing the increasing relative value of guid- of the models produces a wide variety of analyses and ance in the forecast process. products on regional, national, hemispheric, and global scales. An evaluation of the overall performance of the Hurricane and Extratropical Storm Predictions NCEP global numerical weather prediction operation over the period 1985 to 2009 is presented later in this The National Hurricane Center (NHC) uses a chapter (Figure 4.11). variety of models as guidance in the forecasting process. It is a major step to go from a numerical model Figure 4.6 illustrates the performance of the various prediction to information that can be used as guidance models showing annual average forecast track errors to forecasters producing general weather forecasts out for the period 1994 to 2009. The solid black line is the to about 10 days. The NWS has developed Model annual average track errors for the resulting Official Output Statistics (MOS) procedures that downscale 48-h NMC Forecast that is issued to the public. Over NWP model output through a statistical interpretation the 16-year period the performance of the various mod- of the model parameters in terms of surface weather els has converged, with less scatter later in the period, variables appropriate for the forecast time in question. reflecting better data and improved model physics as MOS relates observations of a weather element to be well as improved computing power. The improvement predicted (e.g., maximum or minimum temperature, in the Official Track Forecast is apparent, especially or probability of precipitation) to appropriate variables after 2001. (e.g., model outputs, initial observations, and geocli- The long-term trend in the Official Hurricane matic data such as terrain, and normal conditions) using Track errors for the period 1970 to 2009 is illustrated in multiple regression techniques. Figure 4.7. The dramatic improvement in the forecast At the time of the MAR, the MOS were calculated skill is apparent for all forecast lead-times although the each forecast cycle for specific forecast points and the record for the 96-h and 120-h lead times is short (since model output was interpolated to observation locations. 2001) and does not extend back to the MAR period. Much of the progress in the Official Forecast can be attributed to the major advances made in numerical 6 Prior to 1994, the principal national center was the National prediction models and the improved data resources as Meteorological Center (NMC).

OCR for page 45
62 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT offices and the creation of the Warning Coordination climate predictions, although other national centers Meteorologist position. still outperform NCEP by certain measures of numeri- cal modeling skill. One objective way to evaluate the performance of NWS global medium-range forecasts is NATIONAL CENTERS to compare their accuracy to that of model-based fore- casts made by other operational weather centers of the Each of the goals of the MAR directly affected and state of the atmosphere at around 18,000 ft (500 hPa). was affected by the research, technological develop- Figure 4.11 compares the upper atmosphere forecast ment, and services conducted within all of the NWS performance of several operational centers, including national centers, particularly those within NCEP. As NWS, for 1985 to 2009, averaged over the Northern the science and technology of weather, climate, and Hemisphere. hydrologic prediction evolved, the demand for more Clearly, most of the models, including the GFS, quantitative, accurate, and precise forecast products have exhibited steadily increasing skill over the post- from local forecast offices and national operational MAR period, although the European Centre for forecast centers increased. To develop and deliver such Medium-range Weather Forecasts (ECMWF) consis- products, work undertaken at the National Centers had tently outperformed NCEP (and all other operational to evolve, and the products needed to be better dissemi- global medium-range forecast models) throughout this nated to local forecast offices. By most accounts the period. NWS national centers in general, and the reorganized Wedam et al. (2009) compared surface forecasts of NCEP in particular, have made significant progress in sea level pressure along the East and West Coasts of the development and delivery of such products. the United States during the winters of 2005 through The current scientifically and technologically 2008. On average, the NCEP errors were 26 percent advanced state of NWS could not have been achieved greater than those of the ECMWF. without the significant influence of the National Cen- Both NCEP and the ECMWF have been produc- ters and an information infrastructure to provide data ing probabilistic ensemble forecasts operationally since and forecast products to forecast offices. Furthermore, December 1992. In order to compare performance, the reorganizing of NCEP appears to have enabled a verification exercise was carried out jointly by staff an environment that can evolve as computational from both agencies and also from the Meteorological capacity and scientific advancements evolve. Since the Service of Canada (MSC) using 2002 data (Buizza et reorganization of NCEP, several major supercomputer al., 2005). The ECMWF ensemble outperformed the acquisitions as well as the development of ‘backup’ NCEP ensemble at all lead times. computational facilities have occurred. Numerical Froude et al. (2007) and Froude (2010) compared modeling and data assimilation algorithms and the the performance of the NCEP and ECMWF ensemble database and computational architectures on which forecasts for forecasting extratropical cyclones in the they depend have, in turn, evolved significantly since Northern Hemisphere. The ECMWF consistently the MAR. The continuing evolution of NCEP and produced better forecasts than NCEP. its capabilities underscore the success of the MAR in Recent reports have made steps toward assessing the enabling a more evolutionary paradigm to prediction reasons NCEP is still outperformed by other national operations as opposed to a move to a new narrow or centers, and point to important future directions for singular operational paradigm. These successes can be enhancing the GFS (e.g., NRC, 2010; UCAR, 2010). measured in terms of the continually improving skill of weather, climate, and ocean models. There has been a Finding 4-4 great broadening of the user base and breadth of prod- Numerical weather forecasts produced by the National ucts being generated by the National Centers now, as Centers for Environmental Prediction (NCEP) and opposed to the pre-MAR period. the associated guidance information and products, Progress in NWP at NCEP has been significant improved steadily over the course of the Moderniza- and MAR-era improvements have placed the NWS tion and Associated Restructuring. However, the as one of a handful of world leaders in weather and

OCR for page 45
63 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING FIGURE 4.11 Seasonal mean anomaly correlations of 5-day forecasts of the 500 hPa heights for different forecast models (NCEP’s Global Forecast System [GFS], ECMWF [EC in figure legend], UK Meteorological Office [UKMO], Fleet Numerical Meteorology and Oceanography Center [FNMOC], the Coordinated Data Analysis System [CDAS], and Canadian Meteorological Centre [CMC]) from 1985 to 2009. A higher anomaly correlation indicates better model forecast performance. Seasons are three-month, non-overlapping averages. The green shaded areas at the bottom are the difference between the ECMWF and GFS performance. The data show that performance of all models increased steadily over the period, but GFS performance still lags that of the ECMWF. SOURCE: National Centers for Environmental Prediction. performance of some NCEP models, particularly from ASOS, NEXRAD, and GOES-Next to the pri- the Global Forecast System, continues to lag behind vate sector and public has been critical in expanding some other national centers, including the European the weather enterprise, and likely contributed to the Centre for Medium-range Weather Forecasts. increase in both quantity and quality of meteorology research. Because of its open-source nature, AWIPS is used by other federal agencies, universities, and research PARTNERSHIPS institutions, which facilitates scientific collaboration. In general, the MAR strengthened the relation- ships between the NWS and other members of the Other Federal Agencies weather enterprise. This was particularly true of the Among the many successes of the partnerships of partnership between the NWS and the private sec- NWS with FAA, DOD, and NASA in financing and tor, a relationship that historically had difficulties. implementing the MAR was the capability of the indi- For example, the availability of weather information

OCR for page 45
64 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT vidual agencies to achieve a step-function increase in Federal Highway Administration (FHWA), and the technological capability at a relatively smaller per-agency U.S. Coast Guard (NRC, 2002). cost. Individuals close to the MAR generally believe that Since the time period of the MAR, declining costs the joint acquisition also resulted in a closer working for instruments and widespread availability of afford- relationship between meteorologists associated with the able digital data communication has led to a prolifera- four agencies (Bjerkaas, 2011; Misciasci, 2011), although tion of remote and in situ sensor networks. Such net- there is still room for improvement in the relationship works, owned by a variety of public and private sector between the NWS and its federal agency partners. entities, now exist alongside the national observing According to the 1995 NRC report on aviation technologies established by the MAR. Research into weather: ways of forming new partnerships that organize and share the large volume of information from this totality the NWS now realizes that the FAA did not serve as an of observing infrastructure is currently ongoing (e.g., effective intermediary between the NWS and aviation NRC, 2004, 2009, 2010). weather users with regard to generating performance requirements for the Automated Surface Observing System (ASOS). Partly as a result of this situation, Private Sector the NWS has had to augment some ASOS units with human observers and develop plans for increasing the One MAR legacy is a greatly improved relation- capabilities of deployed ASOS units to meet aviation ship between NWS and the private sector, based on needs (NRC, 1995a). the personal experience of various committee members A finding from that report was the need to develop and the limited testimony received from private sector a common understanding of aviation weather require- participants (Friday, 2011; Myers, 2011). It took at ments between the FAA and NWS as a critical first least five years after the formal end of the MAR (and step in planning improvements. Revisions to the 1977 at least two years after the NRC’s Fair Weather report) FAA-NWS umbrella Memorandum of Understanding for any kind of noticeable improvement, but today both (MOU) in the late 1990s and in 2004 have helped to the NWS and the private sector view the relationship develop this understanding. As a result, maintenance as more synergistic than competitive. Implementation and capability upgrades to the NEXRAD system of recommendations from the Fair Weather report has are viewed as reflecting the needs of both agencies, played a role in this improvement, along with signifi- although the ASOS system receives negative marks in cant efforts from professional weather associations such this regard as discussed above (Heuwinkel, 2011). as the American Meteorological Society (AMS), the The MAR observing systems were designed specif- American Weather and Climate Industry Association ically to meet the mission requirements of NWS, FAA, (AWCIA), and the National Council of Industrial and DOD. Other federal, state, and local government Meteorologists (NCIM). The long-term constructive agencies meet their observational data needs to varying institutional role of the American Meteorological Soci- degrees with data from these systems. For example, a ety, specifically the Commission on the Weather and 2004 NRC report that focused on road weather notes Climate Enterprise, has been critical. that “[a]ltough the ASOS provides useful data, it was Increasingly, the private sector understands the never intended to be used to characterize the roadway important role of NWS both as a source of basic data environment; therefore, additional networks that target and forecasts and as the nation’s authoritative weather the roadway environment are needed” (NRC, 2004). information source and the NWS understands the Other federal agencies that use and rely on weather data value of the private sector as both a channel for effec- to help meet their operational responsibilities include tively distributing weather information and a source for the Federal Emergency Management Administration innovative added value. As an estimated 90 percent of (FEMA), Environmental Protection Agency (EPA), weather information used by individuals and businesses Nuclear Regulatory Commission (NRC), Department originates with NWS but is transformed and delivered of Energy (DOE), U.S. Geological Survey (USGS), by the private sector, this has been an important accom- U.S. Forest Service (USFS), Bureau of Land Manage- plishment (Myers, 2011). With the private sector leading ment (BLM), NOAA, National Park Service (NPS),

OCR for page 45
65 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING implementation of emerging technologies such as social the NWS improve their performance. Students often networking and smart phones, its position as an inter- participate in these meetings as well. face to users will likely expand further. In this decade’s The MAR resulted in an improved relationship budget environment, it is also increasingly recognized between the NWS and the academic and research that a synergistic relationship can extend and leverage communities. However, there are still concerns that the NWS budget, providing better value for the nation. the structure put in place after the MAR is still not as This relationship is still fragile, depending largely open or as collaborative as it could be. Insufficient sup- on individual attitudes and informal agreements. The port for collaborative weather research programs such NRC’s Fair Weather report concluded that there should as the U.S. Weather Research Program (USWRP) or not be a well-defined separation of NWS and the the Collaborative Science, Technology, and Applied private sector (NRC, 2003a), but rather a process for Research (CSTAR) Program suggests that the NWS is promoting the partnership, and a de facto distinction not fully engaged with the research community. Greater has been emerging. One means of stabilizing the roles support for such programs would aid the transition of is to document successful examples of public-private research-to-operations. collaboration and to use this literature to define the To further assess the impact of colocation of NWS overlapping domains of each (Myers, 2011). Issues offices with universities, and to determine how well this remain, such as broader access to data by the private arrangement has worked for the offices concerned and sector, which is both a technology and a policy issue. for NWS as a whole, the committee sent a questionnaire It is generally recognized that neither the private sector to the relevant NWS offices (including both WFOs nor the NWS can do all things for all people, so extract- and National Centers such as the National Hurricane ing the best of both groups is critical for the success of Center). A detailed summary of the responses is given the enterprise. Accomplishing this requires ongoing in Appendix D. In general, the NWS offices report improvements in relationships and collaboration meth- that colocation has provided a positive experience with ods along with direct inclusion of the private sector in mutual benefits to both NWS and the host universities. R&D and operational improvement planning. In addition to operational, scientific, educational, and outreach benefits, the ability for NWS staff to live in a college town and work in a vibrant and forward-thinking Research Community campus environment helps to foster innovation and leads Several of the new WFOs have been located on or to attracting, hiring, and retaining high quality staff. near university campuses. This enhances the interac- Further, WFO staff report that at such locations, many tions between NWS staff and university faculty and students are recruited as NWS employees. It is certainly students. The NWS staff (particularly the SOOs) gen- possible for NWS staff to work with researchers and erally benefit from closer contact with developments in universities at a distance, but the casual, more-frequent the research field. This leads to earlier implementation interactions easily enabled by colocation add tremendous of advances in scientific understanding of weather value to the advancement of the science and the opera- phenomena as well as improved forecasting techniques. tional application of that science. When there is a lack Often the SOOs and university staff collaborate in of true colocation (as in an office being nearby, but not research efforts pointed in those directions. Students on campus), this appears to be a disadvantage. have opportunities to see close hand what goes on in a Of course, the level of interaction varies from office WFO; some work as volunteers alongside NWS staff, to office. However, the achievements of the WFOs at enhancing their experiences and preparation for jobs. Denver/Boulder, Colorado; Raleigh, North Carolina; Some students (and staff ) undertake research that can Albany, New York; and Seattle, Washington stand out as lead to results benefitting the local forecasting staff. A positive examples of the tremendously positive benefits series of regional meetings generally organized by a that can be achieved through colocation (see Appendix group of SOOs brings the NWS staff and members of D for more detail). In contrast, at one reporting office the research community together to talk about current (the National Hurricane Center [NHC] collocated with problems and learn about recent advances that can help Florida International University) there appears to be a

OCR for page 45
66 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT poor match between the university foci and operational tion, including emergency managers, fire fighters, law mandate of the NWS office leading to less than optimal enforcement, and the private sector. Through this interactions. This suggests that more care may be needed initiative, NWS products became more usable by more in selecting partners for colocation. groups. The complexity of the MAR and all its systems Strong relationships with the federal and academic could have been a detriment to its usefulness to the research communities contribute to enhanced NWS public. By including this human element, the NWS forecasting and warning capabilities. This is especially created and sustained effective partnerships between true of the NEXRAD system; research into the capabil- those who observe, forecast, and warn of the weather, ities and advantages of polarimetric radar, summarized and those who need those products for the safety of life in Bringi and Chandrasekhar (2001) and more recently and continuity of the economy. with specific reference to NEXRAD in Ryzhkov et al. Finding 4-5 (2005), has led to the implementation of a polarimetric Improved relationships with other agencies and exter- upgrade to the NEXRAD radars. Partnerships with the nal partners have proven to be one of the more impor- National Severe Storms Laboratory (NSSL) and other tant outcomes of the Modernization and Associated research groups have introduced numerous advances Restructuring (MAR). These relationships increase in the use of the radar data, a prime example being the National Weather Service’s societal impact and approaches to reduce the range-velocity ambiguities in leverage its limited budget. Success of the MAR radar observations. depended in part on leadership, initiative, and funding by National Oceanic and Atmospheric Administration Emergency Management and National Weather Service units operating outside of the MAR. Though issues remain, partnerships with During the MAR, the NWS began to develop academia and government research institutions have more and better partnerships with state and local emer- increased research-to-operation capabilities, and the gency managers. The partnerships focused initially on MAR elevated the media and emergency management better serving the emergency managers during disasters community from a customer to a partner. The relation- with incident meteorologists. These positions helped ship between the NWS and the private sector took first responders with spot forecasts for responder safety, longer to improve, but it has generally evolved into a trends, and outlooks that may affect the needs of dis- more constructive and productive one. placed survivors, and other tactical information. As part of the restructuring of the workforce, the NWS expanded this emphasis to include the WCM. OVERSIGHT AND ADVISORY GROUPS The WCM became the primary link between the NWS and the customers it serves. As the technologi- The MAR was the focus of many oversight reviews cal and organizational changes from the MAR began and advisory reports (Appendix B). Previous sections to reshape NWS products, the WCM concept began have highlighted specific cases in which the reviews to reshape the relationships with those most affected drew attention to important issues, issues whose reso- by those products. lution was important to the success of the MAR. In The NWS began to accept the philosophy that addition, there are more general benefits that flow from the perfect forecast and the most timely warning are constructively critical expert reviews of complex system worthless unless the individual and the community deployments. These benefits include ongoing relation- receive the information and take the necessary action to ships with congressional staffs, with technical colleagues save lives and property. Many state and local emergency in other federal agencies, and with other sectors of the managers embraced this outreach from the NWS and weather enterprise, such as academia and the private sec- integrated into plans and operations many of the new tor. Successful reviews not only help NWS management products and capabilities the MAR created. understand and react to technical and/or schedule and The WCM reached out to many users who depend budget issues, but help build communities of knowl- on rapid and dependable access to weather informa- edgeable support. In large part, these benefits accrue to

OCR for page 45
67 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING managements that are receptive to outside advice, and staged implementation, including responsibilities of the are able to avoid a defensive response to constructive RFCs, WFOs, and national and regional headquarters criticism. During the course of the MAR, the manage- (NWS, 1996b). The 1996 report reflects considerable ment of NWS was generally receptive to oversight and evolution in the direction and specificity of plans from able to benefit from it. This does not mean, however, that the beginning of the MAR (Fread, 1996). the committee believes more would have been better. We The MAR restructuring of the HSP was intended do believe that outside review and oversight was useful to increase the integration of day-to-day hydrology and that utility was determined primarily by the techni- and meteorology operations (NWS, 1996b). All RFCs cal quality of the oversight and by NWS management’s were colocated with a WFO; in some cases, relocation receptivity to that oversight. moved RFCs away from key clients (e.g., the Army Corps of Engineers in Portland, Oregon). RFC staff Finding 4-6 profiles were changed to include overall management Expert advice and oversight from outside the National by a Hydrologist in Charge (HIC), equivalent to a Weather Service (NWS), and the receptiveness of MIC, science and technical development by a Devel- NWS management to such advice, contributed to opment and Operations Hydrologist (DOH), similar the success of the Modernization and Associated to a SOO, and hydrologic analysis and forecasting Restructuring. by a substantially larger staff, up to a doubling in some RFCs, of degreed meteorologists and hydrolo- gists with cross-disciplinary training. Selected WFOs ADDITIONAL IMPACTS received a degreed Service Hydrologist to support the The committee limited the bulk of its analysis participation of all WFO forecasters in preparation of to those aspects of weather services that were explic- hydrologic forecast products. The restructuring did not itly included in the MAR planning and execution. provide RFCs with a services coordination position However, there are some other key areas that were similar to the WFO WCMs. significantly affected by the MAR, including hydro- The restructuring assigned responsibility for issu- logic services, coastal observations and forecasts, and ance of flood and flash flood watches and warnings to the climate record. NEXRAD observations of non- the WFOs, as well as the generation of Quantitative meteorological targets also provide data valuable to Precipitation Forecasts (QPFs) for use by RFCs. RFCs some unrelated fields of investigation. were charged with providing hydrologic forecast guid- ance to the WFOs in their region at least twice daily (rather than once) over a longer service day, producing Hydrologic Services gridded hydrometeorological products that smoothly The NWS has two principal service areas: meteo- c ross WFO boundaries from multiple automated rology and hydrology. Much of the emphasis of this sensor networks and QPFs, and assimilating high assessment has been on meteorological services. How- resolution datasets and QPFs into hydrologic model- ever, the MAR greatly improved the observation of pre- ing operations. NCEP units (e.g., HPC and SPC) cipitation through the deployment of the NEXRAD were charged with providing routine and event-based network and allowed for increased coordination of hydrometeorological forecasts and analyses (e.g., QPFs, WFOs with River Forecast Centers (RFCs), thus probabilities of exceeding RFC flash flood guidance) allowing NWS to expand its hydrology mission and from NCEP modeling activities. Other NWS units services (NRC, 1996b). The NWS Hydrologic Services (e.g., the Office of Hydrology, regional headquarters) Program (HSP) had two roles within the MAR: as an were also assigned hydrologic services responsibilities integral participant in the restructuring, and as a key under the MAR. customer of the modernized technology (e.g., NWS, Each RFC received multiple AWIPS workstations 1989). The report Hydrometeorological Service Opera- to obtain and use the hydrometeorological informa- tions for the 1990s describes pre-MAR hydrometeoro- tion, forecasts, and guidance products from the WFOs logical operations within the NWS and details plans for and NCEP. Additional software tools were needed by

OCR for page 45
68 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT the RFCs to interactively analyze, quality control, and complement of information processing tools required assimilate the dramatically increased flow of hydrome- to fulfill those functions. Nor did it include any assess- teorological data and forecasts from multiple WFOs ment of RFC needs for AWIPS capabilities, limiting for use in hydrologic modeling operations. The tools the capability of RFCs to request additional capabili- were not provided as part of AWIPS, although they ties, such as storage or processing speed. RFCs use were needed to fulfill RFC responsibilities to support dynamic hydrologic models that must be calibrated, WFO operations. requiring large archives of data much like the National The RFCs were also key customers of the MAR. Centers, and substantial data analysis and quality The intent was for the NWS hydrological services control. RFCs must also consider unique hydrometeo- program to capitalize on the MAR’s technological rological processes within their region, and they have improvements to increase the specificity and accuracy unique partnerships, such as agencies with regulatory of flood and flash flood guidance to WFOs, and to responsibilities and hydropower production entities develop a significantly expanded suite of hydrometeoro- that need highly interactive access to RFC forecasts, logical products and services. During the MAR, NWS products, and even computing resources. The RFCs was engaged in planning and early implementation of shifted personnel hired or trained through the restruc- the Advanced Hydrologic Prediction Service (AHPS), turing to information technology software develop- which also aimed to improve and expand hydrologic ment, delaying development of advanced hydrologic forecasts and services. The MAR and AHPS were very model capabilities, calibration, forecast verification, much intertwined, with the MAR being considered as and probabilistic and ensemble forecasts. For example, one of four components of AHPS, and AHPS as an RFC hydrologic professionals developed hardware integral component of a modernized NWS. Hydrologic configurations and software for producing gridded model development, calibration, and forecast verifica- products, remote ensemble processors, and massive tion were considered activities under the MAR and relational databases with high speed performance. In AHPS. Although AHPS wasn’t funded until midway one RFC, 7 out of 10 hydrologic staff were focused through the MAR, it was essential for enabling the on information technology rather than hydrologic RFCs to capitalize on MAR advancements. science and development during the MAR. The MAR clearly improved coordination among An ongoing, challenging legacy of the MAR is hydrologic and meteorological operations, and enabled that the qualifications for hydrologist positions were significant expansion of products and services. For not upgraded to require degreed hydrologists, but example, the RFCs moved from forecasting only the instead allowed meteorologists to move into hydrol- traditional peak flows at select forecast points to 6- to ogy positions, even within RFCs. While much work of 10-day hydrographs that predict the continuous flow the RFCs (70 to 90 percent in recent estimates across at points within a specific watershed. Improvements three RFCs according to onsite interviews) focuses on began even pre-MAR, as some RFCs and the Office quality control of hydrometeorological records where of Hydrology participated in early demonstrations of meteorological training is effective, negative conse- the complementary aspects of operational hydrology quences of this staffing challenge include limitations and meteorology planned under the MAR (e.g., QPFs, in the capability of RFCs to calibrate their hydrologic flash flood guidance, through the Prototype RFC models. This issue was noted in a mid-MAR review Operational Test, Evaluation, and User Simulation of hydrometeorologic operations (NRC, 1996b). The [PROTEUS] project). staffing profile for hydrologists is imbalanced; of 600 It appears that MAR planning did not fully hydrologist positions, only about 200 are degreed account for the unique characteristics of RFCs and hydrologists and the limited opportunities for career hydrologic operations compared to WFOs, NCEP, advancement of hydrologists create difficulty in recruit- and meteorological operations. Collectively, RFCs ing new employees (Carter, 2011). were intended to serve the WFOs in a manner similar As a whole, the MAR had a positive impact on to NCEP, but at a regional scale (NRC, 1996b). How- hydrologic forecasts and services. The hydrologic ever, the MAR did not provide the RFCs with the full services program took some lessons from the MAR

OCR for page 45
69 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING Climate Record and has used them to inform the design of their institutional approach to implement AHPS and the Reviews of early plans for the MAR noted that Community Hydrologic Prediction System (CHPS). little attention had been given to issues of long-term Key lessons acted upon include the need for organi- management of the vastly greater stream of observa- zation and planning, the need for “full buildout of tions from MAR technology or to the quality of the limited cases” with full interface development, and c limate record, and the reviews repeatedly stressed bottom-up input about the resources needed to imple- the importance of preserving the climate record as ment the larger vision. The recent addition of Service A SOS was deployed (NRC, 1991, 1992b, 1993). Coordination Hydrologists (SCHs) at the RFCs was Recommendations were clear and strongly worded, based on their evaluation of the success of the WCM e.g., “. . . the preservation of data quality for climatic in coordinating with external partners and customers. purposes should have equal priority with its mission of Further, the hydrologic services program desires to providing forecasts” and “[w]hen instrument sites are have a hydrologic-centric MAR, especially to address changed, simultaneous operation at the old and new current staffing profiles. sites should occur until adequate statistics on the dif- ference of observations between sites can be developed. Coastal Observations and Forecasts These statistics should be recorded carefully and made readily available” (NRC, 1991). The 1992 NRC report Although the MAR did not explicitly include tech- included a separate appendix about data for climate nological enhancements and capabilities for the U.S. studies from a standing NRC Climate Research Com- buoy and coastal observing network, there were aspects mittee, which expressed concern that ASOS observa- of marine observations and analysis that benefitted. tions of cloud types and cloud cover, present weather, Approximately 30 percent of the U.S. population is snowfall and water equivalent, total sunshine, radiation, concentrated in coastal communities that border the and turbidity would be insufficient for climate studies ocean (Crowell et al., 2007). Because of the geographical (NRC, 1992b). The 1993 NRC report noted that prior prominence of the coastal regions, an NRC review panel recommendations relating to the climate record had (NRC, 1999a) highlighted the need for NWS assess- not been addressed (NRC, 1993). Those same reports, ment of the AWIPS system for coastal marine weather though, also noted that the MAR provided opportu- forecasts and warnings, which had not been part of the nities to enhance the climate record by providing new testing that took place within the MAR. The ASOS kinds of data not previously available (e.g., NEXRAD and NEXRAD deployments significantly enhanced the precipitation estimates). observing capabilities in coastal regions. In addition, For this assessment, comments were sought from the higher spatial and temporal observations obtained the NWS Climate Services Division (CSD) and the with the GOES-Next satellites over data-sparse ocean National Climatic Data Center (NCDC). Siting of regions improved forecasts of, for example, Pacific ASOS stations was clearly driven by aviation require- Ocean storms approaching the west coast. Even given ments, not considerations of the climate record. Con- some of the documented shortcomings previously dis- tinuity plans for concurrent observations at limited cussed (e.g., reliability issues associated with ASOS; sites were developed by the NWS Office of Science siting of NEXRAD radars at high altitudes), the new and Technology, following NWS Directive 10-21, capabilities provided forecasters with unprecedented but according to the CSD, those studies were never observations of the mesoscale coastal weather phe- completed. However, a series of commissioned overlap- nomena in real time. The AWIPS capability gave the ping observation studies were conducted in the 1990s forecasters for the first time an integrated depiction of at a number of sites throughout the United States for coastal mesoscale meteorology that included the new varying periods of time, in all cases less than one year. ASOS and NEXRAD observing systems and GOES- Other studies provide additional insight (Brazenec, Next, blended with the existing observing network, 2005; Butler and McKee, 1998; Doesken and McKee, including coastal buoys (Reynolds, 2011). 2000; Kauffman, 2000; McKee et al., 2000; McKee

OCR for page 45
70 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT et al., 1994a, b, 1996a; Schrumpf and McKee, 1996; ally by staff. Over time, ASOS has become one of the Sun et al., 2005). Comparison of ASOS and manual most robust data collection systems ever fielded and the observations are complicated by differences in gauge advantages of the greater number of high quality sta- locations, ranging from just a few hundred feet to more tions, the station-to-station uniformity, the improved than one mile, although with little elevation differences. instrument siting and the rigorous (in most cases) However, local exposure and vegetation differences can maintenance is providing the community with a rich be significant (Guttman and Baker, 1996; McKee et al., dataset for future climate studies. 1995). Many performance issues are associated with the Further, the MAR more broadly, ultimately had a ASOS instrumentation package. A detailed description positive impact on the climate record as the emphasis of the ASOS impacts on the climate record for different on data stewardship and preserving weather observa- observed variables is provided in Appendix E. tions for climate-quality records increased. Some of this Converting to ASOS has had a significant impact improvement required adjustments that took place after on the climate record. Discontinuities in temperature the formal completion of the MAR, including quality data occurred due to changes in instrumentation as control tools, such as NCDC’s Datzilla. In addition, the well as changes in the observing location that occurred NWS Climate Services training program has been used at most airport locations. There was also a significant to inform NWS staff of proper data stewardship prac- tices. Lastly, the climate services9 outreach program has impact on the cloud record with the elimination of manual observations and use of automated ceilometers. expanded the overall knowledge base of users regarding This was especially detrimental with cloud observations the climate data record. limited to 12,000 feet above ground level. Relative humidity was affected as well, due to instrumenta- NEXRAD Observations of tion changes. The negative impact on precipitation Non-Meteorological Targets measurements was severe with the conversion from the universal gauge to tipping buckets, which had dif- A weather radar receives echoes not only from ficulty accurately capturing medium to high rainfall hydrometeors but also from other objects suspended rates and solid precipitation. Alterations in wind shields in the atmosphere—including dust or smoke particles also affected the continuity of precipitation measure- if they are sufficiently dense and close enough to the ments. Observations of snowfall, snow equivalent, radar, as well as insects and birds. Many such echoes, total sunshine, and active weather phenomena are no once referred to as “angel echoes” (Battan, 1973), have longer available. These impacts have created a special come to be recognized as arising primarily from insects challenge for climatologists. Changes in instrumenta- (e.g., Gauthreaux et al., 2008; Russell and Wilson, tion, in the locations of these instruments, and in the 1997; Wilson et al., 1994). Those echoes can provide observational methodology (resulting from the removal useful tracers of the winds (provided the insect motions of the human observer) have created inhomogeneities do not differ greatly from the winds), and also provide in the climatic records at these NWS and FAA airport data useful to entomologists studying insect move- sites. Without homogeneous records, computation of ments or migrations (e.g., Chapman et al., 2004, 2011). long-period means and frequencies of observed vari- The sensitivity of the NEXRAD system has greatly ables becomes meaningless as abrupt step changes in enhanced the value of the NWS network data for such the time series are introduced. investigations. From another perspective, however, ASOS did Echoes from birds are more likely to contaminate offer something unprecedented within the climate wind velocity estimates, because the birds often move observing community: near real-time data collection with appreciable velocity differentials (e.g., Serafin and and archival. Data observations could now be elec- Wilson, 2000). However, those echoes have proven quite tronically transmitted and readily available, as opposed useful to biologists studying bird and bat behavior (e.g., to the historical record keeping, which took a month Gauthreaux and Belser, 1998; Horn and Kunz, 2008). or longer to publish data that were hand-recorded on 9Climate services include observations, monitoring, forecasting, paper, mailed to the data center, and keyed in manu- and assessments of climate.

OCR for page 45
71 IMPACT OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING Finding 4-7a community. The testbed concept and risk reduction The Modernization and Associated Restructuring activities emerged out of the MAR framework as (MAR) improved collaboration among hydrologic well. One of the lessons of the MAR was the value of and meteorological operations within the National prototypying new operational concepts (e.g., PROFS Weather Service, and allowed significant expan- and the pre-NEXRAD and pre-AWIPS systems). sion of hydrologic forecast products and services. This pre-operational prototype paradigm has been However, the challenges facing the River Forecast advanced following the MAR and embraced by the Centers were magnified because the MAR did not NWS, which now has a number of successful testbeds adequately take into account the unique require- including ments of hydrologic data management, modeling, and partner collaborations. • the Developmental Testbed Center, • the Hydrometeorology Testbed, Finding 4-7b • the Hazardous Weather Testbed, The Automated Surface Observing System (ASOS) • the Joint Hurricane Testbed, was not implemented in such a way that the climate • the Aviation Weather Testbed, and record was preserved. Discontinuities that degrade • the Joint Center for Satellite Data Assimilation. computation of long-period statistics, created by changes in instrumentation and observing locations, Testbeds have accelerated the transfer of technol- are still a concern. However, the Modernization and ogy from research-to-operations; successful examples Associated Restructuring continues to offer pros- include the Joint Hurricane Testbed and the Hazardous pects for improvement of the overall national climate Weather Testbed (jointly operated by NWS and the record over the long term. Office of Oceanic and Atmospheric Research [OAR]). These testbeds improved capacity and better separation between development and operational systems for run- FRAMEWORK FOR EVOLUTION RATHER ning models (Hayes, 2011). Nevertheless, the testbeds THAN REVOLUTION primarily have a focus on transition of research-to- operations, not the broader scope needed to prototype In many respects, the changes that the NWS new concepts for methods of operations that was pres- experienced as a result of the MAR can be viewed as ent in the prototyping and risk-reduction activities of revolutionary. The MAR brought dramatic improve- the MAR. The current generation of testbeds tend ments in weather services to the nation. New technol- to operate largely independently of one another, and ogy including ASOS, NEXRAD, GOES-Next, and provide little capacity to experiment with multi-office AWIPS provided forecasters with an unprecedented collaboration on delivery of new services. Provision of set of observational and analysis tools. The new NWS new services will likely be an increasingly important organizational strategy transformed the forecast offices requirement in the future. Some of the current testbeds into a modern national network of WFOs. The work- have limited capacity to engage key stakeholder groups, force transitioned from two-thirds technicians, to including emergency managers, media, and commercial two-thirds professional meteorologists. Many WFO weather service providers, in developing and evaluating staff now have a college degree, and many SOOs have new service concepts. advanced degrees. It is also becoming more common Despite some of the shortcomings of the current for staff in other positions to possess advanced degrees testbed system, the framework established during the (Sokich, 2011). MAR provides the NWS excellent opportunities for Following the official end of the MAR in 2000, new collaborations and partnerships, responding to the a framework was left in place so that the technology ever-increasing interdisciplinary nature of meteorology and NWS organization could continue to grow in and hydrology. The MAR established a foundation for a n evolutionary manner. Examples of this evolu - evolution that will allow the NWS to better meet the tionary framework are post-MAR upgrades to the future needs of the United States. ASOS, NEXRAD, and AWIPS systems, occurring in tandem with technological advances in the wider

OCR for page 45