This chapter focuses on the effects of the Modernization and Associated Restructuring (MAR) of the National Weather Service (NWS) on the provision of weather services to the Nation after 2000. The actual impact of the MAR is compared to the promised benefits presented in Chapter 2, and summarized in specific findings about the major aspects of the MAR. The MAR brought such major changes in the capabilities and operation of the NWS that its effects took, in many cases, several years after 2000 to be realized. This is particularly true about the skill of atmospheric analysis and forecasting, which has improved steadily since the end of the MAR, as well as the relationship of the NWS with both the private sector and the emergency management communities. Therefore, this chapter contains several rather extensive discussions of MAR impacts extending up to the present day.
In addition to the planned system improvements that were the objective of the MAR, execution of the project itself left a legacy of institutional and cultural changes at NWS, largely for the betterment of the organization. Critical to understanding this legacy is differentiating between the near-term impacts during the MAR (influenced by the challenges of dealing with change) and the longer-term impacts (after the changes had been institutionalized).
Management Context and Constraints
The capability of NWS to function within the greater context of issues discussed in Chapter 3 was considerably improved as a result of the MAR. The staff perception now is that NWS is widely seen as more authoritative, it is doing its job better,1 it manages relationships more effectively, and it is more focused on customers and understanding their needs (committee member WFO site visits, see Appendix C for list of WFOs visited). That is not to say that contextual issues have disappeared. Technology is still evolving more rapidly than the NWS can respond, particularly in the area of communications (e.g., social media) and applications. Infrastructure put in place during the MAR is now as much as two decades old, and could present a cost liability as it requires replacement. There is also an increasing need to leverage partnerships; these partnerships bring new challenges, such as quality and standards arising in the case of data partnerships (e.g., weather observing networks from a variety of sources; NRC, 2003b).
Budget and Schedule
NOAA and NWS’s experience with the budget and schedule challenges of the MAR could have resulted
1Employee comments from one WFO included “the reorganization allowed us to focus better, and the modernization allowed us to do a better job within that focus.”
in an improved capability to manage large, complex procurements, but it is not clear whether that was achieved. Issues with upgrading the NEXRAD system to dual-polarization radars and the implementation of AWIPS-II, suggest that either many of the MAR lessons were not internalized within the agency or that they are not relevant in the current environment. There have also been issues with the National Polar-orbiting Operational Satellite System (NPOESS) program. Although it was managed under an Integrated Program Office and not according to the typical NOAA program approach, it appears to also not have benefited substantially from lessons learned from the MAR.
Organization and Staff
One of the most important results of the MAR was the organizational transition of the meteorological staff. The ratio of technicians to professional meteorologists evolved from 2:1 to 1:2 and the number of staff was reduced by about 10 percent (Sokich, 2011). Based on committee member visits to WFOs (see Appendix C for list of WFOs visited), the transition is now viewed positively by employees, but a wide range of issues persist, presenting opportunity for further improvement. Employees appear to have learned and retained the value for ongoing innovation and change, recognizing that it is essential to ongoing organizational survival and improvement. The NWS focus on extensive staff training has been retained as well, but much of the training is now done online or on the job due to budget limitations (Spangler, 2011). A key issue that arose during the MAR, the balance between standardization of office size and structure and local flexibility, remains a central tension within the organization. Activities and process development to better achieve the correct balance are an ongoing focus. For example, there is some concern that requirements of some staff positions vary from office to office, and that these variations are not adequately reflected in job descriptions and staffing levels. Field office location continues to be a concern in some cases, particularly where the WFO is not sited close to the primary community within its area of responsibility. The MAR had clear and lasting impacts at the field level that are discussed in more detail later. It is less clear whether the MAR improved the overall organizational efficiency at the executive level. However, the realignment of the National Centers that restructured several of the core technological activities of the NWS presumably had a streamlining effect on the management of NWS activities.
Many of the processes introduced to execute the MAR have been retained in one form or another. One key process, the research-to-operations transition, continues to be a major issue (MacDonald, 2011) and has been the subject of numerous NRC reports (e.g., NRC, 2000, 2003b, 2010). Partner relationships, (Congress, private sector, the National Weather Service Employee Organization, media, emergency managers) have been substantially improved in most cases (Friday, 2011; Hirn, 2011; Myers, 2011). Overall NWS processes are now more flexible and responsive to evolving context, though there is considerable room for further improvement.2 Among the more important legacies is the capability to assess performance,3 instituted in the early 1980s and used during the MAR in part to satisfy the Congressional mandate for no degradation of service. However, it is often difficult to obtain performance statistics outside the NWS. The government procurement process, within which NOAA and NWS have limited flexibility to configure procurement to their particular needs, continues to be a major constraint. There is evidence of this in the upgrade of the NEXRAD system to dual-polarization radars and the implementation of AWIPS-II.
A notable issue with the processes involved in implementing the MAR was the lack of a systems architecture (GAO, 1994). Necessary elements include developing a system-of-systems architecture and concept of operations based on defining achievable, quantifiable mission goals and prioritizing user needs. Such an architecture would have tied the top-level goals and objectives for each individual system with the specific user needs for each individual system, and synthesized these into a set of system-of-systems goals that are
2An example is the ASOS system, which logs data every minute but only reports hourly summaries. To access minute-scale data, each WFO must connect with each ASOS site using dial-up access.
3Performance and performance degradation can involve subtle issues, as with changes in WFO performance when an office is moved, or changes are made in the storm reporting system (Smith, 1999).
functions of the separate systems all working together. In the case of the MAR, the individual systems were ASOS, NEXRAD, GOES-Next, the National Centers computer systems, and AWIPS. One possible goal of this system-of-systems could have been a specific, measurable improvement in Probability of Detection for various types of severe weather or even an overall scorecard that encompasses multiple metrics that are deemed important to the NWS and made available to the public. Exercise of this rigorous process would next have led to development of an optimal set of top-level requirements for the respective individual systems’ contractors.
Tying the mission goals and key performance metrics to specific user needs via top-level requirements analysis and documentation is essential to enable the contractor to develop a design against a set of requirements that meets both the mission goals and the user needs. Providing a clear and concise set of documents to the contractor early in the process is crucial so that they can execute efficiently and be held accountable for meeting budget, schedule, and performance goals. Lack of such a systems architecture introduces considerable risk to a program the size and complexity of the MAR. The larger and more complex a program effort is, the more important it is to utilize effective systems engineering processes. Without this the program manager (whether government managing contractor or contractor managing subcontractor) loses a major management tool. Setting mission performance metrics also allows for a quantitative assessment of the success of a program upon completion. An illustration of the lack of adequate application of the system engineering process is the AWIPS program. AWIPS requirements were based on user needs, but they were apparently not tied to mission-based goals (GAO, 1996c). Also, the large, complex set of over 20,000 requirements indicates a lack of prioritization of user needs. The contractor failed to develop a design that met the needs of the primary users, but with a lack of prioritization and overwhelming number of requirements, this is understandable.
Many of the institutional changes (management structure, culture, processes, partner relationships) introduced to implement the Modernization and Associated Restructuring (MAR) have been retained by the National Weather Service (NWS). Most of these “institutional byproducts” have been as valuable as the MAR improvements themselves and will help the NWS to continue to modernize. However, from viewing more recent projects, implementation of a rigorous systems engineering process to facilitate more effective management of the procurement and development of large, complex systems appears not to have been institutionalized within the National Oceanic and Atmospheric Administration. The systems engineering process needs to start at the beginning of the program, in the agency’s program office.
Although the technologies improvements of the MAR fell behind schedule and had larger than anticipated costs, they contributed to the capability of the NWS to provide improved weather services to the nation. This improvement is particularly evident in the forecasting and detection of severe weather such as tornadoes and flash floods, and will be discussed at the end of this section.
Automated Surface Observing System
The replacement of human observers with the Automated Surface Observing System (ASOS) was quite controversial at the time of the MAR, and continues to be controversial. Through the years, a number of conflicting reports from a variety of sources have both lauded and criticized ASOS and its implementation. From the outset, the ASOS implementation was designed to provide a more robust, hourly, and automated surface observation capability at over 1,000 airports (Figure 4.1). The manual observations being collected at the time of the modernization were at 250 airports, and the observations were only taken during the hours that each airport was open. Although automation was seen as both a cost cutting measure and an opportunity to collect more data, the numerous stakeholder groups that were destined to use the data questioned the quality of the data collected. In addition, each of the primary users of ASOS, the NWS, FAA, and DOD, had a different set of requirements for the data and clear, cohesive metrics to evaluate the success of ASOS were never determined. Each user group had a different set of metrics, and therefore judged the suc-
FIGURE 4.1 Locations of Automated Surface Observing System (ASOS) sites in the United States. The 315 ASOS sites managed by the National Weather Service are indicated by red diamonds. Blue triangles, blue circles, and green triangles indicate the 571 Federal Aviation Administration, 75 Navy, and 48 Air Force ASOS sites, respectively. SOURCE: National Weather Service.
cess of ASOS through its own lens. Another key issue in the implementation of ASOS was the lack of field testing. This lack of preliminary reliability testing led to problems with the sensor suite remaining undiscovered until after ASOS was deployed (GAO, 1995h). Furthermore, ASOS algorithm development likely could have benefited from the type of prototyping that occurred for AWIPS through PROFS.
At the end of the MAR era, there were still internal reports being commissioned by both the FAA and NWS to examine ASOS. For example, a 1999 FAA document claims, “…after years of development, ASOS correlates quite closely with human observations most of the time” (AOPA, 1999). No references are listed, no studies are cited, and it is only an anecdotal statement.
NEXRAD resulted in the ability for NWS forecasters to observe weather phenomena at higher resolutions than its pre-MAR technological predecessors, but the advancement for weather and climate forecasting that was realized from the deployment of ASOS was not as dramatic. Because ASOS was designed primarily to support airport aviation needs, and because of well documented issues with sensor performance as they pertain to weather and climate studies, many scientists turned to developing their own networks for surface observing. These regional and state networks, called mesonets, were typically operated by state entities and agencies. Examples include the Oklahoma Mesonet (commissioned in 1994) and the Florida Automated Weather Network (FAWN; commissioned in 1997). Data from these mesonets have become important resources for the NWS severe weather warning operations as well as research. Because the mesonets are state initiatives the coverage is not even across the nation, or sometimes even across a region. Such uneven coverage needs to be addressed as the weather enterprise further develops. A 2009 NRC report provided recommendations for creating a “network of networks” with even coverage across the nation (NRC, 2009).
By 2003, the NRC Fair Weather report outlined nine major examples of ASOS failures where climate studies are concerned. The report states, “[i]n the ideal, the ASOS observations would be error-free and representative of actual conditions. Therefore the interim climate summary, daily climate summary, preliminary climate data, and final official climate record would all agree with each other and all reflect the best possible estimate of conditions. As the nine representative cases make clear, this ideal situation is not always met” (NRC, 2003a). In addition, Horel et al. (2002) state that the widespread use of ASOS will continue to impede efforts to monitor Earth’s climate and study its variability. The impact of ASOS on the climate record is discussed later in this chapter and in Appendix E.
Next Generation Weather Radar
The 1-degree beamwidth and Doppler capability of the NEXRAD radars provided forecasters with enhanced ability to identify weather features of concern. The NEXRAD network is largely responsible for the improvement in the NWS capability to detect severe weather such as tornadoes, as discussed below. The broad national coverage of the NEXRAD radars was also a distinct improvement over that of the predecessor WSR-57, WSR-74, and Air Weather Service FPS-77 systems (Figure 4.2). Maddox et al. (2002) provide a more recent analysis of NEXRAD coverage.
The NEXRAD Product Improvement Program has continued to capitalize on continuing advances in technology and science underlying the processing and use of the radar data. An “R&D” NEXRAD at NSSL provides a testbed for evaluating proposed system improvements (Zahrai and Zmic, 1993).
Implementation of the recommendations from NRC (1991) in the NEXRAD program benefitted the nation through an organized research-to-operations program leading to a series of substantial upgrades to the basic NEXRAD system. Examples range from the conversion from the initial proprietary computational system to an open architecture to the forthcoming polarimetric upgrade.
Another recommendation related to the need for ongoing training programs for NWS personnel was as well not implemented:
The National Weather Service should develop a continuing comprehensive training and education program so that the skills of the Next Generation Weather Radar maintenance and operational staffs, as well as the meteorologists and hydrologists, reflect the ever-changing state of the art (NRC, 1991).
The intensive on-site training provided at the outset of the MAR has been gradually reduced in scope; the Warning Decision Training Branch does provide a comprehensive program, but the number of people who can take advantage of it is limited. A series of COMET modules provides some online training, but these lack the hands-on element provided by the on-site experience and are not regarded as comparable. This is an item of special concern as the polarization upgrade to the NEXRAD system comes online.
Perhaps the main remaining radar coverage issue had to do with the difficult problems encountered in complex terrain. A mountaintop site provides a long-range view, but cannot see down into many of the valleys where most people would live (a problem exacerbated by the NEXRAD restriction to a minimum elevation angle of 0.5 degree). A valley site may address that problem for one or a few valleys but cannot provide broad area coverage. The NEXRAD site selection generally opted for the mountaintop; for some purposes such sites provide adequate support of forecasting and warning functions (e.g., NRC, 2005), but for others they are less satisfactory (e.g., Reynolds, 2011; Westrick et al., 1999).
NEXRAD radars are owned and operated by the USAF and the FAA, in addition to the NWS. Missions of those agencies differ from those of the NWS, and this occasionally led to some nonuniformity of operations. For example, availability of the USAF radars was an issue in the early days. Archiving of the data on a routine basis is of interest to the NWS, while the other two agencies are concerned mainly with data related to some event such as an aircraft incident; this has led to gaps in the archival record. In a similar vein, distribution of wideband data or products to neighboring installations is more important to the NWS functions than to the USAF or FAA operations; the latter tend to focus on specific airfields. Again, this has led to unevenness in the export of data from different NEXRAD sites. At the same time, it must be said that the FAA has pushed for development of capabilities (such as a
FIGURE 4.2 (a) Composite pre-NEXRAD coverage at 10,000 feet above site level for CONUS is indicated by white circles. Radar locations are indicated by diamonds (WSR-57 and WSR-74S) and circles (WSR-74C). The pink shading indicates areas that have no radar coverage below 10,000 feet above site level. (b) Composite NEXRAD coverage at 10,000 feet above site level for CONUS is indicated by white circles. WSR-88D radar locations are indicated by + (National Weather Service radars) and × (Department of Defense radars). The pink shading indicates areas that have no radar coverage below 10,000 feet above site level. The striped blue shading indicates areas where coverage at the 10,000 feet level is reduced compared with the pre-NEXRAD network. SOURCE: U.S. Department of Commerce.
gust-front algorithm) that would aid the aviation mission but are of less interest to the NWS.
NOAA’s objectives for GOES-Next were continuous Earth-viewing with retention of the existing visible imaging, higher resolution infrared (IR) imagery, improved Earth location capabilities, and a separate sounder. Despite the difficulties in program design and execution, GOES-Next introduced substantial data and product improvements. On earlier geostationary satellites, the imager and sounder could not simultaneously collect data because they used the same telescopic view-
ing apparatus, and the spin-stabilized satellite rotated on its axis viewing Earth only six percent of the time on each 360-degree rotation (GAO, 1997c). Although the initial development of a three-axis, body-stabilized spacecraft design for GOES was problematic, it ultimately resulted in successful establishment of a valuable approach. These improvements together enabled continuous, simultaneous, independent imaging and sounding. Each instrument had flexible scan control, allowing for coverage of small areas, hemispheric, and full disk global scenes. Meteorologists were able to access close-up, continuous observations of dynamic, short-lived weather phenomena, such as local severe storms and tropical cyclones, as well as obtain data on the atmospheric temperature and water vapor structure.
The implementation of GOES-Next resulted in substantial improvements to the frequency, spatial resolution, data quality, and spectral resolution of NWS geostationary satellite data. Specific impact areas include
• Imagery. Due to the Earth-pointing capability of the GOES-Next satellite, the five-channel imager could produce imagery every 5 to 10 minutes for local-scale severe weather events and every 15 minutes for CONUS coverage, and scan the full disk northern hemisphere in less than 30 minutes (with images provided every 3 hours). The continuous viewing capability is critical for monitoring severe storms (GAO, 1991b). Improvements were made in the spectral resolution and signal-to-noise performance, as shown in Table 4.1. New uses of imager data were developed. For example, the data were combined with the NEXRAD radar data to enhance winter snowstorm forecasting, nighttime fog detection was enabled using two IR channels, and the higher resolution IR imagery was useful in predicting and monitoring severe thunderstorms. Additional results include
- best 6.7 μm (IR water vapor channel) imagery ever; an order of magnitude improvement enables identification of mesoscale disturbances embedded within synoptic scale features;
- better wind data inferred from cloud drift with 4 km image resolution for better edge detection and improved target selection;
- improved wind data inferred from water vapor imagery in clear regions with 8 km spatial resolution and better signal-to-noise at 6.7 µm;
TABLE 4.1 Comparison of Measured Imager Performance for GOES-7 (Pre-MAR Generation Satellite) and GOES-8 (GOES-Next Generation).
|Wavelength||(km [E/W ×||(km [E/W ×|
|0.55-0.75||0.75×0.86||0.75×0.86||6 bit data + 2|
|counts (3 sigma)|
|3.84-4.06||13.8×13.8||3.0×13.8||6.0 K @ 230 K|
|0.25 K @ 300 K|
|6.40-7.08||13.8×13.8||3.0×13.8||1.0 K @ 230 K|
|10.4-11.3||6.9×6.9||3.0×6.9||0.2 K @ 230 K|
|0.10 K @ 300 K|
|12.5-12.8||13.8×13.8||3.0×13.8||0.8 K @ 230 K|
|0.40 K @ 300 K|
|0.52-0.72||1.0×1.0||0.57×1.0||10 bit data + 8.1|
|counts (3 sigma)|
|3.78-4.03||4.0×4.0||2.3×4.0||4.0 K @ 230 K|
|0.16 K @ 300 K|
|6.47-7.02||8.0×8.0||2.3×8.0||0.27 K @ 230 K|
|10.2-11.2||4.0×4.0||2.3×4.0||0.40 K @ 230 K|
|0.12 K @ 300 K|
|11.5-12.5||4.0×4.0||2.3×4.0||0.40 K @ 230 K|
|0.20 K @ 300 K|
a Instantaneous Geometric Field of View: The detector IGFOV (or footprint) is the size of a pixel on Earth's surface that a single detector, in the array of detectors associated with a specific wavelength, is able to “view” when looking directly below the spacecraft (the sub-satellite point).
b Sampled Subpoint Resolution (SSR): Because the combination of the imager's scan rate and detector sample rate exceeds the pixel E/W IGFOV, the viewed scene is oversampled. An IGFOV of 4 km oversampled by a factor of 1.75 provides an effective resolution, or SSR, of 2.3 km.
SOURCE: Purdom (1996).
- fog, water, and ice cloud detection both day and night using continuous 3.9 μm imagery with other channels;
- identification of super-cooled cloud;
- monitoring of snow and ice cover and the detection of cloud over snow;
- improved detection of forest fires and biomass burning;
- useful imagery well beyond the satellite’s 60-degree zenith angle making possible the detection and tracking of sea ice and polar lows;
- improved low light imaging capability with 10-Bit visible-channel data;
- enhanced land and sea surface temperature monitoring capability using 30-minute interval multispectral IR capability (Purdom, 1996).
• Soundings. With the launch of GOES-8 in 1994, continuous geostationary sounder data was available for the first time. The new, independent sounder produced 18 channels of IR data in addition to one in the visible, yielding improved vertical resolution. Soundings retrieved from the GOES-Next data proved to be useful aids in qualitative interpretation. They provided timely information about changes in atmospheric moisture and stability and gradients were better delineated. In 1997, measurements of precipitable water from the sounder were included for the first time in the input to numerical weather prediction (NWP) models (GAO, 1997c).
• Systematic Impacts. The Advanced Weather Interactive Processing System (AWIPS) included the capability to display GOES-Next satellite data on the Weather Forecast Office (WFO) workstations and to combine this imagery data with other data to aid the forecaster. The satellite improvements were critical to WFOs along the west coast, improving their capability to analyze approaching weather over the data-sparse Pacific Ocean as well as moisture surges and tropical disturbances from Mexico, the Gulf of Mexico, and the Caribbean. Many recommendations were made to NOAA by the GAO and others regarding the approach to satellite system procurement. NOAA and NASA apparently took these recommendations into consideration when planning the follow-on series of geostationary satellites, after GOES-Next. Approaches considered included procuring “clones” of prior satellites and/or instruments via sole source contracts. NOAA weighed the potential benefits of significant technological advances against the schedule and budget risks involved. NASA was positioned to once again act on behalf of NESDIS and manage the instrument contracts directly (GAO, 1996a).
National Centers Advanced Computer Systems
The strategic and operational planning for the MAR emphasized the need for dramatic upgrades in the computing capabilities of the NWS. The cited rationale included the capability to run ever more complex general circulation models and data assimilation algorithms. New computational capacity was required to assimilate the new observations available as part of the MAR, particularly the satellite retrievals and later radiances, into the various global and mesoscale numerical weather prediction models.
Public Law 100-685 called for “detailed plans and funding for meteorological research to be accomplished under this title to assure that new techniques in forecasting will be developed to utilize the new technologies being implemented in the modernization” (U.S. Congress, 1988). The Strategic Plan stated that “[f]undamental model improvements are necessary to satisfy these requirements and provide guidance products of sufficient quality and frequency to support the warning and forecast operations at each office” (NWS, 1989). The Development Division within the National Meteorological Center (NMC), together with the research in numerical modeling being undertaken at the Geophysical Fluid Dynamics Laboratory (GFDL; Princeton, NJ) were both continually involved in model development. But, so far as the committee can ascertain, the MAR planning did not explicitly include benchmarks, or a timeline, for the very extensive software development effort involved in dramatic improvements in modeling and data assimilation. It seems that it was assumed these developments would occur, without specific planning as a component of the MAR. However, by the end of the MAR, there had been substantial improvements in atmospheric modeling and data assimilation, as well as the development of an evolutionary capacity of computing technology within NWS.
Advanced Weather Interactive Processing System
Development of an advanced computer and communications system to help forecasters in field offices integrate all sources of weather data, to assist them in analyzing fast-breaking storms, and to aid in the timely preparation of warnings and forecasts was a major accomplishment of the MAR. The system provides a communications network that interconnects each WFO and includes the capability for distribution of centrally collected data and centrally produced analysis and guidance products, as well as satellite data and imagery. Together this system is termed the Advanced Weather Interactive Processing System (AWIPS) and NOAAPort (NWS, 1989).
By the end of the MAR, AWIPS was, and continues to be, used to
• provide computational and display functions at operational NWS sites;
• provide open access, via NOAAPort, to extensive NOAA datasets that are centrally collected and/or produced (e.g., NCEP NWP products, products from other centers such as NHC and SPC, and international centers producing global analyses and predictions);
• acquire and process data from an array of meteorological sensors including ASOS, NEXRAD, GOES instruments, local sources (e.g., mesoscale networks, river flow gauges, atmospheric sounders) and other sources (e.g., sensor data from commercial aircraft via the Aircraft Communications Addressing and Reporting System [ACARS]);
• provide an interactive communications system to interconnect NWS operations sites and to broadcast data to these sites; and
• assist forecasters in preparation and dissemination of warnings and forecasts in a rapid, highly reliable manner.
With the implementation of AWIPS, forecasters are now able to sit down at one workstation and view a large, complex set of weather data in as many as twelve windows. The total spectrum of weather information can be overlaid and integrated on a single map to get a unified picture of what’s happening and aid in forecasting. According to one forecaster,
AWIPS has greatly improved [forecasters’] ability to quickly ingest, manipulate, and analyze immense amounts of data. One of the most important capabilities introduced with AWIPS has been to combine graphical data (e.g., geopotential height analyses) with imagery (including satellite imagery), then view these data on a loop with easy zoom and pan capability. This has been an important function of WFO workstation technology given the rapid increase in available numerical model data (Jackson, 2011).
Although AWIPS met the meteorological forecaster needs, it did not adequately address hydrologic applications. This inadequacy reflects issues in both the requirements development and AWIPS build and test processes. The forecaster-user was very well integrated into the entire development and build cycle; the hydrologist-user needs were unable to be as well addressed due to time and budget constraints, among other issues. The lesson here is that, if all users are equal, all user-needs need to be equally addressed from program initiation throughout all processes, and this effort needs to be reflected in the planned schedule and budget. In addition, an important component, GFE (Graphical Forecast Editor), was not integrated into the AWIPS core software package. Short term warnings use AWIPS; long term forecasts and hydrology use GFE. However, because AWIPS uses the open source Linux operating system, additional software development and integration is facilitated. As a result of the MAR, forecaster workstations and some servers were also upgraded. Prior to the MAR, offices as part of AFOS had a few unlinked workstations connected via “store and forward” regional communications loops. WFOs now have half a dozen workstations linked by a high speed national data network.
Performance of Post-Modernization Forecasts and Warnings
The performance of post-MAR forecast and warning operations of the NWS were dramatically improved by the MAR. The following review is limited to tornado and flash flood warnings; numerical weather prediction and its application to general weather forecasts; and hurricane and extratropical storm forecasts, as these are the types of weather of most interest to the public (winter weather forecast performance data from before the MAR are not available).
Tornado and Flash Flood Warnings
As proposed in the Strategic Plan, one major goal of the MAR was to provide more reliable detection and prediction of severe weather and flooding. Perhaps the most striking result of the MAR has been the improvement in the probability of detecting and issuing warnings for severe weather events (e.g., Figure 4.3a,b). The probability of detection (POD)4 and warning for
4From the AMS Glossary of Meteorology: “POD and FAR are useful evaluators for binary, yes/no kinds of forecasts, and detection techniques. For example, if A is the number of forecasts that rain would occur when it subsequently did occur (forecast 5 yes, observation 5 yes), B is the number of forecasts of no rain when rain occurred (no, yes), and C is the number of forecasts of rain
FIGURE 4.3 Probability of detection (POD)4 and False Alarm Ratio (FAR)4 for (a) tornado warnings and (b) flash flood warnings. Lead times5 for (c) tornado warnings and (d) flash flood warnings. The POD and warning lead times for both tornadoes and flash floods increased steadily over the course of the MAR, while the FAR for tornadoes and flash floods remained relatively constant. SOURCE: Based on data provided by the National Weather Service.
tornadoes and flash floods increased steadily from the beginning of the MAR until it was completed in 2000. At the same time, the average lead times5 of tornado warnings issued on the basis of observations increased from under five minutes to over 12 minutes (Figure 4.3c) and flash flood lead times increased from about ten minutes to over 40 minutes (Figure 4.3d). The failure of the accompanying false alarm ratios (FAR)4 to decrease at the same time as the POD has increased has been a disappointment. This problem could have several causes including the unreported occurrence/confirmation of predicted tornadoes or the occurrence of funnel clouds that did not reach the ground, or the common problems of atmospheric sampling caused by the limitations of the void under the radar beam caused by Earth’s curvature. In any event, the warning system continues to issue many warnings that are not reported and/or realized. Increased scientific understanding of these severe weather processes and integration of such understanding into operations, as well as further improvements in technology—especially radar and radar coverage—could help improve the false alarm ratio.
The NWS severe weather warning system continues to be impacted by problems associated with the dissemination of the warnings to the population at risk. Again in the severe weather occurrences (tornado outbreaks) in 2011, timely warnings were issued but the loss of electrical power due to earlier severe weather left many households in the path of the storms without adequate means to hear the warnings and take necessary lifesaving actions.
when rain did not occur (yes, no), then POD = A/(A1B) and FAR 5 C/(A1C). For perfect forecasting or detection, POD 5 1.0 (or 100 percent) and FAR 5 0.0 (or 0 percent).”
5Lead time is the time from when the warning is issued until the time the event is reported within the warned area.
Numerical Weather Prediction and its Application to General Weather Forecasts
In addition to improved severe storm and flash flood detection resulting from the MAR technologies and service restructuring, one of the promised benefits of the MAR was to improve NWS forecasts and warnings, making them as accurate and timely as possible. Forecasts result from a complicated process that starts with obtaining all possible observations, such as direct measurements of surface and upper-air properties and remote measurements by satellites and radar. These observations are assimilated into the initialization process for numerical models, and the model output is post-processed using Model Output Statistics (MOS) procedures to develop guidance which is used, along with real-time observations and the model output itself, by field office forecasters to make forecasts.
The NWS has performed numerical prediction operations at NCEP6 beginning in the mid-1950s and continuing to today. The four-times per day execution of the models produces a wide variety of analyses and products on regional, national, hemispheric, and global scales. An evaluation of the overall performance of the NCEP global numerical weather prediction operation over the period 1985 to 2009 is presented later in this chapter (Figure 4.11).
It is a major step to go from a numerical model prediction to information that can be used as guidance to forecasters producing general weather forecasts out to about 10 days. The NWS has developed Model Output Statistics (MOS) procedures that downscale NWP model output through a statistical interpretation of the model parameters in terms of surface weather variables appropriate for the forecast time in question. MOS relates observations of a weather element to be predicted (e.g., maximum or minimum temperature, or probability of precipitation) to appropriate variables (e.g., model outputs, initial observations, and geoclimatic data such as terrain, and normal conditions) using multiple regression techniques.
At the time of the MAR, the MOS were calculated each forecast cycle for specific forecast points and the model output was interpolated to observation locations. MOS were applied to most surface weather variables. Dallavalle and Dagostaro (2004) examined the application and value of MOS alone compared to the local forecasts produced by NWS field forecasters utilizing their professional judgment as well as the MOS guidance for the period 1966 to 2004. The study analyzed forecasts for 80 stations distributed across the CONUS. Recently updated results, with data through 2010, for 36- and 60-hour forecasts of minimum temperature are shown in Figure 4.4, and for 24- and 48-hour forecasts of probability of precipitation (PoP) are shown in Figure 4.5. The study reported by Dallavalle and Dagostaro (2004) included other parameters (e.g., maximum temperature) and other forecast lead times and for the cold season as well as the warm season, with similar results as shown in Figures 4.4 and 4.5. The results clearly show the improvement in the quality of both the local forecasts and the guidance—largely reflecting the model improvement. The skill between the MOS and the locally generated product converges later in the period, showing the increasing relative value of guidance in the forecast process.
Hurricane and Extratropical Storm Predictions
The National Hurricane Center (NHC) uses a variety of models as guidance in the forecasting process. Figure 4.6 illustrates the performance of the various models showing annual average forecast track errors for the period 1994 to 2009. The solid black line is the annual average track errors for the resulting Official 48-h NMC Forecast that is issued to the public. Over the 16-year period the performance of the various models has converged, with less scatter later in the period, reflecting better data and improved model physics as well as improved computing power. The improvement in the Official Track Forecast is apparent, especially after 2001.
The long-term trend in the Official Hurricane Track errors for the period 1970 to 2009 is illustrated in Figure 4.7. The dramatic improvement in the forecast skill is apparent for all forecast lead-times although the record for the 96-h and 120-h lead times is short (since 2001) and does not extend back to the MAR period. Much of the progress in the Official Forecast can be attributed to the major advances made in numerical prediction models and the improved data resources as
6Prior to 1994, the principal national center was the National Meteorological Center (NMC).
FIGURE 4.4 Mean annual absolute error for warm season local forecasts and MOS guidance forecasts of tonight’s (36-hr; light blue and red, respectively) and tomorrow night’s (60-hr; dark blue and pink, respectively) minimum temperature generated during the 0000 UTC cycle. Mean is calculated for 80 stations distributed across CONUS. The improvement in both local forecasts and guidance is indicated by the decline in the mean absolute error. SOURCE: Meteorological Development Laboratory, National Weather Service.
FIGURE 4.5 Improvement in Brier score (Brier, 1950) of locally generated forecasts and MOS guidance probabilities of precipitation for the 12- to 24-hour (24-hr; light blue and red, respectively) and 36- to 48-hour (48-hr; dark blue and pink, respectively) forecasts during the warm season. In this analysis, results from both the 0000 and 1200 UTC cycles were combined. The improvement in both local forecasts and guidance is indicated by the decline in the Brier Score (a perfect Brier score is 0.0). SOURCE: Meteorological Development Laboratory, National Weather Service.
FIGURE 4.6 Annual average guidance 48-hr track errors for the Atlantic basin tropical cyclones for the period 1994 to 2010 from all available models. The solid black line shows the annual average 48-hr errors for the National Hurricane Center official forecast and the dashed line is the error for the “CLIPPERS” Climatology and Persistence model, which provides a statistical baseline for comparison. The colored symbols identify the various individual models. More information about the individual models is available from the source. Availability of better data, improved model physics, and improved computing power led to less scatter in the performance of the various models. The annual average track error has declined. SOURCE: National Hurricane Center, National Weather Service.
well as the growth in computing power available to NCEP and to other centers both in the United States and around the world. The skilled application of the guidance to the operational analysis by the NHC forecasters contributes to the improvement as well.
Forecasts for extratropical storms have improved as well. Charles and Colle (2009) compared the quality of NWS storm forecasts over time from 1978 to 2007 on the basis of displacement errors in the forecast positions of the centers of extratropical cyclones, compiling results from previous literature. The results are shown in Figure 4.8, and show that there was a steady improvement over that period, which includes the MAR.
Forecasts of hurricane intensity, however, have not seen the marked improvements of hurricane track forecasts. The lack of progress in the prediction of hurricane intensity is illustrated in Figure 4.9. Considerable gains in observations, especially from within the eye of the storms, and a much more concentrated research effort are required before improvements can be expected.
The Modernization and Associated Restructuring (MAR) provided for more uniform radar coverage and surface observations across the United States. The Next Generation Weather Radar network and Geostationary Operational Environmental Satellites dramatically improved the quantity and quality of data available to forecasters and enhanced the numerical weather prediction capabilities of the National Weather Service (NWS). Replacing human observers with the Automated Surface Observing System introduced significant gains, despite possible adverse affects on the climate record and the loss of some important visual elements of the observation. The Advanced Weather Interactive Processing System has been a
FIGURE 4.7 Annual average official track errors for Atlantic basin tropical storms and hurricanes for the period 1970 to 2010 with least squares trend lines imposed. The different forecast times are indicated by red (24-hr), green (48-hr), yellow (72-hr), gold (96-hr), and blue (120-hr). Data for the 24-hr, 48-hr, and 72-hr forecast show a steady decline in the annual average track error. Data for 96-hr and 120-hr forecasts are only available after 2001, so the trend in the forecast error is more difficult to discern. SOURCE: National Hurricane Center, National Weather Service.
FIGURE 4.8 Extratropical cyclone displacement errors (km) versus forecast hour for different National Weather Service forecasts for the period 1978 to 2007. The solid, black line represents the North American Mesoscale (NAM) model and the solid, light gray line represents Global Forecast System (GFS) model for the period 2002 to 2007. The black, long-dashed line represents the Limited Area Fine Mesh-II (LFM-II) model displacement errors for the 1978-1979 cool season. The data were originally published in 1982 by Silberberg and Bosart (labeled as S&B 82). The black, short-dashed line and the dark gray line represent the Nested Grid Model (NGM) and Aviation Model (AVN) displacement errors for the 1987-1988 and 1989-1990 cool seasons. These data were originally published in 1993 by Smith and Mullen (S&M 93). The results show a steady improvement in the performance of extratropical cyclone forecasts. SOURCE: Charles and Colle (2009).
FIGURE 4.9 Annual average official intensity errors for Atlantic basin tropical storms and hurricanes for the period 1970 to 2010 with least squares trend lines imposed. The different forecast times are indicated by red (24-hr), green (48-hr), yellow (72-hr), gold (96-hr), and blue (120-hr). Data for 96-hr and 120-hr forecasts are only available after 2001. Data for all forecast times show a lack of improvement in hurricane intensity forecast errors. SOURCE: National Hurricane Center, National Weather Service.
critical technological advancement that integrates the data and information provided by other MAR elements and makes them easily accessible by forecasters.
The Probability of Detection (POD) for both tornadoes and flash floods improved over the course of the MAR and after the MAR. Likewise the Lead Times of the warnings increased. However, the False Alarm Ratios (FAR) were not reduced and remain high.
Restructuring of the NWS involved substantial changes in both the office and workforce distributions. Many of these changes were viewed negatively by some NWS employees during the MAR period (NRC, 1994a), but hindsight has shown that they have greatly improved the capability of the NWS to provide weather services to the nation, and the changes are now viewed favorably by the staff (committee member WFO site visits; Hirn, 2011).
Consolidation of Offices
With the completion of the MAR, weather forecast and warning services are provided to the nation by 122 WFOs, with distribution more or less uniform across the CONUS (Figure 4.10). Each WFO has an associated NEXRAD radar, and the WFOs provide more uniform distribution of forecasting and warning services. Though there are now fewer “local” offices, the forecast and warning services are provided by staff with higher skill levels and with more advanced technology at their disposition.
Provision for much better linkages to the user community are in place as a consequence of the MAR. However some forecast offices are not necessarily optimally located within their community. Since the MAR, the availability of inexpensive wideband communication has eliminated the need to site WFOs close to their
FIGURE 4.10 (a) Locations of the 204 Weather Service Offices (red diamonds) and the 52 Weather Service Forecast Offices (green squares) before the MAR. (b) Locations of the 122 Weather Forecast Offices (red diamonds) after the MAR. SOURCE: National Weather Service.
radar. In some cases, relocating the WFO closer to the primary community in the area of responsibility would provide better service. Such primary communities vary by location, but include media markets, emergency management, and university or research facilities. Locations could be determined on the basis of predetermined service criteria. The full consequence of forecast office relocations on staffing is not entirely clear. There are examples of both the remoteness of some locations negatively affecting recruitment of meteorologists, and of journeyman meteorologists who view positions at remote WFOs as opportunities to gain important field experience and advance their careers.
While achieving the goal of an agency-wide staff reduction, the restructuring of the field office management and staff positions had a profound impact on the services provided. Placing professional meteorologists on staff throughout the country, instead of solely at a smaller number of centralized forecasting facilities, allows for increased use of numerical modeling and a more scientific approach to weather forecasts and warnings. The NWS can now maximize the utilization of new science and evolving technologies like NEXRAD and AWIPS. The proportion of professional meteorologists increased significantly and the GS pay grades were increased, making the work more professionally rewarding (Hirn, 2011). As discussed in Chapter 3, the increase in the average pay grade, and thus salary costs, likely balanced out any cost savings from a reduced workforce.
The creation of the Science Operations Officer (SOO) and the Warning Coordination Meteorologist (WCM) positions resulted in dedicated staff for two critically important tasks. The SOO serves as the focal point for the integration of new science and technologies into WFO operations. The SOO also leads research relevant to local weather issues, and coordinates the continuing professional development of the WFO staff through training. As the NWS has moved away from training at a centralized facility and toward remote educational efforts in each WFO, staff training has become one of the primary tasks of the SOO (Santos, 2011). Since the end of the MAR, an Information Technology Officer (ITO) has been added. The addition of the ITO led to the full utilization of AWIPS capabilities, and has helped maintain the still-evolving AWIPS technology.
The quality of the NWS’s warning capability corresponds with its capability to muster an ample,
fully trained local staff at its WFOs as severe weather unfolds. With current staff levels, there are always two people working each shift, 24 hours a day, 7 days a week. Though this works well in fair weather, it can become problematic during severe weather, particularly when events develop rapidly under seemingly benign conditions. While managers at individual WFOs generally plan ahead to add sufficient staff to cover forecasted dangerous weather situations, more innocuous weather scenarios that suddenly and unexpectedly “blow up” often lead to shortcomings that are directly attributable to having insufficient manpower. Several recent Service Assessments (e.g., NWS, 2003, 2009, 2010) illustrate the critical role that adequately enhanced staffing (or lack thereof) plays in the success (or weakness) of NWS warning performance during major events. Appropriate levels of staffing, beyond the normal fair weather staffing, during major weather events, are critical for fulfilling the NWS’s “protection of life” mission.
Changes in Customer Linkages
By creating a liaison between NWS and the media and emergency management communities in the WCM position, the MAR significantly improved customer service. Innovations such as NWS-Chat,7 although not officially part of the MAR, now allow for direct communication between NWS forecasters, broadcast meteorologists, and emergency managers.
This strengthened relationship between NWS and media came at a time when electronic media outlets invested millions of dollars in technology that allowed broadcast meteorologists to track dangerous weather in real-time and provide continuous on-air coverage of breaking weather situations. Increasingly, NWS Service Assessments (e.g., NWS, 2007, 2009) point to the importance of TV and radio broadcasts in providing awareness and a call to action to protect life and property. Each WFO is somewhat unique in its approach, but after the MAR, there are several examples of WFOs and local media outlets sharing resources, including Doppler radar imagery and mesonets.8
The WCM also brought the NWS much closer to the emergency management community. The core missions of emergency management and the NWS are very similar, and WCM efforts to provide continuing education and maintain strong relationships with local emergency managers have facilitated rapid sharing of crucial information during the severe weather threats. The MAR elevated the emergency management community from merely a user of weather services to a partner in the protection of life and property, therefore the post-MAR relationship between NWS and emergency management is discussed in greater detail in the later section on Partnerships.
National Weather Service staff was reduced, but technical capabilities and career paths were substantially upgraded, leading to little or no cost savings from the workforce reorganization.
The staffing level that resulted from the Modernization and Associated Restructuring allows for at least two people on duty for all shifts, but timely planning and coordination by field office managers and supervisors are required to be able to increase the staffing level for times when severe weather threatens life and property.
The Science Operations Officer position created as part of the Modernization and Associated Restructuring, in principle, allows advancements in the science community to be more rapidly integrated into operations. Communication and dissemination of weather information at the local level has been much improved by the restructuring of the forecast
7NWS-Chat is an Instant Messaging program that enables communication between media and emergency management and NWS operational personnel, and is particularly useful during hazardous weather situations. It allows sharing, in both directions, of data, weather observations, and spotter reports.
8For example, the WFO in Miami uses and relies on the WeatherBug mesonet, not only via direct ingest into the AWIPS workstations, but through the Web sites of the stations in each market who have the local contract with WeatherBug (Channel 4 in Miami, Channel 12 in Palm Beach, and Channel 2 in Fort Myers). While there is excellent radar coverage on the southeast coast of Florida, NWS does occasionally use NBC2’s weather radar imagery from the Fort Myers area in southwest Florida. Additionally, NWS-media collaboration is greatly enhanced by NWS-Chat.
offices and the creation of the Warning Coordination Meteorologist position.
Each of the goals of the MAR directly affected and was affected by the research, technological development, and services conducted within all of the NWS national centers, particularly those within NCEP. As the science and technology of weather, climate, and hydrologic prediction evolved, the demand for more quantitative, accurate, and precise forecast products from local forecast offices and national operational forecast centers increased. To develop and deliver such products, work undertaken at the National Centers had to evolve, and the products needed to be better disseminated to local forecast offices. By most accounts the NWS national centers in general, and the reorganized NCEP in particular, have made significant progress in the development and delivery of such products.
The current scientifically and technologically advanced state of NWS could not have been achieved without the significant influence of the National Centers and an information infrastructure to provide data and forecast products to forecast offices. Furthermore, the reorganizing of NCEP appears to have enabled an environment that can evolve as computational capacity and scientific advancements evolve. Since the reorganization of NCEP, several major supercomputer acquisitions as well as the development of ‘backup’ computational facilities have occurred. Numerical modeling and data assimilation algorithms and the database and computational architectures on which they depend have, in turn, evolved significantly since the MAR. The continuing evolution of NCEP and its capabilities underscore the success of the MAR in enabling a more evolutionary paradigm to prediction operations as opposed to a move to a new narrow or singular operational paradigm. These successes can be measured in terms of the continually improving skill of weather, climate, and ocean models. There has been a great broadening of the user base and breadth of products being generated by the National Centers now, as opposed to the pre-MAR period.
Progress in NWP at NCEP has been significant and MAR-era improvements have placed the NWS as one of a handful of world leaders in weather and climate predictions, although other national centers still outperform NCEP by certain measures of numerical modeling skill. One objective way to evaluate the performance of NWS global medium-range forecasts is to compare their accuracy to that of model-based forecasts made by other operational weather centers of the state of the atmosphere at around 18,000 ft (500 hPa). Figure 4.11 compares the upper atmosphere forecast performance of several operational centers, including NWS, for 1985 to 2009, averaged over the Northern Hemisphere.
Clearly, most of the models, including the GFS, have exhibited steadily increasing skill over the post-MAR period, although the European Centre for Medium-range Weather Forecasts (ECMWF) consistently outperformed NCEP (and all other operational global medium-range forecast models) throughout this period.
Wedam et al. (2009) compared surface forecasts of sea level pressure along the East and West Coasts of the United States during the winters of 2005 through 2008. On average, the NCEP errors were 26 percent greater than those of the ECMWF.
Both NCEP and the ECMWF have been producing probabilistic ensemble forecasts operationally since December 1992. In order to compare performance, a verification exercise was carried out jointly by staff from both agencies and also from the Meteorological Service of Canada (MSC) using 2002 data (Buizza et al., 2005). The ECMWF ensemble outperformed the NCEP ensemble at all lead times.
Froude et al. (2007) and Froude (2010) compared the performance of the NCEP and ECMWF ensemble forecasts for forecasting extratropical cyclones in the Northern Hemisphere. The ECMWF consistently produced better forecasts than NCEP.
Recent reports have made steps toward assessing the reasons NCEP is still outperformed by other national centers, and point to important future directions for enhancing the GFS (e.g., NRC, 2010; UCAR, 2010).
Numerical weather forecasts produced by the National Centers for Environmental Prediction (NCEP) and the associated guidance information and products, improved steadily over the course of the Modernization and Associated Restructuring. However, the
FIGURE 4.11 Seasonal mean anomaly correlations of 5-day forecasts of the 500 hPa heights for different forecast models (NCEP’s Global Forecast System [GFS], ECMWF [EC in figure legend], UK Meteorological Office [UKMO], Fleet Numerical Meteorology and Oceanography Center [FNMOC], the Coordinated Data Analysis System [CDAS], and Canadian Meteorological Centre [CMC]) from 1985 to 2009. A higher anomaly correlation indicates better model forecast performance. Seasons are three-month, non-overlapping averages. The green shaded areas at the bottom are the difference between the ECMWF and GFS performance. The data show that performance of all models increased steadily over the period, but GFS performance still lags that of the ECMWF. SOURCE: National Centers for Environmental Prediction.
performance of some NCEP models, particularly the Global Forecast System, continues to lag behind some other national centers, including the European Centre for Medium-range Weather Forecasts.
In general, the MAR strengthened the relationships between the NWS and other members of the weather enterprise. This was particularly true of the partnership between the NWS and the private sector, a relationship that historically had difficulties. For example, the availability of weather information from ASOS, NEXRAD, and GOES-Next to the private sector and public has been critical in expanding the weather enterprise, and likely contributed to the increase in both quantity and quality of meteorology research. Because of its open-source nature, AWIPS is used by other federal agencies, universities, and research institutions, which facilitates scientific collaboration.
Other Federal Agencies
Among the many successes of the partnerships of NWS with FAA, DOD, and NASA in financing and implementing the MAR was the capability of the indi-
vidual agencies to achieve a step-function increase in technological capability at a relatively smaller per-agency cost. Individuals close to the MAR generally believe that the joint acquisition also resulted in a closer working relationship between meteorologists associated with the four agencies (Bjerkaas, 2011; Misciasci, 2011), although there is still room for improvement in the relationship between the NWS and its federal agency partners.
According to the 1995 NRC report on aviation weather:
the NWS now realizes that the FAA did not serve as an effective intermediary between the NWS and aviation weather users with regard to generating performance requirements for the Automated Surface Observing System (ASOS). Partly as a result of this situation, the NWS has had to augment some ASOS units with human observers and develop plans for increasing the capabilities of deployed ASOS units to meet aviation needs (NRC, 1995a).
A finding from that report was the need to develop a common understanding of aviation weather requirements between the FAA and NWS as a critical first step in planning improvements. Revisions to the 1977 FAA-NWS umbrella Memorandum of Understanding (MOU) in the late 1990s and in 2004 have helped to develop this understanding. As a result, maintenance and capability upgrades to the NEXRAD system are viewed as reflecting the needs of both agencies, although the ASOS system receives negative marks in this regard as discussed above (Heuwinkel, 2011).
The MAR observing systems were designed specifically to meet the mission requirements of NWS, FAA, and DOD. Other federal, state, and local government agencies meet their observational data needs to varying degrees with data from these systems. For example, a 2004 NRC report that focused on road weather notes that “[a]ltough the ASOS provides useful data, it was never intended to be used to characterize the roadway environment; therefore, additional networks that target the roadway environment are needed” (NRC, 2004). Other federal agencies that use and rely on weather data to help meet their operational responsibilities include the Federal Emergency Management Administration (FEMA), Environmental Protection Agency (EPA), Nuclear Regulatory Commission (NRC), Department of Energy (DOE), U.S. Geological Survey (USGS), U.S. Forest Service (USFS), Bureau of Land Management (BLM), NOAA, National Park Service (NPS), Federal Highway Administration (FHWA), and the U.S. Coast Guard (NRC, 2002).
Since the time period of the MAR, declining costs for instruments and widespread availability of affordable digital data communication has led to a proliferation of remote and in situ sensor networks. Such networks, owned by a variety of public and private sector entities, now exist alongside the national observing technologies established by the MAR. Research into ways of forming new partnerships that organize and share the large volume of information from this totality of observing infrastructure is currently ongoing (e.g., NRC, 2004, 2009, 2010).
One MAR legacy is a greatly improved relationship between NWS and the private sector, based on the personal experience of various committee members and the limited testimony received from private sector participants (Friday, 2011; Myers, 2011). It took at least five years after the formal end of the MAR (and at least two years after the NRC’s Fair Weather report) for any kind of noticeable improvement, but today both the NWS and the private sector view the relationship as more synergistic than competitive. Implementation of recommendations from the Fair Weather report has played a role in this improvement, along with significant efforts from professional weather associations such as the American Meteorological Society (AMS), the American Weather and Climate Industry Association (AWCIA), and the National Council of Industrial Meteorologists (NCIM). The long-term constructive institutional role of the American Meteorological Society, specifically the Commission on the Weather and Climate Enterprise, has been critical.
Increasingly, the private sector understands the important role of NWS both as a source of basic data and forecasts and as the nation’s authoritative weather information source and the NWS understands the value of the private sector as both a channel for effectively distributing weather information and a source for innovative added value. As an estimated 90 percent of weather information used by individuals and businesses originates with NWS but is transformed and delivered by the private sector, this has been an important accomplishment (Myers, 2011). With the private sector leading
implementation of emerging technologies such as social networking and smart phones, its position as an interface to users will likely expand further. In this decade’s budget environment, it is also increasingly recognized that a synergistic relationship can extend and leverage the NWS budget, providing better value for the nation.
This relationship is still fragile, depending largely on individual attitudes and informal agreements. The NRC’s Fair Weather report concluded that there should not be a well-defined separation of NWS and the private sector (NRC, 2003a), but rather a process for promoting the partnership, and a de facto distinction has been emerging. One means of stabilizing the roles is to document successful examples of public-private collaboration and to use this literature to define the overlapping domains of each (Myers, 2011). Issues remain, such as broader access to data by the private sector, which is both a technology and a policy issue. It is generally recognized that neither the private sector nor the NWS can do all things for all people, so extracting the best of both groups is critical for the success of the enterprise. Accomplishing this requires ongoing improvements in relationships and collaboration methods along with direct inclusion of the private sector in R&D and operational improvement planning.
Several of the new WFOs have been located on or near university campuses. This enhances the interactions between NWS staff and university faculty and students. The NWS staff (particularly the SOOs) generally benefit from closer contact with developments in the research field. This leads to earlier implementation of advances in scientific understanding of weather phenomena as well as improved forecasting techniques. Often the SOOs and university staff collaborate in research efforts pointed in those directions. Students have opportunities to see close hand what goes on in a WFO; some work as volunteers alongside NWS staff, enhancing their experiences and preparation for jobs. Some students (and staff) undertake research that can lead to results benefitting the local forecasting staff. A series of regional meetings generally organized by a group of SOOs brings the NWS staff and members of the research community together to talk about current problems and learn about recent advances that can help the NWS improve their performance. Students often participate in these meetings as well.
The MAR resulted in an improved relationship between the NWS and the academic and research communities. However, there are still concerns that the structure put in place after the MAR is still not as open or as collaborative as it could be. Insufficient support for collaborative weather research programs such as the U.S. Weather Research Program (USWRP) or the Collaborative Science, Technology, and Applied Research (CSTAR) Program suggests that the NWS is not fully engaged with the research community. Greater support for such programs would aid the transition of research-to-operations.
To further assess the impact of colocation of NWS offices with universities, and to determine how well this arrangement has worked for the offices concerned and for NWS as a whole, the committee sent a questionnaire to the relevant NWS offices (including both WFOs and National Centers such as the National Hurricane Center). A detailed summary of the responses is given in Appendix D. In general, the NWS offices report that colocation has provided a positive experience with mutual benefits to both NWS and the host universities. In addition to operational, scientific, educational, and outreach benefits, the ability for NWS staff to live in a college town and work in a vibrant and forward-thinking campus environment helps to foster innovation and leads to attracting, hiring, and retaining high quality staff. Further, WFO staff report that at such locations, many students are recruited as NWS employees. It is certainly possible for NWS staff to work with researchers and universities at a distance, but the casual, more-frequent interactions easily enabled by colocation add tremendous value to the advancement of the science and the operational application of that science. When there is a lack of true colocation (as in an office being nearby, but not on campus), this appears to be a disadvantage.
Of course, the level of interaction varies from office to office. However, the achievements of the WFOs at Denver/Boulder, Colorado; Raleigh, North Carolina; Albany, New York; and Seattle, Washington stand out as positive examples of the tremendously positive benefits that can be achieved through colocation (see Appendix D for more detail). In contrast, at one reporting office (the National Hurricane Center [NHC] collocated with Florida International University) there appears to be a
poor match between the university foci and operational mandate of the NWS office leading to less than optimal interactions. This suggests that more care may be needed in selecting partners for colocation.
Strong relationships with the federal and academic research communities contribute to enhanced NWS forecasting and warning capabilities. This is especially true of the NEXRAD system; research into the capabilities and advantages of polarimetric radar, summarized in Bringi and Chandrasekhar (2001) and more recently with specific reference to NEXRAD in Ryzhkov et al. (2005), has led to the implementation of a polarimetric upgrade to the NEXRAD radars. Partnerships with the National Severe Storms Laboratory (NSSL) and other research groups have introduced numerous advances in the use of the radar data, a prime example being approaches to reduce the range-velocity ambiguities in radar observations.
During the MAR, the NWS began to develop more and better partnerships with state and local emergency managers. The partnerships focused initially on better serving the emergency managers during disasters with incident meteorologists. These positions helped first responders with spot forecasts for responder safety, trends, and outlooks that may affect the needs of displaced survivors, and other tactical information.
As part of the restructuring of the workforce, the NWS expanded this emphasis to include the WCM. The WCM became the primary link between the NWS and the customers it serves. As the technological and organizational changes from the MAR began to reshape NWS products, the WCM concept began to reshape the relationships with those most affected by those products.
The NWS began to accept the philosophy that the perfect forecast and the most timely warning are worthless unless the individual and the community receive the information and take the necessary action to save lives and property. Many state and local emergency managers embraced this outreach from the NWS and integrated into plans and operations many of the new products and capabilities the MAR created.
The WCM reached out to many users who depend on rapid and dependable access to weather information, including emergency managers, fire fighters, law enforcement, and the private sector. Through this initiative, NWS products became more usable by more groups. The complexity of the MAR and all its systems could have been a detriment to its usefulness to the public. By including this human element, the NWS created and sustained effective partnerships between those who observe, forecast, and warn of the weather, and those who need those products for the safety of life and continuity of the economy.
Improved relationships with other agencies and external partners have proven to be one of the more important outcomes of the Modernization and Associated Restructuring (MAR). These relationships increase the National Weather Service’s societal impact and leverage its limited budget. Success of the MAR depended in part on leadership, initiative, and funding by National Oceanic and Atmospheric Administration and National Weather Service units operating outside of the MAR. Though issues remain, partnerships with academia and government research institutions have increased research-to-operation capabilities, and the MAR elevated the media and emergency management community from a customer to a partner. The relationship between the NWS and the private sector took longer to improve, but it has generally evolved into a more constructive and productive one.
The MAR was the focus of many oversight reviews and advisory reports (Appendix B). Previous sections have highlighted specific cases in which the reviews drew attention to important issues, issues whose resolution was important to the success of the MAR. In addition, there are more general benefits that flow from constructively critical expert reviews of complex system deployments. These benefits include ongoing relationships with congressional staffs, with technical colleagues in other federal agencies, and with other sectors of the weather enterprise, such as academia and the private sector. Successful reviews not only help NWS management understand and react to technical and/or schedule and budget issues, but help build communities of knowledgeable support. In large part, these benefits accrue to
managements that are receptive to outside advice, and are able to avoid a defensive response to constructive criticism. During the course of the MAR, the management of NWS was generally receptive to oversight and able to benefit from it. This does not mean, however, that the committee believes more would have been better. We do believe that outside review and oversight was useful and that utility was determined primarily by the technical quality of the oversight and by NWS management’s receptivity to that oversight.
Expert advice and oversight from outside the National Weather Service (NWS), and the receptiveness of NWS management to such advice, contributed to the success of the Modernization and Associated Restructuring.
The committee limited the bulk of its analysis to those aspects of weather services that were explicitly included in the MAR planning and execution. However, there are some other key areas that were significantly affected by the MAR, including hydrologic services, coastal observations and forecasts, and the climate record. NEXRAD observations of non-meteorological targets also provide data valuable to some unrelated fields of investigation.
The NWS has two principal service areas: meteorology and hydrology. Much of the emphasis of this assessment has been on meteorological services. However, the MAR greatly improved the observation of precipitation through the deployment of the NEXRAD network and allowed for increased coordination of WFOs with River Forecast Centers (RFCs), thus allowing NWS to expand its hydrology mission and services (NRC, 1996b). The NWS Hydrologic Services Program (HSP) had two roles within the MAR: as an integral participant in the restructuring, and as a key customer of the modernized technology (e.g., NWS, 1989). The report Hydrometeorological Service Operations for the 1990s describes pre-MAR hydrometeorological operations within the NWS and details plans for staged implementation, including responsibilities of the RFCs, WFOs, and national and regional headquarters (NWS, 1996b). The 1996 report reflects considerable evolution in the direction and specificity of plans from the beginning of the MAR (Fread, 1996).
The MAR restructuring of the HSP was intended to increase the integration of day-to-day hydrology and meteorology operations (NWS, 1996b). All RFCs were colocated with a WFO; in some cases, relocation moved RFCs away from key clients (e.g., the Army Corps of Engineers in Portland, Oregon). RFC staff profiles were changed to include overall management by a Hydrologist in Charge (HIC), equivalent to a MIC, science and technical development by a Development and Operations Hydrologist (DOH), similar to a SOO, and hydrologic analysis and forecasting by a substantially larger staff, up to a doubling in some RFCs, of degreed meteorologists and hydrologists with cross-disciplinary training. Selected WFOs received a degreed Service Hydrologist to support the participation of all WFO forecasters in preparation of hydrologic forecast products. The restructuring did not provide RFCs with a services coordination position similar to the WFO WCMs.
The restructuring assigned responsibility for issuance of flood and flash flood watches and warnings to the WFOs, as well as the generation of Quantitative Precipitation Forecasts (QPFs) for use by RFCs. RFCs were charged with providing hydrologic forecast guidance to the WFOs in their region at least twice daily (rather than once) over a longer service day, producing gridded hydrometeorological products that smoothly cross WFO boundaries from multiple automated sensor networks and QPFs, and assimilating high resolution datasets and QPFs into hydrologic modeling operations. NCEP units (e.g., HPC and SPC) were charged with providing routine and event-based hydrometeorological forecasts and analyses (e.g., QPFs, probabilities of exceeding RFC flash flood guidance) from NCEP modeling activities. Other NWS units (e.g., the Office of Hydrology, regional headquarters) were also assigned hydrologic services responsibilities under the MAR.
Each RFC received multiple AWIPS workstations to obtain and use the hydrometeorological information, forecasts, and guidance products from the WFOs and NCEP. Additional software tools were needed by
the RFCs to interactively analyze, quality control, and assimilate the dramatically increased flow of hydrometeorological data and forecasts from multiple WFOs for use in hydrologic modeling operations. The tools were not provided as part of AWIPS, although they were needed to fulfill RFC responsibilities to support WFO operations.
The RFCs were also key customers of the MAR. The intent was for the NWS hydrological services program to capitalize on the MAR’s technological improvements to increase the specificity and accuracy of flood and flash flood guidance to WFOs, and to develop a significantly expanded suite of hydrometeorological products and services. During the MAR, NWS was engaged in planning and early implementation of the Advanced Hydrologic Prediction Service (AHPS), which also aimed to improve and expand hydrologic forecasts and services. The MAR and AHPS were very much intertwined, with the MAR being considered as one of four components of AHPS, and AHPS as an integral component of a modernized NWS. Hydrologic model development, calibration, and forecast verification were considered activities under the MAR and AHPS. Although AHPS wasn’t funded until midway through the MAR, it was essential for enabling the RFCs to capitalize on MAR advancements.
The MAR clearly improved coordination among hydrologic and meteorological operations, and enabled significant expansion of products and services. For example, the RFCs moved from forecasting only the traditional peak flows at select forecast points to 6- to 10-day hydrographs that predict the continuous flow at points within a specific watershed. Improvements began even pre-MAR, as some RFCs and the Office of Hydrology participated in early demonstrations of the complementary aspects of operational hydrology and meteorology planned under the MAR (e.g., QPFs, flash flood guidance, through the Prototype RFC Operational Test, Evaluation, and User Simulation [PROTEUS] project).
It appears that MAR planning did not fully account for the unique characteristics of RFCs and hydrologic operations compared to WFOs, NCEP, and meteorological operations. Collectively, RFCs were intended to serve the WFOs in a manner similar to NCEP, but at a regional scale (NRC, 1996b). However, the MAR did not provide the RFCs with the full complement of information processing tools required to fulfill those functions. Nor did it include any assessment of RFC needs for AWIPS capabilities, limiting the capability of RFCs to request additional capabilities, such as storage or processing speed. RFCs use dynamic hydrologic models that must be calibrated, requiring large archives of data much like the National Centers, and substantial data analysis and quality control. RFCs must also consider unique hydrometeorological processes within their region, and they have unique partnerships, such as agencies with regulatory responsibilities and hydropower production entities that need highly interactive access to RFC forecasts, products, and even computing resources. The RFCs shifted personnel hired or trained through the restructuring to information technology software development, delaying development of advanced hydrologic model capabilities, calibration, forecast verification, and probabilistic and ensemble forecasts. For example, RFC hydrologic professionals developed hardware configurations and software for producing gridded products, remote ensemble processors, and massive relational databases with high speed performance. In one RFC, 7 out of 10 hydrologic staff were focused on information technology rather than hydrologic science and development during the MAR.
An ongoing, challenging legacy of the MAR is that the qualifications for hydrologist positions were not upgraded to require degreed hydrologists, but instead allowed meteorologists to move into hydrology positions, even within RFCs. While much work of the RFCs (70 to 90 percent in recent estimates across three RFCs according to onsite interviews) focuses on quality control of hydrometeorological records where meteorological training is effective, negative consequences of this staffing challenge include limitations in the capability of RFCs to calibrate their hydrologic models. This issue was noted in a mid-MAR review of hydrometeorologic operations (NRC, 1996b). The staffing profile for hydrologists is imbalanced; of 600 hydrologist positions, only about 200 are degreed hydrologists and the limited opportunities for career advancement of hydrologists create difficulty in recruiting new employees (Carter, 2011).
As a whole, the MAR had a positive impact on hydrologic forecasts and services. The hydrologic services program took some lessons from the MAR
and has used them to inform the design of their institutional approach to implement AHPS and the Community Hydrologic Prediction System (CHPS). Key lessons acted upon include the need for organization and planning, the need for “full buildout of limited cases” with full interface development, and bottom-up input about the resources needed to implement the larger vision. The recent addition of Service Coordination Hydrologists (SCHs) at the RFCs was based on their evaluation of the success of the WCM in coordinating with external partners and customers. Further, the hydrologic services program desires to have a hydrologic-centric MAR, especially to address current staffing profiles.
Coastal Observations and Forecasts
Although the MAR did not explicitly include technological enhancements and capabilities for the U.S. buoy and coastal observing network, there were aspects of marine observations and analysis that benefitted. Approximately 30 percent of the U.S. population is concentrated in coastal communities that border the ocean (Crowell et al., 2007). Because of the geographical prominence of the coastal regions, an NRC review panel (NRC, 1999a) highlighted the need for NWS assessment of the AWIPS system for coastal marine weather forecasts and warnings, which had not been part of the testing that took place within the MAR. The ASOS and NEXRAD deployments significantly enhanced the observing capabilities in coastal regions. In addition, the higher spatial and temporal observations obtained with the GOES-Next satellites over data-sparse ocean regions improved forecasts of, for example, Pacific Ocean storms approaching the west coast. Even given some of the documented shortcomings previously discussed (e.g., reliability issues associated with ASOS; siting of NEXRAD radars at high altitudes), the new capabilities provided forecasters with unprecedented observations of the mesoscale coastal weather phenomena in real time. The AWIPS capability gave the forecasters for the first time an integrated depiction of coastal mesoscale meteorology that included the new ASOS and NEXRAD observing systems and GOES-Next, blended with the existing observing network, including coastal buoys (Reynolds, 2011).
Reviews of early plans for the MAR noted that little attention had been given to issues of long-term management of the vastly greater stream of observations from MAR technology or to the quality of the climate record, and the reviews repeatedly stressed the importance of preserving the climate record as ASOS was deployed (NRC, 1991, 1992b, 1993). Recommendations were clear and strongly worded, e.g., “…the preservation of data quality for climatic purposes should have equal priority with its mission of providing forecasts” and “[w]hen instrument sites are changed, simultaneous operation at the old and new sites should occur until adequate statistics on the difference of observations between sites can be developed. These statistics should be recorded carefully and made readily available” (NRC, 1991). The 1992 NRC report included a separate appendix about data for climate studies from a standing NRC Climate Research Committee, which expressed concern that ASOS observations of cloud types and cloud cover, present weather, snowfall and water equivalent, total sunshine, radiation, and turbidity would be insufficient for climate studies (NRC, 1992b). The 1993 NRC report noted that prior recommendations relating to the climate record had not been addressed (NRC, 1993). Those same reports, though, also noted that the MAR provided opportunities to enhance the climate record by providing new kinds of data not previously available (e.g., NEXRAD precipitation estimates).
For this assessment, comments were sought from the NWS Climate Services Division (CSD) and the National Climatic Data Center (NCDC). Siting of ASOS stations was clearly driven by aviation requirements, not considerations of the climate record. Continuity plans for concurrent observations at limited sites were developed by the NWS Office of Science and Technology, following NWS Directive 10-21, but according to the CSD, those studies were never completed. However, a series of commissioned overlapping observation studies were conducted in the 1990s at a number of sites throughout the United States for varying periods of time, in all cases less than one year. Other studies provide additional insight (Brazenec, 2005; Butler and McKee, 1998; Doesken and McKee, 2000; Kauffman, 2000; McKee et al., 2000; McKee
et al., 1994a, b, 1996a; Schrumpf and McKee, 1996; Sun et al., 2005). Comparison of ASOS and manual observations are complicated by differences in gauge locations, ranging from just a few hundred feet to more than one mile, although with little elevation differences. However, local exposure and vegetation differences can be significant (Guttman and Baker, 1996; McKee et al., 1995). Many performance issues are associated with the ASOS instrumentation package. A detailed description of the ASOS impacts on the climate record for different observed variables is provided in Appendix E.
Converting to ASOS has had a significant impact on the climate record. Discontinuities in temperature data occurred due to changes in instrumentation as well as changes in the observing location that occurred at most airport locations. There was also a significant impact on the cloud record with the elimination of manual observations and use of automated ceilometers. This was especially detrimental with cloud observations limited to 12,000 feet above ground level. Relative humidity was affected as well, due to instrumentation changes. The negative impact on precipitation measurements was severe with the conversion from the universal gauge to tipping buckets, which had difficulty accurately capturing medium to high rainfall rates and solid precipitation. Alterations in wind shields also affected the continuity of precipitation measurements. Observations of snowfall, snow equivalent, total sunshine, and active weather phenomena are no longer available. These impacts have created a special challenge for climatologists. Changes in instrumentation, in the locations of these instruments, and in the observational methodology (resulting from the removal of the human observer) have created inhomogeneities in the climatic records at these NWS and FAA airport sites. Without homogeneous records, computation of long-period means and frequencies of observed variables becomes meaningless as abrupt step changes in the time series are introduced.
From another perspective, however, ASOS did offer something unprecedented within the climate observing community: near real-time data collection and archival. Data observations could now be electronically transmitted and readily available, as opposed to the historical record keeping, which took a month or longer to publish data that were hand-recorded on paper, mailed to the data center, and keyed in manually by staff. Over time, ASOS has become one of the most robust data collection systems ever fielded and the advantages of the greater number of high quality stations, the station-to-station uniformity, the improved instrument siting and the rigorous (in most cases) maintenance is providing the community with a rich dataset for future climate studies.
Further, the MAR more broadly, ultimately had a positive impact on the climate record as the emphasis on data stewardship and preserving weather observations for climate-quality records increased. Some of this improvement required adjustments that took place after the formal completion of the MAR, including quality control tools, such as NCDC’s Datzilla. In addition, the NWS Climate Services training program has been used to inform NWS staff of proper data stewardship practices. Lastly, the climate services9 outreach program has expanded the overall knowledge base of users regarding the climate data record.
NEXRAD Observations of Non-Meteorological Targets
A weather radar receives echoes not only from hydrometeors but also from other objects suspended in the atmosphere—including dust or smoke particles if they are sufficiently dense and close enough to the radar, as well as insects and birds. Many such echoes, once referred to as “angel echoes” (Battan, 1973), have come to be recognized as arising primarily from insects (e.g., Gauthreaux et al., 2008; Russell and Wilson, 1997; Wilson et al., 1994). Those echoes can provide useful tracers of the winds (provided the insect motions do not differ greatly from the winds), and also provide data useful to entomologists studying insect movements or migrations (e.g., Chapman et al., 2004, 2011). The sensitivity of the NEXRAD system has greatly enhanced the value of the NWS network data for such investigations.
Echoes from birds are more likely to contaminate wind velocity estimates, because the birds often move with appreciable velocity differentials (e.g., Serafin and Wilson, 2000). However, those echoes have proven quite useful to biologists studying bird and bat behavior (e.g., Gauthreaux and Belser, 1998; Horn and Kunz, 2008).
9Climate services include observations, monitoring, forecasting, and assessments of climate.
The Modernization and Associated Restructuring (MAR) improved collaboration among hydrologic and meteorological operations within the National Weather Service, and allowed significant expansion of hydrologic forecast products and services. However, the challenges facing the River Forecast Centers were magnified because the MAR did not adequately take into account the unique requirements of hydrologic data management, modeling, and partner collaborations.
The Automated Surface Observing System (ASOS) was not implemented in such a way that the climate record was preserved. Discontinuities that degrade computation of long-period statistics, created by changes in instrumentation and observing locations, are still a concern. However, the Modernization and Associated Restructuring continues to offer prospects for improvement of the overall national climate record over the long term.
In many respects, the changes that the NWS experienced as a result of the MAR can be viewed as revolutionary. The MAR brought dramatic improvements in weather services to the nation. New technology including ASOS, NEXRAD, GOES-Next, and AWIPS provided forecasters with an unprecedented set of observational and analysis tools. The new NWS organizational strategy transformed the forecast offices into a modern national network of WFOs. The workforce transitioned from two-thirds technicians, to two-thirds professional meteorologists. Many WFO staff now have a college degree, and many SOOs have advanced degrees. It is also becoming more common for staff in other positions to possess advanced degrees (Sokich, 2011).
Following the official end of the MAR in 2000, a framework was left in place so that the technology and NWS organization could continue to grow in an evolutionary manner. Examples of this evolutionary framework are post-MAR upgrades to the ASOS, NEXRAD, and AWIPS systems, occurring in tandem with technological advances in the wider community. The testbed concept and risk reduction activities emerged out of the MAR framework as well. One of the lessons of the MAR was the value of prototypying new operational concepts (e.g., PROFS and the pre-NEXRAD and pre-AWIPS systems). This pre-operational prototype paradigm has been advanced following the MAR and embraced by the NWS, which now has a number of successful testbeds including
• the Developmental Testbed Center,
• the Hydrometeorology Testbed,
• the Hazardous Weather Testbed,
• the Joint Hurricane Testbed,
• the Aviation Weather Testbed, and
• the Joint Center for Satellite Data Assimilation.
Testbeds have accelerated the transfer of technology from research-to-operations; successful examples include the Joint Hurricane Testbed and the Hazardous Weather Testbed (jointly operated by NWS and the Office of Oceanic and Atmospheric Research [OAR]). These testbeds improved capacity and better separation between development and operational systems for running models (Hayes, 2011). Nevertheless, the testbeds primarily have a focus on transition of research-to-operations, not the broader scope needed to prototype new concepts for methods of operations that was present in the prototyping and risk-reduction activities of the MAR. The current generation of testbeds tend to operate largely independently of one another, and provide little capacity to experiment with multi-office collaboration on delivery of new services. Provision of new services will likely be an increasingly important requirement in the future. Some of the current testbeds have limited capacity to engage key stakeholder groups, including emergency managers, media, and commercial weather service providers, in developing and evaluating new service concepts.
Despite some of the shortcomings of the current testbed system, the framework established during the MAR provides the NWS excellent opportunities for new collaborations and partnerships, responding to the ever-increasing interdisciplinary nature of meteorology and hydrology. The MAR established a foundation for evolution that will allow the NWS to better meet the future needs of the United States.
This page intentionally left blank.