Weather information, specifically forecasts and warnings, has a great impact on the U.S. economy and is critical for the protection of life and property. The National Weather Service (NWS) serves as the nation’s authoritative source of such information, providing routine public, marine, and aviation forecasts, as well as advisories and warnings when conditions warrant. Under its operating model, the NWS provides these services as well as atmospheric and hydrological data free of charge to other agencies, the research community, the private sector, and the public. The NWS also provides seasonal climate forecasts, and its observations are an essential part of the forecast process as well as part of the long-term climate record. As the primary provider of weather data in the United States, it is crucial that NWS operations stay at the forefront of available technologies for observing, forecasting, and understanding the weather.
The 20th century saw an exponential growth in the technological capabilities of weather observations and forecasting. Because of the rapid rate of change, it was difficult for the NWS to keep pace and in the 1980s it became clear that to take advantage of new technologies in the most cost effective manner, and to provide better weather services to the nation, the NWS needed to change dramatically. The concept of a modernized and restructured weather service emerged.
Between 1989 and 2000, the nation invested approximately $4.5 billion to implement the Modernization and Associated Restructuring (MAR) of the NWS. New observational and computational systems were planned and deployed, and the NWS field office structure was redefined around new concepts for observing, forecasting, and service delivery to capitalize on the investments in these new systems. The NWS workforce was restructured around these concepts and substantial investments in training and recruitment developed a more professional workforce with the skills necessary for the modernized NWS.
To modernize its operations, the NWS developed and implemented five major technologies:
• Automated Surface Observing System (ASOS): an automated electronic sensor instrument system to replace manual weather observations at all NWS (and many other) surface observing locations;
• Next Generation Weather Radar (NEXRAD): a network of advanced Doppler radars to measure motions of the atmosphere responsible for severe weather such as tornadoes, detect heavy rainfall and hail, and increase lead times for prediction of severe weather events and flash floods;
• Satellite Upgrades: a new series of geostationary meteorological satellites to provide higher spatial and temporal resolution imagery and data to aid shorter-range forecasts and warnings, and a new series of polar orbiting meteorological satellites to provide improved, all-weather, atmospheric data to assist in longer term forecasting;
• National Centers Advanced Computer Systems: a tenfold increase in computing power to support the National Centers. Along with numerical weather prediction model improvements, this improved national guidance for forecasts and warnings; and
• Advanced Weather Interactive Processing System (AWIPS): a workstation-centric, advanced com-
puter and communications system to help forecasters integrate, visualize, and analyze all sources of weather data. The system allowed communication between each weather forecast office and distribution of centrally collected data and centrally produced analysis and guidance products, as well as satellite data and imagery.
To take advantage of these modern technologies, the NWS restructured their field office organization. Prior to the MAR, the NWS had a two-tiered office structure: 52 Weather Service Forecast Offices (WSFOs) had a core component of professional meteorologists and 204 Weather Service Offices (WSOs) were staffed with observers and meteorological technicians. This structure was replaced with a single-tiered system of 122 Weather Forecast Offices (WFOs). The intent was for WFO locations to be more evenly distributed across the nation, to provide more uniform provision of weather services and greater interaction with communities, specifically local media and emergency management. The combination of modernized technology and a reorganized operational structure contributed to improvements in forecasts on time scales of minutes to weeks, time scales that were the focus of the MAR. For example, the probabilities of detection and forecast lead times for both tornadoes and flash floods improved after the MAR. However, the false alarm ratios for tornadoes and flash floods have remained high. Hurricane track forecasts improved after the MAR, whereas hurricane intensity forecasts still need improvement.
No comprehensive assessment of the MAR plan and its execution, or comparison of the promised benefits of the MAR to its actual impact, has been conducted. Therefore, Congress asked the National Academy of Sciences to conduct an end-to-end assessment that addresses the past modernization as well as lessons learned to support future improvements to NWS capabilities. This report contains Phase I of the committee’s work, a retrospective assessment of the MAR with a focus on lessons learned from the effort to plan, deploy, and oversee the MAR. Phase II will apply the lessons learned from the MAR to develop actionable recommendations for the NWS on how best to plan, deploy, and oversee future improvements, and will be presented in a later report.
Overall, the MAR led to a greater integration of science into weather service activities and improved outreach and coordination with state and local government, emergency management, and communities. The technological improvements provided forecasters with a wealth of new data and observations, allowing them to provide more accurate and timely forecast and warning services to the nation. The stated objective of the MAR in the Strategic Plan prepared by the NWS was
to modernize the NWS through the deployment of proven observational, information processing and communications technologies, and to establish an associated cost effective operational structure. The modernization and associated restructuring of NWS shall assure that the major advances which have been made in our ability to observe and understand the atmosphere are applied to the practical problems of providing weather and hydrologic services to the Nation.
It is clear that the NWS succeeded in the deployment of observational, information processing, and communications technologies that have improved weather and hydrologic services. The MAR significantly increased the amount of data and information available to field forecasters, the private sector, and the general public. The forecast and warning products produced by the post-MAR NWS are greater in both quantity and quality. However, the cost-effectiveness of the operational structure is difficult to assess quantitatively, because of the challenges involved in assessing the value of decreased loss of life and property as a result of improved forecasts and warnings.
This summary presents the committee’s findings and lessons about the MAR as a whole, as well as more detailed findings and lessons about six specific elements of the MAR: (1) management and planning; (2) modernization of technology; (3) restructuring of forecast offices and staff; (4) national centers; (5) partnerships; and (6) oversight and advisory groups. The evidence and analysis supporting these findings and lessons are contained in the main report.
FINDINGS AND LESSONS LEARNED
The committee has two findings and one lesson about the MAR as a whole:
• The National Weather Service (NWS) had been unable to keep up with the pace of technological advances and had nearly become obsolete by the 1980s.
Therefore the NWS was not utilizing the full potential available to provide the best possible meteorological services to the nation. The $4.5 billion national investment in the Modernization and Associated Restructuring (MAR) was both needed and generally well spent. Overall, the MAR was successful in achieving major improvements for the weather enterprise.
• A framework was created and left in place following the Modernization and Associated Restructuring that allows and encourages the technology and to some extent the workforce composition and culture of the National Weather Service to continue to evolve.
Lesson 1: If a science-based agency like the National Weather Service, which provides critical services to the nation, waits until it is close to becoming obsolete, it will require a complex and very expensive program to modernize.
MANAGEMENT AND PLANNING
• During the Modernization and Associated Restructuring (MAR) period from 1989 to 2000, the major components of the MAR were well planned and completed largely in accordance to that plan. Established processes were extensive and generally followed. However, notable budget overruns and substantial schedule delays occurred for nearly all of the project elements. This was due in large part to the MAR aggregating four major technology programs that had been separately initiated during the 1980s. Many of the MAR’s cost and schedule issues were set in place by decisions that occurred during this pre-MAR period.
• Many of the institutional changes (management structure, culture, processes, partner relationships) introduced to implement the Modernization and Associated Restructuring (MAR) have been retained by the National Weather Service (NWS). Most of these “institutional byproducts” have been as valuable as the MAR improvements themselves and will help the NWS to continue to modernize. However, from viewing more recent projects, implementation of a rigorous systems engineering process to facilitate more effective management of the procurement and development of large, complex systems appears not to have been institutionalized within the National Oceanic and Atmospheric Administration. The systems engineering process needs to start at the beginning of the program, in the agency’s program office.
Lesson 2: The budget, schedule, and technological issues encountered during execution of the Modernization and Associated Restructuring of the National Weather Service (NWS) reflected traditional challenges of large projects: inexperience of the government project-level leadership, shifting budget constraints, ambitious technology leaps, multiparty stakeholder pressures, cultural inertia, contractor shortcomings, and oversight burdens. Each represents important lessons for the NWS with regard to future projects of a similar nature:
- Expertise in system design, procurement, and deployment is essential to successful implementation of any complex technical upgrade.
- Dedicated leaders are crucial for resolving road-blocks and ensuring ultimate project success.
- Clearly defined system-level requirements, and competent management of those requirements, are essential to any contractual acquisition of a major system.
- Statistical indicators of forecast and warning performance are a major element for gaining and maintaining support for implementing new technologies.
- It is necessary to establish comprehensive performance metrics at the beginning of a process, evaluate them throughout the process, and reevaluate them after the process is complete.
MODERNIZATION OF TECHNOLOGY
• The various technological problems that were encountered included lack of preliminary analysis and ensuing design problems, inadequate program management, and poor contractor performance. These problems were generally overcome and the major technology system upgrades were successfully executed.
• The Modernization and Associated Restructuring (MAR) provided for more uniform radar coverage and surface observations across the United States. The Next Generation Weather Radar network and Geostationary Operational Environmental Satellites dramatically improved the quantity and quality of data available to forecasters and enhanced the numerical weather prediction capabilities of the National Weather Service (NWS). Replacing human observers with the
Automated Surface Observing System introduced significant gains, despite possible adverse affects on the climate record and the loss of some important visual elements of the observation. The Advanced Weather Interactive Processing System (AWIPS) has been a critical technological advancement that integrates the data and information provided by other MAR elements and makes them easily accessible by forecasters.
• The Probability of Detection for both tornadoes and flash floods improved over the course of the MAR and after the MAR. Likewise the Lead Times of the warnings increased. However, the False Alarm Ratios (FARs) were not reduced and remain high.
Lesson 3: The time scale for implementing major change in government systems is very long compared to the time scale for major technological change. The pace of technological progress complicates the planning, procurement, and deployment of large, complex systems. While technology is changing so rapidly, in every aspect of the project where it is feasible, it is crucial to
- establish clear metrics for evaluating improvement in forecasts and warnings at the beginning of a major technological upgrade;
- use rapid prototyping and system demonstrations. An example includes the Program for Regional Observing and Forecasting Service (PROFS) and their Denver AWIPS Risk Reduction and Requirements Evaluation (DAR3E) effort, which proved critical to the success of the Modernization and Associated Restructuring;
- evaluate such prototype systems under a variety of actual operational situations with multiple classes of users and stakeholders in order to refine the system design;
- establish the capacity for continual upgrades of complex systems, particularly those involving digital technology (e.g., high performance computing, and communications); and
- continually assess and apply the lessons of past systems, whether successful or unsuccessful.
RESTRUCTURING OF FORECAST OFFICES AND STAFF
• The restructuring of offices and upgrading of staff brought more evenly-distributed and uniform weather services to the nation.
• During the early stages of the Modernization and Associated Restructuring, there was insufficient communication between National Weather Service management at the national level and the field office managers and their staff, as well as the employee union.
• National Weather Service staff was reduced, but technical capabilities and career paths were substantially upgraded, leading to little or no cost savings from the workforce reorganization.
• The staffing level that resulted from the Modernization and Associated Restructuring allows for at least two people on duty for all shifts, but timely planning and coordination by field office managers and supervisors are required to be able to increase the staffing level for times when severe weather threatens life and property.
• The Science Operations Officer (SOO) position created as part of the Modernization and Associated Restructuring, in principle, allows advancements in the science community to be more rapidly integrated into operations. Communication and dissemination of weather information at the local level has been much improved by the restructuring of the forecast offices and the creation of the Warning Coordination Meteorologist position.
Lesson 4: The Modernization and Associated Restructuring (MAR) of the National Weather Service (NWS) faced initial resistance from NWS employees and, to some extent, the general public. This resistance could have been lessened by, very early in the planning stages,
- engaging those whose career and livelihood were to be affected in planning the changes; and
- better engaging a diffuse public, and to some extent Congress, regarding the benefits of improved weather forecasts and warnings as opposed to the perceived cost of losing a forecast office in their community.
The restructuring dictated a degree of standardization between forecast offices, however it has become apparent that this needs to be effectively balanced with the flexibility needed to allow for customization at individual offices to respond to local requirements.
The MAR increased the overall education level
of the workforce and set in place the need for routine training to keep the staff on pace with technological and meteorological advancements in the community. Staff development through in-person, hands-on training in a centralized classroom or laboratory of the type that occurred during the MAR has great value. Where relevant, online courses or self-directed study can be a useful supplement, but can sacrifice quality of learning and the connections made with colleagues that are essential to the overall operations of the NWS.
NATIONAL CENTERS
• The overarching Modernization and Associated Restructuring goal to integrate science-based approaches to weather, climate, and hydrologic prediction, and to rapidly assimilate evolving facets of information technology, led to the formation of the National Centers for Environmental Prediction (NCEP), which have become a key part of the National Weather Service.
• Numerical weather forecasts produced by the NCEP and the associated guidance information and products, improved steadily over the course of the Modernization and Associated Restructuring. However, the performance of some NCEP models, particularly the Global Forecast System (GFS), continues to lag behind some other national centers, including the European Centre for Medium-range Weather Forecasts (ECMWF).
PARTNERSHIPS
• Partnerships between the National Weather Service and other National Oceanic and Atmospheric Administration line offices, other Federal Agencies, state and local governments, academia, the research community, and to some extent the private sector through contractor relationships, while not perfect, especially in the early years, were essential to successful execution of the Modernization and Associated Restructuring.
• Improved relationships with other agencies and external partners have proven to be one of the more important outcomes of the Modernization and Associated Restructuring (MAR). These relationships increase the National Weather Service’s societal impact and leverage its limited budget. Success of the MAR depended in part on leadership, initiative, and funding by National Oceanic and Atmospheric Administration and National Weather Service units operating outside the MAR. Though issues remain, partnerships with academia and government research institutions have increased research-to-operation capabilities, and the MAR elevated the media and emergency management community from a customer to a partner. The relationship between the NWS and the private sector took longer to improve, but it has generally evolved into a more constructive and productive one.
Lesson 5: The execution of the Modernization and Associated Restructuring required working with many partners, which provided cost-sharing and improved understanding of user needs. However, the relationships with the partners were not always as well conceived or managed as would have been desirable. This could have been avoided by involving all known stakeholders (e.g., other agencies, academia and the research community, the private sector, media, and emergency management) from the outset. The National Weather Service (NWS) operational staff is also a stakeholder, and need to be involved early in the design and procurement process to ensure system functionality and practicality. Engagement with stakeholders from both inside and outside the NWS would help the NWS better understand user needs and secure ‘buy-in’ to new initiatives.
OVERSIGHT AND ADVICE
• Independent oversight and technical guidance helped draw attention to important issues and impediments that otherwise may have inhibited the success of the Modernization and Associated Restructuring (MAR). This external oversight provided accountability of the technical, scheduling, and budget metrics during the MAR process.
• Expert advice and oversight from outside the National Weather Service (NWS), and the receptiveness of NWS management to such advice, contributed to the success of the Modernization and Associated Restructuring.
Lesson 6: The Modernization and Associated Restructuring of the National Weather Service (NWS) showed
that candid yet non-adversarial advice from outside experts and other interested parties was useful in the design and deployment of a large complex system. Because NWS management was receptive to such oversight and advice, the outside input was effective.
ADDITIONAL IMPACTS
• The Modernization and Associated Restructuring (MAR) improved collaboration among hydrologic and meteorological operations within the National Weather Service, and allowed significant expansion of hydrologic forecast products and services. However, the challenges facing the River Forecast Centers were magnified because the MAR did not adequately take into account the unique requirements of hydrologic data management, modeling, and partner collaborations.
• The Automated Surface Observing System (ASOS) was not implemented in such a way that the climate record was preserved. Discontinuities that degrade computation of long-period statistics, created by changes in instrumentation and observing locations, are still a concern. However, the Modernization and Associated Restructuring continues to offer prospects for improvement of the overall national climate record over the long term.
The MAR was a large, complex process that lasted a decade, and cost approximately $4.5 billion. Despite issues, some more significant than others, in the end the MAR was an unqualified success. New technologies deployed during the MAR now provide forecasters with more observations of higher quality. NWS forecast and warning products were dramatically improved, in both quality and quantity. NWS now has stronger relationships with many of its partners in the weather enterprise. Changes in the distribution of field offices have allowed stronger connections with local communities. Weather services have great value to the nation, and the MAR was well worth the investment.
