Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 19
3 Execution of the Modernization and Associated Restructuring T his chapter focuses on the implementation of port coordination activities between all the NWS offices the Modernization and Associated Restructur- involved in the MAR. Contracting, personnel manage- ing (MAR) of the National Weather Service ment, external relations, and facilities construction was (NWS) during the period of 1989 to 2000. The chapter overseen by NOAA headquarters and the Department provides an overview of the management and planning of Commerce (DOC; NRC, 1991). issues, technology upgrades, and the reorganization of field offices and the work force. The actual implemen- Management Context and Constraints tation is compared to the MAR execution objectives presented in the preceding chapter, and summarized in To understand the MAR management, it is helpful specific findings about the major aspects of the MAR. to first identify key context and contemporary issues within which the MAR was implemented (NRC, 1980, 1991; NWS, 1989): MANAGEMENT AND PLANNING The MAR was “the most complex project ever car- • Perception. The perspective was that NWS was ried out in the Department of Commerce” at the time in need of substantial improvement (Kraus, 2011; (Hayes, 2011). Implementation occurred during a period NRC, 1980); there were high expectations that the MAR would improve the agency.1 of rapid technological change (including the emergence of the Internet), and involved a number of major systems • Mission. The MAR did not seek to change the deployed across a geographically diverse nation, as well primary NWS role to be the nation’s authoritative as several federal agencies and the direct participation source of weather information. However, the MAR of three National Oceanic and Atmospheric Admin- did change the manner NWS interacted with other istration (NOAA) line offices (NWS, the National weather information sectors. Environmental Satellite, Data, and Information Service • Operating Model. The NWS operating model [NESDIS], and the Office of Oceanic and Atmospheric of free weather-related services to the nation was not questioned and did not change during the MAR.2 Research [OAR]). Any such undertaking requires rigor- ous management. A NOAA Deputy Assistant Admin- istrator for Modernization was appointed to oversee the 1 This was true both formally and informally; the MAR was ex- NEXRAD Joint System Program Office, the Office of pected to provide a substantially better cost-benefit ratio than “busi- Systems Development (which included the ASOS and ness as usual” with payback of investment in 1.6 years (NIST, 1992). 2 It had been questioned during the 1980s, with substantial discus- AWIPS projects), the Office of Systems Operation, the sion regarding privatization of some or all elements of NWS (Booz Office of Hydrology, and the Transition Program Office. Allen & Hamilton Inc., 1983). Many national weather services in NWS established the Transition Program Office to sup- other countries use operating models that differ from NWS. 19

OCR for page 19
20 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT • International Obligations. NWS needed to main- • Labor Relationships. NWS had strong union tain its international obligations, most notably through participation at the field office staff level (National the World Meteorological Organization (WMO), and Weather Service Employees Organization; NWSEO). none were altered by the MAR. The MAR did not include plans to change the role of • Budget. NWS and NESDIS are parts of NOAA NWSEO, but the proposed change in workforce struc- and the DOC, and thus subject to NOAA and DOC ture meant NWSEO and its members were strongly considerations as well as their own.3 Furthermore, affected. Prior to the MAR, NWS had generally NWS worked with the Federal Aviation Administra- maintained limited interaction with NWSEO (NRC, tion (FAA) on the Automated Surface Observing 1994a). System (ASOS), and FAA and the Department of • Partnerships. NWS depended on many partner- Defense (DOD) for the Next Generation Weather ships with government, academia, media, and private Radar (NEXRAD). sector entities. At the time of the MAR, some of these • Downsizing Government. The NWS expected were generally strong (e.g., government, academia, the MAR to increase the efficiency of its operations research institutions, technology firms), others such as and downsize its organization with no degradation of the media and private sector meteorology firms were weather services. The agency planned to reduce the informal to a fault, or simply absent. number of field offices from 256 to about 120, and to • Shared Responsibilities. The MAR elements of reduce its staffing levels from a pre-MAR level of 5,100 ASOS and NEXRAD required shared responsibil- to about 4,000 through restructuring and automation ity with FAA and DOD. This inevitably introduced (GAO, 1995d; NWS, 1989).4 The long term net sav- challenges from authority and coordinated budgeting. ings in staffing costs was used as part of the justification Within NOAA, the shared responsibility with NESDIS for the MAR (NWS, 1989). for satellites was important, but mostly handled in a • Performance Guarantee. Congressional Lan - cooperative and constructive way. guage (Public Laws 100-685 and 102-567) required • Public-Private Interaction. A growing private certification that services did not degrade. This was sector marketing weather products was increasingly an important factor in deciding how the MAR would performing functions of data acquisition, modeling, and be executed, with several key processes tied directly to delivery of customized products. At the time, there was this issue. considerable friction between NWS and the private sec- • Congressional Politics. In addition to agency-level tor regarding perceived conflict of roles (NRC, 2003a). political issues, NWS was highly sensitive to state, • Completeness. The MAR did not focus primarily district, and local politics because of the national dis- on some elements of the enterprise, such as the River tribution of field offices and the plan to close or move Forecast Centers (RFCs). These, while proceeding many of them. There was high potential for politically- along in development, did not receive the same priority influenced congressional and Administration involve- in planning, implementation, and oversight as the other ment, and resulting risks to the overall plan and delay, elements of the MAR. that played out in numerous Congressionally-requested reviews of individual office relocation plans (OAR, Budget and Schedule 2010) and even specific legislative direction for the location of particular offices (U.S. Congress, 1992). Information sources available to the commit - tee are surprisingly poor for assessing budget and schedule performance of the MAR. The generally 3 One anecdotal comment was that “[i]t is sometimes easier to accepted authoritative source is GAO reports published get funding for new programs than for sustaining existing ones” throughout the MAR, which are rather sparse in their (Kraus, 2011). supporting details. The annual National Implementa- 4 Staff was ultimately reduced from 5,200 to 4,700 while chang- ing the mix from one third meteorologists and two thirds techni- tion Plans of the MAR documented budget requests; cians to the opposite (NRC, 1994a; Sokich, 2011). There were while not always identical to the funds expended, they proposals for more dramatic staff reductions early in the planning provide some ability to interpret the GAO numbers. stages (Booz Allen & Hamilton Inc., 1983).

OCR for page 19
21 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING TABLE 3.1 Cost and schedule performance of the MAR as documented in GAO reports. MAR Element Planned Cost ($M) Final Cost ($M) Planned Completion Actual Completion 72*,a 150**,b ASOS 1990a 1998c 800†,b NEXRAD 340a 1989a 1996b Satellite Upgrades†† 640a,d 2,000d 1989a 1994d Advanced 47.5e 106b 1994e 1999b Computer Systems AWIPS 350a 539b 1995a 2000b ~1,000‡ Other Costs ~500 n/a n/a (facilities, staff, R&D) TOTAL 2,000a 4,500 f,g 1994a,h 2000 aGAO (1991a); bGAO (2000); cNadolski (2011); dGAO (1997c); eGAO (1994); fGAO (1997a); gGAO (1998a); hGAO (1995b). Detailed information about each of these information sources can be found in the Reference list at the end of the report. The planned cost for ASOS is in 1986 constant dollars; for NEXRAD is in 1980 constant dollars; for satellite upgrades is in 1991 constant dollars; and for AWIPS is in 1985 constant dollars. *This cost was for 250 NWS locations initially planned. **This cost was for the 314 NWS locations. The total cost of the 314 NWS, and 678 FAA and DOD locations was approximately $350 million (GAO, 2000). †This cost was for the 125 NWS radars. The total cost of the 125 NWS, 12 FAA, and 29 USAF radars was approximately $1.2 billion (GAO, 2000). ††These budget figures are for the total GOES-Next system, including the government part of the effort, as well as the SS/L prime contract, the ITT sub - contract, and various other subcontracts under SS/L (GAO, 1997c). The completion dates are for the launch of the first satellite in the series. ‡The actual amount of the Other Costs is hard to determine, but appears to be in the range of $900 to $1,200 million, as discussed in the text. According to GAO initial MAR planning antici- 1. Do the budget and schedule numbers reported pated completion within 5 years of the formal MAR in GAO reports and summarized in Table 3.1 accu- start5 within a budget of $2 billion (GAO, 1991a). rately reflect the cost and schedule performance? From this and other GAO reports, it is possible to con- 2. Were the cost and schedule issues encountered struct the overall view of cost and schedule performance during the MAR out of the ordinary for projects of shown in Table 3.1. Each element is described in more comparable magnitude? detail later in this chapter. 3. What were the root causes of the cost and sched- Executing on budget and schedule was among the ule issues? biggest challenges of the MAR. From the start, cost 4. What lessons can be learned for the future? overruns and schedule delays received considerable visi- bility in the GAO and at the Congressional level. Prob- Question 1: Do the budget and schedule numbers lems persisted throughout the duration of the MAR; reported in GAO reports and summarized in Table 3.1 it even achieved the GAO designation of a high-risk accurately reflect the cost and schedule performance? W hile Federal program for 1995 and 1997. The many GAO GAO cost and schedule numbers appear correct as reports addressing these issues are discussed later in this cited, assessment of MAR cost and schedule perfor- chapter and listed in Appendix B. mance is highly dependent on the GAO’s definitions Unlike the GAO, this committee had the luxury of of when program elements started and what they reviewing cost and schedule issues in hindsight. Given this included. The committee believes that the chosen freedom, the committee identified a framework of four definitions lead to a distorted picture of MAR schedule questions within which the review was accomplished: and budget performance. First, GAO chose to compare actual costs in real year (inflated) dollars to planned costs in fixed year 5 W hile several GAO reports state that initial planning estimated dollars for all program elements. The NEXRAD sys- that the MAR would be completed in 1994, the anticipated date tem, for example, was proposed in 1980 to not exceed of completion was in flux during the early stages of the MAR. The $340 million in 1980 dollars ( JSPO, 1980). By the first National Implementation Plan, for example, estimated that the 1988 planned completion, inflation had contributed MAR would be completed in 1996 (NWS, 1990).

OCR for page 19
22 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT approximately a factor of 1.75 (the actual value depends 1996c, 1997, 1998, 1999). If so, that would imply the on details of the year-by-year spend), suggesting the total MAR cost was approximately $4.5 to 4.7 billion, proposed cost should be adjusted to at least $600 mil- comparable to the $4.5 billion cited by GAO. Other lion (even if it assumed that the original schedule had internal NWS costs essential to the MAR, such as the been maintained). Overall, inflation likely accounted 1980s R&D work done on PROFS ultimately needed for $800 million of the cited $2.5 billion cost overrun; to implement AWIPS, are also not included. These planned costs should have been adjusted for this infla- would grow the cost further, though it is readily argued tion by GAO for proper comparison. that such R&D should fall under normal operating Second, GAO chose a cost and schedule baseline budgets rather than the MAR. (project start) going back as far as a decade before In conclusion, the GAO cost numbers and schedules formal MAR initiation (e.g., 1980 for NEXRAD); appear to be largely accurate based on a strict reading of although these projects were executed by NOAA, they GAO’s assumptions, but the ability to draw conclusions preceded MAR management. An alternate approach about MAR cost and schedule performance is limited by might have been to use the date of MAR initiation these assumptions. The strict GAO accounting implies and compare final cost and schedule to those estimated a total MAR cost growth of 150 percent. The consider- at MAR initiation. The cited figure of $2 billion for ations described here suggest the actual value is consid- planned cost was updated to $4.6 billion as early as erably less under assumptions deemed more appropriate 1991; an updated baseline might substantially change by the committee, but any particular number depends the assessment of actual performance. Indeed, by the subjectively on the assumptions used. end of the MAR the GAO calculated the completion Question 2: Were the cost and schedule issues encountered cost of the four major systems (ASOS, NEXRAD, during the MAR out of the ordinary for projects of compa- Next Generation Geostationary Environmental Satel- rable magnitude? The answer to this question depends lite [GOES-Next], and AWIPS) at $3.5 billion (GAO, to some extent on the interpretation of Question 1 as 2000), well under the $4.2 billion expected by GAO in to what cost and schedule issues should be attributed to 1991 (GAO, 1991a).6 the MAR. For comparison, recent studies of NASA pro- Third, some costs appear to have been improperly grams having roughly comparable complexity show an accounted for by GAO, such as inclusion of facilities average cost growth ranging from 33 percent (Emmons in the NEXRAD cost prior to FY1992 (the original et al., 2007) to 45 percent (CBO, 2004), while trans- NEXRAD cost estimate explicitly excludes such costs). portation infrastructure projects have had average cost This cost was as much as $63 million per year in subse- overruns of about 28 percent (Flyvbjerg et al., 2002). quent years; it is unclear how much from prior years is These account for the cost of inflation, whereas the improperly included in the NEXRAD completion cost. GAO numbers for the MAR do not. When the inflation Fourth, it is not clear that all costs, such as the tran- difference is included, and the external factors (such as sient staff increase needed to execute the MAR, were the Challenger failure) are accounted for, MAR cost and properly included in the GAO reports. The difference schedule issues appear to be high but not substantially between summing the program element costs shown out of line with experience on similar projects. There is in the table and the cited MAR total cost appears to no question that issues with virtually all MAR elements correspond to MAR-related cost elements not included persisted through MAR completion as documented in by the GAO but referenced in the NIP budgets. These GAO reports. But one argument might be that while internal R&D, construction, and temporary personnel these were all the responsibility of NOAA, many of the costs were originally expected to be about $500 million. issues were inherited by the MAR and should not be The actual cost is difficult to determine, but it appears attributed to it. As much as $1 billion had been spent to have been between $900 million and $1,200 mil- prior to the formal MAR initiation, and many of the lion (NWS, 1990, 1991a, 1992b, 1993, 1994b, 1995, issues that subsequently plagued these program elements were already committed by that time. Question 3: What were the root causes of the cost and 6 GAO included the cost for the entire NEXRAD system in the schedule issues? GAO reported extensively on the prob- 1991 estimate but only the NOAA portion in the 2000 summary.

OCR for page 19
23 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING lems with MAR elements in a contemporary context, government and the planned use of non-market-cost but the root causes are not well described and are still shuttle launches, played a role. But no singular issue difficult to identify from other sources. More than stands out as a clear MAR-specific lesson for the future half of the total overrun occurred within the satellite readily identified in the history. The following should upgrade program element alone. This overrun has thus be viewed as informed opinions of the committee been widely attributed to poor government oversight rather than a definitive analysis of MAR performance. and technical problems encountered by the contractor The MAR clearly suffered from poor ‘project (GAO, 1989, 1991b). While correct, a deeper analysis initiation’ when its roots in the early 1980s are consid- reveals two major external contributing factors that are ered. It was pulled together from previously initiated poorly referenced in GAO summaries. program elements with different management teams, The first is inadequate initial costing of the launch varying procurement experience, and only partially component, a result of the lack of full cost-accounting aligned objectives. There was no integrating archi- associated with Shuttle launches that was used to help tecture until well into the MAR. At some level, the justify the Shuttle program at the time. Following the problems with each program element were independent Challenger accident in 1987, GOES-Next switched to of the others. But a common theme was an attempt to expendable launch vehicles and had to adjust launch do complex development with procurement processes costs to reflect market values. not up to the task; ASOS: (GAO, 1995h); NEXRAD: The second is the cost-constrained government (GAO, 1995f ); GOES-Next: (GAO, 1991b); AWIPS: environment within which GOES-Next was conceived, (DOC, 1992). Weak procurement processes lead to leading to an ill-advised procurement plan, which elimi- poorly-defined objectives, incomplete understanding of nated a critical development phase while at the same technical and programmatic risks, inadequate mitiga- time requesting substantial technology advances.7 While tion processes, overly rigid processes, and selection of each of the MAR elements had distinct issues, the com- contractors without sufficient experience or with design mon internal contributing factor appears to have been flaws in their proposals. Once these problems are set weakness of the procurement process. In all cases, it is in place, program execution becomes a series of recov- difficult to separate the relative roles of an inadequate ery actions. With the MAR, these issues had almost government contracting process and poor contractor a decade to develop before coming under the MAR performance within the procurements. Examples of both auspices. After MAR initiation, individual initiative can be identified. What can be said is that these issues seems to have been a critical element in completing were largely set in place prior to MAR initiation. MAR the planned technological changes without further management appears to have taken repeated steps to c ost growth, although additional schedule delays recover; the fact that the accepted MAR expenditure of occurred. The parallel development of a PROFS-based $4.5 billion (GAO, 2000) is actually lower than the 1991 approach to replace the AWIPS contracted solution is estimate of $4.6 billion (GAO, 1991a) is a testament. one example—an excellent case of flexibility built into the process to recover from unanticipated problems. Question 4: What lessons can be learned for the future? Practical lessons unique to the MAR are difficult to Decisions during the MAR undoubtedly contributed to identify beyond those that apply to the challenges of further cost and schedule issues, but the most important executing all large projects, of which there were many. lesson appears to be the need to establish a procure- These lessons could fill their own report. Certainly, con- ment process with sufficient definition yet adequate temporary issues, such as the 1980s debate about limited flexibility to accommodate the challenges of complex system development. 7 Specifically, the Phase B development phase was eliminated, something usually done only for systems that have little or no new Organization and Staff technology development. The planned improvements included a switch from a spinning spacecraft to one that is three-axis stabilized The MAR implemented significant changes in and the corresponding switch from instruments that scan based both organization and staffing. Prior to the MAR, on spacecraft motion to those that stare and perform scanning the NWS culture was resistant to change. This was internally.

OCR for page 19
24 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT understandable, based on the experience with Auto- It is appropriate to ask what ongoing cost savings mation of Field Operations and Services (AFOS), the were achieved by this staffing reduction. The staffing only other significant technological upgrade that was mix was about one-third meteorologists and two-thirds implemented NWS-wide. Therefore, as the MAR technicians prior to the MAR and the reverse after- plan was introduced the staff generally accepted that ward, with an overall reduction from 5,100 to 4,700 change was inevitable. They were motivated to evolve (GAO, 1995d; NRC, 1994a; Sokich, 2011). Meteo- the culture (Glackin, 2011), though they were anxious rologists are grade GS-12 to GS-14 employees while about the uncertainties of change. Planners anticipated technicians are GS-9 to GS-11. With typical GS pay these issues, but it is not clear that the human dimen- rates, this implies an increase in overall staff cost of sions of the change were fully appreciated. Staffing about 7 percent, though a more thorough analysis with levels underwent a temporary increase: 5,100 prior to actual personnel data could reach a slightly different the MAR, about 5,400 during the MAR, and evolving conclusion. Had the originally planned reduction to to 4,700 today (Friday, 2011; GAO, 1995d; Sokich, a staff level of 4,038 (GAO, 1995d) been achieved, a 2011).8 Such a temporary increase was to be expected savings of 8 percent would have been obtained instead. during the changeover from pre-MAR to post-MAR W hether this originally planned staffing reduction operations (GAO, 1995d), while at the same time was a target or a commitment is unclear. The MAR ensuring the Congressional mandate for no degrada- S trategic Plan (NWS, 1989) stated ambiguously tion of service (U.S. Congress, 1988). NWS promised “. . . lower costs associated with more accurate and employees and NWSEO that any staff reduction would timely warning and forecast services are accomplished occur by attrition only (Friday, 2011).9 The stated while concurrently increasing the benefits. . .” Further- commitment to retain and formally retrain staff was more, cost savings are measured against a baseline, and essential to maintaining morale as well as enlisting NWS argued in part that the deployment of new tech- cooperation of NWSEO, with the shared story being nology would otherwise have required additional staff. that staff would be better off as a result. Many NWS “If the new technological network were constrained field office staff members recall that the change they by the current field office structure, required staffing encountered was hard at the time, but with years of levels and overall costs would increase unnecessarily” hindsight they now see the change as worthwhile (com- (NWS, 1989). mittee member WFO site visits, see Appendix C for list of WFOs visited). Staff at RFCs was also affected by Processes changes in office locations and staffing profiles, as well as new technologies and procedures for working with The MAR was executed using a wide variety of the WFOs. Other staff, such as those at the National processes. These included the following: Centers, was also affected through the consolidation of the centers. • Planning and Documentation . Several NWS and National Research Council reports (e.g., NRC, 1980) preceded the MAR and set the stage for what 8 It is noteworthy that this staffing level is small compared to was expected from it. Execution plans were docu- weather agencies in some other industrialized countries, such as mented in a strategic plan (NWS, 1989), a sequence of Japan and China and certainly for Europe as a whole where each annual implementation plans (e.g., NWS, 1990) that country has its own meteorological service and several countries tracked progress, and a well-defined set of site-specific operate an equivalent of the NWS National Center for Environ- mental Prediction (e.g., United Kingdom, France, Germany, a joint and transition plans. External reviews (e.g., General Scandinavian Center) as well as the European Centre for Medium- Accounting Office [GAO], Modernization Transition range Weather Forecasts. For example, Japan cites staffing of 5,555 Committee [MTC], NRC) also contributed. during FY2008 and countries such as Germany, United Kingdom, • Plan Execution. Analysis of these reports shows and France typically fall in the range 2,000 to 4,000. 9 Primarily retirement, though some staff left because they did t hat execution largely followed the original plan. not like the required relocation or personal changes (such as retrain- Real-time issues forced some key changes. One good ing from being a meteorological technician to being a professional example is the transition of the majority of the devel- meteorologist).

OCR for page 19
25 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING Finding 3-1 opment of the Advanced Weather Interactive Process- During the Modernization and Associated Restruc- ing System (AWIPS) from a contracted provider to a turing (MAR) period from 1989 to 2000, the major NOAA entity. All of the major system procurements components of the MAR were well planned and com- required frequent adjustments to respond to technical pleted largely in accordance to that plan. Established and programmatic issues. processes were extensive and generally followed. • Organizational Dynamics. The NWS placement However, notable budget overruns and substantial within NOAA and DOC determined which processes schedule delays occurred for nearly all of the project were employed and how. In contrast to a major tech- elements. This was due in large part to the MAR nological procuring agency like DOD, DOC, possibly aggregating four major technology programs that with the exception of NESDIS, rarely undertakes an had been separately initiated during the 1980s. Many effort the size and scope of the MAR, and therefore of the MAR’s cost and schedule issues were set in must create essentially a one-time process and assemble place by decisions that occurred during this pre- staff to undertake the unique systems acquisitions. It MAR period. follows that DOC has essentially no room for extended evaluation or internal budget and program adjustment. Each decision becomes a budget decision. MODERNIZATION OF TECHNOLOGY • Process Flexibility and Individual Initiative. A critical contribution to MAR success was the individual As described in Chapter 2, the MAR included the initiative to deviate from process where it made good development, procurement, and deployment of tech- sense. Persistence and individual initiative from senior nologies in five major areas: surface observations, the staff and the general workforce was in many cases criti- radar network, satellites, computing upgrades, and a cal to success when process alone could not overcome forecaster interface to integrate the data and informa- impediments. tion made available by the other elements of the mod- • Oversight. Many oversight bodies examined and ernization. The systems procured as part of the MAR influenced the MAR process. This topic is addressed all involved major technology upgrades, which require more completely later in this chapter. long lead times, on the order of many years, and in the • Communication. The original MAR plan encour- case of satellite systems, on the order of a decade. One aged active communication channels with Congress, of the strengths of the MAR was the development, the private sector, NWSEO, oversight entities, and prototyping, and demonstration of operating concepts other stakeholders. The continuing communication through a number of risk reduction activities. The MAR and outreach to partners through these channels was planning included the Modernization and Associated critical to MAR success. Restructuring Demonstration (MARD), which was • Validation. The AFOS program of data collec- intended to showcase the new capabilities of the mod- tion established a performance baseline that enabled ernized NWS (NWS, 1989, 1990). The Program for performance improvement validation. By the final Regional Observing and Forecasting Services (PROFS) MAR annual report (NWS, 1999), several statistics created a laboratory that used prototypes of NEXRAD for improvements in tornado warning accuracy and and AWIPS to develop operating concepts for the lead time, flash flood warnings, hurricane landfall post-MAR weather offices. These included the Denver prediction, and other metrics were available. However, AWIPS Risk Reduction and Requirements Evaluation publically available, systematic, long-term validation of (DAR3E) and the Norman AWIPS Risk Reduction and surface weather forecasts over the United States is not Requirements Evaluation (NAR3E), which assisted in widely available outside the NWS. transitioning PROFS prototypes into operation. • Commissioning . The commissioning process evolved from an initial ad hoc effort to a regular and Automated Surface Observing System repeatable process as the MAR progressed. This pro- cess satisfied the Congressional language mandating no As part of the MAR, the NWS cooperated with degradation of services. the FAA and the DOD to change the paradigm for

OCR for page 19
26 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT surface weather observing in the United States. The When AAI, Inc. was let the contract for full pro- new observation strategy deployed automated sen- duction of ASOS in the early 1990s, there were already sors to perform much of the work previously done by 55 “limited production”-run ASOS sites located in the human weather observers. The instrumentation suite southern/central plains. McNulty et al. (1990) studied was labeled the Automated Surface Observing System the Kansas ASOS sites and tried to determine whether (ASOS). At the time of the MAR, staff at about 250 ASOS resulted in improved forecasts. Although the airports across the nation manually gathered surface air- results were inconclusive, it was clear that it was left way observations (SAO). Staffing limitations prevented to the scientific community to determine what metrics some SAO sites from operating 24 hours per day. The would be used to evaluate the success of ASOS. Over ASOS deployment plan increased the number of surface the next decade, numerous publications appeared that observation sites to about 1,000. In addition, ASOS redefined the metrics, as well as gauged ASOS against allowed for the possibility of 24-hour operations, and those metrics. Some examples follow. more frequent observations than its SAO counterparts. In 1993, an NRC report found problems with the ASOS automatically collects surface weather data reliability of ASOS (NRC, 1993), and in November and electronically provides observations to weather 1994, commissioning of ASOS sites was halted (GAO, observers, weather forecasters, airport personnel, pilots, 1995h). Also in 1994, then NWS Director Joe Friday air traffic control specialists, and other users. The sys- stated, “[o]perational use of ASOS has allowed the tem automatically collects, processes, and error checks NWS to review ASOS performance in a real-world data; and formats, displays, archives, and reports the e nvironment. This experience has confirmed that weather elements included in the basic Aviation Rou- ASOS can provide timely and accurate observations for tine Weather Report (METAR) and Aviation Selected the aviation and meteorological communities” (Friday, Special Weather Report (SPECI). These data typically 1994). On behalf of itself and its partner agencies, include temperature, pressure, wind, type and intensity NWS had bought 617 units as of December 1994, and of precipitation, runway visibility, sky condition, and 491 of those had been accepted. Forty seven of the 491 ceiling height. To date, there are 1,009 ASOS stations accepted units had been commissioned (GAO, 1995h). deployed. These include 315 operated by NWS, 571 No human observers had yet ceased recording surface operated by the FAA, and 123 operated by the DOD observations. (Nadolski, 2011). NWS electronics technicians (52 Full In 1995, a General Accounting Office (GAO) Time Equivalent [FTE]) conduct the operations and report was commissioned that was the most critical of maintenance for NWS and FAA ASOS sites through ASOS to date, stating that “ASOS’ overall reliability an interagency memorandum of agreement (Nadolski, during 1994 winter testing, measured in terms of mean 2011). hours between critical system failures and errors, was The ASOS Preproduction Development contract only about one-half and one-third of specified levels, ($34M) was awarded to competing industrial sources respectively” (GAO, 1995h). The report stated that reli- in April 1988. Program reviews were completed in ability testing was not performed before deployment, October 1988 (Preliminary Design Review), in March so this problem surfaced after ASOS was deployed. 1989 (Hardware Critical Design Review), and in May The report documented that six of the eight ASOS 1989 (Software Design Review). The release of the system sensors did not meet contract specifications for Request for Proposals for the Deployment Phase of accuracy or performance. the ASOS contract occurred in June 1989. In 1990, The 1995 GAO report led the NWS to develop a a “ limited production” run of 55 ASOS units for the proposal to conduct limited tests comparing ASOS with three participating agencies were created (NWS, 1990). manual observations for a period of six months at 22 These limited production units supported other mod- commissioned and four noncommissioned ASOS sites. ernization prototype activities, primarily in the central This ASOS Aviation Demonstration was designed to and southern plains. AAI, Inc. won the production assess the “operational representativeness and system contract in February 1991 and provided for the balance performance” of ASOS in different weather regimes of all required ASOS systems (Nadolski, 2011). (NWS, 1996a). At the time, “operational representa-

OCR for page 19
27 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING tiveness” was defined as “the ability to provide accurate to inflation contributing to the cost increase included and timely weather observations in support of aviation an increase in the number and change in the types of operations,” and “system performance” was defined units to be procured; inclusion of costs such as WFO as “the ability of ASOS to generate and transmit construction, training, and logistics not incorporated complete observations through the communications in the original estimates; and technical and contractual network” (NWS, 1996a). The Demonstration occurred problems. in 1995, and the results were reported in an internal Efforts to deal with these problems continued NWS document in February 1996 (NWS, 1996a). The through the spring of 1991, when the tri-agencies and Demonstration found that while there were some dif- contractor reached a comprehensive settlement of con- ferences between automated and manual observations, tract claims and deficiencies. Meanwhile, in 1990 the “the operational representativeness and availability of option to start Full-Scale Production had been exer- the ASOS system was, in general, very good.” The cised and the first Limited Production Phase unit had Demonstration also highlighted a higher number of been delivered. Further Operational Assessment took short duration failures than expected. Modifications to place with that unit in the spring of 1991. However, the sensor suite were developed to address this problem, the reliability problems continued into the mid-1990s and while they were not deployed during the Demon- (GAO, 1995f ). stration, commissioning of ASOS sites resumed based The prototype and the first half-dozen fielded on expected improvements in the sensor suite (NWS, systems operated with circular polarization, mainly 1996a). to facilitate the suppression of ground-clutter echoes The main impetus behind the deployment of ( earlier operational weather radars operated with ASOS was achieving the cost and staff reduction goals linear polarization). However, research on microwave of the MAR. This contributed significantly to gaining propagation through rain had revealed a difference in Congressional approval for the MAR. The deployment the propagation velocity (and hence in the phase shift) of ASOS enabled a reduction in the number of NWS of horizontally versus vertically polarized waves (e.g., field offices and reduced the staffing levels needed to Oguchi and Hosova, 1974; Seliga and Bringi, 1976), a make surface observations. The deployment of ASOS property of the medium that would gradually degrade also shifted the NWS workforce toward one with fewer the circularly-polarized signal as it passes through. A technicians and more professional meteorologists. circularly-polarized research weather radar had been operating in Alberta for some 15 years (McCormick, 1968) and this behavior of the circularly-polarized Next Generation Weather Radar waves was known (e.g., Humphries, 1974). This unac- As noted in Chapter 2, the tri-agency NEXRAD ceptable feature necessitated a redesign of the system program was well under way prior to the official begin- and conversion of the already-fielded systems to linear ning of the MAR. The NEXRAD program initially polarization. The failure to account for the results of did not provide for adequate prototype demonstrations prior research in this case was a shortcoming of the under operational conditions. An Initial Operational JSPO operation. Test and Evaluation (Part 2) carried out by the USAF The NEXRAD program was supported from the (1989) using the Unisys NEXRAD prototype provided beginning in both engineering and scientific mat- an independent test that highlighted a number of prob- ters, first with an Interim Operational Test Facility lems requiring attention (NRC, 1991). These ranged (established about the time the NTR was issued) to from reliability concerns, software algorithms and assist in the development of hardware, software, and documentation issues, to training programs. Accord- operational concepts. This organization transitioned to ing to the GAO (1991a), since 1980 the schedule for an Operational Support Facility (OSF; later renamed completion of the NEXRAD system had slipped by Radar Operations Center) to support deployment, seven years and the estimated cost escalated by a fac- maintenance, operation, application, and upgrade of tor of more than four (though the latter was based on the WSR-88Ds. As NWS field sites began making use current-year dollars on both ends). Factors in addition of the Limited Production Phase radars in late 1991,

OCR for page 19
28 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT the OSF began operating a Hotline (eventually 24/7) NEXRAD Information Dissemination Service to provide consultation with the field staff as questions In the pre-MAR era, the NWS did not collect and problems with the new system arose. The OSF radar data at a central location and had limited capac- supported deployment of the NEXRAD systems with ity for redistribution of data from remote radar sites. a vigorous training program to help ensure effective In addition, users (researchers, universities, commercial operation and use of the new systems in the field. At companies, broadcasters, etc.) interested in collecting the same time maintenance training was conducted at radar data, analyzing and studying it, and/or potentially the NWS Technical Training Center. The NEXRAD redistributing it, had to provide their own communica- Technical Advisory Committee monitored the evolving tion equipment and the appropriate transmission lines program and provided engineering and scientific advice (Baer, 1991). During the development of NEXRAD, a and recommendations. OSF began issuing a series more robust capability to disseminate WSR-88D data of software builds in 1995 to introduce solutions to to users was part of the design. The NWS outsourced identified problems and upgraded capabilities. More- this capability, through a competitive procurement, to over, a NEXRAD Product Improvement Program was four companies (Alden Electronics Inc., Kavouras Inc., established to capitalize on continuing advances in Unisys, and WSI Corporation) and called the contrac- technology and science underlying the processing and tual agreement the NEXRAD Information Dissemina- use of the radar data. tion Service (NIDS). Through the NIDS agreement These aspects are pursuant to a trio of recommen- a suite of select WSR-88D base and derived radar dations in the second report of the NRC’s National reflectivity and velocity products (NIDS products) Weather Service Modernization Committee (NRC, were made available to subscribers such as television 1992b): stations, private weather forecasting companies, energy companies (gas and electric utilities), airlines, and other Modernization must continue beyond the imple- mentation of systems now being procured. Provision industries (Baer, 1991; Klazura and Imy, 1993; Morris should be made to . . . take advantage of scientific et al., 2001; Pirone, 2011). Special subscriber status developments as well as improved computational and was provided via the NIDS contract to universities, and information systems as they become available. federal, state, and local government agencies. NIDS Steps should be taken to ensure the continued develop- providers were allowed to charge such special subscrib- ment and improvement of Next Generation Weather ers for only the cost of delivery of the NIDS products, Radar processing algorithms as new developments and with restrictions on data redistribution. Alden, Kavou- operational experience accumulate. . . . ras, Unisys, and WSI each paid a one-time access fee The National Weather Service and the National Oce- of $780 per radar site and a recurring maintenance fee anic and Atmospheric Administration should create of $1,395 per site via a NIDS Access Agreement (Baer, technical advisory panels for each of the major systems 1991). The four NIDS providers were given exclusive that contribute to the technical modernization. . . . rights to redistribute the radar data to recover their costs of collecting the data from all sites and providing The first Full Scale Production NEXRAD was it on a display terminal for quality control purposes at delivered in mid-1992, and the last of the initially NWS headquarters. planned NWS radars was installed in 1997. An NRC During the transition from the WSR-57/74 radars panel reviewed the nationwide coverage of the network to the WSR-88D radars, NEXRAD data was merged in the mid-1990s and noted a few locations for which into the value-added radar products, including radar coverage was less satisfactory than that provided by the data mosaics, winter storm mosaics, and other innova- earlier systems (NRC, 1995b). Under the Congres- tive reflectivity-based radar products that have become sional “no degradation of service” mandate, action was commonplace and easily accessible through a multi- taken to provide better coverage to those locations. tude of media. It is clear that this acquisition strategy Three NEXRAD systems were added to the network for radar data via NIDS allowed competitive market in 1997-1998; another radar was installed in 2000, and forces to provide benefits not only to the government, yet another is to be added in 2012.

OCR for page 19
29 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING but also to the weather industry, and ultimately the (NPOESS; GAO, 1995c). This system and the associated public. Despite these benefits, such dedicated vendor program effort reflected the complexity involved when arrangements were problematic from a user perspec- a single system is to meet the needs of multiple, diverse tive. Such arrangements have the unintended side communities with differing requirements. NOAA did not effect of impeding hydrometeorological research and manage NPOESS. Instead an Integrated Program Office innovations in calibration and correction methodolo- had that responsibility. Thus it was not a part of the MAR gies because they can make data difficult or costly to and will not be addressed here. obtain. These arrangements are antithetical to the free The MAR included development and launch of the flow of scientific data and information upon which the GOES-Next satellite system. NESDIS is the line office scientific enterprise is founded, as well as the operating within NOAA responsible for satellite systems. Acting model of the NWS. The NIDS contract expired on on behalf of NESDIS, the National Aeronautics and December 31, 2000, and with the intervening advances Space Administration (NASA) awarded a cost-plus- in communication technologies the NWS became the award-fee contract in 1985 to Space Systems/Loral, sole provider for NEXRAD data (NRC, 2003a). Inc. (SS/L, formerly the Ford Aerospace Corporation), with an instrument subcontract to ITT Corporation (GAO, 1991b). Five new satellites were to be developed Satellite Upgrades and built, each with an imager and a sounder. GOES- The life cycle of a multi-satellite system procure- Next system improvements ultimately resulted in the ment can be long relative to the upgrade or development collection of substantially more weather data of higher of some of the other assets of NOAA. A full system quality. However, the program experienced several procurement, including planning, design, build, integra- technical issues, and substantial cost and schedule over- tion, pre-launch test, launch, and on-orbit operational runs. The official estimate of the overall development test activities, can easily extend over 10 years for a five- cost increased over 200 percent, from $640M in 1986 satellite system. Factors affecting the schedule include to $2.0B in 1996 (GAO, 1997c). The costs include the launch requirement date for each satellite, the number government effort as well as the contractor effort. The of satellites and instruments involved, changes in product launch of the first satellite was delayed from July 1989 requirements, and the design complexity of spacecraft to April 1994, leading to a potential gap in geostation- and instruments. The upgrade goals for the geostationary ary satellite coverage. Fortunately, NESDIS obtained system stated in the MAR Strategic Plan (NWS, 1989), use of a European METEOSAT, and avoided the as well as the plans for the NEXRAD network, had been threatened outage (NRC, 1997b). The second satellite under development well before the MAR, and may have (GOES-9) exhibited signs of imminent momentum been implemented in any case. However, it is likely that wheel failure three years after launch and was taken out the MAR made the realization of the NEXRAD and of operation (GAO, 2000). All five satellites were ulti- satellite upgrade goals possible by gaining the necessary mately launched, becoming GOES-8 (launched April public support and financial support from Congress. 1994), GOES-9 (May 1995), GOES-10 (April 1997), The satellite system that was part of the MAR planning, GOES-11 (May 2000), and GOES-12 ( July 2001). referred to as GOES-Next, will be addressed here. The development, execution, and technical prob- The desired polar system upgrades foreseen in the lems that occurred during the program effort can be MAR Strategic Plan included all-weather atmospheric summarized as follows: data (by implementing microwave imagers and sound- ers, for example). However, in May 1994 President • Lack of preliminary analyses and ensuing design Clinton signed a directive requiring DOD and DOC complexity. The typical engineering analyses usually to integrate their separate satellite systems. The Defense required for a technical program were not authorized by Meteorological Satellite Program (DMSP) and the Polar NESDIS or required by NASA prior to GOES-Next Operational Environmental Satellite (POES) converged development work. They concluded there was sufficient into a single, national system, the joint National Polar- proof-of-concept in “body-stabilized” spacecraft and orbiting Operational Environmental Satellite System instrument design heritage, and NOAA was facing

OCR for page 19
34 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT tions technologies that were occurring independently NEXRAD radars. Each WFO was assigned responsi- of the MAR. bility for forecasts and warnings for a county warning In hindsight, telecommunications costs need not area (CWA) covered by its NEXRAD. have been a major factor motivating the colocation of The transition to the new organizational structure radars and WFOs. The time scale for major govern- required closing more than half the existing offices, ment technical initiatives is quite different than the a politically sensitive issue. The “no degradation of time scale over which change occurs in digital tech- service” requirement of Public Law 100-685 called nologies. Hence, federal programs with a large digital for a certification of no degradation before any office technology component need to be aware that prevailing could be closed. An elaborate certification proce - costs and capabilities during the planning period are dure was established to meet this requirement (U.S. not appropriate for their forecasts of eventual costs and Congress, 1988). It included commissioning of the capabilities. This dilemma is not easily overcome and is newly-installed technologies (ASOS, NEXRAD, and a factor that needs to be taken into account. An agency AWIPS) and demonstration that forecasting and warn- capability for rapid prototyping and user-feedback ing services could be provided to the CWA before the during a major acquisition is one way of dealing with WSO or WSFO previously serving that area could the reality of rapid technological change. Another can be closed. The certification process was overseen by be leasing of computation capabilities as opposed to the Modernization Transition Committee (MTC), a purchase, because provision can be made for constant Federal Advisory Committee. upgrading of agency capability. Workforce Finding 3-2 The various technological problems that were The field staffing was changed from a mix of about encountered included lack of preliminary analysis one-third professional meteorologists and two-thirds and ensuing design problems, inadequate program meteorological technicians before the MAR to the m anagement, and poor contractor performance. reverse after the MAR (Sokich, 2011). Meteorological T hese problems were generally overcome and the technicians, while required to become certified in several major technology system upgrades were successfully important meteorological tasks, are not required to have executed. a professional atmospheric sciences degree. Before the MAR, they were mainly responsible for observations, including radar, aviation surface weather, and upper air RESTRUCTURING OF (via radiosonde) observations. In the WSOs, they were FORECAST OFFICES AND STAFF also responsible for issuing severe weather warnings Restructuring of the NWS involved substantial (e.g., tornado, severe thunderstorm, flash flood) based reduction in the number of field offices, relocation and/ on radar observations. Other duties included answering or realignment of the functions performed at many of phones and attending to the NOAA Weather Radio. those offices, and staff changes including reduction Meteorologists have professional atmospheric sciences in total numbers along with upgrading of the overall degrees. Before the MAR, meteorologists were mostly professional levels. found only at WSFOs. Generally at WSOs, the Meteo- rologist in Charge (MIC) was the staff person with a meteorology degree. At WSFOs, journeyman and lead Consolidation of Offices forecasters held degrees in atmospheric sciences and The 52 Weather Service Forecast Offices (WSFOs) were responsible for severe weather warnings within and 204 Weather Service Offices (WSOs) were replaced their area of responsibility, in addition to statewide by 122 Weather Forecast Offices (WFOs). The distri- aviation, marine, and public forecasts, discussions, and bution of WFOs was based on attaining an even dis- summaries. The lead forecasters at WSFOs also served tribution of offices across the nation for equal service as shift supervisors at their office while also overseeing provision, and it generally followed the distribution of the work of all WSOs under their jurisdiction.

OCR for page 19
35 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING With the revised makeup of WFO staff planned C ommunication about the MAR between NWS under the MAR, the question of bringing staff meteo- management and field level staff was perceived as rological technicians up to the required levels of train- inadequate (NRC, 1994a). A 1994 NRC survey of ing arose. A program was established at San Jose State employee attitudes about the MAR found that while 66 University to provide training equivalent to a B.S. percent of respondents felt they received enough infor- degree in meteorology; support was offered to any of mation about the new technologies, 61 percent felt they the meteorological technicians who wished to qualify received too little information about the implementa- for meteorologist positions. The program was free of tion process and timing of the MAR (NRC, 1994a). cost to the technicians who participated. While not This lack of communication with field office employees many went into the program (Sokich, 2011), it did likely contributed to some of the initial resistance to the allow some to upgrade their skills and thus bring the MAR. The 1994 NRC survey found that within job benefit of their experience into the modernization era. categories, meteorological technicians were the least NOAA also initiated a Cooperative Agreement with optimistic about the MAR (NRC, 1994a). the University Corporation for Atmospheric Research Before the MAR, each WSFO was led by a (UCAR) to implement the Cooperative Program for Meteorologist in Charge (MIC) and a Deputy MIC Operational Meteorology, Education, and Training (DMIC). The DMIC had a diverse set of responsibili- (COMET). COMET, which still exists, provided pro- ties, from personnel management of the WSFO staff fessional development courses for operational forecast- and staff scheduling, to attending to media requests ers (NWS, 1991a). Most of the training was intended and educational outreach. The Deputy could well have to be taken through “distance learning” facilities. This been the most multidimensional person on staff. The was initially a challenge, but the advent of the Internet MAR’s groundbreaking division of deputy duties into created a truly flexible capability for distance learning. the Science Operations Officer (SOO) and the Warn- NEXRAD training was provided in Norman, Okla- ing Coordination Meteorologist (WCM) allowed for a homa (NWS, 1991a), and was viewed favorably by the more focused approach to two critically important tasks workforce (NRC, 1994a). Training on the new tech- at the WFOs. These new positions were responsible for nologies was also provided for the electronics personnel. incorporation of ongoing scientific advances into WFO The initial plan was for maintenance, at least for ASOS, operations, and communication with the external user to be contracted out. However, it was determined that community, respectively. The SOO in essence was the retraining existing electronics technicians would be office’s lead scientist, typically holding a Ph.D. or M.S. more cost effective (Sokich, 2011). and a strong scientific background. This enhanced In addition to training, the change in field office WFO staffing provides for improved forecast and distribution required relocation of many staff, which warning performance by enabling increased situational c aused some dissatisfaction within the workforce awareness and recognition of evolving severe weather, (NRC, 1994a). However, the upgrading of staff was speed and accuracy of issued warnings, and frequency accomplished without forced termination of any of the and quality of “follow up” severe weather communica- in-place personnel. The reduction in total staffing level tions that augment the initial warning messages. was achieved primarily through retirements. The National Weather Service Employees Organi- Changes in Customer Linkages zation (NWSEO) played a crucial role in the process, becoming more engaged than ever before in defending Customer service advanced significantly with the and helping define the future role of its constituent creation of the WCM position at WFOs. Before the members. One proposal (Booz Allen & Hamilton Inc., MAR, outreach from NWS to the user community was 1983) was to reduce staff to less than 3,000 employees, spotty at best. As noted, this was one of many func- down from the pre-MAR figure of 5,200. However, tions of the DMIC at most WSFOs. WSO sites were the final number of employees after the MAR was far staffed by technicians focused on data acquisition and greater (4,700) due in part to the efforts of NWSEO the issuance of storm-based warnings. In the few U.S. and NWS management (Friday, 2011; Hirn, 2011). cities with a WSFO, ad hoc staff efforts to reach out

OCR for page 19
36 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT Finding 3-3b to the general population often superseded infrequent During the early stages of the Modernization and efforts from the MIC or DMIC (Santos, 2011). Associated Restructuring, there was insufficient Before the MAR, communication links were mostly communication between National Weather Service one-directional. For example, media requests were typi- management at the national level and the field office cally handled on a case by case basis. Professional meteo- managers and their staff, as well as the employee rologists were located at WSFOs in less than a quarter union. of the country’s main media markets. The creation of the WCM increased and strengthened the linkages between the NWS and media outlets. A strong partner- NATIONAL CENTERS ship between the NWS and the media and emergency management community is crucial to facilitate timely Concomitant with the goals of the MAR was the and accurate delivery of lifesaving messages. need to implement and sustain more science-based The field of emergency management was under- approaches to weather, climate, and hydrological pre- going its own modernization during the decade of the diction, and to rapidly assimilate evolving facets of MAR. The end of the Cold War provided the final information technology. To do so required restructur- incentive to transition away from the civil defense ing of the relationship between WFOs, RFCs, and the posture of earlier decades. The 1990s saw a significant various National Centers. At the time of the MAR, shift to preparing for all hazards that face communi- the National Meteorological Center (NMC) had six ties. There was a greater emphasis on preparedness by components: Automation, Development, and Meteo- individuals and communities and on mitigation against rological Operations Divisions; the Climate Analysis future disasters. C enter; the National Hurricane Center; and the The services of the NWS continued to be of great National Severe Storms Forecast Center (McPherson, value to emergency managers (EMs) during the MAR. 1994). The National Centers, as they exist today, serve W hile at some field offices special telephone hot lines to support many core activities of the NWS through the or radio communication devices provided a direct link collection, ingest, analysis, and archival of weather, cli- between the NWS and EMs, there was no uniformity mate, oceanographic, space environment, and hydrol- of linkages or services to the EM community. In some ogy data; the development of data assimilation and offices, state and local EMs were customers in the numerical modeling systems; and the generation of same manner and priority as an individual citizen. many forecast products. Those charged with first response to disasters received Although most of the Centers were affected the NWS warnings at the same time and in the same by the MAR in some manner, the most significant manner as the general public. The local authorities then changes were experienced through the development issued their own instructions about evacuation, shelter- of the National Centers for Environmental Prediction ing, and other emergency measures. (NCEP). The overarching mission of NCEP has been The improvement in NWS warning times for to deliver science-based environmental predictions to tornadoes, flash floods, and other fast breaking events the nation and the global community. As the principal contributed to the overall time needed for action by developer of prediction models and forecast products, local governments and individuals, but the process the specialized centers within NCEP played, and remained linear, with information passing from NWS continue to play, critical roles in the evolution of the to local governments, to individuals and households. science-based prediction methodologies upon which most forecast products are based. Timely and efficient Finding 3-3a access to products being generated by NCEP was one The restructuring of offices and upgrading of staff of the motivating factors in the development of the b rought more evenly-distributed and uniform AWIPS system. The education and perpetual training weather services to the nation. of NWS staff is also driven by the rapidly evolving technology being incorporated by NCEP as well as other National Centers.

OCR for page 19
37 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING FIGURE 3.2 Organizational structure of the National Centers for Environmental Prediction (NCEP) that resulted from the 1994 reorganization. The reorganization was not officially part of the MAR, but the concept for reorganization was based largely on the principles of the MAR. Data provided by the NWS indicate that the number of FTEs was largely unchanged by the reorganization. The Space Weather Prediction Center did not become part of NCEP until 2005 when it moved from the Office of Oceanic and Atmospheric Research. SOURCE: National Weather Service. In 1995, NCEP was formed out of existing NOAA provides nationwide analysis and forecast guidance components (McPherson, 1994; UCAR, 2010). The products out through seven days; individual components, many of which were previously • NCEP Central Operations (NCO): sustains and housed at the NMC, now include (see also Figure executes the operational suite of numerical analyses 3.2): and forecast models and prepares NCEP products for dissemination; • National Hurricane Center (NHC): provides fore- • Office of the Director (OD): provides overarching casts of the movement and strength of tropical weather management to the nine centers; systems and issues watches and warnings for the United • Aviation Weather Center (AWC): provides avia- States and surrounding areas; tion warnings and forecasts of hazardous flight condi- • Ocean Prediction Center (OPC): issues weather tions at all altitudes within domestic and international warnings and forecasts out to five days for the Atlantic air space; and Pacific Ocean north of 30ºN; • Climate Prediction Center (CPC): monitors and • Space Weather Prediction Center (SWPC): pro- forecasts short-term climate fluctuations and provides vides space weather alerts and warnings for distur- information on the effects climate patterns can have on bances that can affect people and equipment working the nation; in space and on Earth; and • Environmental Modeling Center (EMC): devel- • Storm Prediction Center (SPC): provides tornado ops and improves numerical weather, climate, hydro- and severe weather watches for the contiguous United logical, and ocean prediction through a broad program S tates (CONUS) along with a suite of hazardous in partnership with the research community; weather forecasts (NWS, 2011a). • Hydrometeorological Prediction Center (HPC) :

OCR for page 19
38 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT As stated by McPherson (1994), “the operations vate sector through contracted work, and the research concept for the [MAR] includes a vertically-integrated community played a large role in the development of forecast process in which the national centers will MAR technologies, particularly AWIPS. provide a suite of products consisting of output from As a complex program, the MAR required a major, numerical models (EMC), statistical adjustments to the coordinated effort across the NOAA line offices. A model fields (HPC, CPC), and value added products key partnership of the MAR consistsed of NWS and (essentially all units) . . . ” These products would then be the other NOAA line offices involved in carrying out digitally transmitted to forecast offices via the AWIPS the MAR: NESDIS and OAR/ERL. The leadership system and used as the basis for issuance of local (Directors) of NWS, NESDIS, and ERL formed an forecasts, watches, and warnings. Therefore, follow- ad hoc group that was referred to as the “Troika.” The ing restructuring, NCEP served as the principal data principal function of the group was to pull together ingest point for all available domestic and international the programmatic, budget, and technological aspects of meteorological data. the MAR to address, coordinate, and direct the various Prior to the MAR and the formation of NCEP, offices working on the MAR activities within each line many of the centers such as the CPC, EMC, and OPC office, and to present a unified program, budget request, focused more on model development and evaluation and technological picture to NOAA and DOC man- than forecast product generation. The pre-NCEP modus agement. The work of the Troika required very intense operandi of the centers was necessitated by the fact that support, especially from the NWS staff overseeing many numerical prediction techniques were still in the the MAR. After the last AWIPS was commissioned early stages of development and possessed very modest and critical staff left the NWS, the Troika activity levels of skill and reliability. However, scientific advances diminished. While the end of the MAR removed the in NWP, coupled with the enormous revolution in motivation for an arrangement as formal as the Troika, observational capability and computational capacity coordination across line offices was a strength of the occurring in the late 1980s and 1990s, was accelerating MAR, and the NWS could benefit from similar coor- prediction science. Thus, the restructuring of the NWS dination today. national centers into the present structure, particularly the creation of NCEP, was motivated by the need to Other Federal Agencies make new observations and forecast products instantly available to NWS forecasters. The FAA, DOD, and NASA all participated with the NWS in the financing and implementation Finding 3-4 of the MAR. The FAA, DOD, and NWS cooper- The overarching Modernization and Associated ated in the development and deployment of ASOS R estructuring goal to integrate science-based and NEXRAD. NASA managed the development a pproaches to weather, climate, and hydrologic and procurement of GOES-Next, and that relation- prediction, and to rapidly assimilate evolving facets ship was discussed earlier (Bjerkaas, 2011; Misciasci, of information technology, led to the formation of 2011). the National Centers for Environmental Predic- The FAA, DOD, and NWS formed a tri-agency tion, which have become a key part of the National program to manage and fund the development and Weather Service. deployment of ASOS. These agencies planned to install 868 ASOS units by FY1997 at a cost of $351 mil- lion (GAO, 1995h). The FAA and NWS anticipated PARTNERSHIPS that the ASOS installations would allow the NWS to Execution of the MAR by the NWS depended eliminate human weather observers at many airports. on the involvement of many partners. Development However, many aviation users perceived that this would and deployment of all the observational systems of have the effect of reducing the overall quality of surface the MAR involved other NOAA line offices as well as observations. To some extent this is the case; the ASOS other federal agencies. The NWS worked with the pri- data are essentially point observations, while human

OCR for page 19
39 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING observers were able to incorporate some information weather enterprise. The NRC report Aviation Weather about the surrounding area. Services: A Call for Federal Leadership and Action sum- Of the 163 planned NEXRAD radars, 144 were marized the situation in 1995. The report noted that to be located at sites within the CONUS operated by aviation weather is a specialized area that falls outside both the NWS and the DOD, while 19 were to be the mainstream of general-purpose weather services placed at locations in Hawaii, Alaska, the Caribbean, and makes the point that aviation meteorologists need the Atlantic, the Pacific, and Korea (GAO, 1995f ). The to be sure to spend enough time with weather users to CONUS sites were selected to provide coverage to sup- develop a detailed understanding of the operational port the respective missions of the three agencies. The situation and information needs of those users. The non-CONUS sites were selected to support FAA and report found that in 1995 there was insufficient time USAF aviation safety and resource protection tasks. spent in this area, and noted that the MAR had the Several of the non-CONUS sites also provided needed potential to exacerbate the problem by moving many information to the NWS about approaching off-shore weather service offices away from local airports (NRC, weather. Data from the radars are shared among the 1995a). three agencies. The three agencies originally planned to purchase Private Sector and deploy 175 NEXRAD radars (including 115 for NWS, 44 for the Air Force, and 16 for the FAA). The private sector plays two critical roles within That number was eventually decreased to 163 due the weather enterprise: (1) contracted development to changes in requirements and funding limitations of NWS and NESDIS systems and contracted provi- (GAO, 1995f ). The radars were to be purchased, sion of supercomputer capability, and (2) provision of operated, and maintained by the respective agencies, weather services to end users. The first of these is a but data from the radars were to be shared among the relatively normal aspect of government projects, and is agencies. The NEXRAD JSPO, organizationally resid- addressed elsewhere in this report within the relevant ing within NOAA, but staffed and funded by the FAA, technology system descriptions. The second, provision DOD, and NWS, was established with responsibility of useful products based on NWS data, is more unusual, for managing the acquisition of the radars. particularly with regard to the relationship between A 1995 GAO report noted that the USAF public and private roles and deserves focused treatment. NEXRAD radars, while essential to the NWS to issue The private sector has been an important element quality forecasts and warnings, were not available as of the weather enterprise for many decades, with at much as the three agencies had agreed was necessary least 50 companies operating in 1980 (NRC, 1980). (GAO, 1995f ). The requirement specified that each At the start of the MAR, the private sector was grow- NEXRAD radar be operationally available 96 percent ing rapidly as an important element of the weather of the time. In 1994 and 1995, only 38 to 90 percent enterprise for provision of services. Weather services of USAF radars met this requirement on a month-to- companies, such as AccuWeather (founded 1962) month basis (GAO, 1995f ). The report specifically and The Weather Channel (founded 1982), had been identified the inefficiency of the USAF’s supply and around for some time. They served both the media logistics process for spare NEXRAD parts as com- market for communicating weather to the public and pared with the NWS. This problem was resolved after the enterprise market that uses weather information the USAF identified NEXRAD as a unique support for operational and decision-making needs (including responsibility in their supply system and converted financial instruments such as insurance). In part, this from contracted to government employee maintenance. helped fill a role the NWS did not want to serve, such Aviation weather services in the United States as providing TV visualization and newspaper layouts. are provided to nonmilitary aircraft by the FAA, the Other important private sector roles included providing NWS, and the private sector. The DOD provides its software and other tools for meteorologists, building own tailored weather services for its military operations and operating private sensor systems such as for road and shares its weather data with other elements of the

OCR for page 19
40 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT weather, supplying communications infrastructure, and ings and information were also beneficial. However, building major NWS systems (ASOS, satellites, etc.). the transitional MAR period was challenging, as cost Nevertheless, within NWS many viewed the pri- overruns and schedule slips made it difficult to plan for vate sector as competition. This perception was exac- availability of new NWS datasets to support innovative erbated by private sector lobbying during the 1980s for private sector products. substantial privatization of NWS services. Although this position was embraced by only a portion of the Research Community private sector, the environment polarized the two com- munities. At the time of the MAR, the convergence of Partnership with the research community was growing market needs for advanced weather services key to the success of the MAR. There was debate (beyond what was available from NWS) and emerging about whether the modernized NWS should include information technologies (cable TV, computers, mobile a research function, but in the end the primary long- telephony) greatly stimulated private sector growth. term research role remained with OAR. However, The tension initially worsened as the private sector the introduction of the SOO positions (many have was poorly integrated by NWS into MAR planning doctoral degrees) at the WFOs allowed a substantial or execution. effort in applied research related to ongoing problems The first substantive step toward improving the faced in carrying out the WFO missions. Of course the relationship between NWS and the private sector was technical developments (e.g., ASOS and NEXRAD) publication of the 1991 Public-Private Partnership evolved from many prior years of research. However, it Policy (NWS, 1991b). It defined the relationship and was important to avoid the pitfalls that beset the earlier respective roles of the NWS and private sector. The AFOS system; the developer of AFOS worked largely policy statement’s primary purpose was to strengthen independently of the intended users and the result was a the foundation of a public-private partnership that system that, at least initially, failed to meet the real needs had evolved over 50 years. The goal was a partnership of those users. To avoid that hazard, many features of that enhances total service to the American public, the MAR underwent rapid prototype testing; PROFS, government, and industry. It resulted in more frequent operated by FSL, was a major component of this effort. interaction between NWS and the commercial weather The PROFS staff had access to a Doppler radar and providers (e.g., NWS director meeting with providers, experience in merging radar, satellite, and other data in computer and display systems. The DAR3E effort in the interactions at the American Meteorological Society’s annual meeting; Ritchie, 2011). However, to members 1980s and 1990s (e.g., Rasmussen et al., 1992; Wilson et of the private sector, these interactions were insuffi- al., 1988) subjected many of the modernization concepts cient. The commercial weather sector was in a growth to critical assessments by prospective users. Experienced curve. There was pressure on both sides to find a way forecasters were brought in to work with and evaluate to properly capture opportunities. During the MAR, the capabilities of the prototype systems, and suggest the private sector was not brought in on a foundational shortcomings and desired improvements. Data and basis (Frederick, 2011). In the view of many, the NWS products from PROFS were transmitted to the WSFO was trying to compete with (even undermine) the in Denver for evaluation of utility in their ongoing real- commercial weather sector (Myers, 2011). It wasn’t time forecast and warning mission. until the NRC’s Fair Weather report in 2003, which Ultimately software developed by PROFS and presented recommendations for strengthening the tested in this manner was chosen to replace that devel- public-private partnership, that these issues began to oped by the AWIPS contractor; the latter had been be fully addressed (NRC, 2003a). developed pursuant to a lengthy set of requirements Despite these conflicts, by the end of the MAR the but without ongoing assessment of how well it could commercial weather sector had greatly benefited from meet the actual needs of the eventual users, as discussed the increase in NWS’s foundational data elements and in the AWIPS section. The PROFS system was devel- numerical weather forecasting improvements. Better oped in parallel with the contractor software as a risk accuracy, timeliness, and localization of weather warn- reduction effort (NRC, 1996c).

OCR for page 19
41 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING through contractor relationships, while not perfect, The effectiveness of the NWS in carrying out its especially in the early years, were essential to success- mission depends upon integrating advances in science ful execution of the Modernization and Associated and technology into its processes for producing and Restructuring. disseminating weather information, forecasts, warn- ings, and other products and services. Many of those advances originate in the academic community, and OVERSIGHT AND ADVISORY GROUPS that community is also the primary source of the pro- fessional staff of the NWS. Thus close linkages with The magnitude of the MAR was large, both in the academic community can facilitate the education of scale and cost. The reorganization of staff and reloca- NWS staff and the transfer of research-to-operations, tion and closure of offices introduced political issues. as well as stimulate the researchers to investigate The NWS received a large amount of oversight and problems of concern to the NWS. The NOAA system technical advice both from within and outside the gov- of cooperative institutes on university campuses that ernment, throughout the execution of the MAR. existed prior to the MAR is one way of maintaining an effective relationship with the research community.11 Modernization Transition Committee The COMET Program, established in 1989 by an agreement between NOAA and UCAR, provides one The Modernization Transition Committee (MTC) mode of enhanced collaboration with the research com- was mandated by Congress in Public Law 102-567 munity. Training is a major component of the COMET and chartered pursuant to the Federal Advisory Com- program; some on-site short courses are offered to mittee Act in July 1993 (DOC, 1993). The committee meteorologists and hydrologists, and an extensive set of consisted of 12 members appointed by the Secretary modules is accessible online. COMET also supports a of Commerce, with five members from federal agen- number of small collaborative research efforts, related cies that provide or use weather services, and seven to problems of concern to the NWS, which involve members from the academic, research, private sector, NWS staff as well as university faculty and students emergency management communities, as well as repre- (Auciello and Lavoie, 1993; Johnson and Spayd, 1996; sentatives of the workforce (DOC, 1993). The primary Waldstreicher, 2005). role of the MTC was to ensure that no degradation of weather services would occur with the closure of Finding 3-5 any WFSO or WSO, by reviewing the certifications Partnerships between the National Weather Ser- prepared by NWS. The MTC was also responsible for vice and other National Oceanic and Atmospheric advising the Secretary and Congress on implementa- Administration line offices, other federal agencies, tion of the Strategic Plan and the annual development of state and local governments, academia, the research National Implementation Plans. The committee served community, and to some extent the private sector as an advisory body for implementation of the mod- ernization criteria mandated by Public Law 102-567, 11 The cooperative institutes that were in existence at the time and matters of public safety in the provision of weather of the MAR and conduct some weather-related research include services (DOC, 1993). the Cooperative Institute for Research in Environmental Science at the University of Colorado (CIRES; est. 1967), the Cooperative Institute for Marine and Atmospheric Studies at the University National Research Council of Miami (CIMAS; est 1977), the Joint Institute for Marine and Atmospheric Research at the University of Hawaii ( JIMAR; est. An NRC National Weather Service Moderniza- 1977), the Joint Institute for the Study of the Atmosphere and tion Committee, mandated by Congress in Public Law Ocean at the University of Washington ( JISAO; est. 1977), the Cooperative Institute for Mesoscale Meteorological Studies at 102-567 (U.S. Congress, 1992), provided oversight and the University of Oklahoma (CIMSS; est. 1978), the Cooperative review of various aspects of the MAR from 1990 (NRC, Institute for Meteorological Satellite Studies at the University of 1991) until the MAR was declared completed (NRC, Wisconsin (CIMSS; est. 1980), and the Cooperative Institute for 1999b). Over that decade the committee produced 15 Research in the Atmosphere at Colorado State University (CIRA; est. 1980). reports that provided findings and recommendations

OCR for page 19
42 THE NWS MODERNIZATION AND RESTRUCTURING: A RETROSPECTIVE ASSESSMENT to guide the NWS and NOAA in putting the new Appendix B includes a list of the GAO reports pro- technologies and new organizational structure in place duced during the decade-long period of oversight. The to improve weather services to the nation. Topics of GAO provided oversight of four major MAR pro- the reports included staffing and services as well as grams: ASOS, NEXRAD, GOES-Next, and AWIPS. the technologies being introduced (Appendix B). This The GAO reports often highlighted key design and committee provided external expert review that, while implementation inadequacies of the MAR systems not always welcomed, was generally considered an asset acquisitions and provided recommendations to address to the MAR. The NEXRAD Product Improvement these problems. Also, the GAO addressed schedule Program discussed previously is an example of imple- delays and budgetary issues. mentation of a committee recommendation. At least six GAO reports (GAO, 1989, 1991b, c, 1996a, 1997c, 2000) addressed problematic aspects of the GOES-Next program. The reports highlight that NEXRAD Technical Advisory Committee G OES-Next experienced technological problems, A NEXRAD Technical Advisory Committee significant cost overruns, and schedule delays in the (TAC) was established in 1981 to review technical development of the five GOES satellites. aspects of the tri-agency program, provide recommen- The AWIPS program provides another example dations for needed scientific and technical advances, of the extent of GAO oversight. AWIPS, which in review proposals to accomplish those advances, and many respects was the integrator of the modernization, review the readiness of new developments for full encountered significant costs overruns and schedule implementation. This committee continues to serve delays. The original budget for AWIPS was $350 mil- under a somewhat revised charter; the “modernization” lion and it was expected to be fully deployed by 1995 of the NEXRAD system is an ongoing process that (GAO, 1991a). In 1995, the NWS estimated that the did not end in 2000. The TAC comprises primarily cost to develop AWIPS was $525 million, and that it representatives from the three involved agencies (four would be fully deployed by 1999. The GAO investigated from each); however, it is chaired by an outside engineer and found the estimate to be inaccurate (GAO, 1996b, or scientist and includes two at-large representatives d). Subsequently, the DOC committed to a $550 mil- appointed by the chair to provide external input. The lion budget for AWIPS, however the GAO noted that committee establishes technical needs that, if met, the costs were likely to exceed that amount due to the would enhance the NEXRAD capabilities, and evalu- complex nature of the system (GAO, 1997e). AWIPS ates proposed hardware or software improvements to was initially deployed with less than full functionality the system for readiness for implementation. Though and required additional upgrades to be added to future the NRC National Weather Service Modernization software builds, adding additional costs. The GAO Committee recommended similar technical advisory reported that a systems architecture that described the committees for each of the major MAR systems, none overall blueprint for AWIPS was lacking (GAO, 1994), were formed during (or subsequent to) the MAR. and the NWS embraced the GAO recommendation to develop such a systems architecture (GAO, 1999) near the end of the MAR. U.S. General Accounting Office and A number of other GAO reports also highlighted Department of Commerce Inspector General design and implementation flaws, as well as schedule Oversight by the U.S. General Accounting Office delays and budgetary issues for the ASOS (GAO, (GAO12) and Department of Commerce Inspector 1995h) and NEXRAD (GAO, 1995f, g) programs. General (IG) was thorough and significant, with over Other reports addressed concerns regarding person- 20 reports addressing various aspects of the MAR. nel issues and staffing (GAO, 1995d), or evaluated the MAR progress from a broader perspective (GAO, 1995a) including identifying future risks at the mid- 12 Effective July 7, 2004, the GAO’s legal name was changed way point (GAO, 1995i). from the General Accounting Office to the Government Account- The DOC IG also provided oversight of the MAR. ability Office.

OCR for page 19
43 EXECUTION OF THE MODERNIZATION AND ASSOCIATED RESTRUCTURING For example, early in the program, concerns about that their constituents would have no degradation of NWS management of the MAR were raised by the IG services. Public comments identified 32 areas of con- (DOC, 1990). Other IG reports addressed concerns cern about degradation of service related to NEXRAD about fair competition for the modernization develop- coverage. An NRC assessment of NEXRAD cover- ment program awards (DOC, 1999) and addressed the age determined that degradation of radar coverage AWIPS system (DOC, 1992). These IG reports and did not necessarily equate to degradation of services inquiries added further oversight to the MAR, and (NRC, 1995b). There were only a few instances where provided federal guidance when needed. a Member succeeded in preventing the closing of a forecast office or other NWS facility, but on occasion a planned radar location was altered. Some frustration Congress arose from the different interpretations of degradation To the benefit of the NWS there were a number of of service: whether degradation could be determined members of Congress who supported the opportunities from meteorological criteria, or whether it was related for improvement of weather services presented by the to a change in the number of jobs or money spent in MAR. Having advocates in Congress helped reach the a state or congressional district. It appears that while final decision for implementation and to see the needed such political challenges could have derailed the overall appropriations survive the budget process. Clearly, project, this did not materialize, largely because of good there were champions of the MAR in Congress. These communication between NWS and Congress. members understood the value of new technologies The circumstances that came together in the 1980s that could track weather events in real-time, and allow and brought about major decisions to change the NWS the NWS to issue warnings in time to save lives and were significant. NWS invited oversight by Congress, property. These champions of the MAR were able and Congress welcomed the information about grow- to impress upon their colleagues the importance of ing technological capabilities and saw the benefit of improvements to NWS technologies. improved forecasts and warnings. These positive work- NWS and DOC management and their repre- ing conditions made it possible to obtain the necessary sentatives were able to adequately communicate with appropriations to implement the vision of a modern members of Congress to build the support necessary for NWS and make the difficult decisions about office appropriations of significant funding for the procure- closures and relocation. ment and implementation of new technology, training and retraining of personnel, location and relocation of Finding 3-6 facilities, and to reorganize the NWS. Working with Independent oversight and technical guidance helped members who had the ability and interest to garner draw attention to important issues and impediments support from other members from critical geographic that otherwise may have inhibited the success of areas regardless of party affiliation was important. the Modernization and Associated Restructuring Nevertheless, there were many aspects of the (MAR). This external oversight provided account- MAR that invited Congressional oversight. The clos- ability of the technical, scheduling, and budget met- ing and relocation of many local forecast offices was of rics during the MAR process. particular interest, because members wanted to ensure

OCR for page 19