Executive Summary

The public law that authorized the National Weather Service (NWS) to modernize its technical systems and restructure its field office organization set the requirements for what has become known as the modernization and associated restructuring demonstration (MARD). The law requires testing of the integrated performance, over a sustained period, of the modernized components and an operational demonstration that no degradation in service will result from the modernization and associated restructuring. In this report, the National Research Council's National Weather Service Modernization Committee reviews the September 1998 draft plan for MARD. The committee drew three conclusions from its review of the plan and offers five recommendations to the NWS.

Conclusion 1.

If MARD achieves the goals and objectives stated in the MARD Plan, it will meet the requirement in the public law for a multistation operational demonstration that tests the performance of the modernization in an integrated manner for a sustained period.

Conclusion 2.

The MARD evaluation plan is intended to assess a wide range of local office capabilities, operations, and services. Before this plan is implemented, it will require refinement to link proposed evaluation variables to the evaluation and analysis objectives, provide directions for documenting actual site conditions during MARD, and minimize the burden on staff and operations.

Conclusion 3.

Controlled tests of multistation capabilities, such as interoffice backup, that progressively stress the system up to and including total failure of a field office are needed to ensure system robustness. Tests of this kind for routine coordination and for interoffice backup during failure, covering both system capabilities and operational procedures, can be incorporated into the ongoing process of operational program evaluation, beginning with the Build 4.2 operational test and evaluation (OT&E) for the Advanced Weather Interactive Processing System (AWIPS).

Recommendation 1.

The National Weather Service should test failure modes at progressive levels of failure, including the complete failure of a field office (for both weather forecast offices and river forecast centers). These tests should be conducted during the Build 4.2 operational test and evaluation and demonstrated during MARD. The tests should focus on the effectiveness of capabilities and procedures for inter-office backup and intra-office recovery and document site configurations.

Recommendation 2.

The National Weather Service should continue to make periodic assessments of technical, operational, and service capabilities as part of the ongoing operational program evaluation by expanding the operational test and evaluation process being used for AWIPS builds to include the integrated suite of technical systems. The assessments should include service programs such as coastal marine, fire, and mountain weather forecasts and warnings, which are not likely to be fully tested in the MARD area.

Recommendation 3.

To ensure adequate evaluation of a stable system configuration, MARD should be continued for a reasonable period of time (several months at least) after AWIPS is commissioned at the MARD sites. During this period, the MARD sites should use the modernized systems without significant reliance on legacy systems.

Recommendation 4.

The National Weather Service should use outside experts in program evaluation and survey design:

  • to review the MARD evaluation plan, survey questionnaires, and other data forms
  • to assist in developing a plan for analyzing the evaluation


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Executive Summary The public law that authorized the National Weather Service (NWS) to modernize its technical systems and restructure its field office organization set the requirements for what has become known as the modernization and associated restructuring demonstration (MARD). The law requires testing of the integrated performance, over a sustained period, of the modernized components and an operational demonstration that no degradation in service will result from the modernization and associated restructuring. In this report, the National Research Council's National Weather Service Modernization Committee reviews the September 1998 draft plan for MARD. The committee drew three conclusions from its review of the plan and offers five recommendations to the NWS. Conclusion 1. If MARD achieves the goals and objectives stated in the MARD Plan, it will meet the requirement in the public law for a multistation operational demonstration that tests the performance of the modernization in an integrated manner for a sustained period. Conclusion 2. The MARD evaluation plan is intended to assess a wide range of local office capabilities, operations, and services. Before this plan is implemented, it will require refinement to link proposed evaluation variables to the evaluation and analysis objectives, provide directions for documenting actual site conditions during MARD, and minimize the burden on staff and operations. Conclusion 3. Controlled tests of multistation capabilities, such as interoffice backup, that progressively stress the system up to and including total failure of a field office are needed to ensure system robustness. Tests of this kind for routine coordination and for interoffice backup during failure, covering both system capabilities and operational procedures, can be incorporated into the ongoing process of operational program evaluation, beginning with the Build 4.2 operational test and evaluation (OT&E) for the Advanced Weather Interactive Processing System (AWIPS). Recommendation 1. The National Weather Service should test failure modes at progressive levels of failure, including the complete failure of a field office (for both weather forecast offices and river forecast centers). These tests should be conducted during the Build 4.2 operational test and evaluation and demonstrated during MARD. The tests should focus on the effectiveness of capabilities and procedures for inter-office backup and intra-office recovery and document site configurations. Recommendation 2. The National Weather Service should continue to make periodic assessments of technical, operational, and service capabilities as part of the ongoing operational program evaluation by expanding the operational test and evaluation process being used for AWIPS builds to include the integrated suite of technical systems. The assessments should include service programs such as coastal marine, fire, and mountain weather forecasts and warnings, which are not likely to be fully tested in the MARD area. Recommendation 3. To ensure adequate evaluation of a stable system configuration, MARD should be continued for a reasonable period of time (several months at least) after AWIPS is commissioned at the MARD sites. During this period, the MARD sites should use the modernized systems without significant reliance on legacy systems. Recommendation 4. The National Weather Service should use outside experts in program evaluation and survey design: to review the MARD evaluation plan, survey questionnaires, and other data forms to assist in developing a plan for analyzing the evaluation

OCR for page 1
to assist in the collection of evaluation data in local offices Surveys used in the service assessments of future operational program evaluations should also be reviewed by outside experts. Recommendation 5. The National Weather Service should use an independent evaluator or evaluation board to assist in assessing MARD results.