3
Conclusions and Recommendations

Conclusions

Conclusion 1.

If MARD achieves the goals and objectives stated in the MARD Plan, it will meet the requirement in the public law for a multistation operational demonstration that tests the performance of the modernization in an integrated manner for a sustained period.

Conclusion 2.

The MARD evaluation plan is intended to assess a wide range of local office capabilities, operations, and services. Before this plan is implemented, it will require refinement to link proposed evaluation variables to the evaluation and analysis objectives, provide directions for documenting actual site conditions during MARD, and minimize the burden on staff and operations.

Conclusion 3.

Controlled tests of multistation capabilities, such as interoffice backup, that progressively stress the system up to and including total failure of a field office are needed to ensure system robustness. Tests of this kind for routine coordination and for interoffice backup during failure, covering both system capabilities and operational procedures, can be incorporated into the ongoing process of operational program evaluation, beginning with the Build 4.2 operational test and evaluation (OT&E) for the Advanced Weather Interactive Processing System (AWIPS).

Recommendations

Recommendation 1.

The National Weather Service should test failure modes at progressive levels of failure, including the complete failure of a field office (for both weather forecast offices and river forecast centers). These tests should be conducted during the Build 4.2 operational test and evaluation and demonstrated during MARD. The tests should focus on the effectiveness of capabilities and procedures for interoffice backup and intra-office recovery and document site configurations.

Recommendation 2.

The National Weather Service should continue to make periodic assessments of technical, operational, and service capabilities as part of the ongoing operational program evaluation by expanding the operational test and evaluation process being used for AWIPS builds to include the integrated suite of technical systems. The assessments should include service programs such as coastal marine, fire, and mountain weather forecasts and warnings, which are not likely to be fully tested in the MARD area.

Recommendation 3.

To ensure adequate evaluation of a stable system configuration, MARD should be continued for a reasonable period of time (several months at least) after AWIPS is commissioned at the MARD sites. During this period, the MARD sites should use the modernized systems without significant reliance on legacy systems.

Recommendation 4.

The National Weather Service should use outside experts in program evaluation and survey design:

  • to review the MARD evaluation plan, survey questionnaires, and other data forms
  • to assist in developing a plan for analyzing the evaluation
  • to assist in the collection of evaluation data in local offices

Surveys used in the service assessments of future operational program evaluations should also be reviewed by outside experts.

Recommendation 5.

The National Weather Service should use an independent evaluator or evaluation board to assist in assessing MARD results.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 12
3 Conclusions and Recommendations Conclusions Conclusion 1. If MARD achieves the goals and objectives stated in the MARD Plan, it will meet the requirement in the public law for a multistation operational demonstration that tests the performance of the modernization in an integrated manner for a sustained period. Conclusion 2. The MARD evaluation plan is intended to assess a wide range of local office capabilities, operations, and services. Before this plan is implemented, it will require refinement to link proposed evaluation variables to the evaluation and analysis objectives, provide directions for documenting actual site conditions during MARD, and minimize the burden on staff and operations. Conclusion 3. Controlled tests of multistation capabilities, such as interoffice backup, that progressively stress the system up to and including total failure of a field office are needed to ensure system robustness. Tests of this kind for routine coordination and for interoffice backup during failure, covering both system capabilities and operational procedures, can be incorporated into the ongoing process of operational program evaluation, beginning with the Build 4.2 operational test and evaluation (OT&E) for the Advanced Weather Interactive Processing System (AWIPS). Recommendations Recommendation 1. The National Weather Service should test failure modes at progressive levels of failure, including the complete failure of a field office (for both weather forecast offices and river forecast centers). These tests should be conducted during the Build 4.2 operational test and evaluation and demonstrated during MARD. The tests should focus on the effectiveness of capabilities and procedures for interoffice backup and intra-office recovery and document site configurations. Recommendation 2. The National Weather Service should continue to make periodic assessments of technical, operational, and service capabilities as part of the ongoing operational program evaluation by expanding the operational test and evaluation process being used for AWIPS builds to include the integrated suite of technical systems. The assessments should include service programs such as coastal marine, fire, and mountain weather forecasts and warnings, which are not likely to be fully tested in the MARD area. Recommendation 3. To ensure adequate evaluation of a stable system configuration, MARD should be continued for a reasonable period of time (several months at least) after AWIPS is commissioned at the MARD sites. During this period, the MARD sites should use the modernized systems without significant reliance on legacy systems. Recommendation 4. The National Weather Service should use outside experts in program evaluation and survey design: to review the MARD evaluation plan, survey questionnaires, and other data forms to assist in developing a plan for analyzing the evaluation to assist in the collection of evaluation data in local offices Surveys used in the service assessments of future operational program evaluations should also be reviewed by outside experts. Recommendation 5. The National Weather Service should use an independent evaluator or evaluation board to assist in assessing MARD results.