A Framework for Performance Evaluation Using Exercises

Major outcomes in public health typically involve decreasing mortality and disease rates and progress is measured periodically (e.g., Healthy People 2010 process). Performance measurement in public health is, however, a relatively new field. In the case of public health preparedness for bioterrorism and other events with significant public health impact, outcomes are occasioned by actual events themselves, and the infrequency and huge variation among these events (including the proxy events discussed in preceding pages of this report) make it difficult or nearly impossible to gauge, for example, a decrease in rate of disease from contaminated water or other reductions in mortality and morbidity attributable to the disaster. Due to the nature of disaster-related public health problems, performance measurement in this area is by necessity more process-oriented. When CDC and its state and local partners identify exercise objectives that will be used in evaluating the exercise, these objectives will be most helpful if they are linked with the Evidence-Based Performance Goals for Public Health Disaster Preparedness developed by CDC.

Exercises offer an alternative way to measure performance and finetune preparedness before a crisis occurs. Public health preparedness exercises take place at national, state, and local levels, and it is important that evaluation of exercises take place at all levels. The committee believes it is essential to design and conduct exercises that stress and test CDC’s own performance. As noted in the preceding discussion of proxy events, CDC is a vital part of preparedness and response and it is itself a limiting factor in terms of the resources it provides (e.g., laboratory reagents, information, technical assistance) to state and local counterparts. In asking “what if” questions in a proxy event or in an exercise, the limits of availability of such resources must be probed. In addition, modeling could be used to estimate such things as the rate of producing and renewing the supply of needed laboratory reagents, or the speed with which needed field experts could be moved from place to place. In a more dramatic type of exercise, questions could be asked about the potential effect if CDC itself was the target of an attack and critical facilities destroyed.

After-action reports will play an important role in facilitating continuous quality improvement. They provide an overview of agency or interagency performance in an exercise and identify areas where there are gaps in planning, unforeseen circumstances that are poorly managed, or areas where communication or the flow of information break down, among other issues.

Various types of methods for measuring performance will eventually be determined to be effective and even to have some predictive value (e.g., of future successful response). The link between research and practice requires

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement