Although the assumptions made about exercises are reasonable, and exercises seem like a practical strategy in many circumstances, there are at least two reasons to seek more objective study: the need to compare multiple types of exercises for more targeted use and the potential costs posed by exercises.
As noted above, more research is needed on preparedness (Tierney et al., 2001), but exercises and other means for evaluating performance (and for improving preparedness) form a particularly neglected subset of preparedness. The overall effectiveness of exercises as a preparedness strategy has not been well demonstrated, and research is needed to determine, for example, whether exercises could be considered predictors of successful response, what type of exercise would have the greatest positive influence on preparedness, what exercises are most cost-effective, and the best way to assess opportunity costs posed by conducting exercises (NRC, 2003). The use of scenarios, which may serve as a component of exercises, for training and planning purposes has not been well researched either, but there are “indications that they are an excellent method of teaching rapid response-style thinking, decision-making and the development of managerial skills” (Alexander, 2000; Simpson, 2002:56). Potential research questions would include: how do reality-based scenarios compare with entirely fictional scenarios, and under what circumstances would the use of one be preferable to the other?
In the disaster literature, mention of exercises seems limited to descriptions of how they were utilized by responder agencies and disciplines (EMS, emergency departments, fire departments, etc.), the lessons learned, and changes in operations made as a result (Tierney et al., 2001). A brief review of the medical and health peer-reviewed literature (using the National Library of Medicine’s PubMed search engine) shows that hospitals and public health agencies conduct exercises and find them useful in evaluating the quality of training, the smoothness of emergency operations, and other aspects of disaster response, but there seems to be little or no empirical study of the validity or effectiveness of exercises themselves as a strategy for public health and health care preparedness. The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center recently conducted a review of the literature on hospital exercises and drills and concluded that “the evidence was insufficient to support firm conclusions about the effectiveness of specific training methods because of the marked heterogeneity of studies, weaknesses in study design, and the limited number of exercises that have been reported in the literature” (AHRQ, 2004). The committee hopes that the experience of public health agencies with exercises and drills will not only be reported to CDC, but that there will be increased emphasis on more in-depth studies of the effectiveness of public