Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation, Letter Report #6
July 6, 2004
Dr. Julie Gerberding
Centers for Disease Control and Prevention (CDC)
1600 Clifton Road, NE Atlanta, GA 30333
Dear Dr. Gerberding:
The Institute of Medicine (IOM) Committee on Smallpox Vaccination Program Implementation is pleased to offer you the sixth in a series of brief reports.
This report may seem like a departure from the committee’s previous work, which focused on smallpox vaccination as a part of public health preparedness. However, this report responds to a CDC request for guidance as the agency moves toward comprehensive preparedness for bioterrorism and other public health disasters and toward broad smallpox preparedness efforts. The committee was asked to look specifically at preparedness exercises, which are required by CDC grant guidance and are being conducted by public health agencies. In general, the public health community has somewhat limited experience with exercises, so the committee was asked to describe the state of the science in evaluation of exercises, to identify leadership and experience to build on, and to identify
issues or concerns about the use of exercises as a means to performance measurement.
At its fifth meeting, on March 29, 2004, the committee heard presentations about: CDC’s recent efforts in public health preparedness; the modeling workgroup of the Department of Health and Human Services (DHHS) Secretary’s Council on Public Health Preparedness; the theory and science related to preparedness and exercises1 from both a sociological and a disaster management and response perspective; the perspective of a Center for Public Health Preparedness; and the Department of Homeland Security’s experience with planning, conducting, and evaluating exercises. This letter report contains the committee’s findings and recommendations based on information from that meeting and additional though limited (given time constraints) review of what public health may learn from disaster research and from the practice of disaster response.
Charge to the Committee
One way to measure public health agencies’ performance in achieving preparedness is by performing and evaluating exercises.2 Whereas exercises have been conducted and evaluated in the emergency management field for many years, public health has had less experience with exercises and is currently beginning to assess their value for relationship building, training, and performance measurement. To place the role of exercises appropriately into the broader definition of what it means to be prepared and to identify specific aspects for which measures can be developed, CDC asked the Committee on Smallpox Vaccination Program Implementation to:
Describe the state of the science in exercises and related preparedness strategies;
Identify leadership and experience to build upon, from other fields and other federal agencies; and
Identify issues or concerns about this approach to performance measurement (Sosin, 2004).
To meet the charge presented by CDC, the committee has endeavored to: (1) examine conceptual issues and challenges related to integrating public health into disaster preparedness and response; (2) review some of the evidence base from disaster research and practice that is germane to public health preparedness; (3) learn from the public health response to proxy events; (4) discuss the usefulness of modeling; and (5) discuss the usefulness of exercises, including a description of some of the exercise activities occurring in the federal government.
Summary of Recommendations
The report’s recommendations revolve around the issues of interagency and intersectoral coordination, learning from experience and research, and continuously improving performance.
The committee recommends that all federal entities concerned with bioterrorism preparedness (e.g., CDC, the Health Resources and Services Administration [HRSA], the Office of Domestic Preparedness [ODP]) should more actively coordinate guidance and funding activities. Federal agencies should also work together to develop mechanisms that facilitate coordination and collaboration among their grantees at the state and local levels. Such mechanisms may include, but are not limited to, regular meetings to familiarize CDC and ODP program staff with each other’s program priorities and activities, a database for informing ODP and other partners of exercises planned by CDC grantees, etc. Federal coordination efforts should also include the clarification of primary responsibility and authority in bioterrorism events, to ensure that CDC can fulfill its unique role as the nation’s public health agency.
The committee recommends that CDC should collaborate with all of its partners to strengthen preparedness by applying research findings and experience in public health emergency response, bioterrorism preparedness, and disaster management. In order to strengthen the evidence base for public health preparedness, CDC should:
Strengthen the link between public health research and practice;
Participate in and promote interdisciplinary research about preparedness;
Support a system to assure the ongoing collection, synthesis, and sharing of lessons learned and best practices from public health preparedness exercises and public health response to proxy events; and
In coordination with the appropriate federal-level partners, such as the Agency for Healthcare Research and Quality, evaluate the effectiveness, design, and opportunity costs of preparedness strategies, such as exercises.
The committee recommends that CDC should use the Evidence-Based Performance Goals for Public Health Disaster Preparedness to develop standards against which CDC, states, and localities may regularly measure their performance in exercises and in response to proxy events. Public health agency performance in exercises and proxy events should be used to identify gaps in preparedness and to improve planning, communication, and coordination at the agency and interagency levels, as part of a process of continuous quality improvement in preparedness planning and response. Preparedness drills and exercises should not be evaluated individually, but their cumulative and long-term impact on preparedness, such as generalizability to other potential hazards, must be considered in the evaluation process
INTEGRATING PUBLIC HEALTH INTO DISASTER PREPAREDNESS AND RESPONSE: CONCEPTUAL ISSUES
The public health community has become an active partner in the world of emergency and disaster preparedness and response, joining other members in the traditional emergency management and response field who have defined roles and established ways of doing work (Landesman et al., 2001). Although public health workers and agencies have played active roles after many emergency events (and in some states, the emergency medical services [EMS] entity is part of the state public health agency), public health workers have not necessarily counted themselves or been counted among emergency responders (Landesman et al., 2001; Kahsai and Kare, 2002).
Some important conceptual issues must be considered in the process of more effectively integrating public health into the disaster preparedness and response field. These issues include (1) the history of public health disaster response, and its relevance to contemporary public health preparedness; (2) the unique role of public health in disasters and primary role in disasters that involve biological agents; and (3) the heterogeneity that characterizes the field of emergency and disaster preparedness and response.
A History of Public Health Disaster Response
History provides myriad examples of public health emergencies and disasters (e.g., cholera outbreaks, toxic spills), that wreaked destruction
akin to or greater than that of major natural disasters and to which the evolving discipline of public health responded. Epidemiologic and other public health skills and knowledge also have been advanced through lessons learned in such responses (Landesman et al., 2001).
The threat of bioterrorism has mobilized the engagement of many disciplines and government agencies both to prevent and to respond. The reemergence of infectious diseases in part related to demographic change and globalization has elevated interest in public health’s role as both a responder to and a preventer of epidemics and infectious disease outbreaks. Public health agencies have the ongoing responsibility to prevent disease outbreaks and other emergencies through measures such as immunization, sanitation, and community education. In cases where preventive measures are not successful, or there are barriers to their implementation, or an unexpected threat causes disease, public health becomes a responder, conducting surveillance, controlling the spread of disease, conducting mass immunization, etc. At the same time, public health agencies continue prevention to limit secondary public health problems. The current integration of public health preparedness efforts with those of more traditional “responder” disciplines is based on a growing acknowledgement of public health’s singular capabilities and importance in preparing for and responding to bioterrorism, as well as the health aspects of a range of disasters. These include deliberate attacks with nonbiological weapons, natural disasters that may result in the contamination of food or water supplies and lead to infectious diseases, and technological disasters that may endanger population health with radiation or chemical hazards.
Unique Role of Public Health in Disasters, and Primary Role in Response to Bioterrorism
Public health generally does not have a formal tradition of disaster preparedness and response. However, notable and instructive exceptions are found in the experience of the following types of public health agencies, some of which have developed varying levels of expertise in planning and exercising for disasters and in managing disasters (e.g., the experience of the state of Georgia described by Werner et al., 1998):
Public health agencies located in the vicinity of nuclear facilities and involved in federally mandated training and exercise programs;
Public health agencies located in areas with frequent natural disasters (hurricanes, floods, or tornadoes);
Public health agencies at sites of one-time or recurring major events or entertainment venues (e.g., auto racing, Olympics, amusement parks); and
State public health agencies in states where emergency medical services (EMS) are integrated into the public health agency.
The role of public health in disaster preparedness and response is unique and is not performed by any of the other disciplines that typically respond to disasters and that differ from public health in mission, services provided, and personnel training (e.g., EMS, clinical medicine). Therefore, the role of public health as a responder needs to be formalized and become an indispensable and recognizable part of comprehensive response to disasters. One common thread characterizes the work of public health agencies in relation to most types of disasters: they possess the knowledge and skills required to safeguard the health of the public by limiting morbidity and mortality, whether an event poses a threat to health from the outset (i.e., bioterrorism) or creates secondary threats to health, as in the case of natural disasters. The public health community’s role before, during, and after the occurrence of disasters is to some extent anchored in its capacity to conduct routine, noncrisis activities, and is consistent with public health’s assessment, policy development, and assurance functions, but varies with community resources and interagency agreements and service provision roles (Salinsky, 2002). Carrying out these functions requires public health agencies to collect, evaluate, and disseminate information; cooperate and collaborate with other disciplines (including, but not limited to, the health care sector); and to prevent disease and ensure the continuity of health care (Landesman et al., 2001; IOM, 2003d).
In addition to the public health effects of most other types of disasters, attacks with biologic agents, as exemplified by the anthrax attacks of 2001, require that governmental public health agencies serve as primary responders. Events that involve biologic agents are different from other types of disasters because their emergence is likely to go unnoticed for some time; biologic agents are microscopic and may be more likely to be introduced silently (e.g., through airborne droplets), rather than with explosions, and become evident over time. Also, the fallout from attacks with biologic agents may not remain confined to a specific physical space; in other words, there may not be a “scene” or a “ground zero” (Perry, 2003) and its impact may not be contained but may ripple outward for some time due to contagion. Preparedness for biologic agents therefore involves at least some different requirements from other types of agents and requires the unique knowledge and skills of trained public health personnel (e.g., case identification and containment) and the unique capabilities (e.g., laboratories, surveillance, communication, community education) and statutory responsibilities (e.g., quarantine) of public health agencies, as well as the complementary facilities, skills, and resources of the health care community (Perry, 2003).
In order to integrate the preparedness and response efforts of public health most appropriately with those of the traditional emergency management and response field, some key differences need to be identified. For example, disaster preparedness and response is the central mission of local, state, and national civilian and military response organizations and they train and exercise regularly to test and maintain their response capabilities. They have the dual role of responding to disasters and to routine emergencies in their communities. For public health agencies, responding to major crises has been the exception from their usual work; therefore, conducting regular drills and training to prepare for disaster response has generally not been a common practice. Also, even when public health agencies have gained experience dealing with disease outbreaks, these events do not typically reach the scale of a disaster, and response is largely limited to the public health and health care communities.
Given the statutory responsibilities and special capabilities of public health, and CDC’s leadership role in the provision of essential public health services under all circumstances, it is clear that CDC and the public health community must be ready to fulfill their primary role in responding to bioterrorism and their support roles in other types of disasters, including terrorism with chemical, nuclear, and other types of weapons (see Figure G-1).
The Diverse Field of Emergency and Disaster Preparedness and Response
Public health is not entering a monolithic or homogeneous field of emergency and disaster management. Disasters involve people, physical
structures, and the broader environment, and they may be caused by a wide range of natural, technological, and deliberately introduced agents. This variety of factors explains the complex array of disciplines and organizations involved in the emergency and disaster response field. The category of first responders has typically included personnel from the fire-fighting, EMS, and law enforcement fields, along with state emergency management agencies and federal agencies (e.g., Federal Emergency Management Agency, Environmental Protection Agency), and nongovernmental organizations, such as the Red Cross and the Salvation Army. Other disciplines involved in preparedness include structural engineers, civic planners, public administrators, etc. Clearly, the set of contributors to emergency and disaster preparedness and response is vast and includes a patchwork of methods, cultures, and disciplines that are in some cases themselves struggling to integrate their activities (Kahsai and Kare, 2002; Tang and Fabbri, 2003). In addition to being multidisciplinary, the field of emergency and disaster preparedness and response is undergoing change toward increased professionalization and an all-hazards3 approach, and is evaluating its assumptions and modes of practice, as discussed elsewhere in this report (Alexander, 2003; NRC, 2003).
CHALLENGES AND OPPORTUNITIES INHERENT IN INTEGRATING PUBLIC HEALTH INTO A BROADER FIELD
The integration of a relative newcomer into the large and complex field of emergency and disaster preparedness and response presents challenges and tensions. Disasters require rapid decisions and quick action, which may bring about cross-jurisdictional conflicts, professional differences, and questions about authority, expertise, and the appropriate chain of command.
In its fifth report (IOM, 2003a), the committee discussed at some length the importance of close collaboration between the public health and health care communities, from the level of federal agencies such as HRSA and CDC, to local public health agencies and their health care counterparts (health care organizations, hospitals, private providers, long-term care facilities, etc.). Previous reports by this committee also have called for public health and health care organizations and workers to coordinate and col-
laborate with agencies, disciplines, and entities with which they were previously not well acquainted, including, but not limited to, fire authorities, law enforcement, EMS, voluntary organizations, and communities.
Research and practical experience show that coordination among all agencies involved is one of the fundamental requirements of effective disaster response and that the lack of adequate coordination is one of the major problems encountered in the field (Auf der Heide, 1989; Tierney et al., 2001). Given the large number of federal, state, and local agencies involved in preparedness efforts, establishing adequate coordination across federal, state, and local levels is proving to be a challenge (Advisory Panel to Assess Domestic Response Capabilities for Terrorism Involving Weapons of Mass Destruction, 2003; GAO, 2003a; Clements and Evans, 2004). Within the federal government, preparedness and response activities are coordinated through the Department of Homeland Security (DHS). Coordination at the top levels of the federal government occurs through the Homeland Security Council (HSC), which is charged with ensuring coordination of all homeland security related activities among executive departments and agencies and promoting the effective development and implementation of all homeland security policies (White House, 2001). Day-to-day coordination of homeland security issues—both within the federal government and among federal, state, and local government agencies—is meant to occur through the Policy Coordination Committees (PCCs) of the HSC (White House, 2001). There are eleven PCCs for different functional areas, including a Medical and Public Health Preparedness PCC. The committee was unable to obtain sufficient information to determine whether and how Medical and Public Health Preparedness PCC actions or policy decisions shape CDC’s preparedness program and whether the PCC plays a role in strengthening CDC’s relationship with DHS.
Despite the existence of mechanisms for coordination at the top departmental level, such as the PCCs, it is not evident to the committee that adequate coordination and information sharing are occurring formally at the level of federal program staff involved in the day-to-day work of public health preparedness (GAO, 2003b). Although the creation of DHS holds the promise of streamlined oversight and funding, there are concerns that coordination between DHS and key preparedness functions in DHHS remains a significant challenge (GAO, 2003a). At the committee’s March 2004 meeting, conversation among presenters from federal agencies and the committee revealed that personal relationships and serendipity may be credited with some coordination and information sharing across agencies, but it was not immediately evident that there are sufficient and functioning formal mechanisms for coordination and collaboration between DHS and DHHS. Coordination must be planned with forethought and deliberation, not left simply to chance and the goodwill of program staff. Coordination
also must be planned and implemented during the preparedness or pre-event phase, beginning with effective communication about funding objectives and activities. For example, it is important for CDC staff to be familiar with relevant activities occurring in DHS and its programs funded and/or administered through FEMA and ODP, and for DHS staff to be aware of CDC priorities and activities to ensure the best use of limited federal preparedness resources.
State and local public health agencies receive funding through CDC’s Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism, and health care entities are funded through HRSA’s National Hospital Bioterrorism Preparedness Cooperative Agreements. These cooperative agreement programs require that grantees conduct exercises that test public health and health care preparedness (and the integration between them) for an attack with biologic or chemical agents. Through the DHS Office of Domestic Preparedness Fiscal Year (FY) 2004 Homeland Security Grant Program and FY 2004 Urban Area Security Initiative Grant Program, states and some local emergency management offices receive funding to conduct exercises that test many of the same capacities and interagency collaborations expected by HRSA and CDC (DHS, 2003). Furthermore, FEMA, which is now under DHS, although its activities seem not yet fully coordinated with those of ODP, also oversees exercises relevant to chemical and radiation emergencies, which include public health components. The committee learned that sometimes states pool different sets of resources to conduct a larger drill or exercise involving a larger number of state and local agencies and community partners, and in other cases, the different funding streams are used to fund separate exercises (Schweitzer, 2004).
ODP has released guidelines for exercises and their evaluation through the Homeland Security Exercise and Evaluation Program (HSEEP). Although the committee is not aware of the nature and extent of CDC’s involvement in the development of the HSEEP guidelines, the committee believes it is important that both CDC and DHSODP work to ensure a reasonable level of compatibility and coordination. This is necessary because of the functional overlap between public health and other state agencies, and because some state public health agencies already plan and execute their bioterrorism preparedness exercises in conjunction with their state emergency management offices. While public health preparedness exercises are needed to assess the unique functions and goals of public health, they will ideally be coordinated with other types of exercises where appropriate. Since state emergency management offices will be following the HSEEP guidelines, and some state public health agencies may be participating in exercises that follow these guidelines, a certain level of coordination is necessary between CDC’s public health preparedness exercise guidelines
and the HSEEP guidelines. In order to maximize the knowledge, skills, and relationship building that states and local jurisdictions gain from participating in preparedness exercises supported by limited federal resources, the committee encourages CDC to work closely with ODP (as well as HRSA) to coordinate, where appropriate and consistent with agency goals, the funding and guidelines for exercises provided by all federal agencies to states, local jurisdictions, and to private sector entities, such as hospitals.
Responding to a public health disaster, such as a smallpox attack, will require coordination with other organizations in the private sector and within the health care community. At the March 2004 meeting, the committee heard about the initiatives of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) to engage communities in preparedness planning and exercises. The committee believes it is important that CDC identify other organizations that, like JCAHO, require and set standards for preparedness activities, including exercises, and interact with communities in the area of bioterrorism and disaster preparedness. This is needed to help avoid duplicative efforts as well as ensure the best coordination of preparedness efforts. The range of partners in preparedness should be conceived broadly, to include local community, health care institutions, voluntary organizations, and others.
The committee also heard that state grantees funded by the DHS ODP FY 2004 Homeland Security Grant Program are encouraged to share exercise calendars with other partners and to coordinate or integrate efforts with other state and local exercises (Schweitzer, 2004). The committee suggests that CDC develop and maintain a list or database of exercises funded under the current (and future) cycle of the Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism and to share this resource with ODP. Also, regular communication between CDC and ODP would inform both about planned exercises and would provide opportunities for coordination of exercises within a state and between states.
CDC and DHS guidance to grantees makes some reference to the need for interdisciplinary and intersectoral coordination (CDC and ODP, 2003; CDC, 2003a, 2004a, 2004b; DHS, 2003). However, it is not clearly spelled out how these linkages function at the federal, state, and local levels, and it is unclear whether the need for coordination is more specifically confirmed with and reinforced with grantees. For example, the CDC guidance for FY 2004 calls for integrating efforts and closely coordinating with “activities funded by the Department of Homeland Security and/or other federal agencies” (CDC, 2004a, 2004b). The guidance does not specifically identify relevant programs funded by DHS, and the committee was unable to locate more detailed explication of the formal linkages and coordination mechanisms that exist or are desirable in the relationship between CDC and DHS grantees.
The committee was unable to find a comprehensive resource describing all of the funding streams available for emergency preparedness activities, their purpose, funding amounts, and intended recipients. Such a tool would aid coordination of funding at the state and local level and also would facilitate coordination of all-hazards preparedness activities among national, state, and local partners in the academic, nonprofit, and business sectors. The committee did find a useful matrix of federal all-hazards grants from the International Association of Emergency Managers (IAEM, 2003). If a similar federal resource exists, that has been verified for accuracy and timeliness by the relevant federal agencies, the committee encourages that such a document be shared widely to facilitate coordination among all participants in emergency preparedness.
The committee urges all federal agencies to plan for and implement adequate collaboration and communication to ensure the long-term sustainability and effectiveness of an interagency approach to funding, developing, implementing, and evaluating public health preparedness in general, and exercises in particular.
An Example of Intersectoral Tension and Collaboration
The relationship between public health and law enforcement in responding to bioterrorism illustrates some of the potential tensions inherent in the coming together of different cultures and approaches to address a crisis.
An attack with biologic agents would put into motion two major and divergent systems (in addition to many others): public health, which attempts to deal with consequences and spread of infectious disease, and law enforcement, which targets the commission of a crime implicit in a deliberate introduction of a biologic agent. In the anthrax attacks of 2001, differences between public health and law enforcement became apparent. These included different investigative approaches (inductive versus deductive, respectively), evidentiary standards (scientific versus legal), and communication objectives. Public health tried to share complete and accurate information with the public in a timely manner, while law enforcement sought to disclose little or nothing pertaining to an investigation in order to maintain the integrity of a potential legal case (Ornstein, 2001; Butler et al., 2002; Gerber, 2002). Given these very different objectives and approaches, bioterrorism events would challenge each set of responders to do its own work while allowing the other to carry out its responsibilities. Preparedness efforts must include discussion and clarification of roles and responsibilities in a way that meets the needs of both public health and law enforcement professionals and undermines neither the disease prevention
goal of public health nor the evidentiary standard required by law enforcement (Butler et al., 2002; Richards, 2002).
In the wake of the anthrax attacks of 2001, the position of CDC liaison to the FBI was created (Butler et al., 2002; GAO, 2003c). This seems to be a step in the right direction, but it would be useful only as long as the liaison unit is considered a priority by both agencies and it is given an adequate scope of work and level of authority. The Forensic Epidemiology training program, a joint effort between CDC and Department of Justice to facilitate mutual understanding between law enforcement and public health, is an example of successful and productive collaboration between public health and law enforcement in the area of bioterrorism preparedness and response (CDC, 2003b).
Collaboration between seemingly disparate government agencies and disciplines is not a new need, and there is some history on which to draw to help clarify and streamline these relationships. In the early 1980s, the CDC and the Federal Bureau of Investigation (FBI) created an interagency bioterrorism unit, located at CDC with secure communication capacity in the wake of a botulinum hoax. Plans were developed for the defense of the civilian population in the event of a bioterrorism incident. This unit was later disbanded (personal communication, W. Foege, Bill and Melinda Gates Foundation, March 30, 2004). This is an example of the type of collaboration that must be initiated and sustained to help address deliberate threats with health consequences.
Common Definitions and Terminology Are Needed
The emergency and disaster management field and federal agencies associated with it have developed a great deal of experience planning for disaster response and designing and conducting exercises to promote relationship building and training (GAO, 2001; Kuhr and Hauer, 2001; Landesman, 2001; FEMA, 2003). As disaster response becomes increasingly interdisciplinary, a common language is needed for good communication and interagency coordination in preparing for and responding to a chemical, biologic, radiological, nuclear, or explosive incident.
The committee found that similar terms do not always have the same meaning in documents created by different federal agencies (e.g., CDC, HRSA, FEMA, and ODP) or in the way they are used by the many disciplines conducting disaster research (CBACI, 2002; Hilhorst, 2003). The terms “exercise,” “drill,” and “simulation” in particular can mean different things to different agencies and disciplines.
Language differences go beyond practical terms used to describe specific activities. Definitions for fundamental concepts such as preparedness and response also are not unambiguous and certainly not universally shared
across the disciplines that employ them (Advisory Panel to Assess Domestic Response Capabilities for Terrorism Involving Weapons of Mass Destruction, 2003). Effective communication and coordination are only possible when concepts and terms are used and understood in the same way by all participants. As is the case with any cross-cultural encounter, however, language is only one potential barrier. A more comprehensive kind of harmonization will require a great deal of effort on the part of each federal, state, and local agency, and of all disciplines involved in preparedness to understand each other’s perspective, assumptions, biases, culture, and goals. Meetings between high-level officials will not suffice to bring this about. Regular, institutionalized, and sustained interaction among program staff will be needed, and all preparedness planning would benefit from applying the values and strategies of cultural competence at the interface between the many disciplines and agencies involved.
Speaking the Same Language: the Lexicon Project
The DHS has already recognized the need for a baseline understanding of the terms, acronyms, and phrases regularly used by different federal agencies that are involved in preparedness activities. For example, there are often very different understandings of the terms “first responder” and “surveillance.” The Homeland Security Advisory Council has created a report for the secretary of Homeland Security on the Lexicon Project—a project that would create a homeland security lexicon by identifying the terms, acronyms, and phrases (and their associated definitions) used most commonly by agencies involved in homeland security activities (Moscoso, 2004). The goal is to develop a baseline level of understanding of all the terms and acronyms that are commonly used by different agencies so that communication can be improved (DHS, 2004c). The council has recommended the creation of an electronic database that would be accessible to all federal agencies, Capitol Hill staffers, lawmakers, and state and local agencies as they draft legislation, submit grant proposals, or prepare emergency plans (Moscoso, 2004). The council also has recognized the value in making such a database accessible to the media and other partners so that standard terminology also would be conveyed to the public at large (DHS, 2004c).
Part of the Lexicon Project at DHS involves assembling “foundational documents” from federal agencies that include the terms commonly used when discussing homeland security activities. To the extent that it is not involved already, the committee encourages CDC to work with the DHS to ensure that the commonly used public health preparedness terms and the relevant CDC documents are incorporated into the Lexicon Project and that knowledge of this effort is shared broadly across CDC and DHHS.
In the preceding pages, the committee has outlined challenges and op-
portunities inherent in integrating public health into a broader field. In order to address the challenges and maximize the opportunities, the committee recommends that all federal entities concerned with bioterrorism preparedness (e.g., CDC, HRSA, ODP) more actively coordinate guidance and funding activities. Federal agencies should also work together to develop mechanisms that facilitate coordination and collaboration among their grantees at the state and local levels. Such mechanisms may include, but are not limited to, regular meetings to familiarize CDC and ODP program staff with each other’s program priorities and activities, a database for informing ODP and other partners of exercises planned by CDC grantees, etc. Federal coordination efforts should also include the clarification of primary responsibility and authority in bioterrorism events, to ensure that CDC can fulfill its unique role as the nation’s public health agency.
THE EVIDENCE BASE FROM DISASTER RESEARCH AND PRACTICE
Nature of the Evidence
Although quantitative evidence (with randomized controlled trials as the gold standard) is extremely important in public health and medicine, this level of evidence may be difficult or impossible to obtain in research pertaining to public health disasters. While endeavoring to conduct quantitative, empirical research whenever possible, public health professionals also value other types of knowledge that contribute to decision-making and research methodologies that provide alternate routes to usable evidence. For example, the Task Force on Community Preventive Services, which is a major contributor to evidence-based public health, evaluates population-based health interventions through systematic and rigorous reviews that are not restricted to empirical and quantitative evidence (Briss et al., 2004). Methodologies for research used in public health are drawn from the social sciences, statistics, and epidemiology rather than solely from the biologic sciences.
Disaster research is in a position somewhat similar to public health research; there is some disconnectedness between academic research and practice (i.e., bringing research to bear on practice, and practice to inform and be validated by research), researchers come from diverse disciplines, there are challenges in translating research to practice, and it has been difficult to develop and secure funding for a comprehensive research agenda (Tierney, 1993; Quarantelli, 1994; Peters et al., 2001). Being aware of these similarities may help public health better understand and interpret disaster research and practice and their potential contributions to public health preparedness.
What Has Been Learned from Disaster Research
The following are some examples of major findings identified in two systematic surveys of the disaster and emergency management literature, two literature reviews on the subject of interorganizational coordination in disasters, and several theoretical articles (Auf der Heide, 1989; Tierney, 1993; Quarantelli, 1994; Granot, 1999; Tierney et al., 2001; Drabek and McEntire, 2002). These concise summaries of findings and research gaps are not provided in any specific order or priority. A general observation emphasized in the literature (and reiterated below) is that a comprehensive, systematic research agenda is needed in disaster research, and the committee would add, analogously, in public health preparedness.
Some Key Research Findings and Recurring Themes
Human behavior in disasters is continuous with predisaster behavior patterns; individuals will generally behave adaptively, altruistically, and will not panic (except for rare situations characterized by an identified set of factors4) (Tierney et al., 2001; Auf der Heide, 1989; Drabek and McEntire, 2002). This finding is relevant to every aspect of planning for and responding to disaster, such as defining a role for communities in disaster response, developing communication plans and messages, and allocating resources based on what is likely to happen, rather than on inaccurate assumptions (Quarantelli, 1994).
Collaborative interorganizational planning and preparedness are essential to successful response (Tierney, 1993; Granot, 1999; Burkle and Hayden, 2001; Tierney et al., 2001; Drabek and McEntire, 2002). Contact and coordination must be established pre-event among government agencies, between public and private entities, and among all entities likely to respond to a disaster.
Studies of the preparedness activities of local emergency management agencies show that they are diverse in structure and operate in ways that make them well adapted to local conditions (Quarantelli, 1994). This demonstrates the importance of focusing on local needs and developing local response capacity, within the context of regional and national coordination and standards.
The level of perceived risk among organization leaders is positively correlated with emergency preparedness (Tierney et al., 2001). This would
indicate that conducting regular, accurate risk assessments and communicating this information to all responder agencies would help strengthen the rationale for preparedness.5
Severe disasters lead to the creation of impromptu community organizations that mobilize to address gaps in response capacity or failure of existing systems to surge adequately in situations where their resources are strained excessively (Tierney et al., 2001). This phenomenon, sometimes called “emergence,” is noteworthy because it underscores the tremendous capacity of communities and their social networks and formal associations to respond to crises. Communities are likely to know themselves better than most outside agencies or organizations, and their knowledge and resources should be part of public health preparedness, including planning for bioterrorism and other public health crises (IOM, 2003d). The evidence about inadequacies in certain aspects of postdisaster response, or in addressing the needs of special populations may be helpful in anticipating and planning to correct such inefficiencies in future responses (Quarantelli, 1994; Kreps and Bosworth, 1999; Tierney et al., 2001).
Studies of disasters have shown that when plans exist simply for compliance with administrative requirements but are not part of a dynamic process of learning, planning, and preparing, responders involved are likely to ignore all or most of the plan (Quarantelli, 1988). In some cases, plans have been found to be irrelevant, inaccurate, or simply unfamiliar to responders who did not know the plan or their role in it (Auf der Heide, 1989). An emergency response plan does little good if the participants in the plan have not developed a relationship with their partners, have not practiced the plan, or have not updated the plan as circumstances have changed (Auf der Heide, 1989; Perry, 2003). The importance of an emergency response plan lies in the process of developing, exercising, and improving it rather than in the document itself, for it is the process that allows relationships to be built, understanding of different disciplines to be fostered, and communication barriers to be broken down.
Examples of Gaps in Disaster Research
There is a need to expand what is known about preparedness, since there is more evidence about how first responders respond to a disaster than how they prepare and how preparedness relates to ability to respond (Auf der Heide, 1989; Tierney et al., 2001). “Large-scale studies are needed
to systematically examine the impact of emergency preparedness on the effectiveness of emergency response activities while controlling for differences in disaster impacts and community characteristics” (Tierney et al., 2001).
More research is needed to identify the planning assumptions that result in effective organizational performance (Tierney et al., 2001). Although disaster practice has benefited from research, it is still common for emergency responders and organizations to adopt certain concepts not based on scientific evidence, but on convenience, a concept’s popularity among peers, and perhaps, anecdotal evidence (Quarantelli, 1995). More research is needed to substantiate or discredit the effectiveness of practices and concepts used in disaster planning (NRC, 2003).
Little is known about what makes local emergency management agencies effective and successful (Tierney et al., 2001).
Research is needed to determine what preparedness and response strategies or models are most useful, under what circumstances, and to what extent they should be implemented for optimal results. For example, more research on the Incident Command System (ICS) is needed (Tierney, 1993; NRC, 2003). Although its value has been questioned by some disaster researchers, and there is limited empirical evidence documenting its effectiveness, ICS has been widely embraced by the emergency management field and even in health care and other areas, and forms the foundation of the National Incident Management System adopted by DHS (Tierney, 1993; Quarantelli, 1995; Drabek and McEntire, 2002; NRC, 2003).
Research on police and fire department preparedness was conducted mainly in the 1970s, so it is out of date (Tierney et al., 2001).
Not enough is known about local emergency preparedness networks, their composition, and the relationships among responder agencies (Tierney et al., 2001).
Little is known about the resources or community characteristics that are related to better levels of preparedness (Tierney et al., 2001).
More effort needs to focus on translating research into the practice of emergency and disaster management, for example, identifying the elements of emergency management applicable to all-hazard preparedness (NRC, 2003).
The emergency management field has frequently grappled with the question of whether to ensure generic or specific preparedness capabilities and processes (NRC, 2003). For example, should the training of first responders center on a set of generic capabilities, or should more attention be paid to specific types of disasters? The field of emergency management has moved to an all-hazards approach in the past decade, and although this approach has certain strengths, it also may have limitations and present challenges. During the planning phase, however, research findings support
a greater focus on generic approaches rather than agent-specific ones. Reasons for this may include: the fact that it is impossible, for practical and resource reasons, to plan for every possible contingency; most disasters share a core of common elements; and disasters cause temporary changes in organizational structure and functioning that require flexibility in planning (Tierney, 1993; Kiel, 1995; Quarantelli, 2004). As the public health community joins other emergency and disaster responders, the question of generic versus specific deserves renewed consideration for its implications to public health preparedness. Should public health preparedness be generic, or should it, for instance, focus on specific biologic agents, like smallpox or botulinum toxin? Or is there a way to strike an ideal balance between generic and specific preparedness?
What Has Been Learned from Disaster Practice
Key Lessons Learned in Disaster Practice
Although many of the lessons learned in disaster practice are not in areas clearly relevant to public health preparedness, certain broad themes may translate relatively well.
Technology may fail in disasters; therefore, planning and training should include “low-tech” alternatives for communication and other activities (Tierney et al., 2001).
Communication among responding agencies is essential during a disaster. The experience of disaster responders across the country contains numerous examples of situations where communications equipment or frequencies were not compatible and responder agencies were unable to communicate with each other (fire to law enforcement, EMS to hospitals, etc.). This can present enormous challenges to all involved (Auf der Heide, 1989).
Ongoing needs assessment is needed over the course of a disaster for efficient distribution of resources and prioritization of activities (Auf der Heide, 1989).
Emergency response is sometimes based on myths that research has disproved, with consequences for the success and effectiveness of the response (Auf der Heide, 1989; Tierney et al., 2001). This also underscores the need to better link research and practice.
A Resource for Learning from Past Experience
It has been observed that a great deal of the knowledge available in the disaster management field and in terrorism preparedness reflects a failure to learn from the past (actual events or exercises); the same mistakes are made
again and again both within and across responder organizations (CBACI, 2002; Auf der Heide, 1989). This is partly due to the fact that what is learned is often not shared. One way to ensure that what is learned is disseminated widely is to create a database or other centralized repository of such information. Recently, the National Memorial Institute for the Prevention of Terrorism and the DHS Office of Domestic Preparedness jointly launched a website devoted to sharing lessons learned from exercises, www.LLIS.gov (LLIS stands for Lessons Learned Information Shared). This site summarizes a large amount of information, including lessons learned, best practices, reports, guidelines, and stories from a wide range of what it terms emergency disciplines (e.g., law enforcement, fire, HazMat, veterinary, search and rescue, public health, and medical), and pertaining to a variety of actual events and exercises. It must be noted, however, that certain aspects of preparedness and response are unique to bioterrorism preparedness, so lessons gathered by emergency responders in other areas may not be applicable in their entirety or at all.
The DHS and LLIS MIPT searchable database will undoubtedly prove helpful to government agencies and their partners as they work together to strengthen their capacity to respond to deliberately inflicted and other types of disasters. The architecture of the LLIS database includes public health among emergency disciplines and functions and seems to provide opportunity for entering material relevant to public health preparedness. Given the importance of disseminating knowledge, and the currently limited avenues that exist to facilitate such sharing, CDC and its state and local public health partners may wish to consider the DHS mechanism for sharing lessons learned and develop a similar and connected mechanism to support public health preparedness goals. Such a database may involve, but not be limited to the following activities: developing and gathering after action reports based on public health preparedness exercises and responses to actual events that tested the capacity of the public health system; conducting a retrospective analysis of public health agencies’ responses to infectious disease and other relevant events in the past 2-3 years; and increasing the emphasis on studying the responses to proxy events and the effect of exercises and publishing findings in the peer-reviewed literature.
LEARNING FROM THE PUBLIC HEALTH RESPONSE TO PROXY EVENTS
Studying the Response to Public Health Challenges
Thus far, the committee has highlighted some key points from disaster research and practice that may be useful to CDC and the public health
community. However, public health has its own rich knowledge base, which includes lessons from recent public health emergencies such as food-borne disease outbreaks, emerging infectious diseases, and the anthrax attack of 2001. Unfortunately, there is no systematic, comprehensive agenda for public health preparedness research to provide a structure for public health emergency preparedness and response research. Such an agenda would be a part of the broader public health research agenda that has recently begun to take shape, but it still requires infrastructure and funding (Council on Linkages between Academia and Public Health Practice, 2004). Systematic public health and interdisciplinary research is essential to inform preparedness against bioterrorism and other threats.
In general, the knowledge gathered from recent outbreaks and other public health emergencies is available predominantly in reports (e.g., from GAO, from nongovernmental organizations) or anecdotal assessments (e.g., in media reports). The peer-reviewed literature seems to offer little research on this subject. Recent anecdotal reports about the ways in which bioterrorism planning and training improved response to a crisis are encouraging, but it is important that such observations are documented and somewhat more quantitative and objective studies are undertaken to determine whether the public health system’s performance (and therefore, response capacity) has indeed been enhanced by expanded resources, surveillance and information systems, and linkages with other partners.
In a study of 12 nationally representative communities, respondents acknowledged general improvements made possible by public health preparedness funding and requirements, including more training of personnel and the development of relationships to first responder and other local agencies and organizations (Staiti et al., 2003). Also, state officials in Massachusetts and Virginia attributed their states’ rapid response to Severe Acute Respiratory Syndrome (SARS) to their public health preparedness efforts supported by funding for bio-terrorism (Staiti et al., 2003; Stoll and Lee, 2003). A GAO report (GAO, 2004) also found that some states have increased laboratory capacity and that the coverage by HAN, CDC’s Health Alert Network, has increased to 90 percent of the nation, which can be assumed, would result in improved rapid notification of health care providers and other health personnel. However, the effect of HAN’s expansion is yet to be determined. In 2003, the executive director of the American Public Health Association and former director of the Maryland Department of Health and Mental Hygiene asserted that previous experience with West Nile virus and anthrax taught the state public health agency in Maryland valuable lessons about communication and cross-jurisdictional coordination—lessons which paid dividends during Maryland’s encounter with SARS (Benjamin, 2003). In April 2004, bioterrorism preparedness efforts were credited with the swift response to
two measles outbreaks by state and federal public health agencies (Elliott, 2004).
Though the examples provided suggest that public health preparedness has been improved, more systematic research is needed to examine the link between the application of lessons learned and improvements in performance, the effect of preparedness on routine public health practice and the delivery of the Ten Essential Public Health Services, and the state of the formal and informal collaboration and communication between the health care and public health communities.
Evaluating Performance in a Proxy Event
Some disease outbreaks (e.g., monkeypox, West Nile virus, SARS) that can serve as proxy events for a bioterrorism disaster (such as a smallpox attack) may be valuable in testing preparedness activities because they are likely to possess some of the same characteristics, especially if they are of significant magnitude. These characteristics include:
No prior planning or announcement (unlike most exercises), therefore placing increased stress on human and other resources;
Increased mobilization of resources;
Enhanced surveillance activities;
Frequent communication among all parties involved; and
Increased scrutiny from the media.
Using the “What if?” Scenario Approach
Given that public health disasters such as attacks with biologic weapons or widespread epidemics are infrequent, plans must be put into place to capture important information and facilitate performance measurement during and after a proxy event. However, the magnitude of a public health emergency determines its potential usefulness as a proxy event for measuring bioterrorism preparedness. In significant proxy events, public health agencies should constantly ask themselves: what if the lead in drinking water, the monkeypox cases traced back to exotic pets, the appearance of SARS, or the occurrence of hepatitis A virus in restaurant food were the result of deliberate, ill-intentioned introduction? What if the number of cases of an unusual new disease was not a handful, but a few thousand? What if not one emergency occurred, but three? Would our response have been adequate, sufficiently rapid, or sufficiently well coordinated? Using various health threats as proxies for evaluating public health agency performance and identifying the requirements for an adequate response would
support continuous quality improvement over time. Components of the evaluation of proxy events may include, but not be limited to: formal debriefings after an event; discussion of what went well, what went wrong, and what was learned (i.e., after-action report); and deciding what changes will be made in communication, staffing, training, equipment, facilities, and interagency/intersectoral coordination (i.e., improvement plan).
The committee believes that it is important to evaluate performance during proxy events at the local, state, and federal levels of the governmental public health infrastructure. CDC is the lead public health agency, not only as a standard setter and funder, but also as an important part of public health practice and of public health response to emergencies and disasters. Proxy events test CDC’s resources and ability to respond to crises rapidly, expertly, and in coordination with state and local agencies. CDC may become a limiting factor during a disease outbreak or a deliberate attack with a biologic agent if the agency is the major or sole source of information or supplies. For example, CDC’s Laboratory Response Network is the source of many reagents for laboratories around the country, and state distribution networks are ultimately dependent on CDC. Therefore, CDC’s role in responding to a proxy event must be considered one of the major aspects of a response, the agency’s own performance must be evaluated, and plans for improvement must be developed and implemented.
One way to use proxy events as a means of performance evaluation for states and CDC in particular is to conduct a systematic and careful review of Epidemic Assistance Investigations, or Epi-Aids. Epi-Aids are a mechanism through which CDC provides collaborative assistance to state, national, and international health officials in investigating disease outbreaks and other epidemiologic emergencies (Office of the Federal Register, 2003). The review of Epi-Aids, which are one form of after-action reports, may provide an additional window on public health performance in public health emergencies and disasters. In reviewing Epi-Aids or other types of after-action reports, it may be most useful to focus on incidents that involved infectious agents with bioterror potential and to review them as if the event had been a deliberate introduction. The questions that must be asked in the course of review include:
How was the outbreak detected?
How much time elapsed between the event and detection? How could that time have been reduced?
Who was exposed and how?
How could numbers of exposed have been reduced?
Did the response to the event follow CDC guidelines?
How quick was the response?
How could the response be improved in the future? Based on the experience what would be the lessons for the next time?
As noted above, the research literature is limited in the area of public health emergency response, and much of what is known about the influence of preparedness on responses to recent outbreaks is based on subjective factors. Any proxy event, such as reoccurrences of West Nile virus, foodborne disease outbreaks, or other public health events of note, should be seen as an opportunity to measure progress toward preparedness goals and competent performance as another milestone in a continuing process. As CDC’s Evidence-Based Performance Goals for Public Health Disaster Preparedness, currently under development, are disseminated and implemented, it is important that CDC and its state and local partners take steps to link these with a system for capturing lessons learned from the response to proxy events.
The committee has described some of the knowledge available from the practice of disaster response and from disaster research and the need to strengthen public health preparedness research. The committee recommends that CDC should collaborate with all of its partners to strengthen preparedness by applying research findings and experience in public health emergency response, bioterrorism preparedness, and disaster management. In order to strengthen the evidence base for public health preparedness, CDC should:
Strengthen the link between public health research and practice.
Participate in and promote interdisciplinary research about preparedness.
Support a system to assure the ongoing collection, synthesis, and sharing of lessons learned and best practices from public health preparedness exercises and public health response to proxy events.
In coordination with the appropriate federal-level partners, such as AHRQ, evaluate the effectiveness, design, and opportunity costs of preparedness strategies, such as exercises.
USEFULNESS OF MODELING
Role of Modeling in Policy Decisions
For public health preparedness, models can be useful tools to assist in decision-making, focusing preparedness efforts, and analyzing different response options. Though there sometimes can be a tendency to want to use models to predict outcomes, models have limitations and should not be
relied upon for this purpose. Models should be used as “an aid to understanding, rather than being an end in themselves” (Taylor, 2003).
Modeling can be a useful tool to assist in assessing different policy options, but only with a clear understanding of the limitations of modeling. Models are only as good as the data that are used to develop them. The accuracy and generalizability of models depend on which components are included or excluded, the validity of any assumptions made about them, and the accuracy of modeling of the interactions between them (Taylor, 2003). If the data are timely, accurate, and appropriate, and the model includes all the relevant input parameters and appropriately portrays all the relationships among the input parameters, then models can serve as useful tools in making policy decisions. A good model can assist decision-making before an event by helping policy makers decide where to focus preparedness efforts, or while an event is occurring as current data can be added to the model to fine-tune the model for the particular situation.
For models to have utility, sensitivity analyses should be conducted for each input parameter, to assess which factor has the greatest effect on the outcome or outcomes of interest. Once it is determined how sensitive the outcome is to the different input parameters, preparedness efforts can be focused on the factor that is estimated to have the greatest effect on the outcome. This also will help policymakers determine the factors for which indicators should be developed. Considering the limited resources currently available for public health preparedness activities, knowing which factors potentially have the greatest influence on the course of an outbreak will be extremely valuable to those who decide how to allocate limited resources. Without sensitivity analyses, models are of limited value to policymakers.
Role of Modeling in Exercise Development
Modeling can help inform planning for exercises by elucidating the critical factors that affect the outcome, which in turn helps in designing exercises that stress that particular part of the system. For an exercise testing the response to smallpox, findings from some of the models described below could help inform the focus that should be placed on vaccination of the public compared to contact tracing and containment, the number of staff in mass vaccination clinics, and the need for post-event vaccination of health care workers.
A number of models created in the past few years have examined the potential spread of smallpox under varying scenarios (Meltzer et al., 2001; Halloran et al., 2002; Bozzette et al., 2003; Kaplan et al., 2003). These models used slightly different assumptions for most of the key input parameters, resulting in different conclusions. Meltzer et al. found from their
model that a combination of quarantine and vaccination would be the best option for stopping a smallpox epidemic (2001). Mass vaccination is found to be the best option for limiting mortality and reducing the time it takes to end a smallpox epidemic in the model produced by Kaplan et al. (2003). In the Halloran et al. model, mass vaccination could produce better outcomes than targeted vaccination, and vice versa, depending upon specific components of the particular outbreak, such as preexisting immunity, rate of transmission to contacts, and vaccine supply (2002). Bozzette et al. found that the net benefits of vaccination depend upon the probability of an attack, with prior vaccination of health care workers being favored unless the probability of an attack is very low, and mass vaccination being favored only if the probability of a national attack or multiple attacks is high (2003). Ferguson et al. (2003) compared these four different models and offered reasons why the conclusions that could be drawn from each model were different.
Models have been used to examine different aspects of a smallpox outbreak. Some models have examined the speed of different components of a response to a bioterrorism incident and how this affects outcomes (Giovachino and Carey, 2001; Hupert et al., 2002). Other models have examined an individual’s risk-benefit profile for pre-or postexposure smallpox vaccination (Meltzer, 2003), estimates of historical transmission rates (Gani and Leach, 2001), the course of historical smallpox epidemics (Duncan et al., 1994; Eichner and Dietz, 2003), and the effect of isolation of overt cases of smallpox and surveillance of contacts on the progression of a smallpox epidemic (Eichner and Dietz, 2003).
Each of these models has utility in examining particular aspects of a smallpox outbreak and the corresponding response options, but before a model is used to help make important decisions about a smallpox exercise or a smallpox response, the model’s assumptions and input parameters must be deemed reliable and realistic. This also is another important reason why sensitivity analyses are necessary. They are used to study the effect of varying the range of assumptions that have been made. This is the importance of the model—not its overall conclusion.
There also may be opportunities to learn from other modeling efforts. The military has extensive experience with modeling, potentially providing a rich knowledge base that could aid smallpox modeling. Knowledge gained from modeling other communicable diseases (e.g., measles) also could inform the population dynamics and transmission aspect of modeling smallpox, West Nile virus, and SARS outbreaks, in particular. Recent efforts to model an intentional release of anthrax could shed light on factors that are unique to modeling a bioterrorism event. The committee encourages CDC to draw upon the knowledge and experience of other modeling efforts when developing models for smallpox or any other biologic agent.
Smallpox Modeling Working Group
Recognizing the potential value of modeling in informing policy decisions, the DHHS Secretary’s Council on Public Health Preparedness recently formed a Smallpox Modeling Working Group (Borio, 2004). The Smallpox Modeling Working Group was created to “explore a range of policy options related to smallpox preparedness and response” (Borio, 2004). Three modeling groups were selected to model the effects of different response strategies. To overcome some of the reasons for differing conclusions of previous models (Ferguson et al., 2003), the Smallpox Modeling Working Group decided that a standardized set of biologically realistic input parameters for smallpox natural history and transmission needed to be agreed upon (Borio, 2004). In addition to the standardized input parameters, the working group also developed outbreak scenarios, policy options regarding outbreak containment measures, and outcome measures of interest to DHHS (Borio, 2004).
The three modeling groups ran their models using the agreed-upon parameters. Based on the results of these models, the Smallpox Modeling Working Group reached the following interim conclusions:
Surveillance and containment alone is sufficient to effectively contain an intentional smallpox release.
There is relatively small marginal benefit in prevaccination of hospital workers or mass vaccination of the population after an outbreak begins. Reactive mass vaccination may have additional value in bringing an epidemic under control.
In the absence of any interventions, the strongest controlling factor is people withdrawing to the home when they become “ill” (Borio, 2004).
As was mentioned earlier, models can offer illustrative guidance as to the factors that have the greatest influence on the outcomes of interest, but models have their limitations and should not be used alone for making policy decisions. Of the three models created under the aegis of the Smallpox Modeling Working Group, one is a deterministic model, since it uses single point estimates for each of the input parameters, whereas the other two are stochastic models, using probability estimates for the different input parameters (Borio, 2004). Since recent data on smallpox transmission rates, incubation period, case fatality rate, vaccine efficacy, vaccination adverse event rates, and population dynamics of the current U.S. population are limited, stochastic models may be more illustrative of the range of outcomes that are possible due to a smallpox outbreak. However, to accurately portray the role of modeling in policy decision-making, the sensitivity of particular input parameters on the model’s outcomes must be provided,
and the limitations of the model and uncertainties in the data must be conveyed (Ferguson et al., 2003).
USEFULNESS OF EXERCISES
The Use of and Rationale for Exercises
Exercises are believed to be effective in enhancing preparedness and are widely used by local, state, and national disaster response agencies (GAO, 2001). The emergency and disaster response field’s assumptions that exercises work to improve preparedness have been reinforced by experience that has suggested a link between exercises and good performance in an emergency or disaster (FEMA/EMI, 2003). According to FEMA’s Emergency Management Institute (2003), exercising reveals flaws in planning, clarifies roles, improves individual performance, and tests and evaluates plans, policies, and procedures. Moreover, exercises have become an institutionalized strategy for planning in homeland security. In fact, the DHS HSEEP materials assert that exercises “provide a risk-free environment for jurisdictions to assess if they have the plans, policies, procedures, resources, and agreements in place to enable homeland security personnel to perform critical tasks required to prevent, respond to, or recover from a terrorist attack” (DHS/ODP, 2003).
Exercises contribute to preparedness by fostering relationship building; by providing a context and tool for training; and by providing a method for evaluating performance. The use of exercises for training may originate in the military experience, but they are conducted as part of preparedness in a variety of contexts, from the nuclear plants required by the Department of Energy to use exercises to prepare for the possibility of an accident, to local firefighters training to deal with major fires or with natural disasters, to regional exercises required by FEMA to respond to hazardous materials (HazMat) and natural disasters.
When exercises are conducted in order to educate, train, or develop interorganizational and interjurisdictional relationships, the underlying assumptions may be easily validated. Disasters are complex events that require many different types of responders; therefore, having partnerships is preferable to working in isolation. Furthermore, some level of organization and coordination is essential to help avoid chaos; rehearsing processes may lead to smoother functioning of complex response systems, and in the event of an emergency, for example, a smallpox attack, having personnel that possess certain knowledge and skills (e.g., smallpox diagnosis, vaccination, and search and containment) is better than having personnel that did not receive such education and training. Exercises that test communication across jurisdictions or test certain skills and processes may provide some indication that certain things are likely to work well in a disaster.
Research on Exercises
Although the assumptions made about exercises are reasonable, and exercises seem like a practical strategy in many circumstances, there are at least two reasons to seek more objective study: the need to compare multiple types of exercises for more targeted use and the potential costs posed by exercises.
As noted above, more research is needed on preparedness (Tierney et al., 2001), but exercises and other means for evaluating performance (and for improving preparedness) form a particularly neglected subset of preparedness. The overall effectiveness of exercises as a preparedness strategy has not been well demonstrated, and research is needed to determine, for example, whether exercises could be considered predictors of successful response, what type of exercise would have the greatest positive influence on preparedness, what exercises are most cost-effective, and the best way to assess opportunity costs posed by conducting exercises (NRC, 2003). The use of scenarios, which may serve as a component of exercises, for training and planning purposes has not been well researched either, but there are “indications that they are an excellent method of teaching rapid response-style thinking, decision-making and the development of managerial skills” (Alexander, 2000; Simpson, 2002:56). Potential research questions would include: how do reality-based scenarios compare with entirely fictional scenarios, and under what circumstances would the use of one be preferable to the other?
In the disaster literature, mention of exercises seems limited to descriptions of how they were utilized by responder agencies and disciplines (EMS, emergency departments, fire departments, etc.), the lessons learned, and changes in operations made as a result (Tierney et al., 2001). A brief review of the medical and health peer-reviewed literature (using the National Library of Medicine’s PubMed search engine) shows that hospitals and public health agencies conduct exercises and find them useful in evaluating the quality of training, the smoothness of emergency operations, and other aspects of disaster response, but there seems to be little or no empirical study of the validity or effectiveness of exercises themselves as a strategy for public health and health care preparedness. The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center recently conducted a review of the literature on hospital exercises and drills and concluded that “the evidence was insufficient to support firm conclusions about the effectiveness of specific training methods because of the marked heterogeneity of studies, weaknesses in study design, and the limited number of exercises that have been reported in the literature” (AHRQ, 2004). The committee hopes that the experience of public health agencies with exercises and drills will not only be reported to CDC, but that there will be increased emphasis on more in-depth studies of the effectiveness of public
health preparedness exercises, and more frequent publication of such studies in the literature. The growing partnerships between public health agencies and schools of public health (i.e., the Academic Centers for Public Health Preparedness) would certainly contribute to such an effort.
At the committee’s March 2004, meeting, DHS ODP speakers presented a graphic describing a cyclical or building block approach to planning and training (see Figure G-2). This graphic outlines an incremental set of techniques or methods for preparedness planning and training, from the minimal complexity of a seminar to the significant complexity of functional or full-scale approaches. On the one hand, this building-block approach seems logical, and it is reasonable to select a training and capability testing method based on and scaled to match the complexity of the objective (e.g., a workshop to train for a particular role and then to test it; a functional exercise to rehearse and evaluate a complex interagency process). However, the committee believes this approach requires evaluation. For example, by what means are games determined to be more or less complex than tabletops, and in what circumstances is one method preferable to another? This type of evaluation research would help ensure the implementation of the most effective, evidence-based means for strengthening preparedness.
Exercise-Related Activities of the Department of Homeland Security (DHS)
To help ensure that the country is prepared for a possible terrorist attack or other emergency, DHS has primary responsibility in the federal government for organizing and evaluating preparedness exercises and drills. In DHS, the ODP is responsible for providing training, funds for the purchase of equipment, support for the planning and execution of exercises, technical assistance, and other support to assist states and local jurisdictions in preventing, planning for, and responding to acts of terrorism (DHS, 2004a). In addition to ODP, FEMA also has been incorporated into DHS, and it has brought over its expertise in the area of exercises. Together, ODP and FEMA are responsible for HSEEP, Radiological Emergency Preparedness (REP) Program, Community Hazards Emergency Response Capability Assurance Program (CHER-CAP), and the Chemical Stockpile Emergency Preparedness Program (CSEPP). These programs and others offer a rich history of exercise experience that could inform public health preparedness.
HSEEP, mentioned in preceding pages, details DHS’s comprehensive exercise doctrine and is “a program of financial and direct support designed to assist state and local governments with the development and implementation of a state exercise and evaluation program to assess and enhance domestic preparedness” (DHS/ODP, 2004a). HSEEP resources include four volumes of reference materials to assist state and local jurisdictions with the design, development, conduct, and evaluation of exercises. The first volume of HSEEP includes a uniform approach for exercise design, development, conduct, and evaluation (DHS/ODP, 2004a). The second volume includes a methodology for conducting evaluation of homeland security exercises and implementing an improvement program (DHS, 2004b). The third volume is an exercise development manual, outlining a standardized planning process, adaptable to any type of exercise or scenario (Schweitzer, 2004). The fourth volume consists of sample exercise documents (DHS/ODP, 2004a).
As the committee learned at its March meeting, FEMA has extensive experience with executing and evaluating exercises as part of the agency’s mission to prepare the country for disasters, and its work contributes to the practical knowledge base on exercises and drills (FEMA/EMI, 1995; FEMA, 2004; Kelkenberg, 2004). For example, FEMA has been involved with the REP program, which was established as a direct result of the Three-Mile Island incident and ensures adequacy of emergency plans and preparedness for areas near commercial nuclear power plants (Kelkenberg, 2004). Whereas the Nuclear Regulatory Commission is responsible for ensuring the adequacy of emergency plans onsite, FEMA is responsible for reviewing and evaluating offsite radiological emergency response plans developed by state and local governments and for evaluating exercises conducted by state
and local governments to determine if radiological emergency plans can be implemented (Kelkenberg, 2004). FEMA also plays the significant role of educating the public about radiological emergency preparedness (Kelkenberg, 2004).
FEMA has a memorandum of agreement with the U.S. Army for the CSEPP program. Similar to REP, the Army is responsible for onsite preparedness and response, and FEMA is responsible for offsite preparedness and response in the surrounding community (Kelkenberg, 2004). Some useful documents have been created through this program and may be applicable to other preparedness programs. For example, Oak Ridge National Laboratories has developed training materials on sheltering in place that would be applicable to other emergency preparedness scenarios (Kelkenberg, 2004). CSEPP also includes an exercise component, which involves both the site itself and the surrounding community (e.g., emergency management agency, health department, hospitals). FEMA evaluates the local community’s performance in the exercise and, based on the findings, recommends how the community partners’ emergency response plans should be improved (Kelkenberg, 2004).
FEMA’s Community Hazards Emergency Response Capability Assurance Program (CHER-CAP) consists of a planning, training assessment, and exercise process for all-hazards response operations (Kelkenberg, 2004). It consists of looking at a community’s emergency response plans, coupling the plan with a risk assessment, identifying training to fill in the gaps in the plan, and then doing a tabletop exercise and peer-evaluated full-scale exercise to identify areas for improvement (Kelkenberg, 2004).
Sample Questions, Strategies, and Methodologies for Evaluation Research on Public Health Preparedness Exercises and Proxy Events
The committee has identified several possible questions, strategies, and methodologies that could be considered by CDC in evaluation and experimental research in public health preparedness. These include:
Examining the effect of a proxy event on two similar communities with different public health infrastructure and capabilities.
Conducting “placebo-controlled” trials comparing the response of two similar communities to a proxy event, false alarm, or to an unannounced exercise. One community previously conducted one or more exercises (or employed other methods) to test its preparedness, while the other did not. Compare response times, smoothness of interagency coordination, functionality of communication channels, and other aspects of their response.
Conducting an unannounced exercise and comparing the performance of three different groups of personnel (with the same qualifications and functional roles) who have undergone one of the following: (1) preparedness-related training only; (2) training and a tabletop exercise only; (3) training and participation in a comprehensive functional exercise.
Randomly assigning educational materials to various types of health personnel to determine whether the type and quality of educational materials have an effect on exercise outcomes.
Randomizing the method of preparing responders and the community at large before an exercise is conducted in order to determine the best way to conduct them. Outcomes to be measured would include professional participation rates, community participation, and major desired outcomes of the activity.
The committee also suggests several areas for further study:
A systematic assessment of all lessons learned in the course of the smallpox vaccination program (which could be considered a national-level, multisite, months-long preparedness meta-exercise);
Determine what knowledge is available about public health preparedness and about conducting exercises (and drills) in public health agencies with experience in this area (history of working with EMS, preparing for nuclear accidents, etc.). Many lines of questioning could be followed in gathering information from public health agencies with a variety of linkages to emergency and disaster response. For example, it may be instructive to compare the engagement of public health in emergency and disaster response in states where EMS is part of public health to states where the two agencies are separate.
Systematically assess the lessons learned by state and local public health agencies (perhaps organized by type according to characteristics such as size, urban or rural location, structure and governance) that have conducted exercises and drills. Such an assessment also must include an examination of any evidence that is available or is becoming available in the literature or in the reporting of public health agencies, of the effectiveness of exercises and drills conducted by public health agencies (similar to AHRQ’s recent work [AHRQ, 2004]).
A systematic and comprehensive research agenda for studying the response to public health emergencies and disasters be developed.
An evaluation of patient safety literature to consider the pre- and posthandling of sentinel events.
A Framework for Performance Evaluation Using Exercises
Major outcomes in public health typically involve decreasing mortality and disease rates and progress is measured periodically (e.g., Healthy People 2010 process). Performance measurement in public health is, however, a relatively new field. In the case of public health preparedness for bioterrorism and other events with significant public health impact, outcomes are occasioned by actual events themselves, and the infrequency and huge variation among these events (including the proxy events discussed in preceding pages of this report) make it difficult or nearly impossible to gauge, for example, a decrease in rate of disease from contaminated water or other reductions in mortality and morbidity attributable to the disaster. Due to the nature of disaster-related public health problems, performance measurement in this area is by necessity more process-oriented. When CDC and its state and local partners identify exercise objectives that will be used in evaluating the exercise, these objectives will be most helpful if they are linked with the Evidence-Based Performance Goals for Public Health Disaster Preparedness developed by CDC.
Exercises offer an alternative way to measure performance and finetune preparedness before a crisis occurs. Public health preparedness exercises take place at national, state, and local levels, and it is important that evaluation of exercises take place at all levels. The committee believes it is essential to design and conduct exercises that stress and test CDC’s own performance. As noted in the preceding discussion of proxy events, CDC is a vital part of preparedness and response and it is itself a limiting factor in terms of the resources it provides (e.g., laboratory reagents, information, technical assistance) to state and local counterparts. In asking “what if” questions in a proxy event or in an exercise, the limits of availability of such resources must be probed. In addition, modeling could be used to estimate such things as the rate of producing and renewing the supply of needed laboratory reagents, or the speed with which needed field experts could be moved from place to place. In a more dramatic type of exercise, questions could be asked about the potential effect if CDC itself was the target of an attack and critical facilities destroyed.
After-action reports will play an important role in facilitating continuous quality improvement. They provide an overview of agency or interagency performance in an exercise and identify areas where there are gaps in planning, unforeseen circumstances that are poorly managed, or areas where communication or the flow of information break down, among other issues.
Various types of methods for measuring performance will eventually be determined to be effective and even to have some predictive value (e.g., of future successful response). The link between research and practice requires
strengthening, so that as research validates certain practices, such as types of exercises, and the most effective techniques to communicate to or evacuate the public, they may be rapidly translated into practice. The practices demonstrated to be most effective (e.g., specific types of exercises) need to then be institutionalized and adapted to local circumstances, with particular attention to maintaining and updating staff competence and sustaining readiness. Staff turnover itself, which requires regularly updating training and conducting exercises, could be used to create new cohorts for performance evaluation.
CDC might wish to consider describing the breadth and depth of exercises needed for public health. The HSEEP Building Block Approach illustrates one typology of training and capacity-building methods, including exercises. CDC could develop a similar representation with specific applications to public health. For example, in the area of exercises, some exercises may be external, conducted in coordination with other agencies at the federal, state, and local level (refer to the section on Coordination Issues), while others will be strictly internal exercises on such issues as how to move from normal to emergency operations, including decisions about closing or curtailing planned clinics, outreach, or investigation; decisions about and use of personal protective equipment under various circumstances; establishment of databases for unexpected investigations or unusual outbreaks.
One of the challenges in developing and implementing exercises is to make the mock disaster approximate as closely as possible a real-life one, including as much complexity and unpredictability as possible, and basing scenarios on what is likely to happen according to the microbiological, immunological, epidemiologic, and disaster literature, not on myths or on widely embraced assumptions.
Ensuring Compatibility Between the DHS Exercise Doctrine and Public Health Preparedness Exercises
The DHS HSEEP describes a yearly cycle of planning and development, followed by training, exercises, and the development and implementation of an improvement plan. The committee has learned that CDC intends to implement a similar cyclical process (target goals → exercise → target goals, etc.) with its grantees (Sosin, 2004). The goals of public health preparedness are a distinct subset of overall preparedness, and public health, as noted elsewhere in this report, has its unique capabilities, responsibilities, and information needs. The use of exercises to measure performance and public health preparedness will differ from their use in other fields in the processes being evaluated, in the skills and knowledge being assessed, in the specific relationships and coordination being tested. However, there will be areas of overlap with other disciplines and programs, and there will be
some commonalities in structure and operations (e.g., a type of emergency operations center and/or other mechanisms for interagency collaboration and coordination, communication activities, information infrastructure). It is important to ensure that planning, conduct, and evaluation of public health exercises at the federal, state, and local levels are compatible with those of DHS activities under the HSEEP. For example, HSEEP describes three levels of performance evaluation: task level performance (individual); agency/discipline/function-level performance; and mission-level performance (interagency, interorganizational, and community) (DHS/ODP, 2003).
The HSEEP Exercise Evaluation Guides for tabletop and operational exercises include public health personnel/agencies under the “response element” heading in addition to EMS, law enforcement, fire department, HazMat, hospitals, and others, and though the exercise methodology indicates that public health is one of the agencies/disciplines/functions to be evaluated in HSEEP, it understandably does not go into detail. If CDC intends to coordinate with or make its public health exercise evaluation compatible with the HSEEP model, the committee suggests that existing resources, such as the Public Health Competencies for Bioterrorism and Emergency Preparedness be utilized in customizing the individual-level evaluation and that the Local and State Public Health Preparedness and Response Capacity Inventories be included in customizing the agency-level evaluation.
In preceding pages, the committee has explored the potential of proxy events and exercises as means to performance measurement. The committee recommends that CDC should use the Evidence-Based Performance Goals for Public Health Disaster Preparedness to develop standards against which CDC, states, and localities may regularly measure their performance in exercises and in response to proxy events. Public health agency performance in exercises and proxy events should be used to identify gaps in preparedness and to improve planning, communication, and coordination at the agency and inter-agency levels, as part of a process of continuous quality improvement in preparedness planning and response. Preparedness drills and exercises should not be evaluated individually, but their cumulative and long-term impact on preparedness, such as generalizability to other potential hazards, must be considered in the evaluation process.
In closing, the committee encourages CDC to learn from the experience and research available from other fields, including, but not limited to disaster research and emergency and disaster response, and to develop the evidence base specific to public health preparedness; strengthen and sustain
active coordination and communication with all relevant entities and government agencies at the federal, state, and local levels; and focus on continuous improvement in planning and performance to further the process and the goal of preparedness. The committee wishes to thank you for the continuing opportunity to be of assistance to the Centers for Disease Control and Prevention and its partners as they work to protect the nation’s health.
Brian L. Strom,
Kristine M. Gebbie,
Committee Vice Chair
Robert B. Wallace,
Committee Vice Chair
Committee on Smallpox Vaccination Program Implementation
Advisory Panel to Assess Domestic Response Capabilities for Terrorism Involving Weapons of Mass Destruction. 2003. Fifth Annual Report to the President and the Congress. Arlington, VA: The RAND Corporation.
AHRQ (Agency for Healthcare Research and Quality). 2004. Training of Hospital Staff to Respond to a Mass Casualty Incident. AHRQ Evidence Report/Technology Assessment Number 95. [Online] Available at http://www.ahrq.gov/clinic/epcsums/hospmcisum.pdf. Accessed June 10, 2004.
Alexander D. 2000. Scenario methodology for teaching principles of emergency management. Disaster Prevention and Management 9(2):89-97.
Alexander D. 2003. Towards the development of standards in emergency management training and education. Disaster Prevention and Management 12(2):113-123.
Auf der Heide E. 1989. Disaster Response: Principles of Preparation and Coordination. St. Louis, MO: CV Mosby.
Benjamin G. 2003. SARS: How Effective Is the State and Local Response? Testimony on Behalf of the American Public Health Association Submitted to the Senate Government Affairs Committee, Permanent Committee on Investigations. [Online] Available at http://www.apha.org/legislative/testimonies/sars52103.htm. Accessed May 11, 2004.
Borio L. 2004. Transcript from the IOM’s Committee on Smallpox Vaccination Program Implementation Meeting Five on March 29, 2004, Washington, DC.
Bozzette S, Boer R, Bhatnagar V, Brower J, Keeler E, Morton S, Stoto M. 2003. A model for a smallpox-vaccination policy. New England Journal of Medicine 348(5):416-425.
Burkle F, Hayden R. 2001. The concept of assisted management of large-scale disasters by horizontal organizations. Prehospital and Disaster Medicine 16(3):87-96.
Butler J, Cohen M, Friedman C, Scripp R, Watz C. 2002. Collaboration between public health and law enforcement: new paradigms and partnerships for bioterrorism planning and response. Emerging Infectious Diseases 8(10):1152-1156.
CBACI (Chemical and Biological Arms Control Institute). 2002. What Should We Know? Whom Should We Tell?: Leveraging Communication and Information to Counter Terrorism and Its Consequences. [Online] Available at http://www.cbaci.org/summary.pdf and http://www.mipt.org/cbacilevcomm.asp. Accessed May 11, 2004.
CDC (Centers for Disease Control and Prevention). 2003a. Continuation Guidance for Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism—Budget Year Four Program Announcement 99051.
CDC. 2003b. Forensic Epidemiology: Joint Training for Law Enforcement and Public Health Officials on Concurrent Investigative Responses to Bioterrorism Threats and Attacks. [Online] Available at http://www.phppo.cdc.gov/od/phlp/docs/For%20Epi%20Factsheet.pdf. Accessed May 25, 2004.
CDC. 2004a. Continuation Guidance for Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism—Budget Year Five. Program Announcement 99051, p. 5. [Online] Available at http://www.bt.cdc.gov/planning/continuationguidance/index.asp. Accessed June 29, 2004.
CDC. 2004b. Continuation Guidance for Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism—Budget Year Five. Attachment H: Cross-cutting Benchmarks and Activities, p. 1. [Online] Available at http://www.bt.cdc.gov/planning/continuationguidance/index.asp. Accessed June 29, 2004.
CDC and ODP. 2003. 2003 Statewide Homeland Security Grants Program: Frequently Asked Questions. [Online] Available at www.phppo.cdc.gov/HomelandSecGrants/index.asp. Accessed May 11, 2004.
Clements B, Evans R. 2004. Bioterrorism preparedness coordination: an ataxic saga continues. Public Health Reports 119:16-18.
Council on Linkages between Academia and Public Health Practice. 2004. National Public Health Systems Research Agenda. [Online] Available at http://www.phf.org/Link/projects.htm#NationalPHSRAgenda. Accessed May 25, 2004.
DHS (Department of Homeland Security). 2003. FY 2004 ODP Homeland Security Grant Program. [Online] Available at http://www.ojp.usdoj.gov/docs/fy04hsgp.pdf and http://www.dhs.gov/dhspublic/interapp/editorial/editorial_0356.xml. Accessed May 11, 2004.
DHS. 2004a. Office for Domestic Preparedness. [Online] Available at http://www.ojp.usdoj.gov/odp. Accessed on June 9, 2004.
DHS. 2004b. Office of Domestic Preparedness: A Better Prepared America Through Practice and Preparation. [Online] Available at http://www.ojp.usdoj.gov/odp/exercises.htm. Accessed June 9, 2004.
DHS. 2004c. Summary of Meeting #4—Public Session (U.S. Department of Homeland Security, Homeland Security Advisory Council). [Online] Available at http://www.dhs.gov/interweb/assetlibrary/HSAC_Minutes033104.pdf. Accessed June 1, 2004.
DHS/ODP (DHS/Office of Domestic Preparedness). 2003. HSEEP, Volume II: Exercise Evaluation and Improvement. [Online] Available at http://www.ojp.usdoj.gov/odp/docs/hseep.htm. Accessed April 2, 2004.
DHS/ODP. 2004a. Homeland Security Exercise and Evaluation Program (HSEEP), Volume I (Revised May 2004). [Online] Available at http://www.ojp.usdoj.gov/odp/docs/hseep.htm. Accessed May 25, 2004.
DHS/ODP. 2004b. Homeland Security Exercise and Evaluation Program. Presented at the Meeting of the IOM Committee on Smallpox Vaccination Program Implementation on March 29, 2004.
Drabek T, McEntire D. 2002. Emergent phenomena and multiorganizational coordination in disasters: lessons from the research literature. International Journal of Mass Emergencies and Disasters 20(2):197-224.
Duncan S, Scott S, Duncan C. 1994. Modeling the different smallpox epidemics in England. Philosophical Transactions: Biological Sciences 346(1318):407-419.
Eichner M and Dietz K. 2003. Transmission potential of smallpox: estimates based on detailed data from an outbreak. American Journal of Epidemiology 158:110-117.
Elliott V. 2004. Measles Cases Trigger Actions in 9 States. AMNews, April 26, 2004. [Online] Available at http://www.ama-assn.org/amednews/2004/04/26/hlsb0426.htm. Accessed May 1, 2004.
FEMA (Federal Emergency Management Agency). 2004. FEMA’s Involvement in Exercises: A Historical Perspective. [Online] Available at http://www.fema.gov/rrr/section1.shtm. Accessed March 2004.
FEMA/EMI (Federal Emergency Management Agency/Emergency Management Institute). 2003. IS 139: Exercises Design. Unit 1: Introduction to Exercise Design. [Online] Available at http://training.fema.gov/EMIWeb/IS/is139lst.asp. Accessed April 1, 2004.
Ferguson N, Keeling M, Edmunds W, Gani R, Grenfell B, Anderson R, Leach S. 2003. Planning for smallpox outbreaks. Nature 425:681-685.
Gani R, Leach S. 2001. Transmission potential of smallpox in contemporary populations [Letter]. Nature 414(6865):748-751.
GAO (U.S. General Accounting Office). 2001. Combating Terrorism: Selected Challenges and Related Recommendations. GAO-01-822. Washington, DC.
GAO. 2003a. Major Management Challenges and Program Risks—Department of Homeland Security (Performance and Accountability Series). GAO-03-102. Washington, DC.
GAO. 2003b. Major Management Challenges and Program Risks—Department of Health and Human Services (Performance and Accountability Series), Report No. GAO-03-101. Washington, DC.
GAO. 2003c. Bioterrorism: Public Health Response to Anthrax Incidents of 2001. GAO-04-152. Washington, DC.
Gerber M. 2002, April. Anthrax Delay Blamed on Lack of Ties. The Hill.
Giovachino M, Carey N. 2001. Modeling the consequences of bioterrorism response. Military Medicine 166(11):925-930.
Granot H. 1999. Emergency inter-organizational relationships. Disaster Prevention and Management 8(1):21-26.
Halloran M, Longini I, Nizam A, Yang Y. 2002. Containing bioterrorist smallpox. Science 298:1428-1432.
Hilhorst D. 2003. Responding to Disasters: Diversity of Bureaucrats, Technocrats and Local People. International Journal of Mass Emergencies and Disasters 21(1):37-55.
Hupert N, Mushlin A, Callahan M. 2002. Modeling the public health response to bioterrorism: using discrete event simulation to design antibiotic distribution centers. Medical Decision Making 22(5 Suppl):S17-S25.
IAEM (International Association of Emergency Managers). 2003. Matrix of Federal All-Hazards Grants. [Online] Available at www.iaem.com/grantmatrix.pdf. Accessed June 17, 2003.
IOM (Institute of Medicine). 2003a. Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report #5. Washington, DC: The National Academies Press.
IOM. 2003b. Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report #1. Washington, DC: The National Academies Press.
IOM. 2003c. Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report #2. Washington, DC: The National Academies Press.
IOM. 2003d. The Future of the Public’s Health in the 21st Century. Washington, DC: The National Academies Press.
Kahsai D, Kare J. 2002. Prehospital disaster management: implications for weapons of mass destruction. Topics in Emergency Medicine 24(3):37-43.
Kaplan E, Craft D, Wein L. 2003. Analyzing bioterror response logistics: the case of smallpox. Mathematical Biosciences 185:33-72.
Kelkenberg K. 2004. Transcript from the IOM’s Committee on Smallpox Vaccination Program Implementation Meeting Five, March 29, 2004, Washington, DC:221-253.
Kiel D. 1995. Chaos Theory and Disaster Response Management: Lessons for Managing Periods of Extreme Instability. What Disaster Response Management Can Learn from Chaos Theory . In Koehler, GA, ed. California Research Bureau Conference Proceedings. May 18-19, 1995.
Kreps G, Bosworth S. 1999. Disaster, organizing, and role enactment: a structural approach. American Journal of Sociology 99(2):428-463.
Kuhr S, Hauer J. 2001. Intergovernmental Preparedness and Response to Potential Catastrophic Biological Terrorism. In Novick LF, Marr JS, eds. Public Health Issues in Disaster Preparedness: Focus on Bioterrorism, pp. 104-110. Gaithersburg, MD: Aspen Publishers Inc.
Landesman LY. 2001. Public Health Management of Disasters: The Practice Guide. Washington, DC: American Public Health Association.
Landesman L, Malilay J, Bissell R, Becker S, Roberts L, Ascher M. 2001. Roles and Responsibilities of Public Health in Disaster Preparedness and Response. In Novick LF, Marr JS, eds. Public Health Issues in Disaster Preparedness: Focus on Bioterrorism, pp. 1-56. Gaithersburg, MD: Aspen Publishers Inc.
Meltzer M. 2003. Risks and benefits of preexposure and postexposure smallpox vaccination. Emerging Infectious Diseases 9(11):1363-1370.
Meltzer M, Damon I, LeDuc J, Millar J. 2001. Modeling potential responses to smallpox as a bioterrorist weapon. Emerging Infectious Diseases 7(6):959-969.
Moscoso E. 2004, April 23. Terror Threat: Bureaucratic Babble. Atlanta Journal-Constitution, 1A.
NRC (National Research Council). 2003. The Emergency Manager of the Future: Summary of a Workshop—June 13, 2003, Washington, DC. Washington, DC: The National Academies Press.
Office of the Federal Register. 2003. Smallpox Vaccine Injury Compensation Program: Smallpox (Vacccinia) Vaccine Injury Table. Federal Register 68(166):51492-51499.
Ornstein C. 2001, November 11. FBI, Public Health Agencies Often at Odds Over Anthrax. Houston Chronicle (reprint from Los Angeles Times). [Online] Available at http://www.chron.com/cs/CDA/printstory.mpl/special/terror/front/1127841. Accessed May 20, 2004.
Perry R. 2003. Municipal terrorism management in the United States. Disaster Prevention and Management 12(3):190-202.
Peters KE, Drabant E, Elster AB, Tierney M, Hatcher B. 2001. Cooperative Actions for Health Program: Lessons Learned in Medicine and Public Health Collaboration. Chicago, IL, and Washington, DC: American Medical Association and American Public Health Association.
Quarantelli E. 1988. Local Emergency Management Agencies: Research Findings on Their Progress and Problems in the Last Two Decades. University of Delaware Disaster Research Center.
Quarantelli E. 1994. Draft of a Sociological Disaster Agenda for the Future: Theoretical, Methodological and Empirical Issues. University of Delaware Disaster Research Center Preliminary Papers (#228). [Online] Available at http://www.udel.edu/DRC/preliminary/228.pdf.
Quarantelli E. 1995. Disaster Planning, Emergency Management, and Civil Protection: The Historical Development and Current Characteristics of Organized Efforts to Prevent and to Respond to Disasters. Available at http://www.udel.edu/DRC/prepapers.html.
Quarantelli E. 2004. Transcript from the IOM’s Committee on Smallpox Vaccination Program Implementation Meeting Five on March 29, 2004, Washington, DC.
Richards E. 2002. Collaboration between public health and law enforcement: the constitutional challenge. Emerging Infectious Diseases 8(10):1157-1159.
Salinsky E. 2002. Will the Nation Be Ready for the Next Bioterrorism Attack? Mending Gaps in the Public Health Infrastructure. NHPF Issue Brief No. 776.
Schweitzer R. 2004. Transcript from the IOM’s Committee on Smallpox Vaccination Program Implementation Meeting Five on March 29, 2004, Washington, DC:265.
Simpson D. 2002. Earthquake drills and simulations in community-based training and preparedness programmes. Disasters 26(1):55-69.
Sosin D. 2004. Transcript from the IOM’s Committee on Smallpox Vaccination Program Implementation Meeting Five on March 29, 2004 , Washington, DC.
Staiti A, Katz A, Hoadley J. 2003. Has bioterrorism preparedness improved public health? Issue Brief: Findings from HSC, No. 65.
Stoll M, Lee H. 2003. State health department develops SARS emergency response plan. Virginia Department of Health News Release, June 23, 2003.
Tang N, Fabbri W. 2003. Medical direction and integration with existing EMS infrastructure. Topics in Emergency Medicine 25(4):326-332.
Taylor N. 2003. Review of the Use of Models in Informing Disease Control Policy Development and Adjustment. Reading, UK: University of Reading, School of Agriculture, Policy and Development. [Online] Available at http://www.defra.gov.uk/science/Publications/2003/UseofModelsinDiseaseControlPolicy.pdf. Accessed April 1, 2004.
Tierney K. 1993. Disaster Preparedness and Response: Research Findings and Guidance from the Social Science Literature. University of Delaware Disaster Research Center Preliminary Paper #193. [Online] Available at http://www.udel.edu/DRC/preliminary/193.pdf. Accessed April 25, 2004.
Tierney K, Lindell M, Perry R. 2001. Facing the Unexpected: Disaster Preparedness and Response in the United States. Washington, DC: Joseph Henry Press.
WADEM (World Association of Disaster and Emergency Medicine). 2001. Conceptual Framework of Disasters (Part III, Chapters 3-7). In Health Disaster Management: Guidelines for Evaluation and Research in the Utstein Style. [Online] Available at http://wadem.medicine.wisc.edu. Accessed June 1, 2004.
Werner L, Naud M, Kellogg A. 1998. Quick Response Report #11—Public Health Emergency Response: Evaluation of Implementation of a New Emergency Management System for Public Health in the State of Georgia. University of Colorado Natural Hazards Center. [Online]. Available at http://www.colorado.edu/hazards/qr/qr111/qr111.html. Accessed May 12, 2004.
White House. 2001. Fact Sheet: Homeland Security Council. [Online] Available at www.whitehouse.gov/news/releases/2001/10/print/20011029-16.html. Accessed April 10, 2004.
LETTER REPORT #6, APPENDIX 6-A SUMMARY OF RECOMMENDATIONS
The committee recommends that all federal entities concerned with bioterrorism preparedness (e.g., CDC, the Health Resources and Services Administration, the Office of Domestic Preparedness [ODP]) should more actively coordinate guidance and funding activities. Federal agencies should also work together to develop mechanisms that facilitate coordination and collaboration among their grantees at the state and local levels. Such mechanisms may include, but are not lim-
ited to, regular meetings to familiarize CDC and ODP program staff with each other’s program priorities and activities, a database for informing ODP and other partners of exercises planned by CDC grantees, etc. Federal coordination efforts should also include the clarification of primary responsibility and authority in bioterrorism events, to ensure that CDC can fulfill its unique role as the nation’s public health agency.
The committee recommends that CDC should collaborate with all of its partners to strengthen preparedness by applying research findings and experience in public health emergency response, bioterrorism preparedness, and disaster management. In order to strengthen the evidence base for public health preparedness, CDC should:
Strengthen the link between public health research and practice;
Participate in and promote interdisciplinary research about preparedness;
Support a system to assure the ongoing collection, synthesis, and sharing of lessons learned and best practices from public health preparedness exercises and public health response to proxy events; and
In coordination with the appropriate federal-level partners, such as the Agency for Healthcare Research and Quality, evaluate the effectiveness, design, and opportunity costs of preparedness strategies, such as exercises.
The committee recommends that CDC should use the Evidence-Based Performance Goals for Public Health Disaster Preparedness to develop standards against which CDC, states, and localities may regularly measure their performance in exercises and in response to proxy events. Public health agency performance in exercises and proxy events should be used to identify gaps in preparedness and to improve planning, communication, and coordination at the agency and interagency levels, as part of a process of continuous quality improvement in preparedness planning and response. Preparedness drills and exercises should not be evaluated individually, but their cumulative and long-term impact on preparedness, such as generalizability to other potential hazards, must be considered in the evaluation process.
LETTER REPORT #6, APPENDIX 6-B ACRONYMS AND GLOSSARY
Centers for Disease Control and Prevention
Community Hazards Emergency Response Capability Assurance Program
Chemical Stockpile Emergency Preparedness Program
Department of Health and Human Services
Department of Homeland Security
Emergency Medical Services
Epidemic Assistance Investigation
Federal Bureau of Investigations
Federal Emergency Management Agency
General Accounting Office
Health Alert Network
Homeland Security Advisory Council
Homeland Security Council
Health Resources and Services Administration
Homeland Security Exercise Evaluation Program
Incident Command System
Lessons Learned Information Sharing (www.llis.org)
Memorial Institute for the Prevention of Terrorism
Office of Domestic Preparedness
Policy Coordination Committee (of the HSC)
Radiological Emergency Preparedness Program
Weapons of Mass Destruction
generally contrasted with “agent-specific,” refers to a broad preparedness and response approach to all possible hazards to population health and safety, whether the complete range of known disasters, or specifically the complete range of public health disasters (from naturally occurring to deliberately introduced).
phenomena caused by natural, technological, or deliberate causes. Term is sometimes used interchangeably with emergency, although the two are not only quantitatively but also qualitatively different. A key difference is that while emergencies call upon largely local resources and response, disasters are of sufficient magnitude to require external resources and personnel for response and recovery (Mothershead, 2003).
similar to exercises, but more narrowly focused activities used for training, testing, and refining capacities, and frequently involving a specific area of preparedness within only one agency rather than more complex processes and relationships at an interagency level.
a title used for increasingly professionalized personnel in local or state government who are charged with coordinating or overseeing the jurisdiction’s multiagency response to an emergency or disaster.
Emergency responder/first responder/traditional emergency responder:
term refers to a set of disciplines and responsibilities, including, but not limited to Emergency Medical Services (EMS), fire, law enforcement, hazardous materials specialists, etc. Personnel in such agencies and the practitioners of such disciplines prepare for emergencies and disasters and are responsible for carrying out response when emergencies and disasters occur.