Creating Situational Awareness: A Systems Approach
The following is a white paper prepared for the June 10–11, 2009, workshop on medical surge capacity, hosted by the Institute of Medicine Forum on Medical and Public Health Preparedness for Catastrophic Events. All opinions expressed in this paper are those of the author and not necessarily of the Institute of Medicine.
By Eric S. Toner, M.D.
Center for Biosecurity, University of Pittsburgh Medical Center
WHAT IS SITUATIONAL AWARENESS?
Put simply, situational awareness means understanding what is going on around you. But there is more to this statement than first meets the eye. Understanding is more than information gathering. It implies gathering the right information (all that is needed, but not too much), being able to analyze it, and making projections based on the analysis. In the best of all worlds, it also means being able to do something with the information (i.e., it is useful information).
The first references to the term come from the U.S. Air Force, after the Korean War, and relate to the understanding of the enemy that a fighter pilot needs in an aerial dogfight. The pilot needs to know not only where the enemy plane is, but what its next move will be. This involves gathering information, analyzing it, and making projections based on that analysis. This was described by Col. John Boyd as the “observe-orient-decide-act loop,” or OODA loop, also called the Boyd cycle. To win a
dogfight, the pilot must “get inside” the opponent’s loop; losing one’s own situational awareness was called being “out of the loop.”1
The term is still used extensively in aviation, and it encompasses the idea of understanding the entire environment of the aircraft, both inside the cockpit and out—what the instruments are indicating, what air traffic control is saying, and what is visible through the windshield. The term is also extensively used by the military. According to the Army Field Manual 1-02 (September 2004), situational awareness is:
Knowledge and understanding of the current situation which promotes timely, relevant and accurate assessment of friendly, competitive and other operations within the battlespace in order to facilitate decision making. An informational perspective and skill that fosters an ability to determine quickly the context and relevance of events that are unfolding.2
Again the emphasis is on understanding actionable information.
What does situational awareness mean in the healthcare context? The concept has been applied to the analysis of patient safety and healthcare quality issues.3 However, the most frequent use of the term in health care relates to emergency management and is often used in reference to computer systems to aggregate data in an emergency operations center, or to collect and transmit disease surveillance data. These systems are useful tools, and may even be essential tools, but they do not in and of themselves provide situational awareness. To achieve situational awareness, the right information (without a lot of noise) is needed at the right time, and the right person is prepared to receive it, is capable of analyzing it, and is then able to do something useful with it.
This raises a number of issues, the most fundamental of which is what information is actually needed in a disaster? What information really makes a difference? In fact, the information needed probably varies with the type of event, and different actors involved in emergency response need different information. For example, the physician in the emergency department needs different information than the state’s Director of Emergency Management or the Secretary of Health and Human Services (HHS).
THE CURRENT STATE OF THE ART OF HEALTHCARE SITUATIONAL AWARENESS
The following are some of the existing systems and programs that relate to healthcare situational awareness. It is not a comprehensive list, but it illustrates the diversity of types of systems. For the most part, these systems were designed as stand-alone systems and are not integrated.
Disease Detection and Surveillance
BioWatch (the United States government system to detect certain bioterrorism agents in the air)4
BioSense (the United States government system to gather syndromic surveillance data from hospitals)5
The Centers for Disease Control and Prevention’s (CDC’s) multiple influenza surveillance systems (e.g., ILInet, Emerging Infections Program)9
The World Health Organization’s Global Influenza Surveillance Network (GISN)10
Traditional public health disease surveillance, case investigation, and contact tracing
Laboratory reporting systems
News and Web Trawling
ProMED (distributes disease reports submitted from around the world)
Global Public Health Intelligence Network (GPHIN) (mines global news for disease reports)11
Google Flu Trends system12
Health Alert Network (sends messages from the CDC and state health departments to clinicians)13
Many home-grown or off-the-shelf systems within hospitals
Many systems for reporting bed data to local, state, and federal governments
Many systems to track patients within hospitals
Many systems to track emergency medical services patients
Incident Command Systems
Web emergency operations center (EOC) and others
Electronic Health Records
Within hospitals or clinicians’ offices
In addition, there are emergency operations or information fusion centers at the local, state, and federal levels whose purpose is to merge the various streams of information. These include local and state EOCs, the Secretary’s Operations Center (SOC) in HHS,14 the CDC Director’s Emergency Operations Center,15 the CDC’s BioPHusion Program,16 and the National Biosurveillance Integration Center (NBIC) in the Department of Homeland Security.17 Another center, the National Biosurveillance Integration System (NBIS) created to connect various surveillance streams and agencies.18
Although these surveillance efforts undoubtedly provide information flows that did not exist before, it is not clear to what extent they have enabled a more robust understanding of a rapidly unfolding event. How do, or how can, these diverse systems, programs, and centers work together to provide an integrated picture? Can decision makers use these systems effectively to direct action in a crisis? Do these systems provide the necessary information in real-time? It seems overall that there has been much more emphasis placed on systems to detect outbreaks rather than on systems to manage outbreaks. At this point it appears that there is
a lot of technology, but relatively little science. Systems exist without a clear concept of operation.
Several entities have or are currently studying these issues, including:
The National Biosurveillance Advisory Subcommittee (NBAS) Task Force (mandated by HSPD-21)
Institute of Medicine, or IOM (mandated by Congress)
Government Accountability Office, or GAO (mandated by Congress)
SITUATIONAL AWARENESS AND NOVEL H1N1: WHAT DID WE LEARN?
The current novel H1N1 epidemic has provided a real-life test of our situational awareness capabilities. One thing that we certainly learned from this outbreak is that situational awareness is critically important—it drives policy decisions. Decisions regarding school closings, personal protective equipment guidance, and antiviral use are all dependent on knowing key characteristics of the epidemic in real-time. These characteristics include the severity of illness, the basic epidemiology (e.g., R value, serial interval), the transmission characteristics, and the extent of dissemination of the disease in the community. Traditionally, these characteristics have been determined in retrospect after careful epidemiological investigation; however, intervention strategies now in place presume knowledge of these characteristics. Therefore, as we have seen, in a nascent pandemic that arrived in the United States without warning, this information must be estimated in real-time if interventions are to be attempted. The same would be true for other contagious diseases, such as Severe Acute Respiratory Syndrome. To estimate these characteristics, one must be able to identify those who have died, those who are seriously ill, and those with mild disease. This requires a capability for rapid and reliable diagnostic testing, near real-time disease surveillance, and the ability to quickly reach down to the bedside to get clinical information.
In the current outbreak, we found that that there was inadequate capacity for accurate rapid diagnostic testing. Rapid antigen tests were unreliable (insensitive and non-specific). While polymerase chain reaction (PCR) was useful (“untypable influenza A” had very high predictive
value), PCR is not available in many clinical labs. Moreover, laboratories had difficulty keeping up with the volume of tests.
We also learned that systems for syndromic surveillance did not seem to provide an adequate picture of the scope of the outbreak quickly enough. The CDC’s ILInet data on outpatient visits to sentinel physician offices has traditionally been reported weekly—not timely enough to inform intervention decisions that must be made in a matter of days. Anecdotally, local syndromic surveillance systems either did not show a spike of ILI or showed a “false” spike—worried well and well people seeking testing. Reportedly, BioSense data corroborated the ILInet data, but whether the data provided further useful information is not yet clear.
One unexpected finding is that apparently a large outbreak of clinically mild disease can fly under the surveillance radar because most surveillance systems are designed to look for people sick enough to seek medical care. Some systems collect data on other surge indicators such as the volume of purchase of over-the-counter medication, but it is not known if these systems have been useful in the current outbreak. The Google Flu Trends system that tracks influenza-related Internet searches revealed only a minimal spike. Why do we care about people not sick enough to see a doctor? Because that information is needed to understand the severity and epidemiology of the outbreak and this understanding drives important policy decisions.
One innovative and apparently successful attempt to quickly assess a localized outbreak was the use of Survey Monkey at St. Francis School in Queens, New York. Students and staff were queried by e-mail about influenza-like symptoms. This afforded a very quick determination of the rough scope of the outbreak.19
There was also a need to quickly get clinical information about hospitalized patients—both to understand the severity of the disease and to guide treatment. This seems to have been a particular problem with the cases in Mexico. Key questions to which answers were needed early on include: What were the reasons for hospital admissions? What are the causes of death? Are there particular risk factors for serious illness? What treatments seem to have been effective and what did not? The data collection methodology for this information seems straightforward, at least in the early stages of an outbreak: a simple telephone call to the physician caring for the patient. A process for collating the data, analyzing it, and feeding it back to clinicians quickly is also needed. Real-time clinical trials may also be needed in an outbreak of an unknown disease.
Although it has not yet been a major issue in the current epidemic, states and the federal government need a process to know what is going on at the ground level in hospitals and other healthcare facilities. They need this information to know how best to deploy their limited resources and to identify critical choke points that they may be able to alleviate. This raises the question of what the key data elements are that indicate a hospital or healthcare system is under severe stress. In other words, what are the vital signs for the healthcare system? How can that data be obtained easily and in near real-time? One approach may be to use existing and developing healthcare coalitions to “take the pulse” of their member organizations and report information to the state and federal governments.
There is also a need to collect healthcare facility data to feed into a cycle of continuous improvement. Understanding how hospitals and other facilities performed during this crisis should lead to refinements of the Hospital Preparedness Program and other preparedness programs.
GOING FORWARD: RECOMMENDED SHORT-TERM GOALS
Over the next several months, before the anticipated next wave of the pandemic begins in the fall, it is important to improve the government’s ability to quickly acquire key information needed to manage the response to the epidemic. This includes:
Assessing the number of mild flu cases so that the severity of the epidemic can be estimated. This can be accomplished by expanding or enhancing existing influenza surveillance systems and by using existing tools like Survey Monkey to do very quick cohort studies.
Accessing information on hospitalized patients. This can be accomplished by enhancing existing hospital influenza surveillance systems and assigning clinicians to do very rapid telephone investigations of hospitalized cases.
Identifying “vital signs” for hospitals and healthcare systems and figuring out how to access them.
RECOMMENDED LONG-TERM GOALS
A more integrated approach to federal biosurveillance programs is needed that is based on clearly defined missions, goals, and priorities. This approach should include a scientific analysis of what information is needed to manage various health emergencies, interoperability among various surveillance systems, and a process for continuous improvement of the systems through rigorous evaluation of events and exercises.
The federal and state governments should harness the momentum toward universal health information technology to improve digital linkages between public health and hospitals so as to improve public health access to key clinical data.
The federal government should invest in increasing the surge capacity of clinical and public health laboratories and in the development and dissemination of rapid diagnostic tests.
The hospital Preparedness Program (HPP) should continue to promote the development of healthcare coalitions for a variety of reasons, one of which is to provide a mechanism to collect critical information from hospitals and other healthcare facilities. The HPP, along with the Centers for Medicare & Medicaid Services and the CDC, should continue to promote digital linkages between hospitals and health departments.
1. Hillaker, H. July 1997. CodeOne Magazine. Available at: http://www.codeonemagazine.com/archives/1997/articles/jul_97/july2a_97.html (accessed May 21, 2009).
2. Deputy Under Secretary of the Army Knowledge Center. August 25, 2009. Available at: http://www.army.mil/armyBTKC/focus/sa/about.htm (accessed May 21, 2009).
3. Singh, H., Petersen, L. A., Thomas, E. J.2006. Understanding diagnostic errors in medicine: A lesson from aviation. Quality and Safety in Health Care 15:159–164. Available at: http://qshc.bmj.com/cgi/reprint/15/3/159?ijkey=3bb3b466c596dbd093ab32c5211beacfc58d9cac (accessed May 21, 2009).
4. Lister, S., Shea, D. November 19, 2003. The BioWatch Program: Detection of bioterrorism. Congressional Research Service Report
No. RL 32152. Available at: http://www.fas.org/sgp/crs/terror/RL32152.html (accessed May 21, 2009).
5. CDC (Centers for Disease Control and Prevention). 2009. BioSense. Available at: http://www.cdc.gov/BioSense (accessed May 21, 2009).
6. University of Pittsburgh. 2009. RODS. Available at: https://www.rods.pitt.edu/site/ (accessed May 21, 2009).
7. Lombardo, J., Burkom, H., Pavlin, J. 2007. ESSENCE II and the Framework for Evaluating Syndromic Surveillance Systems. MMWR 53(supplement):159–165. Available at: http://www.cdc.gov/mmwr/preview/mmwrhtml/su5301a30.htm (accessed May 21, 2009).
8. Children’s Hospital Informatics Program. 2009. Automated Epidemiological Geotemporal Integrated Surveillance System webpage. Available at: http://aegis.chip.org (accessed May 21, 2009).
9. CDC. 2009. Flu Activity and Surveillance: Reports and Surveillance Methods in the United States webpage. Available at: http://www.cdc.gov/flu/weekly/fluactivity.htm (accessed May 21, 2009).
10. WHO Global Influenza Surveillance Network webpage. 2009. Available at: http://www.who.int/csr/disease/influenza/surveillance/en (accessed May 21, 2009).
11. Keller, M., Blench, M., Tolentino, H., et al. 2009. Use of unstructured event-based reports for global infectious disease surveillance. Emerg Inf Dis 15(5). Available at: http://www.cdc.gov/edi/content/15/5/689.htm (accessed May 21, 2009).
12. Google.org Flu Trends. 2009. Available at: http://www.google.org/flutrends (accessed May 21, 2009).
13. CDC. 2009. Health Alert Network webpage. Available at: http://www2a.cdc.gov/han/Index.asp (accessed May 21, 2009).
14. HHS. 2009. HHS Concept Of Operations Plan (CONOPS). Availabile at: http://www.hhs.gov/disasters/discussion/planners/conops.html (accessed May 21, 2009).
15. CDC. 2009. Coordinating Office of Terrorism Preparedness and Emergency Response webpage. Available at: http://www.cdc.gov/about/organization/cotper.htm (accessed May 21, 2009).
16. Lenert, L. 2009. Real-Time Biosurveillance: Strategy and Approach. CDC PowerPoint presentation. Available at: www.cdc.gov/biosense/files/BioSense_talk_3.19.ppt (accessed May 21, 2009).
17. Department of Homeland Security. The Office of Weapons of Mass Destruction and Biodefense webpage. Available at: http://www.dhs.gov/xabout/structure/gc_1205180907841.shtm
(accessed May 21, 2009).
18. Margetta, R. August 13, 2007. Leadership shortfalls hinder biosurveillance system, IG says. CQ.com. Available at: http://public.cq.com/docs/hs/hsnews110-000002571374.html (accessed May 21, 2009).
19. St. Francis Prep Update: Swine flu outbreak. April 30, 2009. Available at: http://www.nyc.gov/html/doh/downloads/pdf/cd/h1n1_stfrancis_survey.pdf (accessed May 21, 2009).