In the final session of the workshop, four speakers presented elements akin to a concept of operations—or CONOPS—for the Department of Homeland Security’s National Biosurveillance Integration Center (NBIC), based in part on their analysis of the preceding discussions. Each presented a slightly different view, providing a multidimensional perspective. In this way, they were able to summarize and highlight some important points made from the previous day and a half of presentations and discussion, as well as add their own expertise to the issues.
As standardized by the Institute of Electrical and Electronics Engineers, a CONOPS document has seven components, said Leslie Lenert, University of Utah School of Medicine:
1. Scope 2. Reference documents
3. Current system or situation
4. Justification for and nature of changes
5. Concepts for the proposed system
6. Operational scenarios
7. Summary of impacts
1This section is based on the presentation by Leslie Lenert, University of Utah School of Medicine.
Though NBIC was created in 2004, it has not yet developed the system specifications defined in a CONOPS document. It needs to do so, said Lenert, for several reasons.
First, a CONOPS would define the National Biosurveillance Integration System (NBIS) within which NBIC operates. The workshop uncovered a great deal of information about how agencies communicate with each other, but it did not reveal a game plan for how NBIS operates. “We need an overall strategy for that,” said Lenert.
Second, a CONOPS would define the strategy with which NBIC adds value to NBIS. NBIC can add value in many different ways, said Lenert, but it needs to demonstrate its ability to do so.
Third, a CONOPS would define the rules of engagement of NBIC when using other participants’ data. Given that trust is essential in the communication of biosurveillance data, the rules of engagement need to be clear. “If the releases of information and the political perspectives aren’t represented within the concept of operations, the trust won’t exist and the information won’t flow.”
Finally, a CONOPS would define the information products NBIC produces. In turn, these information products would determine the usefulness of NBIC.
Goals and Outcomes
The goal of NBIS was to create a system where all relevant information was collected into a central fusion center, Lenert observed. There the information would be analyzed, producing a common operations picture that can inform decisions in partner agencies and the National Operating Center. This solution was largely based on technology, according to Lenert. The largest investments were in elaborate information systems that were supposed to integrate information. But the problem was not technological, said Lenert. It was “a communications and a trust problem across different organizations.”
The authorizing legislation for the Implementing Recommendations of the 9/11 Commission Act of 2007 established the primary mission of the NBIC as follows:
To enhance the capability of the Federal Government to (A) rapidly identify, characterize, localize, and track a biological event of national concern by integrating and
analyzing data relating to human health, animal, plant, food, and environmental monitoring systems (both national and international); and (B) disseminate alerts and other information to Member Agencies and, in coordination with (and where possible through) Member Agencies, to agencies of State, local, and tribal governments, as appropriate.
However, the legislation did not empower NBIC to work top down to organize NBIS as an information supply chain. Agencies were to participate voluntarily in the NBIC subject to memoranda of understanding. Funds were provided for the data center and administrative staff but not for the subject-matter experts from the agencies, so agencies were paying for their own people to be detailed. As a result, said Lenert, “agencies had to pay to send their data to NBIC, which created a set of disincentives. . . . But NBIC persisted in operating in this way because of its mission authority from Congress.”
The situation today is problematic, Lenert said. NBIC is out of the loop of early information flows. It receives only final information products from partner agencies, does not have unique independent data sources, and has inadequate expertise and authority. An analysis of the Food and Drug Administration’s and the Department of Agriculture’s emergency operations plans reveals that NBIC has little or no role in the agencies’ operations. And NBIS governance has been ineffective in helping NBIC achieve its aims.
Trust issues also have hampered the NBIC’s mission, Lenert noted. He quoted the Government Accountability Office’s (GAO’s) 2009 investigation: “A related issue that came to light during the tabletop exercise and was a theme in interviews with NBIS officials is the extent to which NBIS partners trust NBIC to use their information and resources appropriately. According to the exercise after-action memo, participants repeatedly raised concerns about trusting NBIC with data, and participants also expressed concern that NBIC would reach the wrong conclusions or disseminate erroneous data or reports” (GAO, 2009).
Three Solution Models
Lenert described three possible solution models, which he summarized as follows:
1. Leadership: Empower NBIC to lead NBIS in a top-down fashion through Presidential leadership and “mission-based” funding to support systems integration with agencies.
2. Service: Change the focus of NBIC to that of being a service organization supporting the NBIS partners’ data fusion activities.
3. Performance Improvement: Refocus NBIS/NBIC on curation of knowledge and information from evolving events for system monitoring and performance improvement.
Lenert expressed some skepticism that the leadership model would work. The problem is cultural, and forcing the culture to change will be very difficult. “I would ask you to dismiss this solution from the beginning based on what we have heard.”
The second model is more plausible but raises the question of whether NBIC has the resources to serve this role. NBIC would have to create the information services and incentives for effective collaboration, with state and local officials also involved. In addition, other agencies have or would need data fusion centers, which would need to communicate with each other. “There is not enough money to do this in the current budget,” Lenert concluded.
The third model is the most innovative. It posits that NBIC could be a source of ground truth. It could capture and save the information state of each participating agency in NBIS on a daily basis, help other people monitor processes, find when processes are going astray, and alert the appropriate management. In essence, it could produce a metadata picture of the response process that would make it possible to identify process failures and communications, retrace the steps of diagnostic errors, and bring new people up to speed. Because it is outside the different communication chains, it could play the valuable role of improving performance for the whole system. “There is a role within its budget for NBIC as a curator of knowledge and information for performance improvement.”
NBIC cannot know all of the facts, Lenert concluded. It is a myth, he said, that all of the data can be gathered into a single place to achieve a God-like view. But NBIC could make sure that things are working according to plan and could fulfill its congressional mandate through improving the performance of the biosurveillance system.
William Stephens, Tarrant County, Texas, Advanced Practice Center, elaborated on the changes that the model proposed by Lenert would accomplish. He, too, started by defining what a CONOPS provides:
• An analysis that bridges the gap between operational needs or visions and the system developer’s technical specifications.
• Documentation of a system’s characteristics and operational needs in a manner that can be confirmed by the user without requiring any technical knowledge.
• Documentation of desires, visions, and expectations without requiring the provision of quantified, testable specifications until later in the system life cycle.
• Opportunity for quality improvement on business processes to satisfy new needs, as well as providing flexibility for satisfying anticipated business drivers.
• A mechanism to express thoughts and concerns on possible solution strategies, to record design constraints and the rationale for those constraints, and to indicate the range of acceptable solution strategies.
Stephens focused on the fourth item in this list: the opportunity for quality improvement to satisfy not only current needs but new needs. The revised system, from this perspective, would function primarily as a quality improvement engine to collect information in a central repository to capture the sequence of events, the timing of decisions, and response outcomes for system and provider utilization and improvement plans. Quality improvement is a comprehensive and quantitative way of establishing system definitions and then changing the system continuously in response to feedback, Stephens explained. “It is something that you build into your systems so that everything you do has a constant evaluation piece in [it] to see if it is meeting the objectives.” This evaluation mechanism is then used to improve the system to achieve the objectives more effectively.
2This section is based on the presentation by William Stephens, Tarrant County, Texas, Advanced Practice Center.
Stephens highlighted six operational impacts of the proposal:
Changes in procedures. Changes in procedures could provide detection and identification processes with earlier, less refined data that are shared with other stakeholders for appropriate mobilization. Information does not need to be perfect, but it needs to move with the event as more is learned. Multidisciplinary teams are needed for data collection and communication, with collaboration on common objectives and messages throughout the response. The ultimate outcome is quality improvement to achieve NBIC’s mission, said Stephens.
Use of new data sources. State and local health department data are critical, and these data depend on the people at the state and local levels. Decisions that are being made at the top level need to flow down to state and local health departments in a timely fashion. Laboratory data are often a leading indicator, and the investigation process is often accelerated by the flow of these data. But much larger quantities of data will be involved in the future—hundreds of terabytes.
Changes in quantity, type, and timing of data to be input into the system. The same data need to be shared with all partners and stakeholders. While some data sets may be translated into metadata for better understanding, there needs to be consistency across the different layers and the different agencies that are sharing data. Data need to add value to keep people engaged.
Changes in data retention requirements. Security needs to be a critical part of data acquisition, storage, and transmission. For example, many data providers are already dealing with this issue on a daily basis because of Health Insurance Portability and Accountability Act and Health Information Technology for Economic and Clinical Health provisions.
New modes of operation based on emergency, disaster, or accident conditions. The new system could be prototyped using data and experiences from regions with mature systems such as Marion County, Allegheny County, or Tarrant County. Stakeholder data structures that are already in place could be refined and adapted for expansion on a national basis. In this learning lab model, best practices and ideas are developed,
refined, and evaluated over any time from 1 to 3 years. “This could be a very good and effective model for developing the next phase of the NBIS and NBIC,” Stephens said.
Changes in operational budget. Budgets may increase but should be tied to a strategic plan and be justified by risk-benefit or return-on-investment models. Collaborative funding will be very important.
Stephens briefly described three organizational impacts:
Modification of responsibilities. There needs to be improved definition of a recognized and respected oversight agency for biosurveillance to accelerate planning and implementation of meaningful data sharing and decision processes. This requires leadership, said Stephens, that recognizes the importance of doing things differently.
Addition or elimination of job positions. Overlapping or redundant positions at higher level agencies may be eliminated to improve decision processes while resources increase for investigative and response roles.
Training or retraining. Cross-training will be needed within and across agencies for continuity of operations. This will help build a human capital system that provides a broad base of expertise for interagency teams.
Stephens concluded that crafting a CONOPS for NBIC oriented around quality improvement provides enough positive impacts to move forward. “Despite some of the challenges that it will represent, it gives a strong motivation for proceeding this way.”
Today’s biosurveillance system is built on several assumptions, said biodefense consultant Bob Kadlec. The first is that relevant data exist in the public health, medical, food, veterinary, and environmental arenas.
3This section is based on the presentation by Bob Kadlec, Biodefense Consultant.
Second, the data are readily accessible. Third, the data can be shared. Fourth, sharing would benefit the involved disciplines, agencies, and departments. And, fifth, NBIS would be a “sharing place” for biosurveillance data.
Each of these assumptions turned out to be at least partially mistaken, Kadlec observed. First, the relevant data often do not exist. “Oftentimes what you want is not what you are going to get, and what you need is not going to be available.” Furthermore, the data may not be available in a timely fashion. Epidemiology does not occur in real time. It is retrospective and based on collections of data that take time to assemble.
Agencies have an obligation to ensure that the information they provide is correct, which involves checking who collected and analyzed the data and how they were verified. More substantively, agencies do not want high-impact decisions to be made based on preliminary information. Thus, agencies are unlikely to share raw information. “A bureaucratic impediment is that knowledge is power,” said Kadlec, and “it impacts our budget.”
An additional complication is that the data do not necessarily reside in the federal government. They can reside at the state and local levels or in the private sector.
A CONOPS for NBIC needs to take national security considerations into account, said Kadlec. During a conflict, leaders rarely have clarity about what is happening. Reports from the field are generally wrong, and the “fog of war” distorts and obscures reality. Furthermore, even the simplest things are hard to do in a crisis. The reason the military tries to simplify its activities or decisions is that simple actions are much more likely to be executed successfully.
Increasing the speed of analysis increases the risk of overlooking the obvious. When a system is pressured to produce information that is only partly analyzed, mistakes are more likely.
With public health issues, decisions can quickly be elevated to the highest levels of government. Public health is increasingly considered a national security issue, and national security information is expected to be accurate and timely. Yet these characteristics are rarely available in an epidemiological investigation.
Kadlec described what he called the OODA loop, which is an acronym for “observe, orient, decide, and act,” and is a process used by military pilots to continually assess their environment as an influence in decision-making. For the U.S. government, the cycle time of the OODA loop is defined by the media. The government attempts to get ahead of a story and set policy before the media send the public in a different direction. Also, the orientation phase of an OODA loop involves existing cultural traditions, bureaucratic procedures, previous experiences, and other factors, some of which act in opposition to information that is currently coming in. Often this orientation is done by people who are not familiar with epidemiology or other factors involved in biosurveillance, particularly relevant to new information, which can lead to faulty decisions.
National security decision makers have a distinct set of priorities, according to Kadlec. They need to be in control or at least appear to be so, since it is bad politics to appear disengaged or uninformed. They need to stop the suffering, bleeding, and dying, because the people being affected “are all registered voters,” Kadlec said. They need to prevent future attacks or incidents. And they need to convey reassurance and confidence, which in turn breeds public comfort and financial market stability.
Catastrophic bioterrorism poses an immense challenge, Kadlec concluded. The 2001 anthrax attacks caused 22 illnesses and 5 deaths. An aerosol release over an American city could cause half a million illnesses, with a substantial portion of those people dying, and incur costs of trillions of dollars. Decision makers would have to respond quickly and effectively. An effective CONOPS needs to reflect these tremendous stakes.
Integrated biosurveillance is a relatively new idea that is still diffusing into practice, said Michael Wagner, University of Pittsburgh, and the rate of diffusion depends on factors such as perceived relative advantage, compatibility, and complexity. Some ideas spread quickly and plateau, while others take a long time to catch on.
The rate of diffusion also depends on the social structure in which an innovation is appearing. In the case of NBIC, 12 federal agencies are
4This section is based on the presentation by Michael Wagner, University of Pittsburgh.
involved that are expected to share information. These 12 agencies are situated within a much broader governmental context that includes not just other parts of the federal government but state and local governments. Government itself is situated with the broader health care system, food and water distribution systems, and so on. “It is no big shock that diffusion has taken as long as it has,” said Wagner.
Biosurveillance requires a focal point and authority if it is going to happen, according to Wagner. These attributes were available in the Manhattan Project and the Apollo Project, but they are not in the case of biosurveillance. Also, the people who know how to build such a system work largely in informatics and in the health care system, and these people for the most part have not been involved.
A Regional Focus
As described by the previous speakers, a CONOPS involves describing the limitations of the existing NBIC, which include the fact that agencies are not sending data to NBIC. A CONOPS also involves proposed concepts for the system being analyzed, which requires a description of the modified system and of the motivation for change.
With the CONOPS process in mind, Wagner proposed that a biosurveillance data integration center should function as a regional data integration center for one or two cities, counties, or states. It also could have a repository function at a national level, but that would depend on the resources available. Focusing on a regional level would enable solutions to be developed for many of the technical problems of data integration. Which other systems would be involved also could be worked out. Finally, it could provide an example of what could be realized at the federal level if resources were devoted to the task. “Integrated biosurveillance is a massive data and knowledge integration problem,” said Wagner. “We can make the most progress toward solving it if we focus on one city.”
During the discussion period, Braden observed that many of the data analyzed as part of biosurveillance are created on the fly. Some data sources are constant and can be monitored, he said, but “a lot of the data that we need at CDC we have to create de novo at the local level, at the state lev-
el, or at our level.” As a result, it is difficult to do quality improvement on the data collection and analysis system because no such system exists. In addition, data should be analyzed by the people in the best position to do so, after which the results of that analysis should be available to others. “It is not just that the data resides here and is owned there. It is a matter of expertise of handling the data.”
In response, Lenert suggested using a framework of “data, information, and knowledge” to analyze these activities. Tracking the state of knowledge of agencies is easiest; tracking the state of information takes more work; and understanding raw data may be very difficult. Kadlec added that, in a crisis, experienced analysts of data are in very high demand but are distributed, and ways need to be found to bring together their expertise. This is the purpose of the biological assessment threat response (BATR) process, which Hepburn described as an iterative interagency process to arrive at a decision about what should happen next. The BATR process is not biosurveillance, he added, but rather is designed to produce decisions.
Ackelsberg reiterated the idea that data need to be collected, analyzed, and converted to information at the local level so the best knowledge possible can be provided to those who need to make decisions. For that reason, fusion centers are just as useful at the local level as at the federal level. Maillard, too, observed that local public health departments create the instruments needed to gather data on the fly and share information with others rather than raw data.
Gibson asked whether the high-level knowledge being created by NBIC would be of use at the local level, and Wagner asked in response whether Marion County would benefit from the kind of information, including intelligence information, that an NBIC working closely with the county could provide. Gibson agreed that more data would help in preparing for the 2012 Super Bowl. In general, he said, additional data provide new and often unanticipated capabilities. “With syndromic surveillance, we never expected it to be valuable in detecting things, but it has proven to be valuable in a lot of different ways that makes it worth doing every day.”
Ashkenazi pointed out that decisions depend on perceptions, understanding, and prediction. About 80 percent of leadership mistakes, he said, are because of perceptions. Too much information can lead to a cognitive overflow that skews perceptions. And people have a tendency to rely on computers to make perceptions, but computers have no ability to do that. Lenert mentioned as well the cognitive flaw of thinking that a
given situation is similar to a past situation, which is where perception and knowledge can go astray.
Tan observed that data emerge from different systems and with different speeds and have different levels of reliability. When data are shared, the uncertainties associated with those data need to be shared as well.
Annelli said that gaps in information sharing need to be identified before they can be filled. During this process, one question that must be asked is whether a gap exists because an existing system broke down. Addressing that question would help determine the need for CONOPS in different agencies.
Wagner replied that it is not yet clear what data need to be collected. Until people have a chance to build an end-to-end system that leads from data to decisions, the necessary links will not be fully known.
Ackelsberg also emphasized the possibility of catastrophic events—a “bio-Katrina type of situation.” The needs in such a situation would be extraordinarily different. “I don’t think any local entity, any local jurisdiction, or any combination of the agencies in a local setting is going to be able to immediately address the analytic requirements for that.” The analytic capacity still needs to be built at the local level to assess what would be needed in such a situation.
Finally, Sally Phillips, Department of Homeland Security, thanked the speakers and workshop participants for their contributions. The challenges are enormous, she said, not only in structuring a biosurveillance system but in funding it. But “we have learned a lot of lessons today and yesterday, and we have some great ideas on the table and some good analysis.” Biosurveillance is needed, she said. People already use on a daily basis the information that exists, and decision makers crave more information. “We need to figure out what is the next step” while keeping the ultimate goal in mind, she said, “because this is certainly going to be longer than a short-haul fix.”