3
Recommendations

The FBI’s current approach to IT modernization is not working as well as it must to support the FBI’s missions in criminal investigation and preventive counterterrorism. Based on recent history, its approach is not likely to be much improved without major changes. The committee believes that a culture change within the FBI relative to IT is absolutely imperative. While the FBI needs to keep the forward thinkers among its force of experts, it will be severely handicapped unless it also culls out the elements in its leadership structure that are unable to lead the changes necessary to effectively implement an information age organization.

The first and most urgent recommendation, indeed critical in light of the impending VCF system rollout, is that the FBI not proceed with deployment of the VCF until it has a validated contingency plan for reverting completely or partially to the ACS, if necessary, and clear and measurable criteria to determine when the ACS can safely be turned off.

Beyond this urgent recommendation, the committee makes a number of recommendations, grouped into four areas, that will significantly increase the likelihood of success in and drive an accelerated pace for the FBI’s IT modernization efforts. These recommendations fall into two categories. Category 1 recommendations are, in the committee’s judgment, imperatives for the success of the FBI’s IT modernization program, and are listed in the Executive Summary. Category 2 recommendations involve “best practices” or sound advice that the committee believes are appropriate to the FBI’s situation.

3.1 REGARDING ENTERPRISE ARCHITECTURE

If the FBI is to be successful in its efforts to exploit IT, the formulation of an enterprise architecture must have the highest priority in its IT efforts. Of course, an enterprise architecture can be expected to grow and evolve to reflect the increasing responsibilities of the FBI and the increasing role that IT will have in satisfying these responsibilities, but a baseline enter-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program 3 Recommendations The FBI’s current approach to IT modernization is not working as well as it must to support the FBI’s missions in criminal investigation and preventive counterterrorism. Based on recent history, its approach is not likely to be much improved without major changes. The committee believes that a culture change within the FBI relative to IT is absolutely imperative. While the FBI needs to keep the forward thinkers among its force of experts, it will be severely handicapped unless it also culls out the elements in its leadership structure that are unable to lead the changes necessary to effectively implement an information age organization. The first and most urgent recommendation, indeed critical in light of the impending VCF system rollout, is that the FBI not proceed with deployment of the VCF until it has a validated contingency plan for reverting completely or partially to the ACS, if necessary, and clear and measurable criteria to determine when the ACS can safely be turned off. Beyond this urgent recommendation, the committee makes a number of recommendations, grouped into four areas, that will significantly increase the likelihood of success in and drive an accelerated pace for the FBI’s IT modernization efforts. These recommendations fall into two categories. Category 1 recommendations are, in the committee’s judgment, imperatives for the success of the FBI’s IT modernization program, and are listed in the Executive Summary. Category 2 recommendations involve “best practices” or sound advice that the committee believes are appropriate to the FBI’s situation. 3.1 REGARDING ENTERPRISE ARCHITECTURE If the FBI is to be successful in its efforts to exploit IT, the formulation of an enterprise architecture must have the highest priority in its IT efforts. Of course, an enterprise architecture can be expected to grow and evolve to reflect the increasing responsibilities of the FBI and the increasing role that IT will have in satisfying these responsibilities, but a baseline enter-

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program prise architecture is a crucial starting point. To deal with the issues related to enterprise architecture, the committee makes the following recommendations. Category 1 recommendations on enterprise architecture The committee believes that if the FBI’s IT modernization program is to succeed, the FBI’s top leadership, including the director, must make the creation and communication of a complete enterprise architecture a top priority. This means that they must be personally involved and invested in the key decisions that the process will require be made, such as the tradeoffs between the security of and access to information in the various data sources that are used in criminal investigation and counterterrorism efforts. Indeed, it is critical that the director be well versed in, and comfortable with, the operational aspects of the enterprise architecture and their overall linkage to the high-level system design. Only when the FBI’s leadership takes intellectual ownership of the bureau’s enterprise architecture can it be used to make top-level management decisions and to ensure that IT investments realize their full potential. While a contractor might well assist the FBI in developing the enterprise architecture, no contractor will fully understand the operational issues that must be reflected in the enterprise architecture, nor be empowered to make decisions about how to make the tradeoffs with competing concerns. The committee believes that a small team, consisting primarily of senior operational managers from the Criminal Investigation Division, the Office of Intelligence, and the Counterterrorism Division, and a senior IT executive (e.g., the CIO) to translate what these managers say into architectural terms, should be able to develop the broad outlines of the operational aspects of the enterprise architecture as well as a top-level schematic view of the systems design in a matter of 4 to 6 months of full-time work. To decide on the many operational and policy tradeoffs that will inevitably arise, this team must have direct access to and the frequent involvement of the most senior management of the FBI, including the director and the deputy director. A reasonable first step in developing the FBI’s enterprise architecture would be to define, by job, the information necessary for each FBI division to accomplish its task. The output of this effort will not be sufficient to guide implementation detail, but will provide an understanding of the information flows for investigative and intelligence data, identify existing resources, and indicate how the information needs of major categories of users can be satisfied. Further, it will provide a basis for the partitioning of implementation tasks, and identify unmet needs. The FBI should seek independent and regular review of its enterprise architecture as it develops by an external panel of experts with experience in both operations and technology/ architecture. When the first draft of the enterprise architecture has been prepared, it should be reviewed by an external panel of independent experts charged with helping the FBI to improve how it uses IT in the long term. This could be the FBI’s Science and Technology Advisory Board, an ad hoc committee or a contractor familiar with successful DOD architecture efforts, or even an ad hoc committee such as this one, but the important point is that an external inspection of the draft enterprise architecture is a sensible safeguard under any circumstances. Given that the counterterrorism mission requires extensive information sharing, the FBI should seek input on and comment from other intelligence agencies regarding its enterprise

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program architecture effort. The reason is that the FBI’s information systems must have interfaces to those agencies to ensure that the information resources of those agencies are appropriately linked to FBI systems so that those agencies are able to work collaboratively. Category 2 recommendations on enterprise architecture The FBI should build on the early efforts under way in the intelligence area in defining a subarchitecture for the intelligence process, rather than begin with the (implicit) architecture of the VCF. The vision for, and basic architecture of, the Trilogy system—including the VCF—predates the post-9/11 restructuring of the FBI’s mission priorities and relationships to intelligence. At that time, the FBI’s (and Trilogy’s) focus was on support for the law enforcement mission. The change in the FBI’s mission since 9/11 underscores the need for agility and for the separation of mechanism and policy in the information system architecture; many functions that would have been prohibited by policy before 9/11 are now accepted as essential parts of the FBI’s operation. The FBI should make heavy use of scenario-based analysis in its development of an enterprise architecture. Scenario-based analysis calls for understanding relevant scenarios in sufficient detail that one can actually understand the information flows, analytic processes, and top-level decisions that must be made in those instances. Doing so will help the FBI to identify specific roles for IT in supporting operational needs. The FBI should give high priority to reducing the management complexity of its IT systems, even at the expense of increased costs for hardware that may appear duplicative or redundant. The successful management of a complex IT system usually requires a large degree of technical sophistication that the FBI lacks at present. As one example, the FBI should avoid the temptation to make the IDW the single repository that contains all data regardless of sensitivity or type. Section 2.2.2 points out that the access and security requirements for intelligence data and investigative data are very different, and storing both types of data on the same system will entail the implementation of a very complex set of access rules and a significant cost in human effort to maintain and enforce those rules. The cost of the extra complexity entailed by the single-repository concept will, in the long run, far outweigh the cost of the “extra” hardware. Box 1 provides a sampling (not an exhaustive or complete list) of some of the elements that the committee would expect to see in the FBI’s enterprise architecture. 3.2 REGARDING SYSTEM DESIGN To deal with the most important of the system issues described in Section 2.2, the committee makes the following recommendations: Category 1 recommendations on system design The FBI should refrain from initiating, developing, or deploying any IT application other than the VCF until a complete enterprise architecture is in place. The committee hopes that the initial operating capability of the VCF will soon be demonstrated. If successful, it will

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program BOX 1: A Sampling of Items That Should Be Present in the Enterprise Architecture An enterprise architecture is generally represented as a set of operational diagrams accompanied by appropriate narratives that show the operating elements or nodes constituting the enterprise, the data/ information flow requirements between the nodes, and other data reflective of the operational structure. It is the central document describing the operational, system, and technical views of the enterprise. Central to an enterprise architecture is a clear description of key processes and how they can be supported and enhanced by appropriate application of technology. The enterprise architecture specifies the overall design and the set of building codes to which deployed systems must conform in order to make the IT investment effective. The following listing is a sampling (not complete in any way) of items that the committee would expect to see in the FBI’s enterprise architecture. All significant information inputs are shown, with identification of sources and restrictions on their use, if any. Processes that are primarily investigative and those that are primarily counterterrorism- or intelligence-specific are identified. Some processes and flows may be in both domains. Some supportive operational processes interact with the investigative and counterterrorism processes and are labeled as being in the support business category. The VCF is shown as one of a number of information sources that feed the intelligence process. The VCF is shown as the primary source of information for the investigative process. Different subarchitectures are reflected for each major operational process found in the FBI (e.g., for the intelligence and investigative processes). The VCF system is seen to span investigative processes, but does not play a primary role in implementing analytic processes. Interfaces between subarchitectures are defined and specify the data flows between them (e.g., the relationship between the records management process and the intelligence and investigative processes). Standards for data exchange are explicitly acknowledged. Responsibility for the contents of all persistent storage systems or their segments (e.g., data cleansing and validation) are assigned, documented, and represented. Data access constraints, such as law-enforcement-sensitive data and levels of classified and open data, are explicitly identified. Different approaches to security (e.g., risk management versus risk avoidance) are articulated explicitly and provide the framework that allows management to make decisions about tradeoffs. Data storage applications are logically disjoint whenever they have different governing policies (or else a credible argument for doing otherwise is demonstrated). The relationship between the VCF and records management systems is explicitly represented, and responsibilities for maintaining evidence for investigations are highlighted. Data models for the different processes are specified. Any significant differences in data models that need to be unified to serve both the investigative and analytic functions are represented. Access requirements for data supporting the various processes are clearly specified, including any audit trail requirements. Release constraints for data and results emanating from the various processes are clearly specified, including any audit trail requirements. Policies for replication of information among data storage applications are specified. The use of an electronic key management system developed by the National Security Agency in the context of the specific needs of the FBI is explicitly rationalized. Process and data interfaces with other law enforcement organizations (international, state local, and tribal) are identified. Specific databases and controlled interfaces between them (if any) are identified. A map between these interfaces and the physical network configuration shows what links/nodes must be encrypted, physically protected, or both. A key-management plan (for managing encryption keys) is created. A vulnerability assessment (i.e., a paper attack on the paper architecture) is conducted.

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program provide such a major step forward from the ACS that it would be a mistake to halt VCF development and deployment at this time. But in the committee’s view, the benefit of designing future applications within the enterprise architecture framework is so large, and the risk of designing without that framework in place so high, that no additional development should proceed until the framework is in place.1 The FBI should develop a process map for information sharing that clearly defines the current state of and a desired end state for the information-sharing process so that the numerous information-sharing initiatives can be coordinated and properly monitored and managed. This recommendation is derived from the DOJ inspector general report2 but is one that the committee fully supports. In a letter to the DOJ inspector general dated December 11, 2003, the FBI stated that it had already completed a detailed blueprint and process map on its intelligence and information sharing process,3 but as of December 2003, neither the committee nor the DOJ inspector general had been able to review this document.4 The FBI should immediately develop plans that address recovery of data and functionality in the event that essential technology services come under denial-of-service attacks (e.g., from viruses and pervasively replicated software bugs). In addition, the FBI should deploy technically distinct platforms (that is, computing nodes that are based on a different operating system) for the hosting or backup of critical services or data, so that in the event of a global attack on the Trilogy network, these services are more likely to be maintained and the uninfected platforms can serve as a “beachhead” from which cleanup operations can be mounted. Category 2 recommendations on system design The FBI should develop a future release plan for the VCF that specifies what capabilities will be added to it, and in what order and time frame. The committee believes that future releases of the VCF, other than those needed to reach initial functionality, should be delayed until an overall enterprise architecture is in place. Further, the FBI should ensure that the first enhancements to the VCF are to make the system consistent with the overall enterprise architecture (rather than only to add additional functionality). After that, capabilities to be considered include the addition of a separate workflow engine (a high priority, as described below) and the creation and integration of interfaces to the IDW. Developing the plan in the context of the enterprise architecture is critical to aligning the development activities of the IDW, SCOPE, and the VCF, and other systems and to optimizing what will likely be significant investments downstream. The FBI should plan to rework the next version of the VCF to include a workflow engine as a high priority. By incorporating a workflow engine, the FBI will make the VCF more agile 1   The GAO position, as documented in the GAO commentary on the FBI response, is that it is acceptable for the FBI to be “pursuing near-term IT upgrades before it completes and is positioned to use an architecture,” even though pursuing these upgrades “without a blueprint that provides an authoritative, commonly understood frame of reference that translates strategy into implemental actions … [will increase] modernization risk.” (Response of the FBI to the GAO report The FBI Needs an Enterprise Architecture to Guide Its Modernization Activities, dated September 22, 2003. Available at http://www.gao.gov/new.items/d04190r.pdf.) The committee concurs with the GAO position only for the VCF application and believes that for all other applications, the risk of proceeding without an enterprise architecture in hand is too high. 2   DOJ report available at http://www.usdoj.gov/oig/audit/FBI/0410/app8.htm. 3   FBI letter available at http://www.usdoj.gov/oig/audit/FBI/0410/app7.htm. 4   See http://www.usdoj.gov/oig/audit/FBI/0410/app8.htm.

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program in its support of evolving FBI policies and practices, reduce the expense of evolving the VCF application, reduce the risk of delaying implementation of new policies and practices, and extend the lifetime of the VCF application. (Note that commercial workflow engines are available that can be configured to support a myriad of workflow arrangements without modifying the application programs they serve. Workflow engines are a common technology that is employed by many organizations, especially those that are large, complex, and geographically dispersed. Furthermore, IT staff might well include specialists trained in human computer interaction who can understand process flows and how to build systems that reflect new protocols of agents and analysts.) The FBI should adopt a risk management approach to security, for only in doing so will it understand the operational penalties it pays for risk avoidance. Acceptance of this premise results in several immediate items of high priority. The FBI should establish a clear policy governing sensitive but unclassified (SBU) and law-enforcement-sensitive communications (and data more generally). Such a policy can be risk-based. The FBI should reconsider, in the light of a better understanding of the risk/benefit tradeoffs to its missions, the very constrained access that FBI staff have to the Internet. The FBI has stated that it is in the process of acquiring a risk management tool that will assist it in determining where IT vulnerabilities should be mitigated through risk/cost trade-offs, thereby ensuring IT continuity of operations, and that this tool will be inter-faced with tools that the FBI uses to develop and manage its enterprise architecture efforts. These efforts should continue, although the committee notes that the threat driving the need to ensure continuity of operations in the face of attack differs in kind from the threat of compromising sensitive information. If the FBI is not comfortable with moving to a risk management approach to security, at the very least it should review security practices in a risk-versus-reward framework with an entire end-to-end consideration of the information gathering and information management requirements of the FBI’s staff. The FBI should encourage creative experimentation with exploitation of IT in the field, such as the PDA experiment mentioned in Section 2.2.4 above. The committee did not review this area in detail, but it believes that such experimentation, with appropriate safeguards, has enormous potential for helping the FBI to understand how its operational processes might be improved through the use of IT. Further, the learning from these efforts to anticipate the technological future should be brought forward and become an important input into the bureau’s strategic planning process in order to accelerate its pace of modernization. (In this regard, a useful philosophy is the one underlying the Department of Defense’s Advanced Concept Technology Demonstration programs, which are based on an integrating effort, undertaken by an ultimate user, to assemble and demonstrate a significant new military capability in a realistic environment, based on maturing advanced technologies, to clearly establish the capability’s military utility.5) 5   For more information, see http://www.acq.osd.mil/asc/.

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program 3.3 REGARDING PROGRAM AND CONTRACT MANAGEMENT To deal with the most important of the program management issues described in Section 2.3, the committee makes the following recommendations. Category 1 recommendations on program management Because testing is such a critical dimension of system development and deployment, the FBI must allow adequate time for testing before any IT application (including the VCF) is deployed, even if dates of initial operational capability are delayed. Testing must include a full systems integration test and adequate scale, volume, and stress tests. Evolution is an essential component of any large system’s life cycle. Future development contracts for user applications should be premised on the use of small-scale prototypes that can be built rapidly and tested with user feedback before committing to large-scale development. Therefore, in future IT applications development, particularly of large-scale end-user-oriented applications, procurement contracts should be conditioned on the development of small-scale prototypes that can be built rapidly and tested with user feedback before committing to large-scale development. For IT applications beyond the VCF, the FBI should exploit proven methodologies of contracting and contract management, including the use of detailed functional specifications, specific milestones, frequent contract reviews, and earned-value metrics. Given the FBI’s problems with the management of the Trilogy program, the next contract review should focus on continuity and availability. Contracts already in place should be renegotiated to include best practices such as code escrow and support-service-level agreements to protect the FBI against operational failures that can adversely impact the availability of and support for products that the FBI will depend on. The FBI should consult with both other government agencies and the private sector to develop a set of best contracting practices before undertaking its next contract review. Category 2 recommendations on program management The FBI’s contracting strategy should be tied to features of its enterprise architecture; e.g., it should identify opportunities for multiple, smaller contracts with well-defined deliverables and major progress checkpoints. This strategy should also highlight areas in which the FBI requires in-house or trusted technical expertise to define and manage key concepts that govern contracts and relationships between contractors. (This kind of information derives from the architectural triad described in Section 2.1.1. For example, the system architecture should define the major systems to be built. The technical architecture should define the key interfaces between systems that have to be carefully managed to get independently developed component systems to work together. In addition, the FBI should have inhouse expertise regarding operations as they relate to the overall infrastructure, whereas the FBI may be able to leave to contractors the expertise relevant to functionality in specific instances and specific applications.

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program 3.4 REGARDING HUMAN RESOURCES To deal with the most important of the skills issues described in Section 2.4, the committee makes the following recommendations. Category 1 recommendations on human resources For Trilogy and subsequent IT projects to have access to the human talent they need to succeed, the FBI must dramatically grow its own internal expertise in IT and IT contract management as quickly as possible. To deal with human resource shortages in key areas (e.g., program management, data architecture, data modeling, and data warehousing), the only feasible short-term fixes are to borrow experienced personnel from other agencies or to obtain assistance through a memorandum of understanding or agreement (MOU/MOA) from another government agency with substantial experience in the relevant matters. The expectation would be that these arrangements would last for a couple of years, during which time the FBI could train permanent replacements in long-term career tracks with the FBI. Note that this recommendation is still consistent with the notion of using outside expertise, when it is appropriate, as long as the FBI does not cede overall management and the making of the key decisions. For example, the FBI could in principle outsource in a secure facility the day-to-day operation of its information systems and thus conserve its scarce IT talent to work on matters more closely related to operations and strategy. Because of their importance to the short- and long-term success of the bureau’s IT modernization efforts, the FBI must permanently fill the positions of chief information officer and chief enterprise architect, and the committee concurs with the director’s judgment that filling these positions with appropriately qualified individuals ought to have the highest priority. At this writing (late March 2004), an acting CIO and an acting chief technology officer have been appointed. The appointment of these individuals is promising, and unverified reports suggest that they are committed to completing the enterprise architecture in a matter of months, although the committee underscores once again the need for the operational management to be heavily involved in the creation of the enterprise architecture. The FBI should develop an improved system for internally reviewing the state of progress in key IT programs and for communicating relevant findings to key stakeholders, thus pre-empting the perceived need for and distraction of constant external investigations. Category 2 recommendations on human resources The FBI should seek relief from excessively tight constraints on reprogramming allocated funds, or at least seek to streamline the approval process. 3.5 CONCLUSION The committee believes that the FBI has made significant progress in some areas of its IT modernization efforts, such as the modernization of the computing hardware and baseline software and the deployment of its networking infrastructure. However, because the FBI IT infrastructure was so inadequate in the past, there is still an enormous gap between the FBI’s current IT capabilities and the capabilities that are urgently needed.

OCR for page 48
A Review of the FBI’s Trilogy Information Technology Modernization Program The committee has made recommendations that, if adopted, will significantly increase the likelihood that the FBI’s Trilogy IT modernization program will enhance the FBI’s effectiveness in carrying out its critical crime-fighting and counterterrorism missions. But it emphasizes the difference between a pro forma adoption of these recommendations and an adoption of these recommendations that is both fully embraced throughout the agency and aggressively executed. The former may be the metric that auditing and oversight agencies and offices often use in assessing agency performance, but it is the attitude and willingness of senior staff to act that really count. The senior management of the FBI has a substantive and direct role to play in the FBI’s IT modernization efforts. This role either has not been understood or it has been given a lower priority based on the perception of more immediate operational priorities. Given the importance of IT to the FBI’s future success in carrying out its missions, the FBI’s senior management must concern itself as much with developing a coherent vision for using IT to advance the bureau’s strategic view as with budgets, training programs, equipment, and organization. As the complexities of the FBI’s evolving role are understood, the committee believes that investment by the FBI’s senior management team in the IT process will yield major enhancements to mission achievement as well as substantial operational efficiencies.