2
IT-Related Issues for the FBI Requiring Immediate Action

Although limited in scope and time, the committee’s review of the approach and methodology currently being used by the FBI to drive the introduction of its new IT systems has raised a number of significant issues. To address these issues requires concerted FBI action in four major clusters. The issues in each of these clusters are serious in and of themselves. Taken in aggregate, the detrimental impact of inattention to these issues on the FBI’s IT modernization efforts is enormous despite the progress that has been made. These four clusters are:

  • Enterprise architecture. An enterprise architecture maps the linkage between the FBI’s strategy and operational needs and its IT program.

  • System design. System design is the engineering of detailed IT solutions driven by the enterprise architecture.

  • Program and contract management. Large-scale program and contract management processes are critical to the success of IT modernization endeavors as massive as Trilogy.

  • Skills, resources, and external factors. The fourth and final cluster cuts across the first three clusters and generally relates to human resources and skills, and to some of the external constraints faced by the FBI in its IT modernization efforts.

The next four sections deal with these issues cluster by cluster.

2.1 ENTERPRISE ARCHITECTURE ISSUES

2.1.1 Creating an Enterprise Architecture That Serves FBI Objectives

What Is an Enterprise Architecture?

An enterprise architecture characterizes the enterprise’s missions, tasks, and operational processes, and relates these tasks, processes, and operational objectives to IT strategy, invest-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program 2 IT-Related Issues for the FBI Requiring Immediate Action Although limited in scope and time, the committee’s review of the approach and methodology currently being used by the FBI to drive the introduction of its new IT systems has raised a number of significant issues. To address these issues requires concerted FBI action in four major clusters. The issues in each of these clusters are serious in and of themselves. Taken in aggregate, the detrimental impact of inattention to these issues on the FBI’s IT modernization efforts is enormous despite the progress that has been made. These four clusters are: Enterprise architecture. An enterprise architecture maps the linkage between the FBI’s strategy and operational needs and its IT program. System design. System design is the engineering of detailed IT solutions driven by the enterprise architecture. Program and contract management. Large-scale program and contract management processes are critical to the success of IT modernization endeavors as massive as Trilogy. Skills, resources, and external factors. The fourth and final cluster cuts across the first three clusters and generally relates to human resources and skills, and to some of the external constraints faced by the FBI in its IT modernization efforts. The next four sections deal with these issues cluster by cluster. 2.1 ENTERPRISE ARCHITECTURE ISSUES 2.1.1 Creating an Enterprise Architecture That Serves FBI Objectives What Is an Enterprise Architecture? An enterprise architecture characterizes the enterprise’s missions, tasks, and operational processes, and relates these tasks, processes, and operational objectives to IT strategy, invest-

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program ment, and design. It provides substantial detail on the structure and standards used to implement the IT system. The enterprise architecture is the framework that describes the way in which an organization such as the FBI conducts its mission(s), how it organizes and uses technology to accomplish its goals and execute key operational processes, and how the IT system is structured and designed in detail to achieve these objectives. In general, it should also include documentation that explains the rationale behind important decisions and why certain alternatives were chosen and others rejected. An enterprise architecture thus contains much more than information about technology. The enterprise architecture becomes the template on which the IT investment is rationalized, and the enhancements to the FBI’s mission achievement, the return on the investment, are defined and quantified using appropriate metrics. When new needs emerge, as with the counterterrorism mission, an enterprise architecture provides a point of departure—a framework within which additional capabilities can be coherently designed. Absent an enterprise architecture, it is essentially impossible for any large organization, including the FBI, to make coherent or consistent operational or technical decisions about IT investments. Among these decisions are the definitions of appropriate data structures and linkages to other systems and data sources, policies and methods of information sharing, issues of security and the tradeoffs with information access, innovation, and the exploitation of evolving technologies, and metrics of effectiveness for the IT system and its use. The close link between good enterprise architecture planning and sound systems engineering practice on the one hand and success in large-scale IT deployments on the other has been demonstrated in numerous examples in the private sector and in the federal government.1 Further, the existence of the enterprise architecture can be a major contributor to the confidence and trust that management, users, and implementers have in a project, as well as facilitating cooperation among them. A well-documented and communicated enterprise architecture is a prerequisite for driving cultural and operational change and innovation at a pace that effectively capitalizes on the pace of technology improvement. Good models depict the as-is (today’s) environment as well as the to-be (future desired) environment, showing the transition. The Structure of an Enterprise Architecture While technology capability is an important input to operational strategy, it is important that IT investment not be driven primarily from the technology end, but rather that operational strategy drive technology investment and system design. To this end, the committee believes that a good starting point for achieving the required linkage is to think about architectures in three conceptually distinct but interrelated forms:2 1   For an example in a medical setting, see Jonathan M. Teich et al., “The Brigham Integrated Computing System (BICS): Advanced Clinical Systems in an Academic Hospital Environment,” International Journal of Medical Informatics 54:197–208 (1999). For a case study in which failure was closely associated with insufficient attention to architectural issues, see National Research Council, Continued Review of the Tax Systems Modernization of the Internal Revenue Service, Computer Science and Telecommunications Board, National Academy Press, Washington, D.C., 1996. For a case study in the private sector that demonstrates similar lessons, see Christopher Koch, “AT&T Wireless Self-Destructs,” CIO Magazine, April 15, 2004, available at http://www.cio.com/archive/041504/wireless.html. 2   The architectural triad used here originates with the Department of Defense, which has used this framework to develop its command and control systems since 1997. (A good reference on this subject is Architecture Working Group, Department of

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program An operational architecture, which describes in graphical and narrative terms the key operational objectives; the operational elements, tasks, and processes used to achieve them; a high-level description of the types of information used and created; how and with what constraints information of various types must be exchanged; and the information flows in these processes. An operational architecture relates to specific mission scenarios and functions and forms the basis for realistic process and information flow representation and prioritization.3 (An analogy for operational architecture in the construction domain is the concept of operations for the various purposes a building will serve, and how various components of the building will serve those needs.) A systems architecture, which describes at a high level the data repositories, systems, applications, connectivity, and communications infrastructure that supports the operational architecture for mission needs. The systems architecture defines the logical and physical connection, location, and identification of the key nodes, circuits, networks, and platforms that are associated with information exchange and specifies some critical system performance parameters. The systems architecture is constructed to satisfy operational architecture requirements using the standards defined in the technical architecture. (An analogy for systems architecture in the construction domain is the blueprints for a building that illustrate how components fit together.) A technical architecture, which captures the key technical standards, protocols, and specifications required to implement the systems architecture. A technical architecture is intended to be a compact set of rules governing the arrangement, interaction, and interdependence of the parts or elements whose purpose is to ensure that a conforming system satisfies a specific set of requirements. It identifies system services, interfaces, standards, and their relationships. It provides the framework for the derivation of engineering specifications that guide the implementation of systems. (An analogy for technical architecture in the construction domain is the set of building codes for connecting subsystems and ensuring the operational effectiveness of the building with the community electrical, water, sewage, and roadway systems.) To indicate what some of the content of these three architectures might be, the committee attempts to describe in Figure 1 some primary and supporting activities in the FBI as they might contribute to one part of the architectural triad described above—an operational architecture.4 The purpose of Figure 1 is to focus attention on key structural elements of the FBI. Figure 2 shows a subset of FBI activities and the potential scope of planned supporting IT     Defense, C4ISR Architectural Framework, Version 2.0, December 18, 1997, available at http://www.defenselink.mil/nii/org/cio/i3/AWG_Digital_Library/pdfdocs/fw.pdf. The descriptions of operational, systems, and technical architectures contained in this report are adapted from this DOD document.) Note, however, that the engineering methodology of defining a “functional architecture” based on processes and information flows and then building a “system architecture” with ever increasing technical detail dates back to the mid-1970s. And the use of structured analysis to form the basis for a technical system design has been in widespread use throughout commercial and government organizations since the late 1970s. 3   As one possible scenario, the FBI told the committee that it has some 6 million documents found in Afghanistan. How will the information in these documents be extracted and managed? Who will have access to what set of those documents? What circumstances will govern access rules? How will the information contained in these documents be used with other information to which the FBI has access? Thinking through this scenario will provide a great deal of insight into information management requirements. The key point is that however the FBI decides to handle the information from these documents, it should be able to articulate the rationale for doing so in terms that make sense from an operational point of view. 4   See, for example, Michael E. Porter, Competitive Advantage: Creating and Sustaining Superior Performance, Free Press, New York, 1998.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program FIGURE 1 FBI value chain—an example of a framework for “the view from 40,000 feet” of a serious enterprise architecture. This adaptation of Michael Porter’s famous business value chain indicates the core processes of the FBI’s two major missions—(1) criminal investigation and (2) counterterrorism. In addition, the primary activities include process management to monitor the performance of these two missions and quality management to provide feedback on the effectiveness of both the primary outputs—information—and how users actually use that information. At the top of the diagram are the traditional resources management areas: administration, procurement, human resources, finance, and IT. These supporting activities ultimately must be integrated with the primary activities shown in the bottom two-thirds of the diagram for the FBI to work most effectively. activities—the ACS, the VCF, SCOPE (the Secure Counterterrorism Operational Prototype Environment, discussed below in Section 2.2.3), and the IDW. Figures 1 and 2 represent the committee’s (certainly imperfect) perspective on FBI operations and the roles of IT systems in supporting FBI missions. These figures are highly simplified (for example, they do not include the operational impact of providing law enforcement assistance to other agencies), and they lack any kind of supporting detail. However, the committee inferred the content of Figures 1 and 2, rather than receiving them from the FBI. One of the committee’s major concerns is that it was not shown anything produced by the FBI or its contractors that approaches the content or substance of even these oversimplified figures. In

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program FIGURE 2 The scope of key applications on the FBI value chain. This diagram overlays on the middle portion of Figure 1 the FBI’s currently existing IT systems (ACS and the SCOPE prototype) and the ultimate next-generation applications (VCF and IDW 1.0) and their purpose in relation to the bureau’s core processes. Clearly, the FBI has hundreds of systems to be arrayed on a chart of this kind for it to be of most value. This task is not complex, and the results would greatly facilitate understanding both the “as is” and the “to be” contemplated by the Trilogy program and other FBI IT planning. other words, the fact that the FBI was unable to present its own version of these figures was telling to the committee about a lack of clear architectural thinking. Without such a top-level description and understanding, all of the subsidiary steps in the design and implementation of an enterprise IT system are at significant risk of failure. It is important for the FBI to create and use its own versions of Figures 1 and 2 to help explain its strategies, and to do the supporting work required to fill out their content. Once this is done, it does not matter if the FBI’s own versions agree with the committee’s figures, and a detailed acceptance or rejection of the committee’s figures is not relevant. The committee recognizes that the framework described above is only one way to conceptualize enterprise architectures. For example, the Federal Enterprise Architecture (FEA) frame-

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program work,5 to which the FBI has already committed itself, is used in many federal agencies. Multi-agency use of the FEA framework enables architectural efforts presented using its five-component framework to be directly compared across agencies. Nevertheless, the committee believes that the architectural triad described above is conceptually clearer and more straightforward than the FEA framework and that the FBI will make more rapid progress using the triad even if some translation between the triad and the FEA framework will likely be necessary at the end of the process. Recognizing the Role of Top Management in Creating an Enterprise Architecture An enterprise architecture has far more operational content than technical content. Accordingly, the FBI’s enterprise architecture must be based on mission needs and must be formulated, propagated, owned, and evolved by the operational leadership of the FBI at the highest levels (the FBI director and the directors of the FBI’s mission-oriented divisions).6 An effective process requires that the enterprise architecture be created by a combined effort involving both senior operational management and key technologists. The CIO plays a key role as the facilitator of the process; however, the creation of the enterprise architecture cannot be handed off to the CIO, and certainly cannot be outsourced. The enterprise architecture is central to an organization’s strategy, as well as important for process analysis, change, and planning. Outside expertise can play a role in assisting and advising the enterprise architecture process, but the FBI’s top management team must manage, buy into, and execute the process. Although this task is critical, it need not be an enormously time-consuming one. With adequate time and focus on the part of senior operational management working with key IT people, major progress could be made in a week of intense work, and the creation of a reasonably complete operational architecture together with a top-level schematic systems architecture—the most critical part of the enterprise architecture—should be possible in a 6-month time frame.7 Further, committee experience indicates that progress is best made with the top-level management team supported by an architecture team composed of a small number of full-time professionals dedicated to the task. These individuals must understand operations and have some appreciation for technology, rather than being technologists with only a loose connection to the operational side. Given the fact that the leadership of any large organization, including the FBI, has many ongoing demands on it, the temptation is large to simply hire a CIO with a great deal of technology experience, delegate the task, and then forget about the problem. Why must an organization’s operational leadership be deeply engaged in the development of an enterprise architecture for IT? Why can’t this function be outsourced? 5   The FEA is a business- and performance-based framework to support cross-agency collaboration, transformation, and government-wide improvement. See http://feapmo.gov/. 6   In principle, it is also possible to begin with a vision of how technology might enable missions to be accomplished—that is, to seek a technology-driven enterprise architecture. But such an approach demands an extraordinarily high level of sophistication about the capabilities and limitations of technology, and for reasons that this report documents, is not appropriate for the FBI. 7   The executive assistant director for intelligence testified to the committee that a small team working full time for about 10 weeks was able to develop a concept of operations for the Office of Intelligence. Thus, 4 to 6 months of work does not seem unreasonable for developing the operational dimensions of a larger enterprise architecture.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program The committee’s experience is that without the drive of operational missions, technology deployments inevitably serve to automate existing functions at a relatively low level, and thus the processes associated with these functions remain static. Even worse, automation of existing functions can degrade the ability of an organization to carry out its missions. The reason is that the functions that an organization must serve are sometimes in tension—consider an organization that must keep sensitive information secure (a driver for restricted access), and yet make it available to everyone with legitimate need (a driver for broad access). When people deal with these competing demands, they make judgments and ad hoc decisions on the basis of their experience and expertise. Despite the existence of functional tensions in a non-automated system, human flexibility and adaptability generally ensure that operations can function at an adequate level of performance. Indeed, in most organizations, what people actually do on the job varies from what is specified by the organization’s formal processes. These variances do not arise because people are poorly motivated or lazy, but rather because people are highly motivated, and the variances are usually necessary for real work to be done. The fact that humans resolve these conflicts so smoothly, however, usually means that the tensions inherent in differing functions are hidden from view. Technology deployments bring out these tensions, and force someone to decide, in advance, how the tradeoff should be resolved. When automation is implemented in an organization without the benefit of operational knowledge (i.e., without taking into account what the organization is trying to do), the tradeoffs involved in deciding what to deploy are made in a vacuum—and often wrongly. Only the senior operational management is in a position to articulate explicitly how these tradeoffs should be resolved. Beyond the need to develop an enterprise architecture to manage the IT investment process, it is important to recognize that the activity of developing an enterprise architecture often results in a (highly desirable) reexamination of operational processes. That is, by focusing on what technology can and should do for an organization, attention is naturally drawn to the processes it is intended to support—and often the processes themselves are seen to be out-moded, unnecessary, or ripe for streamlining, often because they were based on the limitations of older technology or on requirements that are no longer current. Such discoveries are valuable in themselves in that they help an organization function more efficiently even apart from its investments in technology. Such opportunities can be expected to arise when examining the processes of investigation and intelligence at the FBI. Finally, as missions evolve, so must an enterprise architecture—and in that sense, an enterprise architecture is never final. Since it is only the senior operational leadership that can make basic decisions about the organization’s missions, they will need to have a continuing involvement in the evolution of the enterprise architecture and a process to periodically revisit it. This point again reinforces the idea that an articulation of the operational dimensions of an enterprise architecture cannot be outsourced. 2.1.2 FBI Activities with Respect to Enterprise Architecture Based on FBI briefings and presentations to the committee, the committee believes that the FBI’s efforts and results in the area of enterprise architecture are late, limited, and fall far short of what is required. Indeed, in spite of the fact that the Trilogy projects are far along and substantial resources have been expended, the FBI has only very recently begun serious efforts

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program to create its enterprise architecture, and no comprehensive enterprise architecture is in place today.8 For example, the FBI reported to the General Accounting Office that it has completed and approved an enterprise architecture “foundation document.”9 However, the FBI made no reference to this document or its content in its October 2003 and December 2003 presentations to the committee, despite its publication on August 23, 2003, and despite the fact that the committee raised questions about enterprise architecture many times during those sessions. Ultimately, the committee learned of this document by reviewing a DOJ inspector general report, and it obtained a copy of the document in early 2004. Committee members have reviewed this document and believe that the document does begin to discuss at a high level some of the issues discussed above about linking the IT system to missions and operational processes. However, little follow-on work appears to have taken place as of the time of this writing (late March 2004). In addition, the FBI has undertaken other efforts that could reasonably represent seeds of architectural work that needs to be done more broadly. The Office of Intelligence has embarked on a laudable effort to develop the beginnings of its own architectural sub-framework, and spokespeople from that office were able to relate IT requirements to operational process at a high level in a convincing manner. Those responsible for the VCF project have also developed an architectural sub-framework that appears adequate to support the initial rollout of the application in 2004. The VCF may well be a component that can be made, with some effort, to fit into the enterprise architecture when it is created. Nevertheless, these architectural efforts—important though they are—do not substitute for an overall architecture that addresses the missions of the FBI collectively. The committee’s concerns about inadequate enterprise architecture work are reinforced by the fact that only in the FY2005 budget enhancement request will the bureau include very substantial funding for enterprise architecture related activities, and that this request was under consideration at DOJ and the Office of Management and Budget (OMB) as of September 2003. (The FBI is further seeking to obtain an interim system engineering, integration, and test contractor to blend the Trilogy VCF, the Secure Counterterrorism Operational Prototype Environment (SCOPE), and the Integrated Data Warehouse (IDW), and several smaller efforts, into a unified and functioning whole.10) Under the best possible circumstances, this FY2005 budget enhancement request will take effect October 1, 2004, suggesting that FBI effort in this area will not be substantial at least until then. Moreover, the fact that this request has taken the form of a budget “enhancement” for FY2005 implies that the FBI has given this task a lower priority than might be implied, for example, by a request for authority to reprogram existing funds from FY2004 for this effort. In the absence of a serious enterprise architecture effort, even the interim plan to blend the VCF, SCOPE, and the IDW will be very difficult to achieve. 8   In this regard, the committee’s conclusion echoes the primary finding of the General Accounting Office report The FBI Needs an Enterprise Architecture to Guide Its Modernization Activities (General Accounting Office, GAO-03-959, September 2003, available at http://www.gao.gov/new.items/d03959.pdf). 9   See Response of the FBI to the GAO report The FBI Needs an Enterprise Architecture to Guide Its Modernization Activities, dated September 22, 2003. Available at http://www.gao.gov/new.items/d04190r.pdf. 10   See Response of the FBI to the GAO report The FBI Needs an Enterprise Architecture to Guide Its Modernization Activities, dated September 22, 2003. Available at http://www.gao.gov/new.items/d04190r.pdf.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program Beyond the lack of anything resembling a complete enterprise architecture, of most concern to the committee is the fact that most of the senior operational management of the FBI does not seem to have been deeply engaged in either the nascent architecture efforts or in shaping the Trilogy effort. For example, the Enterprise Architecture Foundation Document was produced by an entity called the Architecture Governance Board, whose members are apparently drawn from the FBI IT community—and did not include senior individuals from the major operational units of the bureau. That is, it does not appear to reflect substantial involvement by the senior operational leadership of the FBI and hence cannot be sufficiently complete to guide the FBI’s IT development and deployment efforts. This must change. For the IT modernization to succeed, frequent and engaged participation by the FBI’s senior operational management in the creation and review of an enterprise architecture is necessary, and must be followed up with ongoing and systematic monitoring of the FBI’s and contractor plans and progress through implementation. A properly designed process for developing an enterprise architecture can allow senior management to play their essential role without placing excessive demands on their time, and being actively engaged in the creation of an enterprise architecture does not necessarily mean that the senior operational management is responsible for every step. Rather, as the enterprise architecture is being created, the senior operational management must be involved in sessions at which there is serious discussion of the enterprise architecture’s contents, and where important decisions are made. Many of these issues and decisions involve policy questions, including a determination of whether a relevant policy exists and thus should be reflected in the enterprise architecture, or needs to be created. The committee believes that the absence of an enterprise architecture has been a major contributor to the problems faced by the implementers of the Trilogy program. That is, the lack of an architecture to guide the planning of an information and communication infrastructure has resulted in improvisation that has virtually no chance of resulting in a well-ordered infrastructure for the enterprise to build upon. In fact, merely providing parts (e.g., computers and accessories, piece-part applications, and so on) is like buying brick, mortar, and lumber and expecting a builder to produce a functional building without benefit of building codes, blueprints, or an understanding of how people will use the building. 2.1.3 Data Management Issues Arising from the Absence of an Enterprise Architecture The FBI’s lack of an enterprise architecture has manifested itself in many ways. For example, as noted above, the counterterrorism mission requires access to and the exchange of comprehensive and timely information. Thus, it is easy to imagine that the FBI will, under some set of circumstances, need to receive and/or send information from or to state and local law enforcement agencies (e.g., local police departments), its counterparts in foreign nations, or other members of the U.S. intelligence community.11 The identification of precisely which 11   A proof-of-principle project (the Gateway Information Sharing Project) was established in St. Louis in April 2002 to integrate investigative data from federal, state, and local law enforcement agencies into one database that will ultimately be accessible to all participating agencies via a secure Internet connection. The project merges investigative files and records from all levels of law enforcement into a single, searchable data warehouse, providing investigators and analysts the ability to search the actual text of investigative records for names, addresses, phone numbers, scars, marks, tattoos, weapons, vehicles, and phrases. The project is the result of cooperation between the FBI and a variety of state and local police departments in the St. Louis area and Southern Illinois. See DOJ press release of October 9, 2002, “Attorney General John Ashcroft Unveils Gateway Information Sharing Pilot Project in St. Louis, Missouri.” Available at http://www.usdoj.gov/opa/pr/2002/October/02_ag_590.htm.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program parties the FBI will share information with and the conditions under which information will be shared has profound implications for its enterprise architecture, and is an issue that the senior operational leadership must address. A second issue is a potential confusion between mission and process. In discussions with the committee, senior FBI personnel repeatedly stated that “investigation and intelligence are the same thing.” While it is true that the mission of criminal investigation makes use of intelligence processes, this fact does not mean that an IT infrastructure to support investigation should be the same as an IT infrastructure that can support intelligence. For example, the measure of success of an infrastructure that supports investigation is whether it helps agents to do their investigation more effectively, more efficiently, or with greater quality of output and national impact. Agents are the primary actors in criminal investigations, and they task analysts; thus, the developers of an IT infrastructure for investigation should regard agents as their primary customers. By contrast, the mission of counterterrorism is intelligence-heavy. The measure of success of an IT infrastructure for intelligence is whether it helps the analyst perform analytical functions more effectively/efficiently, e.g., helps analysts identify patterns, clues, or other information generally indicative of events that have not yet occurred. In the intelligence process, analysts are the primary actors, in that they are the ones primarily responsible for analyzing a wide variety of information, from both open and classified sources, in order to identify impending terrorist actions that must be disrupted or pre-empted. Thus, agents will act on the information given them by analysts, and those actions are generally mapped into a criminal investigation. Presumably, agents also serve as collectors in the counterterrorism mission, and therefore can be tasked by analysts. The following are some important differences that must be kept visible in designing IT support for the investigation and intelligence processes: Analysts tend to place a higher priority on access to a broad array of information resources, while investigators tend to place a higher priority on highly targeted information. Analysts may find “bad” or “untrustworthy” information contextually useful, while investigators may regard it as a liability and would want such information purged as soon as possible. Analysts must understand context and how a situation changes over time; investigators often place a higher value on information that can serve as evidence and fact. Analysts must handle and manage classified data, while investigators must generally, though not always, manage “law enforcement sensitive” or “sensitive but unclassified” data, which entails an entirely different set of requirements.12 In general, criminal investigation and intelligence are sufficiently different processes that the FBI would be wise to view the development of an overall enterprise architecture, particularly at the systems level, as calling for two subarchitectures with well-defined interfaces between them. An interface between the two subarchitectures is defined by the data that they exchange and share. These interfaces will require enforcement and oversight so that only 12   There are exceptions to these generalizations, of course (e.g., investigations for counterintelligence purposes often generate classified information). But in any case, these exceptions represent only a small fraction of the total information collected.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program necessary data are passed between them, and it will take senior operational management attention to define what data are “necessary” and what the policies are for the interface engines. Further, these interfaces have profound implications for the systems and technical architectures in areas such as standards that facilitate data exchange. 2.2 DESIGNING IT SYSTEMS TO SUPPORT FBI STRATEGY AND OPERATIONAL NEEDS This section comments specifically on a few important applications design issues, including overall system structure; the design of data repositories; and workflow, security, and access, as well as a collection of specific deficiencies in the VCF and other applications. In most cases, these issues are a consequence of the lack of an enterprise architecture, and thus of the disconnect between enterprise architecture and technology planning and design. 2.2.1 The Virtual Case File Application of Trilogy The Virtual Case File, the user application component of Trilogy, is a custom-designed software application that is intended to facilitate case file management by integrating data from older, separate investigative systems, including the Automated Case Support (ACS) system, and eventually replacing them. The VCF is intended to create efficiencies in entering case-related information by reducing the number of steps in filing documents and to facilitate the storage and retrieval of data for wider access, tracking, and analysis of case-related data.13 The VCF is a new IT application designed to be the primary operations system to support the investigative mandate of the FBI. The design process was under way prior to the addition of the intelligence mission, and the requirements for the intelligence mission were not included in the VCF design. The VCF is a significant step forward from today’s ACS system, and considerable progress on the VCF seems to have been made since September 2002. (The committee underscores the term “seems,” because it was shown only a mock-up of the application, and not the application itself. That is, the committee viewed a canned presentation, because no working application was available at FBI headquarters. The committee notes that there is often a significant difference between the impression conveyed by a mock-up and what an actual application can do in practice.) Perhaps the most important—and commendable—development in the VCF effort is the appointment of a very experienced and computer-savvy FBI special agent as program manager who has played a strong role in driving the design from user requirements. To the committee, this individual appeared eminently capable of articulating user needs based on operational experience rather than speculation, and the committee believes that it is this manager’s operational insights that served as an implicit enterprise architecture (more precisely, a subarchitecture) for the VCF and that have consequently been a primary driver of significant progress. As such, the VCF has the potential to serve agents well in their criminal investigation mission. 13   For further information about the virtual case file, see U.S. Department of Justice, Office of the Inspector General Audit Division, The Federal Bureau of Investigation’s Management of Information Technology Investments, Report No. 03-09, December 2002, pp. 91, 94-99, available at http://www.usdoj.gov/oig/audit/FBI/0309/final.pdf.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program remote agent access in the field; this process is known as a risk management assessment. Nevertheless, the tension cannot be eliminated, and only the FBI’s senior operational management is in a position to judge how this tension should be resolved. A third security tradeoff will manifest itself when mobile computing becomes an issue. The value of mobile computing is likely to be high, but mobile computing is inherently more vulnerable than office-based computing, and the senior operational leadership will have to decide if the increased security risks are worth the added operational flexibility. Empirical data and a formal evaluation process for mobile computing would be useful inputs to the leadership. The FBI’s PDA experiment will help the leadership to think through how to differentiate and measure communications that involve classified information, SBU information, and unclassified information, each of which has different security requirements. In the absence of any specific information on the subject but based on previous experience, the committee suspects that a significant fraction of, if not most, FBI data communications traffic need not be carried at the classified level. If the measurements associated with a PDA trial support this conjecture, the FBI may be in position to take advantage of wireless data communications in designing an infrastructure, systems, and processes to implement its enterprise architecture. Such systems and policies could differentiate between SBU and unclassified communication, handle them differently, and significantly improve FBI metrics for efficiency and effectiveness. If this conjecture turns out to be true, the FBI would also have to establish a clear policy governing SBU communications. The committee is also concerned about a number of security issues related to implementation. These include: The use of passwords for authentication. The weaknesses of passwords as an authentication mechanism are well known (e.g., they are easily compromised without the owner’s knowledge),20 and yet no thought seemed to have been given to alternatives. The lack of consistent security models between the IDW and the VCF. In particular, this inconsistency suggested that searches of data contained in the VCF from the data warehouse side would not be made visible to case owners. Moreover, as long as security mechanisms only provide access control, and do not log the information that is accessed, then misuse of systems by authorized users will not be visible until the information turns up in the wrong places. (And, of course, logs must be reviewed regularly by automated log analysis tools supporting human analysts.) The operating system monoculture. The Trilogy IT modernization put into place a single operating system environment, and the security vulnerabilities of an operating system monoculture are well known.21 Such an environment carries with it the risk that a single exploit, such as a worm or virus, can result in global failure of the Trilogy network.22 The technology 20   See, for example, Department of Defense, Password Management Guidelines, April 12, 1985, available at http://www.radium.ncsc.mil/tpep/library/rainbow/CSC-STD-002-85.html. 21   National Research Council, Cybersecurity Today and Tomorrow: Pay Now or Pay Later, Computer Science and Telecommunications Board, National Academy Press, Washington, D.C., 2002. 22   Note also that security risks may be especially great during a time of transition between old and new systems. The reason is that anomalous system behavior can have multiple causes, and may reflect operator inexperience, inherent system bugs, or adversary-planted exploitation. Uncertainty about the actual cause may lead to inaction for a longer period of time than would be the case during normal operation.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program deployment plan relies on isolation from the Internet and diligent network monitoring and response to protect against such problems. But neither of these approaches, taken singly or in combination, can eliminate the risk of catastrophic failure, and recent worm/virus experience has shown that self-propagating, malicious code can enter isolated networks through an unwitting laptop user connecting to the isolated network after having been connected to, and infected through, the Internet. Furthermore, the propagation speed of modern worm/virus code is sufficiently rapid to overwhelm any monitoring and response center staffed by humans. Seemingly inadequate contingency plans for operating under attack. A basic principle of managing a critical operational network is that plans for maintaining operation in the face of a compromised element must be made in advance. How are nodes in the network protected against other nodes that may have been compromised? Rather than a simple perimeter defense, a principle of mutual suspicion should apply that does not allow an attacker who has breached the perimeter to have free access to the entire network.23 How will the FBI’s public-key infrastructure be managed in the face of an insider threat? How will key revocation be handled? Emergency re-keying? Access overrides (i.e., how will authorized parties obtain access if access tokens and/or information are lost?) How will worms and viruses be purged from the system if they are introduced? How will the network respond to a denial-of-service (DOS) attack? There are many different flavors of DOS attacks, ranging from a deliberate attack on the inside of the network to an attack occurring on the “outside” of the network that merely consumes bandwidth and degrades the performance of the public links that carry the virtual private networks of the FBI. Missing or incomplete validation of a security architecture, which the committee inferred on the basis of presentations to it. Specifically, the FBI did not provide to the committee evidence that it had validated: The putative design principle for security, which seemed to be characterized by an approach of “build to DISA (Defense Information Systems Agency) gold standard and back off until tools work.” The model of the threat that faces the FBI’s IT systems. (Note that a threat model must include threats emanating from both inside and outside the organization.) The use of the electronic key management system developed by the National Security Agency (NSA) in the context of the specific needs of the FBI. In each of these cases (the design principle, the threat model, the use of the NSA electronic key management system), the approach may be plausible and ultimately desirable. But a process of systematic validation is necessary before those conclusions can be drawn. More generally, a security architecture is based on an understanding of the data flows in the organization (derived from the enterprise architecture). Based on these data flows, the databases that need to be isolated are identified. When necessary, controlled interfaces between isolated and non-isolated databases are also identified. These interfaces are then mapped onto the physical network configuration to decide what links/nodes must be encrypted, physically protected, or both. Lastly, the security architecture requires both a plan for managing encryption keys and a vulnerability assessment (i.e., a paper attack on the paper architecture). 23   To illustrate, this principle might call for the use of firewalls on every node of the network and anti-virus protection running continuously on all computers connected to the network to mitigate denial-of-service or intrusion risks.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program Potentially inadequate proactive counterintelligence against inappropriately trusted insiders. As the Robert Hanssen case illustrates, security must be maintained against deliberate exfiltration of sensitive or classified information. Counterintelligence tools for this problem include logging of access, analysis of logs, and possibly filtering of outgoing information. Note also that notifying the owner of a sensitive information item that some other party has accessed that information is a potentially powerful way of raising red flags. Potentially inadequate security of hardware and software implementation. When hardware is deployed and software is written by outside contractors, there are greater risks that hostile parties working for the contractors may seek to insert trapdoors and other ways of bypassing security in the systems being deployed. As a general rule, these risks can be managed, but the weakness of the FBI’s contract management abilities may mean that the FBI’s efforts in this area are not up to par (e.g., as with Defense Department special access projects). The same considerations apply to commercially acquired infrastructure software. Though it might be difficult for an attacker to compromise a component of such software, the possibility cannot be ruled out. Moreover, even security patches and upgrades are not always guaranteed to work properly, which means that installation of such upgrades cannot be undertaken blindly or without explicit thought from IT-savvy managers. 2.2.6 Privacy As part of the FBI’s briefings to the committee, the FBI’s privacy office addressed the committee, from which the committee concluded that the FBI did have some sensitivity to some of the relevant privacy issues. However, the committee has not undertaken a comprehensive study of protection of individual and corporate privacy in the context of the FBI’s IT modernization program, and thus the comments below can only suggest some of the issues that may be involved. The FBI must proceed with great sensitivity to privacy issues as the counterterrorism mission assumes greater importance, and both substance and perception are important in this domain. There are many substantive issues associated with privacy. For example, The FBI privacy office seemed to the committee to be geared primarily to protecting personally identifiable information from being improperly shared outside the FBI (e.g., with contractors), and in the short presentation to the committee, its approaches to this problem seemed reasonable. Another privacy issue not addressed in briefings but of great concern to the privacy-sensitive segment of the public is the protection of the public from the abuse or improper use of such data by rogue FBI employees acting on their own or through some official though perhaps not publicly acknowledged FBI program. The committee has no comment on this issue, other than noting that the simple exhortation “trust us” is not likely to be reassuring to this segment even if in fact the FBI is doing everything possible to prevent such abuses from happening.24 24   Note also that Section 223 of the USA PATRIOT Act allows individuals to recover monetary damages and litigation costs from the United States in the event that information or records are willfully and improperly disclosed, a fact that increases the importance of keeping good records of who is accessing what data and under what circumstances. In addition, federal employees found to have improperly disclosed information are also subject to administrative action under the PATRIOT Act.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program Perception is at least as important as substance. That is, a technically well-conceived privacy protection program solves the substantive problem. But even with such a program in place, the FBI must also pay attention to external perceptions, because those are a major influence on the public trust that the FBI must maintain in order to be effective. 2.3 ENSURING EFFECTIVE MANAGEMENT OF IT DEVELOPMENT AND IMPLEMENTATION This section addresses a variety of serious management issues in the FBI’s IT modernization program. Issues and problems identified here relate to the management approaches and processes used by the FBI across the implementation life cycle, including requirements determination, development methodology, contracting and project management, system rollout, training, evaluation, and metrics of effectiveness. While the committee’s comments regarding implementation are derived from its consideration of the Trilogy infrastructure and VCF projects only, the committee believes that dealing systematically with these issues is essential to success in follow-on efforts such as the IDW and SCOPE. 2.3.1 Overall Development Methodology In the committee’s judgment, the FBI’s management approach to the Trilogy modernization violates a number of basic principles that should govern the development of large systems. Principle 1: Any organization that depends on the continued operation of its IT systems and is modernizing those systems should plan on the simultaneous operation of both old and new systems for some period of time, so that failures in the new system do not cripple the organization. Large-scale deployments involving new technology almost always come with problems (e.g., system crashes, unacceptably slow performance), and there is some risk that the problems will be so severe as to prevent the effective use of the entire system for some period of time. Accordingly, there must be backup and contingency plans in place that anticipate a wide range of failure conditions. However, the committee saw little evidence that such plans had been formalized, and indeed has been informed that the current transition plan does not envision the availability of the ACS after the cutover to the VCF, even though the hardware supporting the ACS is neither being redeployed to support the VCF nor being immediately decommissioned. Effective transitions are usually managed gradually. Even where the transition is abrupt, maintaining the function of the old system for some period as a fallback option is good practice. Any ACS-to-VCF transition without a backup plan is far too risky for the FBI and for the nation, especially in light of the FBI’s myriad and critical operational responsibilities to the nation that will continue during any such transition. The costs of maintaining a fallback plan and a supporting infrastructure are small compared to the operational costs associated with large-scale VCF problems. Moreover, because the hardware supporting the ACS will not be converted to support the VCF, there is a residual and pragmatic disaster-recovery capability to revert to the ACS. But contingency plans that anticipate a continuing if temporary need for the ACS must be made before the transition to exploit this residual capability, or else the use of the ACS in this contingency mode will suffer.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program Expenditures to maintain an infrastructure that can support concurrent use of the ACS and the VCF for a limited transition period would be a worthwhile insurance policy. Principle 2: Any organization undertaking a large-scale IT system development and deployment, but especially an organization without a strong track record in IT development and use, should develop, test, evaluate, and iterate a small-scale prototype before committing itself to an organization-wide program. The development methodology for Trilogy seems to be based on a one-way non-iterative process where rigid specifications are generated in advance. Subsequent changes can then be argued to be “specification changes” that open the door for schedule delay and increased expense. In truth, it is essentially impossible even for the most operationally experienced applications developers to be able to anticipate in detail all of the requirements and specifications in advance. Therefore, development contracts should not make such an assumption, but rather should call for an approach to specification of user requirements that is based on a process of extensive prototyping and usability testing with real users.25 To the best of the committee’s knowledge, apart from SCOPE, no prototype has been developed for any of the major components of Trilogy (the Trilogy network or the VCF) or for the IDW. As the FBI recognized in developing SCOPE, the advantage of a small-scale prototype is that it can be iteratively developed with strong user feedback and involvement, thus increasing the chances that what is ultimately delivered to the end users meets their needs. This point is relevant to many dimensions of system development, including the functionality desired in a new application, the convenience and intuitiveness of a user interface, and the nature and mix of the operating load that the system must support under real operational conditions. In some instances, the intimate involvement of project managers with extensive operational experience can play an important role in some of these dimensions (e.g., defining the required functionality); indeed, the committee believes that it is the involvement of such an individual in the development of the VCF that has been responsible for what success it is likely to have when it is deployed. Nevertheless, in the end there is no substitute for realism in the evolution of a prototype. Principle 3: In large-scale system development and deployment, testing should account for as much as 35 to 50 percent of the project schedule if a successful deployment is to be achieved. Testing is necessary to shake out bugs and flaws in a new system, and flaws will often be invisible until after actual users deal with actual data. The 35 to 50 percent figure above is an experience-based rule of thumb that is widely accepted among those involved with large-scale system design. Moreover, an organization without a strong track record in IT could reasonably be expected to require even more time for system testing and integration. Testing is a 25   These comments do not mean that prototyping is a panacea that guarantees success. While prototyping usually has high value in developing requirements for user functionality, it produces results that are only as valid as the user group selected for involvement. Furthermore, obtaining realistic results from prototyping exercises requires using real-world data. Finally, prototyping is most valuable when understanding user requirements is the task at hand; it is less useful for determining reliability and response times for applications that will be deployed on a large scale.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program critical dimension of system development and deployment requiring a well-developed testing strategy and a progression from unit testing to full-blown integration testing, and usually calls for dress rehearsals for system use. The project schedule for completion of Trilogy as it was represented to the committee in October 2003 appears to leave inadequate time for testing. The committee was shown briefing charts indicating that the FBI allocated less than 10 percent of its schedule for testing, and under schedule pressure the contractor was trimming that amount even further. It was also surprising for the committee not to see, at a very late stage in VCF development, any kind of risk-based prioritization for the items remaining to be done. Instead, these items were lumped together in the category “unfixed bugs.” To provide a simple example of the need for testing, it is important to realize that the term “works” (as in the system “works”) has a wide spectrum of possible meaning. Consider a simple system, consisting of a network transport layer that moves packets and is responsible for encryption and decryption and an application running on top of it. Beginning with the least demanding definition, the expression “the system works” can plausibly mean that: The transport layer “lights up” but does not move application packets. Nodes can find each other, basic protocol functions such as “ping” work, and bits flow between a subset of the network and on the whole network. Definition (1), and in addition data packets move between a subset of the network and on the whole network. Definition (2), and in addition encryption and decryption are successful between nodes on a subset of the network and on the whole network. Definition (3), and in addition the transport layer can move application packets under simulated “realistic” load between nodes on a subset of the network and on the whole network. Definition (4), and in addition users in different locations can use the application to pass the appropriate information among each other. Definition (5), and in addition the system functions with actual users using actual data. Definition (6) with realistic numbers of users and the full complement of files. Definition (7) with some reasonable assurance of the overall security of the applications, as opposed to the encrypted transport of bits alone. In the absence of a clearly specified test plan that is acceptable to all users, it is possible to make a claim that the system “works” according to any of these definitions, even though the point of failure in the development of many systems is encountered when they are required to continue to function under load. It is unclear for Trilogy which definition of “working” is being used by those claiming that a system is working. In the case of the Trilogy infrastructure network, the VCF application is not yet available, and so it is clear that the higher levels of functionality (definition (5) and above) have not yet been attained. 2.3.2 Contracting and Contract Management Both key contracts for Trilogy (Trilogy infrastructure and the VCF) were awarded on a cost plus/cost reimbursable basis. Only optional “tools” were awarded on a fixed-price basis. While the task orders (T0001AJM026 and T001AJM028) for both phases of the Trilogy program

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program detail the firm fixed price to eight or nine significant figures, the task order schedules were almost totally lacking in specifications and a commitment to checkpoints. For example, the documents list a large number of plans to be created, with dates “TBD.” (The committee believes that one reason underlying the lack of detail in these critical documents is that the underlying architectural work has not been done. In the absence of such architectural work, system specifications are hard to develop with any coherence.) Under these circumstances, effective program and contract management becomes essentially impossible. This weakness, aggravated by turnover among key FBI staff (e.g., the FBI chief information officer), makes it unsurprising that Trilogy is significantly behind schedule and over budget. To illustrate some of the problems with contract management, the FBI told the committee that the Trilogy network had been made operational on March 28, 2003, only because FBI personnel in the program office were pressed into overtime service to compensate when contractors failed to meet commitments. While such efforts point to the FBI’s admirable dedication to duty, the need for the program office to stand in for a contractor is a sign of contractor failure. If these “above and beyond” efforts of FBI staff could be converted into penalties for (or dismissals of) the contractors, it would send a very positive signal that the FBI program office is becoming serious about contract management. For a contract of this magnitude and importance to the FBI and to the United States, it is imperative that senior management of the FBI monitor contractor progress closely and step in when necessary to forestall difficulties seen down the road, although day-to-day involvement is not necessary. Senior-level contract managers, experienced and empowered, should be assigned for the duration of the project, and should provide periodic actionable status information to FBI senior management. In briefings, it appeared to the committee that contractors may be viewing this project as governmental “business as usual,” without due regard to the critical importance and congressional visibility of Trilogy’s success or failure. Clear and detailed schedules with intermediate milestones, earned-value metrics, and severe penalties for missed delivery dates and missing functionality are desperately needed. 2.3.3 Program Management In the committee’s judgment, the FBI’s program management of its IT projects is weak, and has been weak for over a year (based on the individual observations of many of the committee members who participated in the September 2002 meeting). Weak program management leads to a reactive mode in dealing with issues and a lack of overall project control. Continued cost overruns and delays often result from a lack of effective program management. By contrast, effective program management with effective management of costs and proactively managed IT projects significantly increases the likelihood of success. For the committee, one of the most serious issues is that many essential tasks have been outsourced to vendors (e.g., development of data models and architectures). In essence, the FBI has left the task of defining and identifying its essential operational processes and its IT concept of operations to outsiders. This is not to say that outside contractors should have no role at all in these tasks, but rather that it should be the FBI, not the contractors, that defines and drives the process. A small but telling example of the FBI’s dependence on contractors is that the FBI reported no contract provisions calling for the escrow of source code for the applications. Escrow calls for the deposit of source code files and appropriate documentation with a mutually trusted

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program third party during and after the completion of the contract. In the event that the vendor becomes unable or unwilling to continue to provide service to the FBI (e.g., the developer must be replaced because of poor performance, or the developer goes bankrupt), the source code is released to the FBI so that it can seek another vendor to assume the first vendor’s responsibilities. Source code escrow is a common, even routine, commercial practice. Program management is a well-understood discipline and set of processes. A strong program management function must be established and supported by FBI management at all levels to provide appropriate oversight and management of FBI IT projects. An effective program management function will provide the FBI with a focal point for monitoring and collecting project data and allow for the reporting of the progress of active IT projects based on well-defined metrics. (Note that program management entails a set of skills and background different from those associated with operational experience in doing investigation or intelligence analysis. That is, a single individual may have strong program management skills and extensive operational experience, and arguably a “program manager” ought to have both qualities, but there is no a priori reason to expect that someone with the former will necessarily have the latter. In any event, a program manager should at least have access to both sets of expertise.) For program managers to be effective, they must participate deeply in the negotiation of contracts with outside vendors. The contracts must include performance measures for key deliverables, milestones, and service levels with penalties and escalation procedures. Contracts should ensure that the vendor suffers severe penalties if it is not meeting the performance measures, and major vendor failure should result in a penalty that allows the FBI to transition to a new vendor with little or no financial impact to the FBI. Program managers are responsible for validating, monitoring, supporting, and assisting in the area of project and life-cycle costs. They may also supply information that can alter project priorities, based on resource availability or interdependent project conflicts. Most important, program managers are responsible for implementing an appropriate methodology for project management, including at least the following: Establishing a framework for all project communications, reporting, procedural, and contractual activity; Developing task-specific project plans with timelines, critical paths, and specific deliverables; Conducting project team meetings, setting schedules and agendas, and managing the meetings; Monitoring milestones and deliverables and ensuring tracking to timelines; Providing project status updates to key stakeholders; Identifying deviations from plan and addressing issues on a timely basis for resolution; Managing project budgets; and Creating an environment where the project team can succeed. An important point is that effective program management generally requires close and frequent interactions between managers and the project team and among team members. Besides well-trained staff, the team also needs an environment that facilitates working effectively, such as proximity of offices, meeting spaces, and areas in which information can be passed informally over lunch or in a chance hallway encounter rather than relying only on

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program formal channels of communication such as e-mail, phone calls, voice mail, teleconferencing, and meetings of high-level managers.26 A particularly useful management tool is a project charter. The charter includes detailed project requirements, resources, and the related roles of the project team, along with identified milestones and deliverables. Risk, issue and scope management, and communication plans are included. The project charter is designed to do several important things. The charter should: Facilitate communication among stakeholders. Document approved purpose, scope, objectives, cost, and schedule baselines. Document the agreement between groups. Define roles and responsibilities inclusive of the overall project governance. Document planning assumptions and decisions. Provide the baseline for scope and expectation management. In effect, the charter establishes a baseline of understanding among all stakeholders and especially between the project manager and the project sponsor(s), business owner(s), and vendor(s). The success of the project may be measured against this charter. Just as importantly, any changes must reference the charter. 2.4 ENSURING THE GROWTH OF FBI IT EXPERTISE AND DEALING WITH EXTERNAL FACTORS The FBI recognizes, and the committee agrees, that the FBI is severely lacking in many of the technical and management skills to successfully plan, contract for, and implement a project of the breadth and scope of Trilogy. This situation must be remedied both to complete the initial phases of Trilogy, and more important, to deal with the issues described in the preceding sections. In addition to filling the top jobs such as the CIO with people who are willing to and capable of effectively managing the larger issues, a number of other gaps and constraints, some not of the FBI’s making, must be addressed. 2.4.1 Human Resources With a few exceptions, the presentations to the committee persuaded it that the FBI lacks a human resource base adequate to deal with its IT modernization program. The FBI lacks experienced IT program managers and contract managers, which has made it unable to deal aggressively or effectively with its contractors. Inexperienced managers generally lack the ability to assume proactive management roles and are often held hostage to the perspectives of the contractor. Given the importance of IT personnel and analysts to the FBI’s broadening missions, such individuals must have career track opportunities that have status and respect that are compa- 26   Although many large organizations are able to effectively manage large-scale efforts with geographically dispersed staff, success in this regard requires a considerable degree of IT sophistication and adherence to well-defined and well-understood IT policies and practices.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program rable to those of traditional personnel tracks (such as agents) within the bureau.27 Such tracks would also enable stability in IT jobs that require extended oversight and long institutional memory. On the positive side, the committee wishes to single out as especially noteworthy the presentations it heard from the head of security for the IT modernization program, the executive assistant director for intelligence, and the VCF project manager. These presentations reflected serious thought about their respective responsibilities from the user perspective. For example, the executive assistant director for intelligence presented a coherent and well-articulated concept of operations, as well as a clear vision of how to draw talent from the rest of the FBI. The presentations of the head of security, though not long enough to provide very much detail, were well-considered. And, the VCF project manager’s operational experience has been invaluable in keeping the VCF intellectually on track. Although the committee met only a few of them, it believes that the FBI has an important IT resource in its younger agents. The FBI agents now being hired are in their late 20’s, and, as with others in their peer group, have been heavily exposed to modern forms of information technology for most of their lives. Given this fluency with information technology, it is likely that these newest agents will be the most enthusiastic about embracing new technologies to enhance their effectiveness. (Of course, the flip side is that if the organization in which they serve is unable to provide technology tools that they believe they ought to have and perceive to be adequate, they are likely to be disappointed in the organization, with all of the consequences regarding morale, work attitude, and retention.) Finally, the committee notes that the FBI has the authority to obtain IT personnel from the private sector.28 Under the Intergovernmental Personnel Act (IPA), the FBI can borrow personnel from other government agencies (federal, state, and local), from institutions of higher education, and from federally funded research and development centers. Generally, these arrangements call for the “home” agency or institution to pay the salary of the employee while on detail status, though the FBI could agree to reimburse the home agency as part of the terms of the cooperative agreement. IPA details cannot exceed 4 years in duration. Under the Information Technology Exchange Program (ITEP), the FBI can enter into a cooperative agreement with a private sector organization for the exchange of personnel who work in the field of IT management, are considered exceptional performers, and are expected to assume increased IT management responsibilities in the future. While on detail to the FBI, the private sector organization employee may continue to receive pay and benefits from his/ her employer, and the assignment may be made with or without reimbursement by the FBI for the pay, or a part thereof, during the assignment and for any contribution to the private sector organization’s employee benefit systems. ITEP details cannot exceed 2 years in duration. The FBI also has the authority to hire IT employees outright. Senior IT positions in the FBI fall under the FBI’s Senior Executive Service (SES) pay scale, the range of which is $103,700 to $157,000. In exceptional cases, the FBI can pay up to $174,500, though this step requires Office of Personnel Management and Office of Management and Budget approval. The FBI is also authorized to pay a lump-sum recruitment bonus of up to 25 percent of the annual rate of basic 27   Note that the problem of enhancing the status of workers in a critical but supporting field is common in many organizations, both inside and outside the federal government. 28   Information on FBI hiring authorities is derived from a memo from the FBI to the committee transmitted on February 11, 2004.

OCR for page 16
A Review of the FBI’s Trilogy Information Technology Modernization Program pay to an employee, including individuals covered by the SES, who are newly appointed to a difficult-to-fill position. The FBI also has available positions classified as Scientific or Professional positions that are above the GS-15 level but do not meet the SES functional criteria. These positions encompass the performance of high-level research and development in the physical, biological, medical, or engineering sciences, or a closely related field. The pay scale for such positions in the Washington, D.C., area ranges from $117,627 to $144,600. In the judgment of the committee, these rates of pay should be sufficient to attract highly qualified IT talent. While it is true that highly qualified senior IT personnel in the private sector can command significantly higher salaries, stock options/bonuses, and the like, the FBI also offers opportunities for public service that compensate, in part, for the lower salary scales. These opportunities include the chance to have a substantial impact in service to the nation in an area of extraordinary import. The recent shift in the opportunity environment for IT people has no doubt increased the relative attractiveness of government service. 2.4.2 External Constraints The FBI also operates under a variety of constraints that originate externally to the bureau. Two of the most significant are the following: Lack of management flexibility. Agile organizations are managerially flexible so that they can take prompt action. It is the committee’s understanding that the FBI is unable to take managerial actions such as reprogramming amounts in excess of $500,000 without explicit congressional approval. This constraint and others with a similar micromanagement flavor are inconsistent with the expectation that the FBI will move quickly and forcefully to reshape itself to deal effectively with the terrorism challenge. Audit pressure. The FBI reported that it had undergone more than 100 investigations and audits in the IT area. While it is true that the FBI’s performance in this area is seriously deficient (and this makes the FBI an attractive target), responding to such investigations uses the time of senior management, both in preparation and presentation of material to those investigators. And because personnel in the IT field are scarce within the FBI, the pace of work is slowed by such investigations. A better use of an equivalent amount of talent and energy would be to assist the FBI in dealing with its problems. Nevertheless, it remains the case that some audits, such as the one prepared by the General Accounting Office as The FBI Needs an Enterprise Architecture to Guide Its Modernization Activities of September 25, 2003 (released to the FBI August 22, 2003), contain valuable guidance. A third area of concern is the presence of inflexible and stringent rules for personnel qualification, though the committee is not certain whether these rules originate within the FBI or outside it. Stringent rules reduce the pool from which potential employees are drawn, and inflexible rules—even undertaken in the name of upholding standards—may inappropriately disqualify otherwise qualified individuals. Of course, job applicants whose history indicates a substantial variance from the standards for employment are unacceptable. But a rule of reason ought to apply, and an otherwise-qualified applicant whose history is only minimally at variance with those standards should not be automatically excluded.