Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report (1996)

Although Tax Systems Modernization (TSM) is much needed, the Internal Revenue Service (IRS) does not currently have a development organization capable of meeting the difficult challenge. This chapter discusses the technical capabilities to be provided through TSM and the IRS’s ability to develop those capabilities successfully and integrate them into its future business practices.

In any development project, regardless of its size and scope, the drive to succeed must be combined with an understanding of the technical, managerial, operational, and political problems in order to achieve success. Furthermore, throughout the development process, leaders must focus continuously on the overall goals to be achieved and must be prepared to modify their course of action in response to the ever-changing environment.

It is the committee’s judgment that TSM is in danger of being ultimately ineffective because in the IRS there is a lack of technical expertise and experience in managing large-scale system development, especially at senior levels, to design, construct, and deploy modern information systems that support the essential mission: collecting taxes more efficiently and providing better service. The committee hastens to note that the IRS, as an organization, has strong capabilities in a number of critical areas such as procurement and large-systems operations, and has made significant progress in recent years in some key areas such as organizational requirements and sensitivity to privacy. However, improvements primarily have been organizational, rather than procedural and cultural, and as such, the IRS has not been very successful in improving its ability to develop the capabilities in certain areas of expertise required by TSM (specifically with regard to the management and execution of large-scale systems development).

The critical problem for the IRS is that of actually creating the information-intensive future that it appropriately envisions. The committee agrees with the IRS with regard to the type of automation that is needed to modernize the tax system, including

• Flexible mechanisms for accepting and processing tax returns;

• An on-line database of information capable of handling most taxpayer interactions over the telephone; and

• Powerful tools for improving taxpayer compliance.

Unfortunately, the IRS’s current technical systems development process for achieving these goals appears to be inadequate. Such a situation is consistent with the facts that the IRS has never before undertaken an effort of this magnitude and that its technology base was woefully antiquated when it started the development process.

The key elements to achieving the stated goals successfully are the following:

• A clear vision of how the mandated activities of the IRS could be carried out in the future with the recognition that what is mandated may change (requirements definition);

• A plan, both technical and organizational, for getting from the current state to the desired future state (the development process);

• Leadership, from the Commissioner on down, that understands (at appropriate levels of detail) how to develop the information systems that are required to support the goals (leadership);

• Technical expertise (internal or contracted) necessary to build the future model (technical expertise); and

• Sufficient funding (funding).

These issues are discussed briefly in turn. In the remainder of this chapter, associated technical issues related to pertinent findings and recommendations are addressed in more detail.

Substantial progress has been made in the past 5 years on articulating a vision of future operations that uses information technology capabilities. The IRS developed, during this committee’s tenure, a business vision that stated concisely the overall goals of the modernization effort and a set of operational requirements derived from the needs of end users. For example, the Service Center Organization Study (SCOS) and the District Office Study (DOS) were conducted using modern business processing re-engineering (BPR) techniques in order to determine the present, near-term, and future needs of the operational units of the IRS. Those studies, responding to early Computer Science and Telecommunications Board recommendations,¹ were used not only to reorganize the operational elements of the IRS, but also to define future skill and corresponding automation require-

ments. As a result of those studies, the IRS defined an Interim Operating Capability (IOC) in which specific capabilities were targeted for delivery in the 1996–1997 time frame, in support of end-user changes being made.

The committee congratulates the IRS on its ability to undertake this type of top-down, business-driven approach; such enterprise-wide reviews seldom are accomplished by most organizations, let alone used to drive major operational reorganizations. However, it must be noted that SCOS and DOS were conducted well after TSM had started² and well after the critical IRS procurement vehicles had been defined and awarded. That is, the technical developers of TSM, involved in a large collection of projects, were already working on designs and systems before the business requirements were determined.

After SCOS and DOS were completed, some amount of retrofitting of the Design Master Plan (DMP) and the overall TSM schedule was performed, but there were no significant changes in the types and number of systems being developed. Such changes are common in any development program, especially one in which the design was performed before the requirements were understood. Clearly, the momentum generated by ongoing projects is difficult to change, but system(s) changes are critical if overall business objectives are to be achieved.

The IRS must develop a modernization process that allows it to adapt and evolve as modernization continues, always marching toward the overall goals. It cannot wait until every last requirement is determined, nor can it push development efforts too far before the fundamental requirements are understood by both users and developers. The IRS must “build a little, test a little, build a little, test a little,…” until its goals have been met. The output of SCOS and DOS is a very good starting point; now the IRS must follow its own analysis, using that to guide development and not the other way around. (This issue is treated below in the subsection titled “Focused Project Development.”)

Throughout the committee’s discussions, the IRS has stated continually that all of the projects within TSM would be “integrated” to form a powerful information processing system. For example, tax returns would be processed initially by a collection of systems, depending on their input format (i.e., Document Processing System (DPS), Electronic Filing (ELF), Telephone Filing (TeleFile)), all of which would generate the same “electronic tax return” and forward those data to central computer systems for storage. Similarly, a collection of end-user applications would retrieve tax data from the central databases and process them according to business needs, with the central databases being updated as necessary. Such an overall design is consistent with an enterprise-wide view of data processing at the IRS and is similar to many modern systems found in other government agencies and in commercial organizations.

In order to achieve such an integrated system, especially when developed over a long period of time, it is important to have a concise plan that can be used to guide the development of each piece. Such a plan is needed whether a “grand-design” approach is taken, in which the overall system is designed to replace an existing system, or an

“evolutionary” approach is taken, in which projects are developed and incorporated into an evolving system.

The IRS has stated clearly that it categorizes TSM as evolutionary, and the committee wholeheartedly agrees that such an approach is needed for the IRS. However, the committee believes that the IRS’s “plan” for guiding system evolution is deficient in some key aspects. Specifically, the committee believes that the following are missing or insufficient:

• An enterprise-wide systems architecture sufficient to support future systems;

• Enterprise-wide data models and standards;

• A plan for creating and evolving the technical infrastructure (telecommunications, standards, personnel training, workstations, and so on) necessary to support systems development and future operation;

• An applications architecture that will permit and regulate the development of a wide variety of applications in the future; and

• A systematic, coordinated project management system that will allow overall tracking and control of modernization efforts.

The committee believes that IRS systems development should be done to at least as high a set of standards as that of large corporations, if not higher (given the importance of its systems to the nation). Anything less puts a large part of the government’s information infrastructure at risk.

It is well understood that significant change in any organization must be led and driven from the top down. Although this is happening in the IRS in terms of revising business practices (SCOS, DOS), the committee believes that it has not yet happened in technical areas. It is essential that technical leaders guide the entire IRS through the technical modernization effort, using the best approaches available.

Technical leadership means more than just making decisions: a leader must also strive to reach consensus among all stakeholders and develop a plan of action to guide all activities. Furthermore, and perhaps most importantly, a leader must follow up on the decisions made, making sure that all of the objectives are understood clearly and eventually are reached. The committee believes that the people of the IRS are, for the most part, sincere, well meaning, and capable. Many honestly want change. Some, perhaps many, are capable software developers, but most lack the kind of design and development leadership experience necessary to inform and guide them on tasks such as those demanded by TSM.

Although there is no shortage of effective general leaders at the IRS, the committee believes that there is a distinct shortage of technical development leaders. This has resulted in the present situation in which the IRS still does not understand the need for an overall, concise plan for all technical developments. It is the committee’s belief that the development of such a plan has not been a priority and that the technical documents that do exist are insufficient in scope or content to allow technical analysis, even though they

are overwhelming in size. Without the support of IRS leadership, it should then come as no surprise that there is no concise technical plan—only pieces of an incomplete effort that do not ever culminate in a coherent picture.

Senior technical management at the IRS has never had control of the TSM process to the level that would be required for TSM to be a reality. The committee believes that, if anything, management has stepped back from efforts to get control. In fact, the committee does not know who, if anyone, has charge of TSM. This observation is supported by the facts that the Systems Architects Office (SAO) has really never been allowed to take technical leadership and that relatively few process improvements have occurred. At a meeting on November 29, 1995, the Commissioner indicated to the committee chair and another member that in response to committee questioning on several points, greater authority for TSM has been vested in the Associate Commissioner and responsibility for members of the SAO has been refocused on architecture issues. The committee has not had an opportunity to assess these developments, although it repeatedly requested timely updates on issues relating to its charge. This lack of leadership could also increase skepticism among IRS employees about future efforts, which would critically damage TSM.

A related problem concerns coordination between headquarters and the field. The process of developing an overall architecture and specification for TSM at the national office level has not used the right priorities for development. It has emphasized making all decisions at the national level, instead of concentrating on issues that affect the overall function and performance of TSM. Although the requirements for additional capabilities should be set at the national level and should be responsive to the needs of customers and servers in the field, design of the application to provide a new capability should be left in the hands of project leaders. The constraints on design are the requirements and the interface standards to access the services of the infrastructure. This is the reason for having a high-quality software development process: to enable the application developers to move freely within a boundary defined from a “total system” perspective. Headquarters cannot build TSM by itself, nor can the application developers. They must all cooperate, using a mature development process, and share the responsibilities and duties.

The IRS has, in-house or under contract, a considerable amount of systems development capability. However, this technical development capability needs significant improvement and, most of all, better leadership.

Part of the rationale for the committee’s previous recommendation for an architects office was to help build the skills and experience needed in the IRS to develop TSM successfully. The intent was to encourage formation of a cadre of very experienced technical developers, capable of both handling the tough problems and, over time, teaching others how to handle them.

The resulting establishment of the SAO was both encouraging and disappointing. To its credit, the IRS quickly obtained the necessary authorization for up to six senior technical billets. These positions were filled with experienced individuals from both inside and outside the IRS. The SAO was attached directly to the Chief Information Officer’s (CIO’s) office, which would presumably have given it sufficient clout to correct serious technical problems and break bureaucratic logjams. Most importantly, the IRS announced that the SAO would lead the development of the TSM architecture, incorporating functional capabilities, data modeling, and security.

Unfortunately, after a relatively short existence, the SAO is essentially a nonplayer in the TSM development process, apparently because of internal IRS politics. Most of the “architects” have moved to other organizations within the IRS in order to be in a position to make a difference. This is not completely bad in that the experience represented by these people is now being spread throughout the development organization on a day-to-day basis. However, as a focal point for technical decisions and leadership, the SAO has not fulfilled its potential. IRS developers desperately need such leadership, and the IRS should quickly move to reinstate the SAO and give it the technical authority and responsibility needed to lead the development effort. In short, the committee wants the IRS to implement its own directions:³

The committee also previously recommended that the IRS acquire, through either hiring or contracting, a range of developers from programmers to project managers who have significant experience in the development of large systems. Again, to its credit the IRS reacted by hosting a job fair aimed at hiring up to 50 experienced software developers, computer scientists, simulation modelers, security architects, and other key experienced technical people. Although more than 30 individuals were hired, the IRS did not fill all of the open positions. The committee was advised that “37 computer scientists, computer engineers or electrical engineers” had been recruited before the freeze was imposed.⁴ Because of the likelihood of restricted funds being appropriated by Congress, the committee is seriously concerned that a freeze placed on hiring as of May 5, 1995, will prevent the IRS from obtaining all of these key people. As it is, 50 experienced people represent a small fraction of the more than 2,000 existing developers in the IRS. This is the last place to impose such restrictions. Cost cutting should be directed principally toward ramping-down improvements to old systems and processes, not to cutting short the key technical skills needed for TSM. The influx of experienced developers must be accelerated, by both hiring and contracting, in order to maintain the development pace and level of quality demanded by TSM.

Even if budgetary constraints do not allow any increase in total head count, acquiring the expertise of people experienced in modern, large systems needs to be accomplished. No management or architecture program can work without the technical staff to support it. Updating the skills of a large group of technical people is a daunting task. Depending on the vintage of the developers in these organizations, they may be unfamiliar with or unconvinced of the need for artifacts such as system-wide data models. They may not be skilled at development paradigms such as object-oriented techniques. The paradox is that developers need experience to be able to change their development paradigm, yet many are not willing to embrace a technique they are not familiar with if there is any schedule pressure. Training classes give book knowledge and minimal experience, but not enough knowledge or experience to actually allow independent use of a technique. Making every developer take a class on a topic such as object-oriented techniques would have some positive effects; however, since classes cannot really guide people as to where to apply the technique or give them enough experience to try it on their own, much of the training will go unused. An organization must have experienced people “planted” in each project making use of a new technique, and the development team has to receive training just prior to using new techniques. Teams will fail if they have a particularly outspoken technical person who does not “buy in” to using the new approach. To permit several hundred new hires to come on board, the IRS may have to initiate a program of out-placement and early retirement of a sufficient number of its current programming staff who are unable to adapt to modern systems development. The IRS should also continue to leverage the talents of its integration contractor by having IRS project managers and developers work side by side with their contractor counterparts in order to learn modern development practices (see Recommendation 2.3 in this chapter and Recommendation 3.7 in Chapter 3).

In any government initiative, the yearly budget cycle plays an important role, allowing oversight organizations to ask questions and guide future direction (at least for the next year). When the committee first began talking with the IRS, one of the major IRS concerns was getting a multiyear commitment out of the budget process. The IRS succeeded in obtaining that commitment, and in the committee’s judgment, the funding levels for the last 5 years have been adequate for the types of activities being done within TSM. That is, the committee believes that the IRS needed to concentrate on re-engineering itself, not simply on deploying hardware, and that the level of funding was sufficient for those purposes.

As the IRS begins to deploy applications to support its vision, it is important to deploy the supporting infrastructure, which includes computer hardware and software, telecommunications equipment, air conditioning, support staff, and so on. Given the size of the IRS, such an infrastructure is extremely costly and will take a long time to deploy. For this reason, the IRS asked for an increasing TSM budget during the initial years of the project, totaling between $500 million and $1 billion per year. The IRS’s FY 1996 appropriations for TSM appear to be $695 million; $1.032 billion was requested.

The hardware infrastructure components are relatively commonplace and, by themselves, represent no risk to the TSM project, regardless of what direction it may take. That is, the IRS is deploying conventional hardware and software capabilities, most of which are purchased as commercial off-the-shelf products. Regardless of the types of

applications being developed now or invented in the future, the IRS will have adequate hardware and operating system platforms on which to build them.

Nonetheless, the committee is concerned about the large investment represented by TSM hardware because the IRS has not demonstrated sufficiently how the primary goals will be advanced once the hardware is in place. The committee agrees that some amount of deployment must be conducted while applications are being developed, but clear and precise “hold” points are needed to appraise the benefits of developing applications before expensive hardware is deployed on a wide-scale basis. To put that statement in perspective, it should be noted that the Integrated Case Processing (ICP) application will be used by almost 56,000 IRS customer service employees to handle a large number of taxpayer issues. It is clearly the driving force for many aspects of TSM, and roughly $20 million per year is being spent to develop ICP. In contrast, the hardware infrastructure needed to execute ICP will cost more than $560 million, if roughly $10,000 per ICP seat is assumed.⁵

The large difference in development and equipment costs must be understood clearly by both the IRS and its oversight organizations in order to discuss what TSM “needs” on a yearly basis. Here again, the committee urges the IRS to connect every TSM project to the overall goals and to show clearly how a given project supports one or more of the goals before obligating a large amount of deployment resources.

Despite almost a decade of activity, the architectural design of TSM either is nonexistent or is hidden in a forest of detail.

The IRS has developed more than 3,000 pages of documentation entitled Integrated System Architecture, Chapter 500, Volume 1 (Parts 1–4). This document was published originally in draft form in September 1993, well after most TSM procurements were awarded and most TSM projects had been started. As might be guessed from its size, Integrated System Architecture (ISA) provides a plethora of details regarding the implementation design of various constituents of TSM. Those details obscure any architectural design that might be hiding within those pages.

Architecture gets no respect in TSM! The “architecture principles” do not appear until Appendix 51C of the 1994 ISA.⁶ The second paragraph of that appendix reads as follows: “Architectural principles are guidelines. They are not requirements…”—a clear statement that the architectural principles can be ignored. In this regard the committee also notes that the TSM Systems Architects Office has no line responsibilities and no control over the development of TSM. Furthermore, the committee was told by front-line developers that they do not make direct use of the ISA. Rather, the developers are given an “Architecture Description Document” that is prepared by Engineering and includes the

definition of the configuration items (CIs) that the ICP project will build. There is apparently no process to make sure that designers, developers, and final systems adhere to the TSM architecture. All of this is clear evidence that architectural design is undervalued by the IRS.

There is still no evidence that the IRS understands the nature of a software system architecture. The ISA document says that four “groups” of “system functionality” exist; these collectively contain 22 “segments” that in turn contain a total of 134 “subsystems,”⁷ but nowhere in any document does the committee find a high-level design that can be rationalized and supported.

To be quite clear about its concerns and avoid any confusion regarding the definition of “architecture” or even “functional architecture,” the committee uses the term “structural description” to refer to a subject area that is of great concern. A software structural description (SSD) is a specification of the software modules of which a system is composed and the informational interfaces or dependencies among those modules. (A “module” is a unit of software—perhaps itself a collection of modules—that has a name, can be manipulated as a unit, and has a defined interface.) At a minimum, a software structural description contains the following:

• Concept of Operations,

• System Model,

• Data Model, and

• Application Framework.

The IRS has produced some documents that, either in whole or in part, map directly or indirectly to these pieces, including

• Integrated System Architecture, Chapter 500, Volume 1 (Parts 1–4),

• IRS Future Concept of Operations, and

• Standards-Based Architecture: Workstation Application Profile.

In addition, the committee has received briefings, supported by presentation charts, on the Corporate Data Model.

However, this assembly of documents is by no means complete, concise, or used widely by the almost 2,000 IRS developers and their contractors. For example, the Corporate Data Model covers taxpayer data that are used by the IRS enterprise to process tax returns. It is only a subset of the overall data model needed for TSM, which must include control data and non-taxpayer data as well.

It has been known for more than 25 years that what the committee calls a software structural description (SSD) ultimately determines most of the critical, long-term characteristics of a system, other than functionality. A system’s complexity, performance, development cost, maintainability, and modifiability are affected strongly by the software structure. Without an SSD it is impossible to evaluate the architecture or the design of a

software system. As of June 1995, the committee had not seen a software structural description of TSM.⁸

The committee’s interim report indicated concern about the representation used in an SSD. The real subject of a software structural description is the set of relationships among the modules. In order to understand the SSD, one must be able to comprehend these relationships. It is very difficult to do so when they are described by using narrative text (including tables). A picture is worth 1,000 words. Diagrams that show the modules and their mutual dependencies in ways that are understood easily must form the core of a good SSD. The Information Engineering Workbench diagrams in the ISA might meet the committee’s requirements for an understandable diagrammatic representation of the SSD.⁹ However, the committee has not seen such diagrams for the high-level description of TSM.

The importance of a concise architecture for a large system being developed over a long period of time cannot be overestimated. In a large system that is undergoing constant change, lack of an architecture means that the addition of new functionality is likely to be a random process. Over time, what little structure does exist will be destroyed. With no overall model or principles, eventually no one person or small group of people will understand what requirements are supported by which projects. As time passes, without a concise architecture there will be no structure to exploit for evolutionary changes and no clear boundaries to exploit when re-engineering parts of the system. This makes the long-term re-engineering task both costly and risky.

An architecture is more than just a set of documents with many words and pictures. An architecture also includes the critical pieces needed to analyze parts of the system before it is built or while it is being prototyped. Such analysis is critical to the ultimate success of a large system.

Performance goals need to be an integral part of systems architecture. For example, if there is a business goal of resolving 80 percent of all taxpayer inquiries on the first call, then the architecture must provide for the system response time needed to support that goal. The existing data retrieval capability allows only the agent to understand the nature of the call, acknowledge to the taxpayer that there is a problem, and initiate corrective action. However, the customer service representative cannot introduce corrections on-line, and so there is a several-week period within which the taxpayer must follow up. A systems architecture that establishes a direct on-line connection for both obtaining the taxpayer’s history and making corrections imposes a response goal on TSM similar to that of a reservation system, which may take anywhere from a few seconds to a few minutes, but not a few hours or days. This necessitates very short delay times on all

system elements, including those related to security. The 80 percent goal clearly affects the design of the overall system and must be considered in all system analysis.

System analysis based on accurate models is critical for determining performance bottlenecks within a complex system. Only then can a bottleneck be detected and corrected. System analysis prior to deployment allows a designer to properly select appropriate hardware and software products capable of reducing or eliminating potentially catastrophic performance problems. Unfortunately, the IRS does not currently have a system model sufficiently detailed to allow such an analysis.

Some amount of analysis is being conducted by the IRS, but it does not seem to be coordinated with the high-level decision-making or development process. For example, the committee was provided with the results of a spreadsheet analysis of the cost per return for various submission processing techniques, including DPS.¹⁰ Examination of those results indicates that DPS may save as little as 10 percent on the cost of processing a return. This may or may not be important, depending on which overall TSM goal is emphasized, but the results of the analysis must be understood by all parties concerned with DPS.

There are numerous other areas in which technical analysis, based on a well-defined system model for TSM, is needed. All aspects of the communications network must be analyzed to determine potential bottlenecks, security weaknesses, and end-to-end delays. The committee has continued to request (informally, including during briefings) the results of such an analysis, without adequate response from the IRS.

The database load generated by end-user applications must be analyzed to determine the types of structures and servers needed on the central database systems to handle that load. When queried on this topic recently,¹¹ the IRS produced database load estimates that were approximately 4 years old. The IRS told the committee that the estimates did not need updating, despite the fact that the estimates were done prior to the SCOS and DOS re-engineering studies and the design of ICP, which is one of the major database applications.¹²

A comprehensive system model for TSM would allow the type of scientific analysis that is needed to answer key technical questions before a heavy investment is made in a particular approach.

The IRS has formed the Infrastructure Task Group in Engineering, which is defining an applications programming interface (API) to be used by application developers.¹³

According to the IRS:¹⁴

It appears that through this approach the IRS is attempting to accomplish what is normally done by defining architectural objectives: defining the architecture and monitoring projects for their compliance with that architecture.

The ITG approach for coordinating the development of a large number of projects is interesting. There are going to be hundreds of applications built on the IRS’s functional information infrastructure in the next decade. Attempting to put all of them into an overall systems architecture is neither feasible nor desirable. The services provided by the infrastructure are enablers of new or improved capabilities of the TSM. A well-defined interface to these services will simplify the design of these capabilities. Presumably, if all projects use the same API for critical functions, all applications will be interoperable and service-wide requirements can be enforced. The establishment of such a standard, durable interface user by all projects would be a major achievement for TSM, providing a high-degree of code reuse. By reusing code between projects, TSM developers could quickly and economically build new or evolving applications, significantly reducing the risks associated with those new applications.

In practical terms, the ITG is an attempt to promote a de facto architecture. Unfortunately, a number of aspects of TSM could hinder a successful outcome. The ITG was formed after most TSM projects were under way. For example, ICP Version 1, which is in the process of being deployed to a number of customer service sites, was developed without any support from the ITG. More importantly, as far as the committee can determine, currently only the ICP project will make use of the output from the ITG in its future versions. The committee was told that the Document Processing System would not be required to use the standard API because it is being developed completely under contract. Furthermore, the project leaders supporting the large database servers have not been “convinced” that the API will be useful to them. Until they are convinced, they will continue to develop capabilities on their own.

Chapter 5 discusses security in detail. This section discusses the way in which security and development issues intersect.

Security has not been integrated into the TSM projects, as exemplified by the following observations:

• A majority of the architecture documents have “TBD” (to be determined) in the security columns;

• The Security Architect was temporarily relocated for an extended period from the Systems Architects Office into the Engineering organization and was responsible for managing the engineering effort; and

• The security guidance from the Infrastructure Task Group is not based on an enterprise-wide security architecture.

The result is a set of security specifications that when implemented may or may not integrate well with the overall project security needs and requirements. Specific design guidance and trade-off tools to select optimum approaches appear to be lacking. The only specific statement made to date is that individual user passwords will be employed for access control, but no specifications for password or key management or distribution were discovered by the committee.

There is no set of security specifications that is produced for each project and program. Moreover, there is no process or mechanism, other than individual effort in Engineering, to translate high-level security requirements into design guidance or detailed project and program security. Finally, there do not appear to be any modeling tools or trade-off processes to provide the specific security analysis required by ongoing projects and programs.

The committee could find no overall security model or technical framework. Project teams have dealt with this issue by relying on the ITG to write and provide any security code that is needed. Without an enterprise-wide concept and technical model, there is little hope that this approach will result in anything more than a patchwork of incomplete and inadequate security measures. Furthermore, there is no indication (as discussed above) that the output of the ITG will be used by all of the projects, which leaves the issue of enterprise-wide security still open.

Given that there appears to be no concise architectural description or consistent use of standard APIs across all development projects, how are the IRS developers proceeding? To answer this question, the committee participated in a number of discussions with front-line developers. Specifically, the committee met with representatives from the ICP and DPS projects. During the ICP meetings, the discussions centered on project management, interactions with other IRS organizations, and the technical capabilities being developed. The DPS meeting focused on the development methodology being used and the technical analysis of various options for imaging. It should be noted that ICP is being

developed primarily by IRS developers, while DPS is being developed primarily by contractors.

The ICP and DPS developer meetings revealed a number of interesting aspects concerning TSM development activities. For the most part, TSM developers are “in front” of the guidance being provided from the “lead” organizations. ICP managers, for example, have been creating new development processes on their own in order to perform their jobs properly. Similarly, they have requested design reviews by Engineering in order to make sure that everything is being interpreted properly. Such activities represent the beginning of a change toward a more mature development environment, but it is only a beginning.

Developers have also formed “bilateral” agreements with other development organizations to resolve integration issues before they become a problem. For example, the DPS team works closely with the developers of the Tax Data Perfection System (TDPS) to make sure that the data entered by DPS can be processed by TDPS and vice versa.¹⁵ Here again, this is the beginning of good development practices, but such activities are the exception, not the norm. More importantly, such agreements are based on the personal initiative of the managers involved, not an agreed-upon, organization-wide process.

Such agreements are helpful when building systems that must be integrated eventually, but they are no substitute for a coordinated process that makes sure that all projects adhere to common standards and practices (e.g., durable interfaces, code reuse, code compatibility between control sets). For example, the committee was told that TDPS might be reimplemented from a workstation platform to a mainframe platform when it is needed to process tax data collected from the Electronic Filing system. Similarly, the committee was told that the data model used by the ICP application was a superset of that produced by the Corporate Data Model project. Apparently, the “corporate” data model, which took about 4 years to develop, covers only “tax return” data, and not the control data needed by systems that are processing return data. Such coordination problems are common in large development projects, but a more mature development approach would highlight such issues before developers were well into their development cycle.

At a minimum, a representative from Engineering should act as an observer and/or mediator during the discussions between two or more development teams. This serves to document properly the decisions being made and to ensure that they are consistent with decisions made in other parts of TSM. Only Engineering has the scope to understand the global issues at hand, but only the development teams have the knowledge and hands-on experience needed to solve problems associated with their systems.

It is the committee’s judgment that there are no well-functioning mechanisms in place to ensure that the IRS will meet its software engineering goals or that it will achieve software process improvement.

The IRS is attempting what might be one of the largest software process improvement projects ever seen—while concurrently trying to develop billions of dollars worth of systems. The tax system needs much modernization. It is antiquated, and the committee has no doubt that it will be improved over the coming years. The question is, How

effectively can funds be applied to this project—and how should they be applied so that they are effective over the entire modernization process?

The term “process improvement” refers to a set of actions that are taken to improve the skills and knowledge of workers, as well as the quality of the tools, equipment, and methods used by those workers. Process improvement is a well-established approach to improve quality, productivity, and cycle time. All of these are needed badly in the TSM project. Software process improvement programs depend on:

• Understanding current software development methods and beginning with good software development practices;

• Establishing good, useful metrics that start with an assessment of current skills;

• Applying methods diligently and measuring results based on an ongoing training program; and

• Proposing and installing improvements in the process.

The committee’s 1992, 1993, and 1994 reports called for process improvement as a high-priority action.¹⁶ Although some token efforts have been made in this area, there is still an urgent need for much better processes in systems and software engineering in both the IRS and its contractors.

In 1993, the IRS evaluated its own software development capabilities using the Capability Maturity Model (CMM) that was developed by the Department of Defense’s Software Engineering Institute (SEI) at Carnegie Mellon University.¹⁷ In the CMM, organizations can be ranked at levels 1 through 5, with CMM level 1 corresponding to the lowest-quality process and CMM level 5 representing a development process that is repeatable, defined, managed, and optimized. In 1993, the IRS ranked itself at CMM level 1. Today, the IRS is still at CMM level 1 in this committee’s opinion. (Generally, it should take about 2 years to reach CMM level 2 once a software development improvement process is begun.¹⁸)

The need for the IRS to move beyond level 1 is critical. Level 1 means that the organization will probably fail to meet cost, schedule, or quality targets because it does not have the ability to control the development process effectively. Level 2 means that the process is repeatable and to some extent predictable. Most development organizations are at least at level 2.

There are also significant benefits that can be gained from a CMM-based software improvement effort, provided the organization is devoted to the approach. For example, a study of 13 organizations from the commercial, government, and government contractor sectors showed a median productivity gain of 35 percent, a 19 percent reduction in the time to market, and a 39 percent reduction in reports of postrelease defects.¹⁹ The importance of the CMM approach is highlighted by the current Air Force policy that requires all contractors bidding on software-intensive projects to be assessed at CMM level 3 or better.

The IRS has formed a number of process improvement teams to focus on a number of areas, including the following:²⁰

• Requirements management,

• Software quality assurance,

• Project planning and tracking,

• Testing, and

• Configuration management.

However, the existence of those teams does not appear to be having an impact on the major projects that are currently being developed; project requirements and schedules have not been modified to accommodate the necessary process improvement activities. Furthermore, the committee has been unable to ascertain any coherent or systematic metrics being used by the IRS to measure technical progress.

The committee also doubts that the process improvement teams are staffed with enough experienced people to make real improvements. Software process improvement is a very difficult task, and experience is gained only by going through the process. The IRS cannot simply send people to “training” classes and expect them to be able to carry forward large-scale improvements.

The committee was briefed on the IRS’s plans for a comprehensive technology curriculum for its employees. However, these efforts were at a very early stage and the committee was unable to determine if such a curriculum would be appropriate to ensure that all IRS employees would be trained to a sufficient level to implement process improvement recommendations.

The committee reiterates three specific suggestions for software and systems engineering process improvement and strongly recommends that the IRS act on these suggestions immediately:

1. The IRS should prepare an overall process improvement plan covering systems and software engineering and both internal and external sources of the systems and software development. This plan should explain needed development capability goals for various groups of developers. These goals should be aggressive, and progress toward achieving the goals should be monitored aggressively throughout the management chains of the IRS and its contractors.

2. The IRS should hire key leaders who have experience with improvement efforts similar to TSM. These leaders should be empowered to procure personnel by direct hire or by contract to carry out an effective and accelerated process improvement activity.

3. The progress in process capability improvement should be assessed regularly in reference to accepted process capability models.²¹ Progress should be reported up through the management chain. Failure to improve at the rate set by the goals in item 1 should result in quick, effective, corrective action. Such monitoring should be conducted in cooperation with external oversight organizations to ensure that the IRS is making improvements to its systems and software development process on a continual, visible basis.

All of these recommendations have been made previously to the IRS by this committee and by other external organizations.²² Unfortunately, the IRS’s response has been disappointing, and there is clearly not enough of a sense of urgency on its part. Accordingly, the committee’s view must be stated quite bluntly:

TSM is an extremely important modernization effort, and the IRS should use the best practices and people to develop it. TSM applications will still be invented and

developed well after most of the current IRS leaders (and members of this committee) have moved on to other initiatives. A mature development approach is critical to ensuring that future development activities fit into the TSM model without exposing the overall system to unacceptable risks. History has shown that developers are capable of heroic efforts when everyone is watching, but the committee is concerned that the hundreds of applications that will come “because the infrastructure exists” will not be so visible. Without a solid development approach, those applications could easily wipe out gains made by the initially deployed applications.

The history of the computing industry shows that a project that spans a decade or more will yield a system that is not near the current state of the art. As a committee member said, “It may be modernized, but it won’t be modern.” Either TSM is trying to move too far too quickly, or many large and somewhat unrelated projects have been gathered into one gargantuan collection that the IRS calls TSM. Moving from a batch, mainframe system to a workstation-based, distributed database, client-server system while simultaneously improving all aspects of software engineering in the organization is a very difficult task. A plan for incremental involvement of projects and process improvements over time would be wiser. The plan should set specific visible milestones and progress should be monitored aggressively. Based on these observations, the committee strongly recommends the following:

This project should be driven by and conform to a well-understood and accepted architecture and should make use of the API and support modules developed by the Infrastructure Task Group. When this project has made substantial progress, an assessment can be conducted; appropriate changes can be made in the methods, techniques, and support modules; and perhaps several more projects can then be commissioned for round two, using the products and processes developed.

From what the committee has seen, the ICP project seems the most likely candidate. The ICP forms the basis for the “modern” IRS, in which taxpayer interactions are handled in a timely manner, with increased accuracy. The information required by ICP represents a large portion of that stored by the mainframe servers and gathered by DPS and ELF. If ICP Version 2 is not done well, then the other, much larger, projects will not do much to change the IRS. The Integrated Collection System may be an alternative project to focus on, but it does not support the goal of improving taxpayer services, nor does it affect an area that is fairly new to the IRS: modern, telephone-based customer service. Once ICP has proven its value to the IRS, through controlled, single-site experimentation, the infrastructure needed to deploy ICP and other TSM applications on a wide scale can be allocated.

To its credit, the IRS is testing a prototype of ICP at a small number of sites, involving a small number of people at each site. However, such experiments do not yield the analytical results needed to appraise ICP in a full-site production mode. In fact, ICP Version 1, which provided basic terminal connectivity to legacy mainframe applications, was tested during only one tax season, on a limited basis, before being deployed to other

sites. Moreover, ICP Version 2, which will provide initial distributed database processing capabilities, is only now being developed.

The IRS must focus its attention on the development of such critical TSM components, making them examples for all other projects to follow. It must resist the temptation to deploy capabilities that provide only limited function when compared with its overall objectives. In short, the IRS should slow down and concentrate on doing a good job, not just doing a job. The modernization of the IRS will take a long time, and near-term successes can be overshadowed easily by long-term failures if the process of developing systems is not learned properly. A more controlled development process will help the IRS to:

• Develop requisite skills,

• Demonstrate feasibility to oversight organizations,

• Improve the development process,

• Decrease overall cost, and

• Reduce the risk of failure.

Based on the information and briefings supplied to it, the committee has come to the following conclusions:

• Overall, there are no technical artifacts (i.e., architecture, proofs of concept, high-level design, system model, software structural description, and so on) capable of supporting quantitative or even qualitative studies of performance, security, capacity, reliability, or implementability. Such artifacts are particularly important for TSM because of its size and scope and because IRS leadership is in a continual state of flux. In short, without such artifacts, no one can substantiate why a specific decision was made or what is being accomplished.

• The IRS has not clearly defined, for its own purposes, how all of the TSM pieces will fit together to yield an “integrated” system. Moreover, the trade-offs among those pieces are not well understood or articulated.

• Overall, as a software or systems development organization, the IRS currently does not have sufficient capabilities or experience to complete TSM successfully. The IRS has made good progress in the last 5 years in improving specific development practices on specific projects, but it is not improving its overall organization at the rate required to keep up with the pace of technology in general and TSM in particular.

• The security portions of the design documents are currently nonexistent, and the security architectures are too high level to drive adequate security measures in the current projects. The IRS plans to retrofit a security infrastructure into the major projects at a later time. At this time, however, there is little adequate security; therefore, no privacy is provided within TSM systems.

• The IRS seems to be moving too quickly to purchase and deploy hardware without fully knowing the characteristics of the applications that will be running on

the hardware and the benefits to be provided by the applications. To its credit, the IRS is testing prototypes of most applications, but deployment of the underlying hardware does not appear to be coordinated closely with the analysis of prototype systems.

Accordingly, the committee makes the following recommendations, in addition to the two presented above in this chapter: