Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 38
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report 3 Management Tax Systems Modernization (TSM) would be a major undertaking for any organization. Therefore, it is necessary to use as much from proven organization and management approaches as possible to help improve the chances for a successful outcome. The statement of task provided to the committee requested an analysis of TSM in three separate areas: Administrative aspects of the TSM program management approach; Plans, schedules, and risk; and Implementation strategies and schedules. This chapter addresses the key issues that the committee has identified in planning, organization, management, and human resources. Rather than addressing these issues individually, the committee has elected to unify all of them to provide a fuller discussion of the formidable management task of completing TSM successfully. CLARITY OF SCOPE AND OBJECTIVES TSM started off as a computer system modernization project. Its scope has evolved over the past 5 years to include the re-engineering of all Internal Revenue Service (IRS) operations, including the computer systems to support these operations. Although it would have been much better to look at new ways of doing business before designing TSM, the committee is pleased that the effort was eventually undertaken. It is important that all stakeholders be aware constantly of the significance of this enlarged scope. The most recent reorganization of the field structures from 63 to 33 District Offices (to be achieved by October 1996), with commensurate removal of many management positions, is tangible evidence of the impact of TSM on operations. Other significant changes will be necessary. The objectives of TSM are not always clear. To the committee, it appears that TSM is being implemented both to increase taxpayer compliance and to improve the effectiveness of the IRS organization. Improving the latter is itself fundamental to improving taxpayer compliance. Most of the communications that the committee has received from the IRS, however, focus on “redoing” the computer systems and implementing “up-to-date” technology. These goals need to be aligned both internally and externally with overall management goals, not just those of the Chief Information Officer (CIO).
OCR for page 39
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report The committee is heartened that there recently has been a shift to a discussion of “implementing,” but it is concerned that IRS executives are not yet coming to grips with how to make this happen. Thus, this chapter focuses on the postplanning issues of organizing, measuring, and controlling. The IRS leadership has made much progress over the years in communicating to its staff the progress of TSM. The Commissioner, Deputy Commissioner, Modernization Executive (now Associate Commissioner), and others have actively participated in this endeavor. As TSM proceeds, an increased amount of effective communication is required at all levels in the IRS. STRONG LEADERSHIP IRS Commissioners change: the recent average has been a new one every 1 or 2 years. (The committee, for example, has worked with three different Commissioners during its 5-year tenure.) Therefore, continuity of leadership for TSM cannot be expected at the Commissioner level. The logical leader of TSM is the Deputy Commissioner, and this structure has been recommended previously to the IRS.1 There is a need to clarify roles and responsibilities, making the Associate Commissioner for Modernization responsible for TSM operations and for systems changes and implementation, with the intent that the Associate Commissioner be the overall program manager. The May 1995 advancement of the Modernization Executive to Associate Commissioner, with the CIO reporting to that person, is a step forward in the committee’s view. Further, it emphasizes the point made above, that the scope includes all operations, not just computer systems. The committee considers that the Associate Commissioner must be vested with the needed power and authority to manage TSM. Recommendation 3.1. The committee recommends that the Associate Commissioner for TSM have responsibility for all aspects of TSM, reporting on these responsibilities directly to the Commissioner. This position should have all project managers accountable to it and should include responsibility for all personnel selections, transfers, demotions, and rewards for those charged with the tasks to make TSM happen. The IRS Commissioner reported on November 29, 1995, that movement in this direction has occurred.2 Although the committee has not examined these changes, it appears that the consolidation of authority and responsibility for TSM under the Associate Commis- 1 The Deputy Commissioner carries a grade level of ES6. See Computer Science and Telecommunications Board, National Research Council. 1993. “Letter Report to Commissioner Margaret Milner Richardson,” Computer Science and Telecommunications Board, Washington, D.C., July 30, p. 3. 2 Following up on that communication and subsequent to the conclusion of the committee’s deliberations, the Associate Commissioner indicated in a December 12, 1995, fax that “upon completion of the rescoping [the IRS] fully expects to assign additional executives to the chief [officers] who will also be responsible to the Associate Commissioner for integrating the new business vision into the day-to-day operations of the IRS.” (See “The Roles and Responsibilities of the Associate Commissioner for Modernization,” an IRS internal use document dated December 11, 1995.)
OCR for page 40
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report sioner is moving toward the recommendations made during the past several years by both the present and the previous committees. It should remove some of the dual-reporting, dotted-line, or ambiguous relationships between organizational entities. The level of accountability for that position needs to be clear to all IRS staff. The committee views this as a very positive change and agrees with the decision to have the new CIO report to the Associate Commissioner as well. Over the longer term, the Commissioner should evolve a total management structure to match the business vision. The committee commented on this in the interim report and drew no overt reaction from the IRS. The three operating chief officers at least, together with the remaining supporting chief officers, should be heavily involved with TSM. However, they seem totally immersed in day-to-day affairs and delegate their input to the site managers for TSM, who are part of the Modernization Executive’s organization. “Buy-in” by the operating chiefs should involve them more directly in implementing the evolving business vision. Each should designate a sponsoring executive to work with the TSM team and accept delivery of the TSM output on behalf of its chief officer. Need for a Strong CIO Although the need for overall clarity of leadership in TSM has been addressed in the changes and recommendation above, the committee is also concerned about the technical leadership embodied in the position of CIO. With the recent retirement of the previous CIO, the opportunity exists to enhance substantially the technical leadership of TSM by bringing into the information systems organization an outside technical manager who is both experienced in and technically knowledgeable about large, integrated information networks. Recommendation 3.2. The committee recommends that the position of CIO be filled by recruiting from industry an experienced and knowledgeable information systems executive. Since the CIO position is extremely important for the successful outcome of TSM, the committee wants to be very specific about the types of qualifications needed for that position. At a minimum, the CIO should have Sufficient technical knowledge to understand why and how TSM is designed and developed; Strong leadership skills, such as the ability to develop consensus among a large group of people, including political oversight groups; Experience managing a large, mature development organization and a large-scale, distributed information system; and A proven ability to lead a large process improvement effort. The CIO must have excellent communications skills and must be capable of explaining all technical aspects of TSM to the Associate Commissioner and working with the Associate Commissioner to ensure that technical issues are successfully resolved.
OCR for page 41
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report Present Matrix Structure Project management at the IRS has an overlay structure, which is common for initial attempts at matrix management but difficult to make succeed.3 Through 1994, the Modernization Executive controlled the funds and planning, but with only a small staff he had to depend on persuasion for control through the project managers, who remain part of the information systems organization. For a program manager to be successful, he or she must have the power to hire, reward, and promote those individuals needed to make the program successful, and the alternative powers (within applicable employment rules) when members of the program organization fail to accomplish their assigned tasks. Throughout the committee’s period of deliberations, the Modernization Executive was able to name only site executives, not project managers, and had little power to reward and promote or to withhold rewards and demote project managers or the manager of the Integrated Project Schedule.4 Based on information received through the committee’s June meeting, the following contrasts can be noted between IRS organization and that of major companies now using matrix management:5 In most matrix-managed organizations, all project managers report directly to the program managers. At the IRS, project managers have remained in the information systems organization; they do not report directly to the Modernization Executive or her organization. The project schedule package used in the private sector is a single overall system, under the control of the program manager; individual project managers do not use their own systems. In the IRS at present, project schedule systems are still in the information systems organization. Sufficient directly reporting staff is on hand in the private sector to carry out the program management task. In the IRS, however, the Site Executive for Submission Processing has only six analysts, which the committee considers an inadequate number to manage a $1.4 billion project. Private sector operating organizations have departments and individuals in place who are prepared to receive and use products from the developing organizations. The committee was encouraged to see the first of these individuals from the Customer Service side of the IRS at a mid-1995 meeting. 3 Kolodny, Harvey. 1979. “Evolution to Matrix Organization,” Academy of Management Review 22(4):453–463. Also, Cleland, David I. (ed.). 1983. Matrix Management Systems Handbook. Van Nostrand Reinhold, New York. 4 The Commissioner indicated in late November 1995 that authority for the Modernization Executive had been increased. The committee was unable to assess the scope of that increase. 5 Cleland, David I. (ed.). 1983. Matrix Management Systems Handbook. Van Nostrand Reinhold, New York, pp. 19, 60, 313, 327, and 336.
OCR for page 42
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report Field Structure The continued consolidation of the field structure makes sense. It is in keeping with the findings of the Service Center Organization Study (SCOS) and the District Office Study (DOS).6 The removal of layers of management by reducing the number of District Offices by October 1996 also demonstrates a determination on the part of IRS senior management to modernize the organization along with the modernization of its information systems. There remains the need to define clearly the role of the District Offices. The organizational role of the districts is also still unresolved in the longer-range business vision, although the districts seem more and more the operating part of the compliance core business systems owner’s (CBSO’s) responsibility. For several years the committee has questioned the role of the regional organizations and has expected their eventual removal.7 The regions become more and more an anomaly if District Offices are part of the compliance CBSO; they should be gracefully eased into a lesser number of assistant chiefs of compliance. The reduction to four regions is a positive step. However, it is not clear whether the four remaining regions represent a step on the road to eliminating all regions, or whether the remaining four are necessary for a proper span of control. If continuation of regions for multiple functions of the chief officers is still intended, the committee suggests that their responsibilities be more clearly defined than they have been in the Business Master Plan (BMP).8 Cultural Integration Historically, the IRS has employed an approach to personnel management and development best described as a preference for “home-grown” skills. The vast majority of middle-level and senior executives have worked their way up the IRS hierarchy, beginning as revenue agents or service center workers and gradually moving into management jobs. Typically, a senior IRS manager is a generalist who has obtained broad (but not deep) expertise through job rotation and career diversity. By and large, this system has served the IRS well. It has provided a level of organizational commitment and a sense of teamwork rare in government agencies. It has the advantage of ensuring that top 6 Internal Revenue Service. 1993. Final Recommendations of the Service Center Organization Study. Document 9111, Internal Revenue Service, Washington, D.C., April. Internal Revenue Service. 1993. Final Recommendations of the District Organization Study. Document 9111-A, Internal Revenue Service, Washington, D.C., April. 7 Computer Science and Telecommunications Board, National Research Council. 1992. Review of the Tax Systems Modernization of the Internal Revenue Service. National Academy Press, Washington, D.C., p. 19. Computer Science and Telecommunications Board, National Research Council. 1994. Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Interim Report. Computer Science and Telecommunications Board, Washington, D.C., p. 18. 8 Internal Revenue Service. 1994. Business Master Plan, Internal Revenue Service, Fiscal Years 1995–2001. Document 9255 (4/94), Internal Revenue Service, Washington, D.C.
OCR for page 43
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report managers have a sense of the “big picture,” inasmuch as their careers have included many different IRS postings, usually in several divisions. Although the system has worked extremely well for the IRS, it undercuts the needs of TSM for technical expertise and specialization. Job rotation gives a picture of the whole and serves to inhibit parochialism, but it also diminishes the ability to specialize and develop the depth of expertise needed for complex, technologically based projects. The home-grown culture also means that the IRS has limited ability to tap relevant experience and knowledge developed in other organizational settings. That is, although the IRS is beginning to hire a few midlevel technical managers, the IRS culture may make it difficult for them to succeed. The IRS is particularly vulnerable to a “not-invented-here” syndrome in which new ideas are discredited because they come from people who have spent much of their work lives at other organizations and are not steeped in the history and various work milieus of the IRS. Recommendation 3.3. The committee recommends that the IRS take steps to ensure that more top-level technical managers involved in TSM are brought in from outside the IRS and that IRS leadership ensures that these experts, who have not taken the usual IRS career paths, are not frozen out by those who are part of the more homogeneous, close-knit IRS culture. PLANNING ISSUES Because the TSM program spans such a long period, the Modernization Executive must understand, accept, and adapt to changing circumstances. The planning organization needs to understand and identify those fundamental TSM objectives that must remain constant, based on plans that show the various parts of TSM and their relationships. Here, the committee discusses in detail such planning issues as the overall status of the project, the program management plan, and operational risk planning. Overall Status The IRS has strengthened its management planning for TSM since 1993, when an update to the 1992 Design Master Plan (DMP) appeared to be needed.9 Not only has it provided a variety of management plans, as mentioned in the interim report,10 but it was 9 “The IRS’s system developers have produced some development schedules and working “updated” TSM design documents. But the schedules need to be reformulated to account for the reengineering plans, and the design documents should be more detailed and should include the new business processes defined by the organizational change studies…. Moreover, the reformulated schedules and design documents must be completed before substantial resources are committed for development of the systems.” SOURCE: Computer Science and Telecommunications Board, National Research Council. 1993. “Letter Report to Commissioner Margaret Milner Richardson,” Computer Science and Telecommunications Board, Washington, D.C., July 30, p. 5. 10 Computer Science and Telecommunications Board, National Research Council. 1994. Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Interim Report. Computer Science and Telecommunications Board, Washington,
OCR for page 44
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report also able to replan swiftly for at least the Fiscal Year 1995 budget cuts—although how these plans have related to the technical plans is less clear. The IRS should also simplify plans so that priority decisions can be made more expeditiously. For example, all data should be put through one planning system for the program manager, rather than being analyzed by different organizational elements as has been the recent practice. Planning data can help with making priority decisions, measuring progress, and communicating changes and progress to all IRS employees. Plans should be structured around regular, deliverable milestones, say every 6 months, to emphasize TSM progress, not only to the IRS staff but also to Congress. This approach may require splitting some of the longer-running multiyear projects into smaller pieces. Program Management Plan A new information systems Program Management Plan (PMP) “for developing, managing and implementing” was provided in late 1994.11 It lists information systems group plans, largely without dates for actions to happen. It calls for monitoring schedules and costs (p. 3–7) but does not address the consequences of, or provide incentives to minimize, deviations. The committee was disappointed to see some of the plans still lumping operating elements together as “business operations,” as if the separate operating chief officers (the core business systems owners, or CBSOs) had never been appointed.12 The committee suggests that the planning organization treat each CBSO as an individual client and structure its plans accordingly. Operational Risk Planning Based on private sector experience, there are two dimensions that need to be considered in planning for integrated networks: the degree of integration and the degree of operational risk. The committee has found a communications gap among the IRS, the General Services Administration, the Office of Management and Budget, and possibly also the Congress, on what operational (as opposed to security) risks an integrated network entails. An integrated network may involve reciprocal, parallel, or serial integration. In a reciprocal network, all nodes must interact continuously, as in an industrial process control system or the air traffic control network. In a parallel network, multiple nodes must D.C., p. 16. 11 Internal Revenue Service. 1994. Program Management Plan (PMP), Version 1.1. Document 9177 (Rev. 10–94), Internal Revenue Service, Washington, D.C., October, cover page. 12 At least in the “Achieving the Business Vision” chart, the functions of Submission Processing, Corporate Computing, and Customer Service/Case Processing were segregated, and the systems serving each were outlined clearly. However, the first two parts of the business vision are focused on the Managing Accounts function, while the Informing/Educating and Ensuring Compliance functions were lumped together, although they seem to belong to two different CBSOs.
OCR for page 45
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report interact with a principal node, such as a telephone central office. In a serial network, processing is “passed” from one node to another, with only local coordination, much like an assembly line. The serial network’s management is the least complex, since problems can be solved one at a time. The degree of interaction in the IRS network is primarily serial, with each process feeding another, downstream process. As such, it consists of several different interfaces, each with relatively straightforward algorithms. Operational risk also can be separated into three levels. The highest operational risk is danger to life and limb, as in air traffic control, rail traffic control, and high-risk industrial process control (e.g., petroleum and chemical plants). This is a far higher degree of operational risk than the IRS will ever face. The middle level of operational risk is one of inconvenience or regulatory violation. If a plant’s wastewater control system fails, polluted water overflows into fresh water but does not necessarily create an immediate public hazard. If a telephone exchange fails to give a dial tone, some members of the public are inconvenienced but not imperiled. The IRS is in the early stages of integrating its network and is nowhere near this level of operational risk. Likely TSM failures will result in blocking or queuing, not catastrophes. Yet the IRS will be at a higher (intermediate) level of operational risk at some later date, when there is no possibility of falling back to manual operations.13 IRS operational risk planning needs to focus on this coming eventuality. Finally, the lowest level of operational risk is the stage at which the IRS is now, where computer systems are being introduced in parallel with manual operations that will not be closed down until the new computer-based modules are fully tested. Nonetheless, the IRS must plan for a time when manual processes cannot be used as a backup. MANAGEMENT ISSUES The key management issues covered in this section are the lessons learned from project implementation to date, program management tools, evaluation and metrics, and the Integration Test and Control Facility. Project Lessons Learned The present state of objectives-based project management within TSM was explored by the committee through an examination of the Integrated Case Processing (ICP) and Document Processing System (DPS) projects. In all discussions, the committee focused on the lessons learned (or not learned) by the IRS. As of early 1993, the essential requirements for ICP Release 1 were well defined.14 During the development process, a number of additional requirements were cited and added to the Release 1 definition, using the change control process established for the ICP project. (In fact, the additional requirements were actually moved forward from the ICP Release 2 definition.) Since no functionality was subtracted, the additional requirements caused the original predicted schedule of 17 months to be extended by 4 months. 13 For example, the Social Security Administration is now facing this middle level of operational risk of integration, with its attendant need for a more rigorous backup strategy. 14 “Integrated Case Processing: Overview,” IRS briefing slides dated February 24, 1994, prepared by Andy Meranda.
OCR for page 46
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report Such a slip might be considered, by some, to indicate a problem. However, since the development group and the site executive agreed wholeheartedly that the added functionality was worth the delay, the slip represents a good example of developer and end-user negotiations. In this case, the committee believes that the ICP project has illustrated the following important management lessons: Proper project definition is important to achieving realistic schedules; and Adherence to change control procedures ensures realistic and thoughtful consideration of requested changes. Release 2 of the ICP was originally scheduled to be completed 8 months after Release 1 and is now scheduled to be completed by January 1, 1996, a slippage of about 5 additional months.15 While some functionality was subtracted from Release 2 by moving it forward to Release 1, more new functionality was brought forward from Release 3, again through the change control procedures. Unlike the previous release, however, Release 2 is not as well understood by either the end users or the developers (Release 2 provides much more complex processing than Release 1), and the uncertainty has led to a Release 2.5 scheduled for June 1996. This juggling of requirements and creation of intermediate releases indicates to the committee that the ICP project has not learned the following: After the original project has been changed once, it is best to reanalyze the collection of requirements and tasks represented by the next phase and proceed forward with the new expectations and evaluation metrics. Using the change control process to slide tasks around within “predefined” releases is not an effective management approach. Overall, the ICP project seems to be on a good foundation with regard to project management; the team has not learned everything necessary for ICP to be considered a mature development project, but team members are learning key lessons that will serve them well in the future. Release 3 of ICP has essentially been redefined and appears to have been given many new capabilities. Its orientation seems to have shifted from Customer Service to Compliance, and thus from one site executive to another. Because of this fundamental change in end-user focus, the IRS must realize the following: It would be best to terminate ICP after Release 2 and to begin the approval process anew with new site executive support, a new project name, and new justification, along with a new task definition, new evaluation metrics, and new dates. The DPS program, however, is in deeper trouble, and 1040A processing has been delayed 4 months from August 1995 to January 1996, postponing the DPS rollout by one processing season (1 year). A critical project review was conducted by the IRS’s Federally Funded Research and Development Center (FFRDC) in early 1995. The results of that review, as provided to the committee, are disheartening for an IRS project that has been under way for more than 4 years. (Although the DPS contract was awarded within the 15 Architecture/Software Subcommittee meeting with ICP project in Dallas, Texas, June 26–27, 1995.
OCR for page 47
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report last 2 years, the project itself has been staffed within the IRS for at least the past 4 years.) Among the most important observations made by the FFRDC were the following:16 Requirements documents need to be completed and placed under configuration management; Planning/scheduling is inadequate; Progress is not monitored against plans; and No definition has been agreed upon for a “successful” pilot. Clearly, as of early 1995, the DPS project team (handling a project much larger in size and scope than ICP) had not yet learned the lessons that had already been learned or were being learned by the ICP project. Given that DPS has been an integral, and presumably well-understood, part of TSM since its inception, there should not have been so many requirements remaining to be completed. There are other indications that the management of the DPS project needs to be improved. As of early 1995, the DPS contractor was working under a compressed development methodology on an 11-month schedule in order to meet the original IRS schedule. Such an accelerated approach is probably not wise, given that the IRS had received schedule estimates ranging from 18 to 36 months for the same task from three other contractors, including the FFRDC. No reason has been given by the IRS for the compressed DPS schedule. The DPS project has also been delayed because of delays in the Tax Data Perfection System (TDPS), which will serve to validate the data collected by DPS and other submission processing systems. The TDPS is being developed solely by the IRS: a relatively low-cost project, it is well within the technical and operational capabilities of IRS developers. Unfortunately, development of TDPS started relatively recently, and it will require longer than expected to develop the essential capabilities. Had it been started when TSM commenced, it could have been the focal point for the “new” tax return processing capabilities being provided by TSM. Again, since the need to validate tax return data has been known since the inception of TSM and since the DPS procurement process stated that DPS would have to integrate with TDPS, it is difficult to understand why the IRS did not start working on the TDPS earlier during the last 5 years. On a technical note, the IRS is not using any simulation early in design to identify architectural bottlenecks. Thus, stress testing is ex post facto. This again is not modern practice, but probably not a major problem for the DPS program. If the DPS pilot were to go onto 1040A forms in January 1996, it would get stress-tested enough by the normal rush of late filings, with the manual line continuing to serve as backup. Recommendation 3.4. The committee recommends that the management process move away from integrated, multiyear projects, with the later years poorly defined, and instead break the development process into a set of smaller projects, with clear, self-contained objectives. 16 “DPS Critical Program Review,” IRS internal use document and memorandum dated January 11, 1995.
OCR for page 48
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report This partitioning will entail a management shift away from depending so heavily on the change control process toward handling more and smaller tasks in the project selection and approval process. Furthermore, all projects must be brought under the same project management process, complete with checks and balances to ensure that individual project managers do not cause major delays in the delivery of their projects because of inadequate advance planning or proactive project management. Program Management Tools The discussions regarding the use of program management tools (i.e., PS6, Artemis, Open Plan) have been confusing. The IRS states that the information from PS6 (and Open Plan) can be made to flow upward into Artemis through extracts. If this is the case, the committee questions why PS6 data cannot be converted into Artemis-compatible formats and flow downward so that the PS6 data can be replaced permanently and one system can be used by everyone. At one point in the committee’s discussions, it was suggested by the IRS that the two tools were not only compatible but necessary and that the committee’s objections represented a differing management philosophy. The experience in industry is, “Don’t throw information over the walls; tear the walls down.” The point of view expressed by the IRS would indicate that the intent is to preserve the walls and have one tool serve as a check or balance on the other. If this is so, it raises questions about the level of trust and common purpose necessary to make TSM happen. It is the committee’s opinion that the need for an integrated project management system for TSM is more important at this time than keeping such a check-and-balance approach. With the change in organizational structure recently announced, which involves the CIO reporting to the Associate Commissioner for TSM, the committee sees an opportunity to simplify program and project management tools into one seamless control system. Recommendation 3.5. The committee recommends that the Associate Commissioner move promptly to bring TSM’s management tools into one common system. Suggestions were made to the committee that collecting and entering the data needed for comprehensive work breakdown structures and critical-path networks are extremely expensive and labor-intensive tasks. The committee’s reaction is, Yes they are and there could be no task more important for TSM to accomplish than to create such a comprehensive database. In fact, it is well recognized that nearly all of the benefit of building these databases is in the identification of issues that result from the interaction needed to create them. Evaluation and Metrics In its 1992 report,17 the committee noted the tendency of managers involved in large-scale technology-based projects to “focus entirely on ‘making things happen,’ neglecting the necessary planning required to accurately determine ‘just how well things 17 Computer Science and Telecommunications Board, National Research Council. 1992. Review of the Tax Systems Modernization of the Internal Revenue Service. National Academy Press, Washington, D.C.
OCR for page 49
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report happen.’” The committee commended the IRS for its early steps in evaluation planning and metrics, cautioning that “[i]t remains for the IRS to follow through with that commitment.” With few exceptions, the IRS has not followed through and, instead, has fallen prey to the common practice of putting off the development of metrics while charging ahead with the more immediate technological challenge. In the committee’s view, this lack of attention to clear-cut objectives, metrics for measuring those objectives, and evaluation approaches for making valid inferences about accomplishments has been one of the major barriers to greater progress in TSM. To the extent that the IRS has developed metrics, they pertain to “inputs” rather than to solid “outcome” criteria. Metrics require a continual focus on the ultimate values a program or project is designed to serve. “Number of persons assigned to a task” is not an adequate metric because it does not reflect progress toward achieving an ultimate value. Similarly, “providing improved service” is not an adequate objective for developing metrics. Metrics should center on such issues as time required to respond to a taxpayer inquiry or costs of processing a form. TSM should, in turn, be assessed on the basis of its contribution to those metrics. If possible, the metrics should reflect a chain of causal reasoning, linking particular elements of TSM to rudimentary performance metrics. Metrics are common in good systems and software design, but metrics are equally appropriate for management and organizational structure change, privacy and security, and other such elements of TSM. Recommendation 3.6. The committee recommends the following: The IRS should begin a comprehensive benchmarking effort to ascertain the appropriate type and magnitude of metrics appropriate for TSM tasks; The IRS should work with other federal agencies (e.g., the National Aeronautics and Space Administration, National Institute of Standards and Technology, Department of Defense) to learn from their more extensive experience in evaluation and metrics; and Any metric employed by the IRS either should measure an ultimate value or should be tied directly to the measure of an ultimate value. Integration Test and Control Facility The Integration Test and Control Facility (ITCF) generated a heated debate among committee members. The ITCF is clearly an attempt to make it easier for the development group to work alone on integration, rather than getting involved with the messiness of launching into a production facility. There are numerous examples of the value of prototyping of applications in a realistic simulated environment. Doing so can reduce the risk of building the wrong product and reduce the disruption of essential operations when prototype systems and software are introduced.18 The IRS managers are convinced that they are right to proceed in this direction. Although some members of the committee agreed, others were ambivalent. A few members remained unconvinced that the IRS can 18 See, for example, the simulations of the F-16 for prototyping and testing block changes to the avionics software.
OCR for page 50
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report clear all problems and make a seamless transition to the user organizations without additional time and cost from making the implementation twice: once from development to the ITCF and a second time from the ITCF to the production facility. (Integration testing was discussed in the committee’s 1994 report.19) HUMAN RESOURCES ISSUES The committee wishes to highlight three continuing sensitive areas in human relations: The effective working relationship between the IRS and the National Treasury Employees Union must continue; Communication of the status of TSM to all levels of the IRS must not only continue but also be expanded; and Succession planning must be carried out for IRS leaders and executives to maintain adequate key technical skills for completing TSM development. Other key human resources issues still of concern to the committee are facilities planning and contractor management. Technical management skill needs are covered in Chapter 2. Facilities Planning and Employee Training To ensure that facilities planning, employee training, and major transition of centers (including relocation and terminations) meet software and hardware implementation schedules, the committee considers it critical that the Facility Planning Models be piloted and validated at a very early date. Contractor Management Absent sufficient internal competency, it appears necessary to use qualified contractors for the management of parts of TSM. Currently, the committee considers that there is insufficient project management and software development talent in the IRS to achieve effective contractor management. Recommendation 3.7. The committee recommends a focused hiring of professionals to fill project management and software development positions, identifying higher-potential IRS employees who can be developed quickly to assume key management and professional roles. This develop- 19 Computer Science and Telecommunications Board, National Research Council. 1994. Continued Review of the Tax Systems Modernization of the Internal Revenue Service. Computer Science and Telecommunications Board, Washington, D.C., pp. 20–21.
OCR for page 51
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report ment can be expedited by assigning trusted and qualified contractors to behave as shadow mentors to effect knowledge transfer.20 HOW TO CONTINUE TO MAKE IT HAPPEN The IRS Commissioner’s Office, and thus its role in TSM, are subject to political change, external pressures involving funding levels, and current operational problems. By its nature, it is not technologically at the forefront. For these reasons, the committee continues to make the following recommendation, as did its predecessor: Recommendation 3.8. The IRS Commissioner should establish a small ongoing group of objective, outside, knowledgeable, and independent advisers. This group would need to be most focused on the Commissioner’s needs and problems with TSM. With regular, private meetings and updates, such a group should be able to serve as a resource for the Commissioner.21 The value of an independent advisory group is that it would help the Commissioner assess and respond to internal reports and recommendations, especially those from more technically fluent parties, which are frequently “filtered” and often slanted in the favor of the unit recommending them. The Commissioner needs assistance in knowing what to ask and how to recognize danger signals, including the camouflage that often accompanies them. SUMMARY The IRS has made management progress in the 2-year period covered by the current committee. It now has a comprehensive set of plans covering its business mission and the implementation of TSM. What remains is to refine and interlock these plans on an ongoing basis to serve as vehicles for making TSM happen. Although operational risk must be planned for, the level of integration and operational risk is not yet high, since a fallback to manual systems is still possible. The committee endorses the latest organizational changes as appropriate to having a unified and seamless organization from the Associate Commissioner down through the CIO and the information systems development groups. It remains for the IRS to follow through on the eight specific recommendations made in this chapter to implement effectively the processes and organizational structure that the committee believes are needed. 20 In November 1995, the Commissioner indicated that a contractor mentoring effort was being implemented. On the other hand, the committee remains uncertain whether the Human Resources staff has become involved on a daily working basis with the various contractors so as to understand the training and retraining requirements necessary to implement the stages of TSM. 21 Whether such a group would have to work under the Federal Advisory Committee Act (FACA), and whether FACA provisions would undercut its effectiveness, need careful consideration. Clearly, some constraints of the FACA on dissemination of sensitive information would have to be invoked.
OCR for page 52
Continued Review of the Tax Systems Modernization of the Internal Revenue Service: Final Report The committee is very much aware of and concerned about three corresponding issues: Organizational and management changes have been painfully slow because of the inherent IRS resistance to change over the years; Organizational change in and of itself does not change actual behavior unless strong leadership from the top down causes such changes to occur; and Reward systems still appear to reward seniority rather than the accomplishment of assigned goals and the metrics thereof. Certainly, most recent organizational changes represent a major step for the IRS and for TSM, particularly when coupled with those that preceded them. However, organizational progress must not be mistaken for real progress, because the former simply sets the stage for the latter.
Representative terms from entire chapter: