Click for next page ( 48


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 47
APPENDIX C Management Guidelines for Future System Development Projects INTRODUCTION The past several decades have seen the rise of large, highly interactive sys- tems that expand the technology for information processing. As a result of this growth, the concept of systems engineering has gained increasing attention. Some of this attention is no doubt due to large program failures that possibly could have been avoided, or at least mitigated, through the use of sound systems engineering principles. The complexity of modern data processing environments such as the ANCS II requires conscious application of systems engineering concepts to ensure producible, operable, and supportable systems that satisfy mission requirements. Since the mid-1950s, the emergence of specialists in engineering began to formalize the life-cycle development processes and record and refine lessons learned in development programs. Although many of these early systems engi- neering concepts were mandated by the U.S. Department of Defense because of the rigor required in weapons systems developments, the general concepts and lessons continue to apply today for more generalized automated data processing system developments. An excellent overview of this development process is the Systems Engineering Management Guide produced by the Defense Sys- tems Management College (DSMC, 19904. Based on both military standards (MIL-STD-499A, Engineering Management, 1 May 1974) and a more general- ized academic treatise (Blanchard and Fabrycky, 1981), the following comments offer an overview of the total life-cycle of a system. 47

OCR for page 47
48 NAUTICAL CHART PROGRAM QUALIFIED MANAGEMENT AND STAFF On programs of the size and complexity of ANCS II, the program manager should have access to senior-level decision makers within the agency on a con- tinuing basis to ensure that the goals and mission of the organization are consis- tent with the direction of the program. The management staff should ideally have continuity across the life cycle of the project. Assignments need to be stable. Similar criteria apply to the use of any contractors to support the management staff; stable and long-term relationships will benefit the project. Successful implementation of any information system upgrade requires the use of a team approach. Even if the information system is developed outside the organization, to be effectively implemented, development requires the involve- ment of users, operators, managers, and every stakeholder affected by the project. ORGANIZATION MANAGEMENT REQUIREMENTS The failure of many systems begins when decisions are put off or not made. Fuzzy thinking leads to fuzzy results. Less-than-perfect decisions are preferable to indecision; the ramifications of such decisions can be dealt with after the fact. The basic building blocks of a system engineering methodology are writing specifications, developing capabilities for approved specifications, testing the re- sults of the development against the original requirements, and building and de- livering the capabilities specified in approved requirements. The heart of a sys- tems engineering methodology is an active configuration management system with high-level review and approval of requirements by senior agency leaders. Such on-going review and approval requires the maintenance of system docu- mentation and the traceability of requirements from conception through develop- ment, test, and delivery. In the case of the ANCS II procurement, the documentation of the system did not reflect an "as-built" status, and, therefore, it did not reflect the currency of the configuration. To migrate ANCS II to the future whether ANCS II itself is a critical component or not requires a documented understanding of the founda- tion on which the system is built and a direction or vision as to where it is going. LIFE-CYCLE COST ASSESSMENT A major function of the project management team is to build and maintain an independent cost estimate for the project. With any commercially available cost analysis tool it is possible to provide accurate cost estimates for the entire life cycle of a computer system. The cost estimates, together with at least a 20 percent management reserve, must then be folded into budget planning for the system. Accurate cost estimates will not occur until system documentation has been written

OCR for page 47
APPENDIX C 49 that describes the desired capabilities. At a minimum, a systems engineer, a soft- ware engineer, a database expert, and a cost analyst are needed to build a good cost model for the ANCS II evaluation. Recognizing that federal agency budgets are controlled by forces external to the agency, it is important that any acquisition strategy be supported by a com- mitment to adequate funds to carry out the projects. Often a practice of "design- to-build" is used when the system boundaries are demarcated by financial consid- erations. This often leads to a system development process that is more focused, in that the development contractor knows that engineering change proposals will not be funded except in extreme circumstances. At the same time, the agency understands that requirements must be clearly articulated, and the process must be managed carefully because of funding limitations. Such an approach could be used by NOAA in future procurements or, alternatively, NOAA could choose to have contractual mechanisms that are firm-fixed-price as opposed to cost-plus. NOAA'S ROLES AND MISSIONS The acquisition strategy must clearly support NOAA's roles and missions in the short and long term. Because the NOAA role in providing nautical informa- tion products is expanding through the provision of digital data in addition to paper products, the acquisition strategy should be focused on achieving this ob- jective. If future objectives can be expressed in concrete terms, the acquisition strategy can be specifically tailored to fit these requirements. The senior agency management's role is to ensure that the system that is being built will support the agency's future roles and missions. Uncertainty in roles and missions will lead to uncertainty and confusion in the implementation of any planned changes and upgrades. Major policy considerations to be included in any acquisition strategy in- clude agency policy on competitive procurement, compliance with federal infor- mation processing standards, and industry standards. Competitive procurement needs to be planned as part of the process. Any other approach (such as sole source) will be difficult in gaining approval and will lead to procurement and technical delays. SYSTEMS ENGINEERING MANAGEMENT APPROACH As discussed above, the use of a dedicated program manager and staff with access to key decision makers with an established requirements database and the discipline to enforce configuration management is basic to any systems engineer- ing approach. Other key factors include the adoption of a realistic migration plan and the use of a team approach to implement the planned system. Systems engineering skills are based on engineering problem-solving tech- niques. These techniques are acquired through experience on similar projects and

OCR for page 47
so NAUTICAL CHART PROGRAM do not require hard-core engineering or computer science skills. Many system engineering methodologies are based on common sense logic and can be taught to individuals with widely diverse training. However, it is generally recommended that a program manager possess either an engineering or a computer science de- gree and related experience. Every systems engineering methodology is based on the maintenance of a comprehensive suite of documentation. An active configuration management sys- tem and the enforcement of a disciplined approach is required throughout the system life cycle. At a minimum, the following documents are required for sup- port of any system: system specification operations concept subsystem specifications interface control documents requirements verification transition plan test plan, procedures, and reports maintenance manuals operator manuals When a system is being developed to replace or upgrade an existing system, additional requirements are imposed on the systems engineering process. The systems engineering methodology must plan the transition to operation of the new system and the retirement from operation of the old system. This requires special attention to avoid any gaps in support of current agency operations and the transition of data from one system to the next without loss of current func- tions. A final step in systems engineering is the evaluation of how effectively the new system is operating. This may be as simple as counting how many maps and charts are produced or may involve more complex metrics requiring feedback from customers and users. A particular problem for users of information technology is the decreasing length of the life cycle for systems that use COTS (commercial off-the-shelf) technology. In the past, systems might have a life cycle of six to eight years before replacement. COTS upgrades now occur every year, and it is typical for a COTS-based system to have a life cycle of only three to four years before re- placement. Developmental Approaches The traditional approach for building new systems is the waterfall methodol- ogy in which, typically, the development cycle is three or four years. In this approach, little or no capability is delivered prior to the end of the cycle. Several alternatives to the traditional waterfall methodology should be considered,

OCR for page 47
APPENDIX C 51 including spiral and prototype development. The main difference among these methodologies is how requirements are handled. Spiral development is the "build a little, test a little, deliver a little" approach. This approach assumes that requirements are reasonably stable, but that incre- mental delivery of capability is feasible. Spiral development provides users with needed improvements sooner than the traditional waterfall approach. Spiral de- velopment also lends itself to situations in which budgetary uncertainty exists if funds are cut off or reduced, there is still something to show for the effort. Spiral development is becoming increasingly popular for use with systems based on COTS integration. Prototype development lends itself to situations in which requirements are not well defined. In this situation the developer can simulate and build prototype pieces of the final system and obtain feedback from the customer to focus the undefined areas and firm up requirements. Prototyping does not usually lead to incremental delivery or early delivery of capability. A variation of the normal developmental approach is to use some combina- tion of the three approaches described above. Spirals may be used for one part of a system while prototyping is used on a different portion. Spirals may be used effectively with prototyping. In addition, waterfall and prototyping can be used effectively together. However, spiral and waterfall approaches should normally not be mixed. Training Requirements Training is a fundamental requirement for any system and is one of the ele- ments of transition planning. Coincident with the delivery and testing of a sys- tem, the training of operators, administrators, and users provides the nucleus for successful operation of the new system. Training requirements should be built into the statement of work and the schedule for development of the system. If training requirements are not known at the time system development is started, the development contractor can be required to provide a training plan that out- lines proposed training courses and materials. After approval of the training plan by the agency, the developer would provide the approved services.