Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Systems Acquisition (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix C What Is Systems Engineering? definitions Before one can develop a definition for systems engineering (SE), one must develop a taxonomy for what constitutes a system, including the hierarchy of the elements of any complex system (Table C-1). Depending on one’s perspective, the term “system” can be used to describe any element of this hierarchy, from a part to a system of systems (SoS). SE has been defined in many different ways, ranging from high-level state- ments to detailed process overviews. These definitions are often tailored to illus- trate a particular background or perspective. However, most accepted definitions have common themes describing a top-down process that is life-cycle-oriented and involves the integration of functions, activities, and organizations. The most widely accepted definitions of systems engineering are presented in Table C-2. To capture all of the essential elements addressed in this report, the commit- tee chose a fairly detailed, three-part definition of SE: (1) SE is the translation of a need or deficiency into a system architecture through the application of rigorous methods to the iterative process of functional analysis, allocation, implementa- tion, optimization, test, and evaluation; (2) it is the incorporation of all technical parameters to ensure compatibility among physical and functional interfaces, and hardware and software interfaces, in a manner that optimizes system definition and design; (3) it is the integration of performance, manufacturing, ­ reliability,   Benjamin Blanchard and Wolter Fabrycky, 2005, Systems Engineering and Analysis (4th Edition), Englewood Cliffs, N.J.: Prentice Hall.   Modified from Systems Design and Operational Effectiveness 625 Class Note—“Systems Design and Operational Effectiveness,” Stevens Institute of Technology, 2007. 124 

AppendiX C 125 TABLE C-1  Hierarchy of Systems Components Term Definition System of A configuration of systems in which component systems can be added or systemsa removed during use, each providing useful services in its own right, and each is managed for those services. Yet together they exhibit a synergistic, transcendent capability. Systemb An integrated set of elements, segments, and/or subsystems that accomplish a defined objective, such as an air transportation system. Subsystemb An integrated set of assemblies, components, and parts that performs a clearly separate function, involving similar technical skills, or a separate supplier. Examples are an aircraft onboard communications subsystem or an airport control tower as a subsystem of the air transportation system. Assemblyb An integrated set of components and/or subassemblies that constitute a defined part of a subsystem, e.g., the pilot’s radar display console on the fuel injection assembly of the aircraft propulsion subsystem. Subassemblyb An integrated set of components and/or parts that comprise a well-defined portion of an assembly, e.g., a video display with its related integrated circuitry of a pilot’s radio headset. Componentb Composed of multiple parts; a clearly identified item, e.g., a cathode ray tube or the ear piece of the pilot’s headset. Partb The lowest level of separately identified items, e.g., a bolt to hold a console in place. aAir Force Scientific Advisory Board, 2005, System-of-Systems Engineering for Air Force Capabil- ity Development, SAB-TR-05-04, July. Available at http://stinet.dtic.mil/oai/oai?&verb=getRecord& metadataPrefix=html&identifier=ADA442612. Last accessed on April 2, 2007. bInternational Council on Systems Engineering (INCOSE), 2004, INCOSE Systems Engineering Handbook (Version 2A), Seattle, Wash.: INCOSE. maintainability, supportability, global flexibility, scalability, interoperability, upgradability, and other special capabilities into the overall engineering effort. Figure C-1 shows an important relationship between three parallel aspects of system development: the functional decomposition of a system shown in the center, supportability and logistics shown on the right, and cost shown on the left. As one follows each process flow, activities across each type of requirement are interdependent upon, and impact, one another. This figure demonstrates the importance of trade-off analysis in developing system requirements to balance performance, cost, and other specialties throughout the system life cycle. SE goes well beyond traditional engineering concepts and tools. In the broad- est sense, it encompasses systems thinking and other related systems disciplines 

126 PRE-MILESTONE A AND EARLY-PHASE SYSTEMS ENGINEERING TABLE C-2  Standard Definitions of Systems Engineering (SE) Source SE Definition International Systems engineering is an interdisciplinary approach and means to enable Council on the realization of successful systems. Systems Engineeringa Military The application of scientific and engineering efforts to: Standard on (1) transform an operational need into a description of system performance Engineering parameters and a system configuration through the use of an iterative Management process of definition, synthesis, analysis, design, test, and evaluation; 499Ab (2) integrate related technical parameters and ensure compatibility of all related, functional, and program interfaces in a manner that optimizes the total system definition and design; and (3) integrate reliability, maintainability, safety, survivability, human, and other such factors into the total technical engineering effort to meet cost, schedule, and technical performance objectives. Department of Systems engineering is an interdisciplinary approach or a structured, Defensec disciplined, and documented technical effort to simultaneously design and develop systems products and processes to satisfy the needs of the customer. Systems engineering transforms needed operational capabilities into an integrated system design through concurrent consideration of all lifecycle needs. NASAd Systems engineering is a robust approach to the design, creation, and operation of systems. aBenjamin Blanchard and Wolter Fabrycky, 2005, Systems Engineering and Analysis (4th Edition), Englewood Cliffs, N.J.: Prentice Hall. bUnited States Air Force, 1974, Military Standard—Engineering Management, MIL-STD-499A, May 1, Washington, D.C.: Department of Defense. cDefense Acquisition Guidebook, Chapter 4, Section 4.1.1. Available at https://akss.dau.mil/dag/ DoD5000.asp?view=document&rf=GuideBook\IG_c4.1.1.asp. Last accessed on December 3, 2007. dNational Aeronautics and Space Administration, 1995, Systems Engineering Handbook, SP-610S, June. Available at http://snebulos.mit.edu/projects/reference/NASA-Generic/NASA-STD-8739-8.pdf. Last accessed August 30, 2007. inherent to the execution of traditional engineering. It is not solely intended for those products described in the academic definition of a system but should also include subsystems, systems of systems, and enterprise-level problems. SE should be applied to all areas that affect the successful completion of a system, including financial management, management of technical risk, political support, and social context. SE practices and approaches have historically been applied to everything from single systems to complex systems of systems. The SE community (e.g., the Institute of Electrical and Electronics Engineers, SoS conferences, the Inter- 

AppendiX C 127 Requirements Requirements Commercial Market Analysis Ops Analysis Manpower/ Personnel Functional Cost Analysis Analysis Analysis Modularity Standards Standards Function/Cost Commonality Establish Cost Allocation Goals Architecture RMA Training Validate Cost Documentation Goals Modeling and Modeling and Trade-Off Alternatives Alternatives Maintenance Monitor Test/Support Achievement Design Facilities Build/Test Supply Deploy Support PHS&T FIGURE C-1  Relationship among the traditional systems engineering functions (center column), cost (left column), and supportability and logistics (right column). SOURCE: C-1 Modified from Systems Design and Operational Effectiveness 625 Class Notes—“Systems Design and Operational Effectiveness,” Stevens Institute of Technology, 2007. national Council on Systems Engineering [INCOSE], and the System of Systems Center of Excellence) is paying increasing attention to issues of SoS, com- plex systems, and enterprise systems domains. In this report, the committee has loosely used the term SE to apply to tools, techniques, and processes for all of these domains. Primary products of good SE are robust and efficient architectures. Archi- tectures are multidimensional representations or combinations of “what, how, 

128 PRE-MILESTONE A AND EARLY-PHASE SYSTEMS ENGINEERING where, who, when, and why.” Regardless of perspective, method, source data, and framework, an architecture description is a representation of a defined domain in terms of its component parts, what those parts do, how the parts relate to one another, and the rules and constraints under which the parts function. It is important to note the difference between an architecture description and an architecture implementation. An architecture description is a representation or “blueprint” of a current or postulated “real-world” configuration of resources, rules, and relationships. It generally contains “views” that are meaningful to each of the multiple disciplines involved in the implementation of the system. Once the blueprint enters the design, development, and acquisition process, the architecture description is then transformed into a real implementation of capabilities and assets in the field. An architecture framework provides guidance in describing architectures, but requires other tools in the tool set to move from representation to implementation of capabilities and assets. The Department of Defense Architecture Framework (DDAF) shown in Fig- ure C-2 is a three-dimensional representation of the multidimensional architecture space. For DDAF, the operational, technical, and system views are critical to developing an understanding of any system, and collectively these should capture the data derived from an analysis of the system. The “All Views” products provide information pertinent to the entire architecture. An interdisciplinary effort (or team approach) is required throughout the system design and development process to ensure that all design objectives are met in an effective manner. This necessitates a complete understanding of the many different design disciplines and their interrelationships. Tools and Methodologies A wide variety of SE methodologies is used within the defense industry, and an even larger collection of tools has sprung up to support SE processes in gen- eral. Information on commercial off-the-shelf (COTS) and government off-the- shelf (GOTS) tools of interest to systems engineers is available from the INCOSE Tools Database Working Group on its Web site. The database categorizes the tools into four general areas: • Requirements Management Tools Survey, • Systems Architecture Tools Survey, • Measurement Tools Survey, and • General Tools Database.   INCOSE Systems Architecture Tools Survey, available at http://www.paper-review.com/tools/sas/ index.php. Last accessed on April 2, 2007. 

AppendiX C 129 T FIGURE C-2  Department of Defense Architecture Framework. SOURCE: William Wood, Mario Barbacci, Paul Clements, Steve Palmquist, Huei-Wan Ang, Loring Bernhardt, Fatma Dandashi, David Emery, Sarah Sheard, Lyn Uzzle, John Weiler, and Art Krummenoehl, 2003, DOD Architecture Framework and Software Architecture Workshop ­Report. ­Carnegie Mellon University: Software Engineering Institute. March. Available at http://www.sei. cmu.edu/publications/documents/03.reports/03tn006.html. Last accessed on April 2, 2007. Copyright 2003 by Carnegie Mellon University. Reprinted with special permission from the Software Engineering Institute. Under the general heading of SE methodologies are the practices that promote and enable success. Some practices to be considered include the following: • Well-documented processes, readily accessible to the users by being Web- based, searchable, and readily available online and in real time; can be tailored on a program-by-program basis prior to program baselining; and configuration can be managed under a synchronized, yet independent process on each program once the program is baselined; • Templates for all engineering tasks organized by function and discipline, with examples of prior successes and links to available experts; • Lists of available, compatible tool sets matched with the engineering tasks that they automate or support, with solved problems in a searchable archive; • Lists of metrics appropriate to each task and program phase, with the mechanisms for automated collection, tools for analysis, examples of detected anomalies, and links to available experts with experience using the tools and in performing the analyses; • Boilerplate work packages validated by prior usage and containing hints for tailoring and expediting, and parametric models of the effort required to 

130 PRE-MILESTONE A AND EARLY-PHASE SYSTEMS ENGINEERING perform the work (cost, schedule, quality, inputs and outputs, required training/ experience, and so on); • A library of architectures and designs used on prior programs with links to the designers and implementers of each (both successful and unsuccessful); • Anecdotes of problems actually discovered and fixes implemented, linked to relevant work package descriptions and to the individuals who performed the work; and • A library of composable models of systems and subsystems (at multiple levels of detail) with links to designers; and for purchased items, to vendors or other available sources of supply. Modeling and Simulation Modeling and simulation continue to be key elements of SE throughout the acquisition life cycle, especially early in programs. Modeling and simulation allow program managers to quickly develop concepts of operations (CONOPS) and analyses of alternatives as part of pre-Milestone A activities. Further along in the life cycle, modeling and simulation can be used for detailed design. Modeling­ and simulation can be used in a distributed collaborative environment that sup- ports authoritative information exchange and rapid refinement of the design or concept, and over the system life cycle to respond to changing circumstances such as technological advances, changing threats, tactics, or doctrine. Much of the modeling and simulation activity during the pre-Milestone A period is the responsibility of the systems engineers and development planning experts in the government acquisition organization. In most cases, they will be supported by systems engineering and technical assistance (SETA) contractors and the feder- ally funded research and development centers (FFRDCs) as they interact with users to fully understand what is needed, identify the existing systems that will coexist and interoperate with the new capabilities to be acquired, and build a modeling and simulation environment adequate to support the acquisition. During the early phases of defining the needed modeling and simulation environment, the SE team must also establish the metrics to be used for evaluating candidate concepts. The specifics of the modeling and simulation environment can then be filled in such a way that meaningful measures of merit can be extracted from the simulations and used to focus further rounds of simulation, critical experiments, and human-in-the-loop testing. Systems Engineering in DoD Acquisition Programs and pre-milestone a systems engineering With budgets becoming tighter, public scrutiny becoming stronger, the increasing focus being placed on advanced technology, and demands arising from the shift toward network-centric warfare, there has been a major emphasis placed 

AppendiX C 131 on SE within DOD. Policies such as the 5000 series and the SoS guide  and the creation of the Systems and Software Engineering Office within the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (OUSD AT&L) point to an understanding of the contributions that SE can make to modern acquisition. Figure C-3 shows the prescribed acquisition process prior to Milestone A as presented in DOD Instruction 5000.2. Before a program can enter formally into the concept refinement phase, a concept decision milestone must be cleared. Typically this consists of an initial concept, an approved analysis of alternatives plan, and an established Milestone A date. After the concept decision, the concept development/refinement is used to refine the initial concept and to help reduce technical risk. The concept devel- opment phase is guided by the Initial Capabilities Document and AoA with continuous feedback to develop a technology development strategy. Modeling and simulation, optimization, and life cycle costing are all needed to conduct a meaningful analysis of alternatives. Systems engineering products such as the systems engineering plan (SEP) have traditionally not been used prior to the Milestone A decision. In a policy memorandum dated February 20, 2004,  the ODUSD (AT&L) directed that the SEP become a requirement for each milestone review. The next version of the DOD 5000 series of acquisition documents will be updated to reflect this policy.   Michael W. Wynne and Mark D. Schaeffer, 2005, “Revitalization of Systems Engineering in DoD,” Defense AT&L: March-April, pp. 14-17.   The 5000 series refers to DOD Directive 5000.1, “The Defense Acquisition System,” and DOD Instruction 5000.2, “Operation of the Defense Acquisition System.”   Department of Defense (DOD), 2006, System of Systems Systems Engineering Guide: Consider- ations for Systems Engineering in a System of Systems Environment, Version 9, December 22. Avail- able at http://www.acq.osd.mil/se/to%20be%20posted/SOSE%20Guide%20Dec%2022%20PDF.pdf. Last accessed June 26, 2007.   DOD, 2006, Systems Engineering Plan (SEP) Preparation Guide, Version 1.02, February 10. Available at http://www.acq.osd.mil/se/publications/pig/sep_prepguide_v1_2.pdf. Last accessed June 12, 2007.   Office of the Deputy Under Secretary of Defense (ODUSD) (AT&L), 2004, Policy memorandum entitled “Policy for Systems Engineering in DoD,” Washington, D.C., February 20. 

132 PRE-MILESTONE A AND EARLY-PHASE SYSTEMS ENGINEERING Overarching Policy NSS/NMS/Joint vision Joint Concept of Operations Functional Functional Area Area Analysis Functional Concept Integrated Architecture D O T Analysis of M Materiel Materiel Feedback Process Approaches L P from CDD DOTLPF and CPD F Process MS A AoA Concept ICD CD Refinement DAB JROC FIGURE C-3  Requirements and acquisition process prior to Milestone A. AoA, analysis of alternatives; CD, concept development; CDD, capabilities development document; CPD, Capabilities Production Document; DAB, Defense Acquisition Board; DOTLPF, doctrine, organization, training, leadership, personnel, and facilities; DOTMLPF, doctrine, organi- zation, training, material, leadership and education, personnel, and ­facilities; ICD, Initial Capabilities Document; JROC, Joint Requirements Oversight Council; NSS, ­ National Security Strategy; NMS, National Military Strategy; MS A, Milestone A. SOURCE: Department of Defense Instruction 5000.2, 2003, Operation of the Defense Acquisition C-3 System. May 12. Available at http://dod5000.dau.mil/DOCS/DoDI%205000.2-signed%20 (May%2012,%202003).doc. Last accessed on April 2, 2007.