4
Basic Research for Air Force Software

Network-enabled systems are by definition dependent on complex software because of the great number of possible states of the networks. Building such software is very challenging. The systems that require such software transcend a range of Air Force applications, from intensive human-machine systems (e.g., command and control, air operation centers) to embedded applications (e.g., avionics systems). Increasingly these applications are connected by networks into a system of systems, and, in fact, the distinction between enterprise and embedded systems blurs as the focus is increasingly on the interconnectedness of all such systems. This chapter develops recommendations for basic research to enable more successful development of network-enabled systems, and those recommendations therefore apply to enterprise systems as well as to embedded systems.

The Air Force’s experience with software over the past 30 years or more has been one of grappling with more and more lines of code to implement increasingly complex functionality in a software-intensive system that itself may contain many interconnected subsystems (e.g., a fire control system and a navigation system on a fighter aircraft). This software complexity has contributed to the challenges encountered in developing and fielding operational systems such as the F-22 aircraft and space systems such as the Space-Based Infrared System (SBIRS).1 The

1  

Department of Defense, Report of the Defense Science Board Task Force on Defense Software (2000). Available at http://www.acq.osd.mil/dsb/reports/defensesoftware.pdf.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs 4 Basic Research for Air Force Software Network-enabled systems are by definition dependent on complex software because of the great number of possible states of the networks. Building such software is very challenging. The systems that require such software transcend a range of Air Force applications, from intensive human-machine systems (e.g., command and control, air operation centers) to embedded applications (e.g., avionics systems). Increasingly these applications are connected by networks into a system of systems, and, in fact, the distinction between enterprise and embedded systems blurs as the focus is increasingly on the interconnectedness of all such systems. This chapter develops recommendations for basic research to enable more successful development of network-enabled systems, and those recommendations therefore apply to enterprise systems as well as to embedded systems. The Air Force’s experience with software over the past 30 years or more has been one of grappling with more and more lines of code to implement increasingly complex functionality in a software-intensive system that itself may contain many interconnected subsystems (e.g., a fire control system and a navigation system on a fighter aircraft). This software complexity has contributed to the challenges encountered in developing and fielding operational systems such as the F-22 aircraft and space systems such as the Space-Based Infrared System (SBIRS).1 The 1   Department of Defense, Report of the Defense Science Board Task Force on Defense Software (2000). Available at http://www.acq.osd.mil/dsb/reports/defensesoftware.pdf.

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs advent of network-enabled systems is in some sense an extrapolation of the old challenge: the development, integration, and sustainment (maintenance) of many interconnected, large, software-intensive systems. And the Air Force expects to continue to develop and sustain systems that are unprecedented in terms of their size and complexity. Therefore, a basic research program in software must address how to develop and sustain such large families of software-intensive systems. Rather than focusing on large-scale code development—a challenge that is being researched by others—the committee recommends that AFOSR focus on a set of important software engineering issues that are key to successful Air Force network-enabled systems but that have received limited attention. This recommended set of issues centers on how to understand what to build and how to ensure that its behavior is relatively predictable and acceptable, both during design and in operational use. Little is predictable about an end product’s behavior when and where it is needed, and this knowledge is critical to understanding both how to engineer a new system and how to sustain and upgrade an existing system. In order to address this challenge, the committee recommends revisiting a concept articulated in 1987 by the Air Force Scientific Advisory Board: Pursue research that helps imbue unprecedented software development projects with some of the attributes that enable success when there is a precedent. Quoting from that report:2 The main philosophy behind risk reduction [for software] is to make unprecedented systems as precedented as practical. Precedented systems are those for which (1) the requirements are consistent and well understood, (2) the system architecture (both hardware and software) is known to be adequate for the requirements, and (3) the acquisition and development teams have worked together to develop a similar previous system. Violation of one or more of these elements of definition causes the system to be unprecedented. Consequently, certain risks arise that require an acquisition-development strategy for their mitigation. A primary assertion … is that precedence is one of the most important elements in the timely development of quality software. In many cases, it is more dominant than many of the more narrowly focused issues that receive more attention, such as language, tools, hardware instruction set, architecture, etc. Simply stated, the concept of precedented systems means that the learning curve applies to software development, as it does in all other areas of human endeavor. 2   Air Force Scientific Advisory Board, Adapting Software Development Policies to Modern Technology (1987). This report is summarized in R.J. Sylvester and M. Stewart, “Unprecedented systems,” Encyclopedia of Software Engineering, J. Marciniak, ed., John Wiley (1994).  

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs FIGURE 4-1 Logical flow of software development in support of Air Force operations. That report went on to note that engineering and management techniques have been developed to address the predictability of software and its behaviors, but only in domains that are fairly well understood and where the team developing and acquiring the software-intensive system has worked together on a similar system. With network-enabled Air Force systems, such experience in software development and maintenance does not yet exist, at least not with actual systems in operational settings (as opposed to tightly controlled and scripted exercises). That gap can be addressed by research into three major questions: How do we discover and understand what is needed? How can critical nonfunctional attributes (those that are desired or necessary but ancillary to the software’s primary functionality) be implemented in a predictable fashion? Can the resulting software, once fielded, evolve to satisfy new needs discovered as it is used? Figure 4-1 shows the logical flow from one question to the next and also indicates an appropriate research theme for each question. These themes, which leverage related research already under way at AFOSR and other research funding organizations, are explored next, and specific research topics are recommended. COEVOLUTION OF AIR FORCE CONCEPTS OF OPERATIONS AND SYSTEM ARCHITECTURES One of the persistent problems in software development is requirement uncertainty. This is particularly problematic with the complex unprecedented systems envisioned by the Air Force. The software management literature is replete with methods to instill disciplined approaches to

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs developing and managing (e.g., the Software Engineering Institute’s Capability Maturity Model Integration),3 but an even more powerful methodology is needed for the Air Force’s next-generation systems; it will require far more than process discipline to avoid repeating recent problems encountered with Air Force software systems. The committee recommends that AFOSR sponsor research to extend the philosophy of software development models, such as iterative development, that support rapid prototyping of a software system so that end users can work with the system to see if it satisfies their needs. Through such interplay, the prototype then becomes an explicit representation of the requirements. To extend this research for, say, a network-enabled C4ISR system, a software model of the proposed system could be constructed and used in simulation-based exercises. The exercises would serve the dual purpose of discovering and validating user requirements, which help build Air Force concepts of operations (CONOPS) while supporting the design and analysis of the architecture. This is consistent with the architecture-first approaches advocated in best practices (e.g., using the Unified Modeling Language (UML) to enable architecture design that is motivated by cases supplied by end users) and the DOD standards for operational, system, and technical views of systems. This model for requirements definition has the additional benefits of building stronger communication between researchers and the user communities and of providing infrastructure to better support team-based problem solving involving humans and automated decision aids, which is particularly valuable for next-generation enterprise systems. Current research in executable architectures and in engineering tools for the design and analysis of functional and nonfunctional attributes provides a basis for enabling this coevolution. Architecture development methods typically rely on standards-focused specifications and policy documents (e.g., those for the GIG, JTA, DODAF, DII COE) and on environments based on industry standards such as real-time CORBA. Those methodologies bring together operational views, which describe how the system will be used from the users’ perspective and which might be expressed by the software or system engineer in a formal language such as UML; technical views, which describe the building codes or standards with which systems will be constructed; and the system view, the actual architecture of the system to be fielded. The system view satisfies the needs as defined in the operational architecture and adheres to the constraints imposed by the technical view. 3   Capability Maturity Model Integration is a service mark of Carnegie Mellon University.

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs Technologies now exist for modeling an architecture, the most popular being UML. Tools and methods exist for evaluating the architecture from the perspective of scalability, real-time performance, survivability, and other nonfunctional attributes; for example, the Architecture Tradeoff Analysis Method (ATAM)4 is a commonly used analysis technique for architecture assessment. Some work has been done on executable and model-based architectures, which enable early execution in a simulation environment. These analysis methods typically work on static representations. A new approach in the software engineering community, the Model Driven Architecture (MDA), is designed to document an organization’s business and business rules and the structure of its information infrastructure so as to create an expression of the business need for software that can guide IT projects. The power of the MDA approach is that the people who know the business will develop and maintain the business model, while the software engineers will develop and maintain the design and documentation of the software. These ideas are also being explored by other communities. For example, the workflow management community has developed an XML-compliant standard to support the explicit definition of business processes and the separation of these processes from underlying applications.5 In addition, the Object Management Group (OMG) recently released a draft version of a new language called the System Modeling Language.6 These recent advances would be extended through the recommended research into methods to allow the coevolution of Air Force CONOPS and system architecture. The committee recommends a general goal of providing executable versions of systems earlier in the development process to ensure that the operational view (the CONOPS) is accurately, effectively, and consistently represented by the system and technical views. To achieve that goal, the committee recommends research into the following: Methods to support rapid composability. Semantic extensions of current modeling languages to enhance composability along with representation of and reasoning about behavior. Development of tools that enable the construction of executable versions of models in system modeling languages. Methods that support experimentation, operational assessment, and the use of initial architecture representations in exercises. An 4   ATAM is described at http://www.sei.cmu.edu/ata/ata_init.html. 5   For example, see http://www.wfmc.org/standards/docs/tc003v11.pdf. 6   See http://www.sysml.org.

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs example might be scripting languages that allow end users to explore early versions of software and help encode their preferences into the final architecture. Approaches that allow user tailoring, definition, and exploration of new processes and automated learning based on past problem solving. Experimentation and demonstration of these research approaches in domains of revelance to the Air Force. SOFTWARE BEHAVIOR ENVELOPES Air Force systems need to be scalable, interoperable, survivable, secure, evolvable, and capable of predictable real-time performance and energy awareness, among other things. These are “nonfunctional” attributes: characteristics of the software that, while necessary for the success of the system, are ancillary to the software’s ability to execute its function(s). The degree to which such attributes can be achieved in a system requires trade-off analysis during design and monitoring during execution. To this end, the committee recommends that AFOSR support a new line of research, extending DARPA-funded, model-based software research to explore the concept of a software behavior envelope. Dynamic analysis of nonfunctional attributes based on embedded models could be viewed as the behavior envelope of a network-enabled system. The extent to which the software could be modified within the envelope, by developers or end users, provides a measure of system elasticity. The analogy is to the flight envelope of an aircraft: Stay within the envelope and performance is safe and assured; wander outside the envelope, and tragedy could follow. This topic would be a new area of research for the software community, but there is related work upon which to build. One related area is the approach called rate monotonic analysis (RMA), which is commonly used for real-time scheduling in operating systems. RMA was credited with enabling the rapid debugging and repair of the Mars Rover during its 1997 mission.7 Another avenue of research is qualitative modeling.8 This is worth reexamination because of the huge increases in computing power over the past 20 years, the lack of which was a limiting factor in those earlier efforts. A third related area comes from a recently completed DARPA program, the Model-Based Integration of Embedded Software 7   See http://www.cs.cmu.edu/afs/cs/user/raj/www/mars.html. 8   For example, see D.S. Weld and J. de Kleer, eds., Readings in Qualitative Reasoning about Physical Systems, Morgan Kaufmann: Los Altos, Calif. (1990).

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs (MoBIES) program, which was largely executed by AFRL’s Information Directorate.9 A fourth area of related research would build on reference frameworks for composable systems (e.g., the work of David Garlan and colleagues at Carnegie Mellon University) and the work at the Software Engineering Institute in predictable assembly from certifiable components.10 Another relevant line of work is that by Janos Sztipanovits and colleagues at Vanderbilt University.11 The union of these directions provides a good starting point for research into how to include executable versions of intended system behavior into actual systems, thus providing a means to establish a software performance envelope. This recommended program of research would leverage and extend ongoing research called out in AFOSR’s BAA 2005-01 and research supported by other DOD laboratories, the National Science Foundation,12 and other agencies. EVOLVABILITY THROUGHOUT THE LIFE CYCLE Once a software architecture has been defined and the performance envelope explored, a logical third capability would be one that supports continued evolution of complex software to continue to adapt it to its fielded context. While most other software engineering research focuses on developing new software-intensive systems, in fact the larger challenge is to learn how to maintain and upgrade the huge amount of Air Force software that is already fielded. Thus, important research areas include methods to infer the architecture of legacy software systems, to identify software components within that architecture, to parallelize legacy system software and applications, and to migrate that architecture and components to new and improved architectures, possibly within a new computing environment. The products of the research outlined in the previous section, for predicting software performance envelopes, could also be applied during such a migration to end users to tailor the software in a new operational environment. 9   Mark Lewis, Air Force chief scientist, described the opportunities and benefits of AFOSR continuing, and thus completing, research projects initiated by DARPA in his October 2005 address to the Air Force Scientific Advisory Board. 10   See http://www.sei.cmu.edu/pacc. 11   K. Chen, J. Sztipanovits, A. Ledeczi, and S. Neema, “Toward a semantic anchoring infrastructure for domain-specific modeling language,” Proceedings of EMSOFT’05, ACM Special Interest Group on Embedded Software, September 2005. 12   For example, see the NSF-sponsored study at http://www.cs.virginia.edu/~sullivan/sodsis.html or a related NSF program at http://www.cise.nsf.gov/funding/pgm_display.cfm?pub_id=13078.

OCR for page 52
Basic Research in Information Science and Technology for Air Force Needs Since network-enabled systems will involve many legacy systems as well as new systems, it is imperative that software be designed in a way that it can be evolved in an affordable manner throughout its life cycle. The committee recommends that AFOSR support research to improve the evolvability of software-intensive systems. The following specific lines of research, which could build on readily available commercial frameworks, are recommended: Improvements to our ability to conduct dynamic, model-based analyses to analyze nonfunctional attributes. In order to improve component integration, research to accelerate the development of abstract design-component systems and code-component-based systems, addressing automated discovery, composition, generation, interoperability, and reuse across hundreds of systems. Research in security in support of the goal of measurable, available, secure, trustworthy, and sustainable network-enabled systems. To attain assured reliability with hard time-deadlines, methods for modeling and analyzing integrated reliability, availability, and schedulability of components and systems in realistic conditions derived from user-specified scenarios. Energy-efficient components of the overall system: (1) network energy on network interface and communication protocols of ad hoc networks, (2) processor energy and process management for scheduling various applications, (3) memory/storage energy and memory/storage management, and (4) display energy. Research into novel integration of methods for verification and validation, such as integration of informal methods (e.g., software testing and monitoring) with formal verification (i.e., model checking and theorem proving) and abstract interpretation and static program analysis techniques. The ability to validate scalability, adoptability, usability, and measurement is also important, and some fundamental breakthroughs in the past 5 years have led to a rapid rise in industry adoption and interest.