1
Recognize the Pivotal Role of DoD Software Innovation

THE ROLE OF SOFTWARE IN DEFENSE

The pivotal role of information technology (IT) in defense has been noted in multiple studies.1,2,3,4 Software is increasingly used to embody the functionality of defense systems of all kinds,5 and IT is used pervasively in the Department of Defense (DoD) for a multitude of different purposes and in a multitude of different program types (Box 1.1). It is a key enabler of overall systems scale and complexity, of integration among systems (net-centricity and “ultra-scale”), and of agility in systems. Mission capability embodied in software has become a unique source of strategic and military advantage, and software producibility is emerging as a key component of military strength, capability, and readiness. The committee uses the term “software producibility” to refer to the capacity to design, produce, assure, and evolve software-intensive systems in a predictable manner while effectively managing risk, cost, schedule, and complexity.

The Defense Science Board’s (DSB’s) Task Force on Mission Impact of Foreign Influence on DoD Software, which explored the essential role of software in defense, released its report in September

1

Defense Science Board (DSB), September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

2

DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.

3

National Research Council (NRC), 2010, Achieving Effective Acquisition of Information Technology in the Department of Defense, Washington, DC: National Academies Press.

4

NRC, 1999, Realizing the Potential of C4I: Fundamental Challenges, Washington, DC: National Academy Press. Available online at http://www.nap.edu/catalog.php?record_id=6457. Last accessed August 20, 2010.

5

DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 17
1 Recognize the Pivotal Role of DoD Software Innovation THE ROLE OF SOFTWARE IN DEFENSE The pivotal role of information technology (IT) in defense has been noted in multiple studies. 1,2,3,4 Software is increasingly used to embody the functionality of defense systems of all kinds, 5 and IT is used pervasively in the Department of Defense (DoD) for a multitude of different purposes and in a multitude of different program types (Box 1.1). It is a key enabler of overall systems scale and complex - ity, of integration among systems (net-centricity and “ultra-scale”), and of agility in systems. Mission capability embodied in software has become a unique source of strategic and military advantage, and software producibility is emerging as a key component of military strength, capability, and readiness. The committee uses the term “software producibility” to refer to the capacity to design, produce, assure, and evolve software-intensive systems in a predictable manner while effectively managing risk, cost, schedule, and complexity. The Defense Science Board’s (DSB’s) Task Force on Mission Impact of Foreign Influence on DoD Software, which explored the essential role of software in defense, released its report in September 1 Defense Science Board (DSB), September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010. 2 DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software , Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord &metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010. 3 National Research Council (NRC), 2010, Achieing Effectie Acquisition of Information Technology in the Department of Defense , Washington, DC: National Academies Press. 4 NRC, 1999, Realizing the Potential of C4I: Fundamental Challenges , Washington, DC: National Academy Press. Available online at http://www.nap.edu/catalog.php?record_id=6457. Last accessed August 20, 2010. 5 DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software , Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord &metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010. 

OCR for page 17
 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Box 1.1 A Taxonomy of Information Technology Program Types Many taxonomies have emerged to assist managers in identifying common patterns among re- quirements and system types and then building on that knowledge to define and implement best practices and standards that are suited to particular categories of systems. Distinctions are made based on the function and role of the system, the types of risk to be addressed, the scale of systems and budgets, and other factors. This report adopts the following taxonomy of IT programs: 1. Business systems and office IT 2. Command and control 3. Computing and communications infrastructure 4. Intelligence, Surveillance, and Reconnaissance (ISR), space, and weapons The classification, which is loosely based on a classification scheme used within the DOD to track IT acquisition programs,1 is primarily functional. But the functional categories also correspond roughly to distinctions among programs based on the extent of innovation (more in categories 2 and 4) and those that are more likely to have precedented requirements and architectures and thus build on established ecosystems (categories 1 and 3). These categories also separate IT that is embedded in weapons or weapons systems or similar plat- forms with potentially high systems risk (category 4), IT in which software and hardware are less tightly integrated (categories 1 and 2), and IT that provides the computing and communications infrastructure (category 3) that can be used by systems identified in the other categories. Because modern larger-scale systems are interconnected and therefore more often integrate across these functionalities, greater numbers of systems may cross these boundaries. For example, many weapons systems incorporate command-and-control functionalities. Finally, despite the differences among these categories, most systems rely on similar development practices, including design and architectural concepts, programming languages, process and measure- ment concepts, and tools. 1 Based on a taxonomy used by the Office of the Assistant Secretary of Defense for Networks and Information Integration to categorize major automatic information system programs. 2007.6 The report notes that “in the Department of Defense, the transformational effects of information technology (IT—defined here broadly to include all forms of computing and communications), joined with a culture of information sharing, called Net-Centricity, constitute a powerful force multiplier. The DoD has become increasingly dependent for mission-critical functionality upon highly interconnected, globally sourced, IT of dramatically varying quality, reliability and trustworthiness.” 7 In other words, at the core of the ability to achiee integration and maintain agility is the ability of the DoD to produce and eole software. This echoes a judgment expressed in many other studies that have considered the role of software in defense.8 The report further notes, however, that “each year the Department of Defense depends more on software for its administration and for the planning and execution of its missions,” 6 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software , Wash- ington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet. dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010. 7 Ibid., p. vii. 8 See referenced studies above.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION and additionally “this growing dependency is a source of weakness exacerbated by the mounting size, complexity, and interconnectedness of its software programs.”9 The rapid growth of software in defense systems is especially significant and parallels the growing role of software in a broad range of application domains. This growth is a natural outcome of the special engineering characteristics of software: Software is uniquely unbounded and flexible, having relatively few intrinsic limits on the degree to which it can be scaled in complexity and capability. Software is an abstract and purely synthetic medium that, for the most part, lacks fundamental physical limits and natural constraints. For example, unlike physical hardware, software can be delivered and upgraded electronically and remotely, greatly facilitating rapid adaptation to changes in adversary threats, mis - sion priorities, technology, and other aspects of the operating environment. The principal constraint is the human intellectual capacity to understand systems, to build tools to manage them, and to provide assurance—all at ever-greater levels of complexity. The extent of the DoD code in service has been increasing by more than an order of magnitude every decade, and a similar growth pattern has been exhibited within individual, long-lived military systems. In addition to this growth in size (as well as growth in other system aspects such as resource usage), there is a corresponding growth in overall system capability and complexity. This chapter addresses the first two of the four questions taken up by the committee: • To what extent is software capability significant for the DoD? Is it becoming more significant or less so? • Will the advances in software producibility needed by the DoD emerge unaided from industry at a pace sufficient to meet evolving defense requirements? Growth in the Role and Significance of Software to Defense The value that software contributes to major systems is increasing rapidly and becoming more fundamental to system capability. The DSB Task Force report on defense software (2000) 10 illustrates this point in the case of combat aircraft. The percentage of system functions performed by software has risen from 8 percent of the F-4 in 1960, to 45 percent of the F-16 in 1982, to 80 percent of the F-22 in 2000.11 Software has become essential to all aspects of military system capabilities and operations, and software-specific investment is critical to them.12 Macroeconomic data show analogous growth in the role software plays in the commercial world. This is significant because commercial vendors are key contributors to the defense software supply chain—for Future Combat Systems,13 for example, 27 million source lines of code (more than 42 percent 9 DSB, September 2007, Report of the Defense Science Board Task Force on Defense Software , p. v. 10 DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord &metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010. 11 Ibid., Table 3.3a. Available online at http://www.acq.osd.mil/dsb/reports/defensesoftware.pdf. Accessed February 25, 2008. 12 Boehm, Kind, and Turner quote an unidentified U.S. Air Force General, “About the only thing you can do with an F-22 without software is take a picture of it.” In Barry Boehm, Richard Turner, and Peter Kind, 2002, “Risky Business: Seven Myths About Software Engineering That Impact Defense Acquisitions,” Program Manager, May 1, 2002. The committee notes, however, that with modern cameras, even taking a picture cannot be done without software. 13 Future Combat Systems (FCS) was “the Army’s modernization program consisting of a family of manned and unmanned systems, connected by a common network, that enables the modular force, providing our Soldiers and leaders with leading-edge technologies and capabilities allowing them to dominate in complex environments.” U.S. Army, “Future Combat Systems.” Avail - able online at http://www.army.mil/fcs/. Accessed March 3, 2008.

OCR for page 17
0 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE of the total delivered executable source lines of code) were commercial off-the-shelf (COTS) or open source.14 It is also significant because software capability has become a strategic source of market differentia - tion in many industries, from financial services and health care to telecommunications and entertain - ment. A 2002 report by the National Research Council’s (NRC’s) Board on Science, Technology, and Economic Policy15 noted that since 1995 the IT and networking industries had accounted for 20 percent of the nation’s economic growth, even though they accounted for only 3 percent of gross domestic product (GDP). Comparable figures exist in the European Community—the information and communications technology (ICT) sector represents just above 5 percent of the European GDP, but reports show that ICT drives 25 percent of overall growth and about 40 percent of the increase in productivity. Finding 1-1: Software has become essential to a vast range of military system capabilities and opera - tions, and its role is continuing to deepen and broaden, including interlinking diverse system ele - ments. This creates both benefits and risks. Software in Systems Military system capability is heavily dependent on software, which has become an enabler for much of the functionality and flexibility of our war-fighting systems. Software has proven to be a differentia - tor in system capability for a wide range of current systems such as the F-22, F-35 Lightning II, and the Aegis Combat System. Software to modify and integrate existing capabilities was a key enabler in the February 2008 successful shoot-down of an errant U.S. satellite as it tumbled back to Earth. This critical role of software in defense is also noted in the more recent DSB Task Force report on foreign software, which states, “The DoD now relies upon networked, highly-interconnected systems for many mission-critical capabilities, and this reliance is projected to increase. The software in these systems is the key ingredient that provides much of the increased capability delivered to the warfighter, just as it represents the key factor in increased productivity and new capabilities for industry today. For the DoD, this advanced technology is a force multiplier.”16,17 A high level of software capability is also important in producing defense systems. For example, very-large-scale, highly networked, and crypto-secured software systems were needed for the robotic design used to construct the production line for F-35 manufacturing. Given the importance of software to the DoD, a vital question is how the department can ensure that it will be able to meet its software needs now and into the future. The subsequent chapters of this report explore significant facets of this issue. Software provides the means to manifest the modeling and simulation capability now essential in the design and testing of advanced military platforms and weapons in all branches of the DoD. These design-focused software capabilities can save millions of dollars—all before the first piece of metal is bent. But perhaps even more importantly, in these early stages this software capability enables the cus - tomer to focus on driving out risks related to the definition of weapons and systems functionality and 14 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software , Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, p. 77. Available at http:// stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA47366 1. Last accessed August 10, 2010. The FCS program was cancelled in 2009, but the experience of that program nonetheless provides valuable insight. 15 NRC, 2002, Measuring and Sustaining the New Economy: Report of a Workshop , Washington, DC: National Academies Press, p. 52. Available online at http://www.nap.edu/openbook.php?record_id=10282&page=52. Last accessed August 10, 2010. 16 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Wash- ington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, p. 12. Available at http://stinet. dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA47366 1. Last accessed August 20, 2010. 17 DSB, April 2009, Creating a DoD Strategic Acquisition Platform, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA499566&Location =U2&doc=GetTRDoc.pdf. Last accessed August 20, 2010.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION architecture. The savings are due to the elimination of the need to build and test numerous prototype designs—this is now done through software. For many systems, this modeling and simulation software implements high-fidelity, massively parallel computational fluid dynamics (CFD) simulations. These simulations enable experimentation with signatures of different missile and aircraft designs without having to resort to expensive physical tests. Wind tunnel testing alone, for example, can cost millions of dollars per week, with months of testing required to settle design issues. Other examples of software-manifest functionality used pervasively in defense systems include onboard prognostic health systems and automated logistic support systems, both of which emerged from Defense Advanced Research Projects Agency (DARPA) research funding in the 1980s. These soft - ware-enabled systems are quietly saving millions of dollars. Software in Organizations In addition to deepening the extent of reliance on software in systems components and in the tooling associated with their development, there is a significant broadening of the role of software in systems and organizations. These changes reflect both a growing centrality in the role of software and also a growing portfolio of associated risks. This combination raises the significance of software-related decisions in systems development—software has become the medium of choice for innovative func - tionalities. These functionalities include not only component capabilities, but also functionalities that are enabled through the use of software to implement interconnections across a family of constituent systems. However, largely as a consequence of both of these roles—innovation and interconnection— software has emerged as the locus for a range of engineering challenges related to reliability, security, and development predictability. The interconnection aspect of this shift has three principal elements. First, there is a shift in emphasis from the development of functionally focused systems to the development of systems that interconnect and integrate capabilities within and across enterprises. The “net-centric” and “ultra-scale” concepts are reflective of this shift. This has great strategic benefit, in that it leverages the value of dispersed assets and enables agile responses at a broad range of echelons in war-fighting situations when multiple sys - tems elements are involved. This interconnection has associated risks, of course, primarily related to the magnitude of failures experienced as a consequence of internal errors, vulnerabilities exploited, etc. In civilian systems, for example, there are widely reported examples of cascading failures of interconnected systems in telecommunications, utilities, and supply chain systems. Second, largely as a consequence, IT staffs are generally less involved in mediating between a system and its users. Much larger numbers of DoD personnel interact directly with systems, and indeed many systems can be (and need to be) accessed through public communication infrastructure. This is more efficient because it removes the delays and inaccuracies caused by intermediation. But it also means that many more individuals—usually inadvertently, but not always—can take actions with wide-reaching consequences, both positive and negative. A third element of the interconnection aspect is that modern systems can support immediate elec - tronic enactment of decisions. This enables agility and fast response in decisions and actions—getting inside the command loop of an adversary, but it also means that failures and compromises can happen very quickly, inside a human decision loop. An example in the civilian context is the recent discussion over the duration in milliseconds of the stock trading look-ahead window. Fourth, interconnection introduces new information security challenges. Software Supply Chains The growth in the role of software, as described above, is enabled in part by a surprisingly recent phenomenon in IT, which is the diversification and enrichment of the supply-chain structure for IT systems. This enrichment is more than just systems outsourcing as experienced in the past half century.

OCR for page 17
 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Supply chains for software systems are both broader and deeper today, and they include commercial as well as defense players and involve technically rich and complex architectures, with frameworks, libraries, services, and other roles contributed by multiple players. The value of outsourcing, which was initially primarily cost reduction and access to expertise, now includes greater agility and ability to respond to changes in the operating environment. The supply-chain structure for modern defense software is evolving in a similar way, and is now significantly more complex and more international than it was even just a decade ago. Indeed, this com - bination of factors motivated the Defense Science Board study mentioned above to assess the impact of this internationalization on defense software systems, including their development and their assurance. 18 The complexity—and the internationalization—are due to a combination of factors, including certain technical developments in software technology, the economic forces and technological enablers of glo - balization, the geographic dispersion of the trained workforce, the minimal capital investment required (not including education and training) for the workforce, and increasing demand for precedented (rou - tinized) projects. The complexity is also enabled by the maturation and acceptance of a diverse set of commercial (COTS) ecosystems with their associated components and infrastructure. From a technological perspective, this richness in the supply chain is enabled by advances in both organizational collaboration technologies and software technology. The collaboration technologies build on Internet infrastructure to provide, for example, messaging, process support, team information servers (document sharing and configuration management), issue databases, servers for software builds, wikis, automated test and analysis tools, and the like. Finding 1-2: The growth in the role of software in systems is due to a combination of technological advances and a maturing of the supply chain structure associated with software systems develop - ment at all levels of scale. PRECEDENT AND INNOVATION IN SOFTWARE Precedented Software and Externalities Software development today relies heavily on established architecture and infrastructure component configurations, which the committee calls software ecosystems (see Boxes 1.2, 1.3, and 1.4 for elaboration). The success of the ecosystem model derives from the natural convergence of component functionalities and the associated architectural elements, chiefly protocols and software/service interfaces, through which these functionalities are delivered within applications and systems. These functionalities are called precedented, in the sense that new users of these functionalities benefit, in terms of design costs and risks, from the experience of existing and prior users. Once ecosystems are established, the development processes associated with them are often characterized primarily by selection of an ecosystem and then, within that ecosystem, tailoring through configuration of settings and the authoring of a relatively very small amount of custom software code. Thus, custom development in these areas of convergence gives way to product selection and procurement. Indeed, because engineering risks are relatively modest, a straight-line sequential process may often be appropriate for development management in these prec - edented portions of larger systems. The emergence of generally accepted ecosystem structures has in recent years become one of the enablers of the growth in richness in the supply-chain structure for software systems, which in the commercial world has promoted a diversity of suppliers, growth of a market for specialists in com - ponent-level innovation, and tools geared to development productivity for particular ecosystems. A 18 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Wash- ington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet. dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION countervailing trend in recent years has been the consolidation at the top end of the software vendor market through mergers and acquisitions. How does this ecosystem phenomenon influence the DoD? First, it should be clear that the DoD derives huge benefit from the ecosystem phenomenon, because so many of the business systems required by the DoD have requirements that are strongly analogous to commercial requirements. In these cases, only modest adaptation of commercial best practices, including choices and configuration of ecosystems, can yield major advantage to the DoD in the form of low costs, managed risk, and predictable outcomes. Additionally, the software elements of an ecosystem are complemented by a supplier ecosystem of expert integrators, consultants, add-in vendors, and the like. In other words, when the DoD’s requirements are similar to commercial requirements, the DoD benefits significantly by adopting commercial best prac - tices where possible. (See Boxes 1.2 and 1.3 for elaboration on the concept and role of ecosystems.) Not only are engineering risks reduced for precedented developments, but also there are benefits from the richness of the supply chain due to network effects—the positive externalities associated with systems adoption.19 When an ecosystem is successful, the commonalities of structures and interfaces enable larger numbers of organizations to participate efficiently in the development of large systems, providing software components, libraries, frameworks, plug-ins, custom elements, and so on. This makes it possible for more suppliers to participate and also reinforces the status of accepted frameworks, broad- ening the benefits for framework adopters and affording them the opportunity to make good choices at the component level while working within the safe conventions of an established ecosystem. Indeed, for many of these categories of requirements, it is nearly intractable, from the standpoint of cost and risk, to develop “separate but equal” approaches. Not only are these expensive, time- consuming, and risky, but also the DoD would then need to bear the entire cost of advancing the idio - syncratic technology, whereas ecosystem participants would benefit from actions by other participants to incrementally advance the performance and capability of the ecosystem, its infrastructure, and its constituent components. (Of course there are also negative externalities associated with widely adopted ecosystems, such as their attractiveness to developers of security exploits and the consequent ease of access by adversaries to offensive capability.) Innovative Requirements and the Activity of Innovative Software Development The situation becomes more complex and challenging when the DoD requires functionality more specifically focused on the defense mission—weapon systems, command and control systems, intel - ligence analysis systems, and other systems more directly supportive of war-fighting and intelligence. They include high-performance embedded systems, large-scale systems with unprecedented architec - ture, and highly interconnected systems. They often require high degrees of software assurance. The functionalities of these systems are more specialized and also, due to the presence of ambitious adver- saries, constantly evolving in response to changing threats. They require the management of complex and evolving requirements. Many of these systems are less precedented in the sense that innovation is required in system architecture, design, infrastructure, linkages with hardware sensors and effectors, and other respects. Technological enablers for the development of these more innovative systems—either with respect to innovative functionality or innovative engineering or both—is the principal focus of this report. This does not exclude considerations regarding precedented systems, however, because larger systems almost always involve a mix of innovative and precedented functionalities and components. Indeed, there are few modern defense systems of scale that do not build on technologies extensively drawn from related defense systems and from the various mainstream software ecosystems. This means that development and acquisition practices must account for this mix of the innovative and the precedented. This mix 19 Carl Shapiro and Hal R. Varian, 1998, “Information Rules: A Strategic Guide to the Network Economy,” Journal of Technology Transfer 25(2):250-253.

OCR for page 17
4 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Box 1.2 The Concept of Software Ecosystems In web applications, there are conventional configurations of server operating systems, relational data- bases, web servers, application server frameworks, business rules, and other elements that are combined to create e-commerce servers. These servers rely, in turn, on the “rich client” ecosystem of a modern client-side web browser, which includes not just HTML and basic HTTP, but also technologies such as JavaScript, XML, DOM access, and asynchronous HTTP. Analogous configurations, with very different sets of interfaces and components, are used to support mobile applications (Apple’s iPhone ecosystem and Google’s Android ecosystem are two recent examples), business intelligence (OLAP, etc.), enterprise resource planning, and other common functionalities. There are competing ecosystems in the commercial world—for example, web application servers can be devel- oped using Java-based ecosystems such as the platform-independent Java EE (formerly known as J2EE) or using the Windows-based .NET Framework, which supports multiple programming languages sharing com- mon runtime services related to memory management, security, etc. In web applications, there are also open source ecosystems, one of which is the so-called LAMP stack, which comprises the Linux operating system, the widely adopted Apache web server, the MySQL relational database, and scripting in a language starting with the letter “P,” most usually PHP, Python, or Perl. The open-source character of this ecosystem means effectively that it operates as a kind of quasi-consortium linking the various stakeholders that participate in the ongoing development of the overall architecture, the details of the interfaces, and the code base. Regardless of particulars, in most of these ecosystems, choice of programming language is often driven by the choice of ecosystem. The committee defines a software ecosystem as a conventional structure consisting of a family of in- frastructural elements that are intended to be combined in a patterned way. Ecosystems include software- architectural structure, but they can also include configurations of hardware and services platforms. Ecosys- tems generally provide a reuse of major elements and infrastructure, which can entail strong structural and semantic commitments.1 Ecosystems often also include documents, tools, practices, and even organizations to accompany these elements. The principal benefits include potentially significant reductions in cost, mitigation of engineering risk, and up-front agreement on representations and meanings for data that are shared within a system or across systems. It is also significant to note that, if we broadly construct the idea of a software ecosystem, then the Internet family of protocols would also be an example. The ecosystem comprising these protocols and its evolution have been much studied—one of the results of this is the “hourglass” model. This model il- lustrates how there can be a diversity of means for provisioning the service associated with a particular interface or protocol, such as TCP, and a separate diversity of client applications that build on the service. For example, “TCP service” is generally provided as a “layer” above IP, which in turn can be provisioned over fiber, wireless, copper, and many other means. TCP, in turn, underlies the web protocol HTTP as well as file transfer FTP and many other higher-level services.2 1 Not all ecosystems involve direct reuse of components. A family of protocols, for example, defines a “means of exchange” among system components. Other examples include instruction set architectures (with multiple vendors providing chips) and agreed-upon XML or other data exchange representations for shared information. 2 See, e.g., NRC, 2001, The Internet’s Coming of Age, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=9823. Last accessed August 20, 2010.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION Another example is the relatively small set of widely adopted ecosystem architectures for embed- ded and real-time control applications (such as the QNX, RTLnix, VxWorks, and Windows CE real-time operating systems), the automotive industry’s AUTOSAR (AUTomotive Open System Architecture), and the SCADA protocols and interfaces used in the electrical grid. Many government users of embedded and real-time capability have taken actions to work with researchers and vendors to develop more capable ecosystems that build on modern concepts and abstractions related to processors, languages, and tools.3,4 Ecosystems are also being adopted or are emerging in areas ranging from robotic systems to data-intensive supercomputing.5 The ecosystem phenomenon is now pervasively apparent in commercial industry, and it is actively pro- moted by leading vendors, in part due to the stability of market structure derived from the network effects. Ecosystem or framework “owners” (from the examples above: Apple, Microsoft, Google, Oracle, several open-source foundations, and many others) control the trajectory of the overall market for components and services associated with that ecosystem, but many firms participate in that “internal” market. Although the risks and costs associated with introducing a new ecosystem or framework may be very high, the risks and costs for niche providers within an established framework can be low. Additionally, once a community of suppliers is engaged within an ecosystem, the overall ecosystem can continue to evolve in response to the broad market trajectory and also to new technology developments. Thus, new languages can be added to .NET (e.g., functional programming with F#), new libraries and language features can be added within Java EE (closures to Java), and so on. This is one of the enablers and sustainers of the global supply chain. The committee notes, in addition, that the structure of ecosystems may become more or less exposed to developers and users, and indeed entire ecosystems may split or merge. For example, a vendor could choose to expose a previously inaccessible internal interface to allow greater customization by customers and integrators. In the case of enterprise resource planning (ERP) systems, for example, vendors expose interfaces that allow “add-in” developers to provide a diverse set of functionalities tailored to particular market segments. This enables a broader diversity of client requirements to be met with less risk to both clients and the framework vendor. The ecosystem structure thus evolves according to changing opportuni- ties and risks for the various stakeholders. Examples of the considerations include network effects (benefits of broader adoption of particular “hourglass necks”), vertical integration (reduction in risks associated with integrating separately developed components), and lock-in (greater friction in interoperation and choices available for component functionalities). 3 See, for example, the workshop convened by the NITRD High Confidence Software and Systems Coordinating Group, “National Workshop on High-Confidence Automotive Cyber-Physical Systems,” April 3-4, 2008, Troy, Michigan. Available online at http://varma.ece.cmu.edu/Auto-CPS/. Last accessed August 20, 2010. 4 See also, Jeanette Wing, 2008, “Cyber-Physical Systems Research Charge,” presented at the Cyber-Physical Systems Summit, April 24, 2008, St. Louis, MO. Available online at http://www.cra.org/ccc/docs/cps-summit.pdf. Last accessed August 20, 2010. 5 NRC, 2009, Assessing the Impacts of Changes in the Information Technology R&D Ecosystem: Retaining Leadership in an Increasingly Global Environment, Washington, DC: National Academies Press. Available online at http://www.nap.edu/ catalog.php?record_id=12174. Last accessed August 20, 2010.

OCR for page 17
 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Box 1.3 The Role of the Software Ecosystems for the DoD There are a number of key advances in software technology that enable the emergence of ecosystems and other architecture-based enrichments in supply-chain structure, with consequent benefits to cost and agility and also the consequent risks of more diverse sourcing. These advances range from the design of software architectures, frameworks, and components to the technical properties of modern programming languages, including first-class encapsulation, advanced typing, interface and package structures, and framework architectures. These and other software technology improvements have also enabled the development and successful implementation of a number of conventional structures of software components used for commercial and government applications. These conventional structures, here called ecosystems, are families of infrastruc- tural elements that are combined in a patterned way to construct precedented applications such as many business and back-office systems, e-commerce and web applications, and mobile applications. Ecosystems include such software-architectural structure as stacks, hardware and software platforms, and software frameworks. They often also include the documents, tools, and practices that accompany these elements (e.g., Eclipse and ASP.NET). The ecosystems also include a diverse array of software and organizational ser- vices that support conventional architectural structures. The ability to successfully define robust and broadly adoptable standards (de facto or ratified) and to stimulate a critical mass of compliant implementations is a significant enabler of ecosystem success. The economics of network externalities1 play a significant role in reinforcing successful ecosystems (often regardless of technical merit) and in guiding the initial stages of promotion of emerging new ecosystems. (This issue is elaborated below.) A particular challenge for the DoD in defining its own ecosystems is to keep up with rapidly evolv- ing technology and to select those interfaces and component capabilities for which certified components and/or trusted suppliers exist. Ecosystems in different categories may share elements and typically support diverse ranges of applications. Over the years, the DoD has attempted to codify its preferences regarding the components and interfaces in these conventional aggregates with efforts such as the Defense Informa- tion Infrastructure Common Operating Environment (DIICOE), the Joint Technical Architecture (JTA), and the System of Systems Common Operating Environment (SOSCOE). Generally speaking, these are broad suites of conventionalized standards and infrastructural elements that have been judged acceptable for adoption in systems generally. These suites may identify particular acceptable ecosystems, but they are not themselves ecosystems by the committee’s definition (or architectures, in the sense of Chapter 3). 1 See, e.g., Carl Shapiro and Hal R. Varian, 1998, Information Rules: A Strategic Guide to the Network Economy, Boston: Harvard Business Press. often creates confusion regarding development processes and other systems engineering choices. “One size fits all” models—even for incremental iterative developments—can be dangerous. 20 Because of the extent to which modern software builds on existing ecosystem and infrastructural elements, modern software development processes entail activities that go well beyond the design and authoring of new code. Modern software development is much more about identifying and defining appropriate and scalable architectures; selecting, using, and adapting infrastructure such as frameworks, components, and libraries; and deploying best practices and tools for collaboration, process support, and 20 With respect to process (Chapter 2), hybrid approaches may best be employed, for example, combining straight-line pro - cesses for precedented elements with iteration and prototyping for innovative elements. With respect to architecture (Chapter 3), modular designs that “concentrate” particular innovative or rapidly evolving functionalities in individual components can greatly reduce overall project risk.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION Modern e-commerce frameworks, for example, are ubiquitous in Internet-based commerce, but they have also been adopted as internal coordination frameworks for large-scale DoD systems. The ecosystems contribute enormous value to software and systems projects that rely on them by al- lowing developers to leverage an enormous investment for which costs are spread across a wide base of users rather than taking on the full effort and expense of developing an entire software system from the ground up. When ecosystems are widely adopted, architectural risks are drastically reduced because prin- cipal architectural commitments are embodied in the successful ecosystem, and, additionally, the extent of value to the DoD can grow over time due to the positive network externalities. Thus, the DoD derives benefits from ecosystem adoption, but it must be attentive both to selection criteria and to its ability to participate in the overall evolution and development of the ecosystems within which it participates. One particular issue is the sustainability of ecosystems that are adopted into systems. In some instances, choices may depend more on appraisals of sustainability and network effects (to use economic terms) than on par- ticular technical characteristics. These sustainability factors may influence engineering risk (see Box 2.2). The DoD may derive great benefits from investing in the evolution of the ecosystems in which it participates, which enhances both technical fit and sustainability. As noted above, these ecosystems are enabled by a wide range of computer science and software engineering advances. The modern software application frameworks essential to Web-based systems, e- commerce, and graphical user interfaces of all kinds are enabled by the same advances in programming language design that led to languages such as Java, C#, and Ada95. Many of these “component” advances and, perhaps more importantly, the principal abstractions and architectural concepts underlying established ecosystems, are legacies of past DoD investment in computing technology R&D, primarily in the form of 6.1 and 6.2 extramural research funding. Because of the rapid pace of infrastructural development, the competitive business environment, and the need to accommodate new functionality, the ecosystems are generally in a state of continuous evolu- tion, carefully managed to stage out new increments of value while minimizing costs and risks for existing adopters—and thus to retain the benefits of the positive externalities. The evolutionary trajectory for some ecosystems is entirely driven by particular vendors, as is the case with Microsoft and .NET or Oracle and its E-Business Suite. Others are driven by complex community processes, as in the case of the open-source LAMP stack (see above), the Internet protocols themselves, and also some commercial ecosystems, as is the case with many of the ecosystems surrounding Java—following the Java Community Process.2 The evolution may include specific component capabilities, architectural and interfaces structures, and associated tooling (as in the case of Visual Studio and .NET).    For more information, see the Java Community Process at http://jcp.org/en/home/index. Last accessed August 20, 2010.  2 validation. Indeed, it is generally recognized that, for large systems in industry and aerospace, the most significant costs are generally associated with gathering functional and non-functional requirements, developing architecture and design, managing process, and achieving assurance—and somewhat less with the writing and evolution of code.21 (Issues related to innovative systems for defense are addressed in two chapters of this report—Chapter 2 focuses on requirements, and Chapter 3 focuses on architecture and agility at scale.) 21 See, for example, economics studies by Barry Boehm, studies of the IBM Rational Unified Process, and other work that shows that the proportion of coding in the overall process is diminishing. See, RTI, 2002, The Economic Impacts of Inadequate Infrastructure for Software Testing, Planning Report 02-3, RTI Project Number 7007.011. Available online at http://www.nist.gov/director/ planning/upload/report02-3.pdf. Last accessed August 20, 2010.

OCR for page 17
4 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Box 1-5 Continued More details regarding the state of research investment are shown in Figures 1.5.1 and 1.5.2. The former shows the investment by year and by agency for HCSS (left side) and SDP (right side). It is evident from Figure 1.5.1, for example, that NASA, DARPA, and the Department of Energy (DOE) have stepped almost completely away from engaging in research related to SDP. Figure 1.5.2 shows the overall extent of the NITRD investment in all eight categories of research related to networking and IT. Two trends are immediately apparent from these charts: there was a surge of SDP and HCSS in- vestment in the mid-2000s by several agencies—DARPA, DOE/National Nuclear Security Administration (NNSA), and NASA—followed by a precipitous drop in their investments and in the total investment. The result is that NSF is now the dominant source of investment in both categories. Figure 1.5.3 shows the percentage of total NITRD investment in SDP and HCSS. It illustrates that while the total NITRD investment more than doubled over the past decade, the percentage of investment in SDP and HCSS fell off sharply. 30.0% 24.6% 24.3% 25.0% 21.4% 20.1% Percentage of Total 20.0% 19.1% 15.0% 13.4% 9.3% 10.0% 8.8% 7.9% 6.5% 5.0% 0.0% 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Years Figure 1.5.3 Percentage of total NITRD investment in either SDP or HCSS. Figure 1.5.3 security systems of all kinds make this kind of shift much more consequential for defense software producibility and for U.S. ability to advance overall defense system capability. In exploring the role of the DoD in advancing software producibility, which is the topic of Chapters 2, 3, and 4, the committee considers the interplay of four factors: 1. Productiity. The DoD benefits from efficiencies gained in the development of innovative function- ality through advances in mainstream software producibility. For mission systems, there are particular challenges relating to requirements and validation, architecture and modeling, process and measure - ment, and tools and language systems. 2. Innoation. The DoD relies on technologically enabled advances in software producibility to enable the more effective creation of unprecedented systems and the interconnecting of existing and new systems to deliver advanced functionality with acceptable cost and risks.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION 3. Assurance. The DoD faces new challenges to addressing the risks associated with diverse and international supply chains, including the development of practices and technologies for software assurance. 4. Ecosystems and infrastructure. The DoD unavoidably relies on mainstream software ecosystems in defense systems and therefore has a stake in the processes by which those ecosystems evolve and are led. THE ROLE OF THE DOD IN ADDRESSING ITS SOFTWARE NEEDS Given the importance of software to the DoD, and to its mission systems in particular, and given also the ongoing rapid advances in software capability worldwide, it is vital to ensure that the depart - ment can not only meet its software needs now, but also sustain its software leadership well into the future. A key question addressed by the committee is to what extent the DoD, without providing its own explicit R&D stimulus, can rely on industry—specifically the domestic defense industrial base and sup - porting vendors—to produce software innovations in areas of defense significance at a rate fast enough to allow the DoD to fully meet software requirements and remain ahead of potential adversaries. This leadership must be with respect to both the capability of systems and effective defense against attacks on those systems. As noted above, the DoD has particular requirements that must be dealt with on systems that are both very large scale and have life-critical mission requirements. Although these areas may overlap with civil and commercial needs, very often the DoD requirements are more sophisticated and cutting-edge than those in the rest of the marketplace. Also, DoD adversaries may choose to “attack” software in the supply chain during development phases of a project—security is much more than about attacks staged over networks during system operations. Additionally, major DoD development projects are structured in a way that often keeps development teams at arm’s length from the key operational mis - sion stakeholders and from overall project management. For these reasons, technological advancement would significantly benefit the DoD’s ability to produce the software it needs. The areas identified in this report, and particularly in Chapter 5, are areas where the committee sees the DoD as having leading demand. The committee notes that the issue is not areas where the DoD has “unique” requirements, but rather the much broader category of areas where it has leading demand with respect to particular kinds of requirements. One obvious example is software assurance, where DoD and national security needs may go well beyond even what is being developed for commercial financial services or health care devices. Another example is the risk-managed development of unprecedented architectural design of the kind required to create high-interconnectivity systems such as FCS, net-centric systems, and many other major defense platforms that have few commercial precedents or analogs. The committee notes that even where industry is aggressively innovative, it may not have sufficient incentives to produce the technology and supporting tools necessary to generalize application-specific software innovations. Additionally, the technologies may manifest innovative concepts, but in a way that cannot be readily adopted by the DoD, for example due to the safety, reliability, and assurance considerations particular to defense applications that, to be addressed, require further technological innovation. It would thus be overly optimistic to conclude that DoD needs such as these will somehow be adequately addressed through a combination of demand-pull from the DoD and technology-push from the defense sector (i.e., firms that primarily supply the DoD) or the broader commercial IT sector. In many other industries and infrastructures, this may be a legitimate conclusion, and in these areas the best policy may be for the DoD to follow the market. However, this is not generally true for software technology, particularly as needed for defense mission systems, where the DoD has leading demand in multiple critical areas, as detailed later in this report.

OCR for page 17
 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Finding 1-4: The DoD’s needs will not be sufficiently met through a combination of demand-pull from the military and technology-push from the defense or commercial IT sectors. The DoD cannot rely on industry alone to address the long-term software challenges particular to defense. The above finding is based on consideration of both the history of software innovation and the set of future needs identified in this study. Commercial R&D Investment Defense contractors do invest extensively in software research, but generally speaking it is focused on manifesting specific capabilities in supporting the competitiveness of bids through differentiated skills and products. Commercial IT firms also invest in software research and form an important part of the defense IT supply chain, but may not necessarily carry out research aimed at meeting the DoD’s needs. Both defense and commercial IT firms lack strong incentives to invest directly in broad-impact areas such as these, particularly when many of the advantages derived from the investment are non- appropriable, in that the associated intellectual property cannot be readily controlled and as a conse - quence the benefits may readily diffuse not only into the supply chain for the contractor but also to competitors (more on this point is given below).27 Indeed, it is important to note that there are certain technological improvements that may in fact not necessarily be good for business, even when the DoD derives capability and acquisition advantage. Additionally, and perhaps most importantly, there is the issue of horizon. Prudent business deci - sions are generally informed by return-on-investment calculations, which depend on (1) appropriability, (2) timeliness, (3) investment risk, and (4) measurability/observability of return. Many improvements in practices, for example, come only after sustained commitments and much technical exploration (after which benefits may rapidly diffuse across the industry). Additionally, benefits, even when judged sig - nificant by technical leaders, may be difficult to quantify, due to the measurement challenges that persist in software and the software-related aspects of systems engineering (see Chapter 2). In other words, in investing in software producibility, all four of the elements above are problematic. And therefore, in a competitive market, individual companies generally have few market incentives for such invest - ments. These factors tend to drive internal R&D investments in contractors toward a combination of addressing business needs expected 1 to 2 years in the future and avoiding technological surprise from competitors. Government research and development in software producibility, from a purely structural perspective, can be less focused on appropriability and investment risk. Additionally, government can invest directly in improving measurement capability, when that is a source of risk (see the discussion of measurement in Chapter 5). The Challenge of ROI and Appropriability It is an economic reality that it is difficult for a firm to make a compelling case of return on invest - ment (ROI) to undertake innovations when those innovations have a non-appropriable character, which is to say that the intellectual property associated with the innovations diffuses broadly into the engi - neering discipline and the economy. This is a familiar issue to those involved in defining industry-wide best practices, standards, and other commonalities. Many of the most important and highly leveraged, government-originated innovations (undertaken by both academia and industry) are in the economic “commons.” This creates challenges in measuring value created, because the value has broad and dif - 27 See, e.g., NRC, 2002, Information Technology, Research, Innoation, and E-Goernment, Washington, DC: National Academies Press, p. 101. Available online at http://www.nap.edu/catalog.php?record_id=10355. Last accessed August 20, 2010.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION fuse benefits.28 A recent NRC report on software and the economy29 notes that “the economic rationale for government investment is based on the non-appropriability of many significant IT innovations, including the most widely used idiomatic data structures and algorithms, as well as design and archi - tectural patterns. Moreover, the IT industry relies on a number of technical and process commonalities or standards such as the suite of Internet protocols, programming languages, core design patterns, and architectural styles.” These innovations effectively “raise everyone’s boat” in the same way as do govern- ment investments in bioscience, health care, and other strategically important scientific disciplines. This includes many of the most highly leveraged areas of software research such as improvements in abstrac - tion mechanisms, design notations, programming languages, software analysis and model checking, basic algorithms, design patterns and architecture concepts, and other core techniques. This is not to say that industry does not invest in non-appropriable results—it does so extensively in the area of standards and also through investment in university projects and other activities. But (for the reasons cited here) the incentives to sustain this investment are lower than for proprietary development efforts. The committee notes, in addition, that when research results are not appropriable, researchers are less likely to be able to secure patents. (Thus, it can be safely hypothesized that direct revenues from licensing university-owned patents are likely to be significant underestimates of the value created by federally funded research, especially in the case of software-related university inventions.) To complicate matters further, the manner in which software is protected as intellectual property is often distinct from what is done in other fields such as biomedicine. A software system is often a combination of differently protected elements, organized following architectural elements of several ecosystems along with custom elements. There is often considerable intellectual property embedded in the architectural designs themselves, even in the absence of components that populate those designs. All this combines to make the non-appropriable, yet valuable, aspects of the work hard to identify. Finally, the committee notes that the software industry is shifting from the development of software products to an increased focus on integration, custom development, and other services. 30 More than half the revenues of software product companies are now coming from services rather than from product sales, but product sales tend to be the most scalable and profitable part of the business. 31 Additionally, the software-product sector is consolidating and thus shrinking in numbers, decreasing from more than 400 to less than 150 publicly listed software product companies on U.S. stock exchanges in the past 8 years.32 Another consequence of the shifts toward services and consolidation may be a reduction in 28 Indeed, fear of value diffusing to competitors that can assimilate it more efficiently into their practices can create a negative incentive to invest, although it can be dangerous to make this judgment when evidence is lacking, because innovation leadership may be less easy to recover once lost. 29 NRC, 2006, Measuring and Sustaining the New Economy, Software, Growth, and the Future of the U.S Economy: Report of a Sym - posium, Washington, DC: National Academies Press. Available online at http://www.nap.edu/openbook.php?record_id=11587. Last accessed August 20, 2010. 30 For quite a few years, about two-thirds of global revenues in the software industry have actually been from services (such as custom software development, maintenance, IT consulting, and technical support), and only one-third of revenues have come from the product companies. See Michael A. Cusumano, 2004, The Business of Software, New York: Free Press, p. 46, footnote 19, citing Standard & Poor’s annual data. An issue in these analyses is how the word “services” is defined—it is used both for custom development/integration by teams of people and also for “software-as-a-service” (SAAS) and cloud-based delivery of software value. Additionally, the distinction between “product” and “service” in software is becoming increasingly muddy as licensed product software delivered to customers is complemented by off-site SAAS and cloud-based software generally. For many end users, for example, there may be relatively little distinction in the experience, say, of Microsoft Office tools on a desktop computer (a “product”) and Microsoft Office Live or Google documents tools on a browser (a “service”). A similar statement could be made at the enterprise level, for example, regarding customer relationship management (CRM) software. This shift is the result of evolving business models enabled by technology and infrastructure developments. 31 Thus, only about one-sixth of global software industry revenues (half of the one-third of revenues from products) are from product sales. See Michael A. Cusumano, 2008, “The Changing Software Business; Moving from Products to Services,” IEEE Computer 40(1):20-27. 32 See p. 22 in Michael A. Cusumano, 2008, “The Changing Software Business; Moving from Products to Services,” IEEE Com- puter 40(1):20-27. This may also reflect post-bubble consolidation in both technology- and media-focused companies.

OCR for page 17
 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE support for new software research. De facto consortia, often but not always manifest as open-source projects, are often the centers for industry-wide innovation. 33 Changes in the DoD Innovation Environment The question regarding the extent to which the DoD can meet its ongoing software needs through innovation-push from the commercial sector has to be answered in the context of three additional fac - tors, which follow three significant shifts in the environment of technology innovation. The first is the growing globalization of the software industry, as noted earlier, with rapid gains in capability in India and China (and elsewhere in Asia) and Russia as well as steady gains in Europe. 34,35 This shift creates competitive pressures with respect to the rapidly increasing proportion of defense mission capability embodied in software. It also amplifies the challenge of mission assurance, given the increasing extent to which DoD software will likely be developed in foreign countries. This topic was taken up by the committee, and Chapter 4 addresses practice and technology issues related to software assurance, including both preventive measures in the engineering process and evaluative measures appropriate for development and for test and evaluation. Other dimensions of cybersecurity are conse - quential but not within the scope of this report—these were discussed at length in the 2007 DSB Task Force report Mission Impact of Foreign Influence on DoD Software.36 The second shift has been the reduction over the past decade of direct DoD investment in advancing software capability within the defense industrial base and its supply chain (see Box 1.5). Although this shift may be under reconsideration, it nonetheless raises a key question that was considered extensively by this committee, which is whether industry, without explicit R&D stimulus from the DoD, will produce innovations in areas of interest to the DoD of the kind and extent that are needed to meet the ongoing rapid growth in DoD software requirements. The third shift is a consequence of the second, which is the reduction in PhD output due to the drop in R&D investment in software producibility. Historically, the DoD has had a significant leadership role in creating and sustaining the innovation advantage of the United States in IT and in fostering new generations of innovators and technical leaders in computer science and IT. This role has been evident from the earliest days of computing during World War II. That this DoD investment has been a sig - nificant driver of U.S. capability and innovation in information technologies is documented in several national studies.37,38,39 This has had the salutary benefit of enabling the United States to develop and retain innovation and ecosystems leadership. In areas related to software producibility, including high- 33 Examples in the realm of open source include Apache, Linux, Eclipse, and other widely adopted open source. In addition, formal consortia are often created to address industry-wide issues, as in the case of W3C (HTTP, HTML, xML, CSS, and other web-related standards) and TCG (TPM, trusted storage, and other trust-related standards for hardware). Finally, expert groups are often convened by standards organizations to address common issues, as in the case of JPEG (ISO and ITU) and MPEG (ISO). 34 See Michael A. Cusumano, 2006, “Where Does Russia Fit into the Global Software Industry?” Communications of the ACM 49(2):31-34. See also, Michael A. Cusumano, 2006, “Envisioning the Future of India’s Software Services Business,” Communications of the ACM 49(10):15-17. 35 In China, there are private and state-connected companies under government sponsorship to develop ecosystems and infra - structure software (China versions of CDMA/GSM, embedded operating systems, and search engines, for example) to reduce dependence on firms such as Qualcomm, Nokia, Microsoft, Google, etc. 36 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Wash- ington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet. dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010. 37 NRC, 1997, Ada and Beyond: Software Policies for the Department of Defense, Washingon, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=5463. Last accessed August 20, 2010. 38 NRC, 2000, Making IT Better: Expanding Information Technology Research to Meet Society’s Needs , Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=9829. Last accessed August 20, 2010. 39 NRC, 1997, The Eolution of Untethered Communications, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=5968. Last accessed August 20, 2010.

OCR for page 17
 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION confidence systems, data from the NITRD coordination reports show that the DoD has reduced its basic research investment (see Box 1.5). In addition to these shifts, there may also be cases where industry has little economic incentive to acknowledge fundamental gaps in knowledge. This is not so unusual. One obvious example is ill- structured contracts, often from the government, that create perverse incentives—for example, where project difficulties (or inadequate tooling and practices) may accrue to a contractor’s economic benefit in the form of increased contract costs and profits, along with long-term revenue streams resulting from costly post-deployment repair and maintenance requirements. A second example is the consequence of poor measurement capability (as noted above), particularly relating to quality and security—“assurance metrics” in the terminology of the 2007 DSB Task Force report.40 When metrics and observables are lacking, it is difficult to construct a business case for improvement of the underlying phenomena of concern—quality and security in this case. A third example is inadequate best practices. When metrics are weak, we must rely disproportionately on folklore-derived best practices and processes and orga - nizational maturity to achieve product-related goals. Process compliance is relatively easier to achieve and certify than quality, but in software it is not always strongly correlated. As noted, fundamental improvements in best practices to enhance what can be achieved in terms of systems capability, pro - ductivity, quality, agility, and other characteristics sometimes fall into the category of non-appropriable innovations, discussed above, and thus may not readily be the subject of industry investment. These factors combine to lower industry economic incentives to address the producibility challenge. Recommendation 1-1: The DoD, through its Director of Research & Engineering (DDR&E), should regularly undertake an identification of areas of technological need related to software producibil - ity where the DoD has “leading demand” and where accelerated progress is needed to support the defense mission. THE NECESSITY OF INNOVATION IN SOFTWARE Is There a Need to Innovate? That global suppliers are moving up the value chain41 suggests the possibility that U.S. leadership may be eclipsed in many of the core technologies related to systems architecture, languages and tools, and software assurance, as well as with respect to key design elements of software ecosystems and infrastructure. This suggests a key question, particularly as we contemplate the commitment of resources to new R&D activity: Is there, in fact, strategic value in retaining U.S. leadership in software produc - ibility? The committee argues strongly in the affirmative, based on the unique role and technological characteristics of software. The DSB Task Force report on foreign software also asks this question, focused on the particular area of software assurance, and offers an affirmative response, noting the essential requirement that the United States maintain advanced capability for “test and evaluation” of IT products. In other words, reputation-based or trust-based credentialing of software (“provenance”) needs more and more to be augmented by direct, artifact-focused means to support acceptance evaluation. The DSB Task Force rec - ommends more effective direct evaluation by consuming organizations throughout the software supply chain, including better ways for producers to create software for which direct evidence of critical quality 40 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Wash- ington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet. dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010. 41 This is in the sense of moving from more routine activities that require minimal technological sophistication (such as first- tier call centers, system-level “black-box” testing focused on the user experience, and similar activities) to more value-enhancing activities (such as custom software design and development). For software, this shift is enabled primarily through increased sophistication in technology and business practices.

OCR for page 17
40 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE and functionality attributes can be provided. It concluded that test and evaluation should be supported by a broad range of software engineering technologies and interventions, not just those employed at the late test phase of development. (These issues are explored more extensively in Chapter 4.) Both the 2007 DSB Task Force report on foreign software42 and the DSB 2000 report on defense software43 also highlight the importance of commercial technology to the DoD, including the essential elements (operating systems, databases, application servers, and so on) of most of the predominant software ecosystems architectures. DoD’s historical investment in basic research has influenced these commercial technologies in ways that have facilitated DoD adoption (for example, early attention to scalability, process separation, interconnection, and survivability in operating systems and distributed systems infrastructure). Without continued research investment, that influence will diminish just when the off-the-shelf technologies are rising in importance to DoD systems. Additionally—and looking beyond software assurance into the other critical dimension of software producibility—without continued research investment, the DoD will lose effectiveness in its ability to undertake custom software engineering to rapidly achieve high levels of capability and to adapt with maximum agility to changes in the operating environment. A significant loss of U.S. leadership in either area could threaten the DoD’s ability to produce and assure the software it requires. Historically, the DoD investment has been an enabler, both directly and indirectly, of U.S. technologi- cal leadership in software innovation (Box 1.6). This has helped enable the United States to be a leader in the development of software ecosystems, which means that the DoD is able to rely on ecosystem archi - tectures and components with greater confidence. It has afforded a high level of capability in software producibility for the necessary custom software that the DoD must develop or acquire. Will Software Capability Reach a Plateau? Some have contended that software capability may be reaching a plateau, and as a consequence there is reduced need for leadership and innovation, because the technologies are inevitably commoditizing and the engineering focus is shifting to optimization and routinization. This suggestion is sometimes offered as an analogy with other technology disciplines, ranging from many specialized materials to display panels and memory chips. The committee views this contention as dangerously incorrect. The reality is that software is not at a plateau, despite the fact that suggestions of the possibility are made on a regular basis. Consider, for example, the ambitious aspirations of those developing Fortran (Box 1.7). A similar story can be told about the “Fourth Generation” database languages introduced a few decades later and, more recently, about languages for business rules. These are all major innova - tions with far-reaching impact, and in all cases they delivered considerable value. But they also inspired computer users to greater ambitions, and thus the limits of these innovations were reached. One of the most significant special characteristics of software, as noted at the outset of this report, is its unboundedness—the lack of natural physical limits on its scale and complexity. As the sophistica - tion of languages, models, tools, and practices increases, the ambitions of computer users continue to be realized. These characteristics also mean that while these developments move us forward, they do not actually get us closer to “being there” at some plateau of capability and emerging commodity status. New software-manifest capabilities are constantly emerging—for example, machine-learning technol - ogy is now used in applications ranging from data mining to robot design and quality-of-life enhance - ment for seniors. The profound fact is that software seems to be limitless. For software, “continuous improvement” in capability (as distinct from process) is less a matter of fine-tuning than an innovation 42 DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Wash- ington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet. dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010. 43 DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software , Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord &metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.

OCR for page 17
4 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION Box 1.6 Lessons About the Nature of Research in IT The following material is reprinted from National Research Council, 2003, Innovation in Information Technology, National Academies Press, Washington, DC, pp. 2-4. The Results of Research —America’s international leadership in IT—leadership that is vital to the nation—springs from a deep tradition of research. . . . —The unanticipated results of research are often as important as the anticipated results. . . . —The interaction of research ideas multiplies their impact—for example, concurrent research programs targeted at integrated circuit design, computer graphics, networking, and workstation-based computing strongly reinforced and amplified one another. . . . Research as a Partnership —The success of the IT research enterprise reflects a complex partnership among government, industry, and universities. . . . —The federal government has had and will continue to have an essential role in sponsoring fundamental research in IT—largely university-based—because it does what industry does not and cannot do. . . . Industrial and governmental investments in research reflect different motivations, resulting in differences in style, focus, and time horizon. . . . —Companies have little incentive to invest significantly in activities whose benefits will spread quickly to their rivals. . . . Fundamental research often falls into this category. . . . the vast majority of corporate R&D addresses product and process development. . . . —Government funding for research has leveraged the effective decision making of visionary program managers and program office directors from the research community, empowering them to take risks in designing programs and selecting grantees. . . . Government sponsorship of research especially in universities also helps to develop the IT talent used by industry, universities, and other parts of the economy. . . . The Economic Payoff of Research —Past returns on federal investments in IT research have been extraordinary for both U.S. society and the U.S. economy. . . . The transformative effects of IT grow as innovations build on one another and as user know-how com- pounds. Priming that pump for tomorrow is today’s challenge. —When companies create products using the ideas and workforce that result from federally sponsored research, they repay the nation in jobs, tax revenues, productivity increases, and world leadership. . . . process that regularly delivers order-of-magnitude jumps in capability. The improvements are not just outcomes from the Moore’s Law curves and other exponentials in the underlying physical infrastructure of processors, memory, and networks. Those improvements are significant enablers, but our ability to produce increasingly complex software artifacts has largely been enabled by a steady pace of technologi- cal breakthroughs in software practices, technology, languages, models, and tooling. (The performance of software has been enabled by the development of new algorithms, by advances in hardware, and by simplification and rescoping of the computational problems being solved.) Similar observations apply to software ecosystems. These are a critical and essential development to provide conventionalized access to infrastructure and capability on which systems are built. They are, in fact, the success of software reuse.44 As technical progress is made, these structures evolve, with increasing levels of infrastructure capability, for example in operating systems, databases, application servers, frameworks of various kinds, data center services, and so on. In a sense, this is analogous to a 44 Butler W. Lampson, 1999, “Software Components: Only the Giants Survive,” st International Conference on Software Engineering (ICSE’), Keynote Address. Available online at http://research.microsoft.com/en-us/um/people/blampson/70- softwarecomponents/70-softwarecomponents.doc. Last accessed August 20, 2010.

OCR for page 17
4 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE Box 1.7 Fortran Was a Breakthough but Did Not Lead to a “Software Plateau” It is instructive to consider the publication more than 50 years ago of the 1958 landmark paper by John Backus describing the first Fortran compiler.1 The title made reference to “automatic program- ming,” and indeed this phrase was widely used at the time, including in the titles of technical confer- ences.2 The point of this phrase, with respect to Backus’s great accomplishment, is that there was a much more direct correspondence between his high-level programming notation—the earliest Fortran code—and pure mathematical thinking than had been the case with the early machine-level code. One can construe that it was imagined that Fortran enabled mathematicians to express their thoughts directly to computers, seemingly without the intervention of programmers. This was an extraordinary and historical breakthrough. But we know that, in the end, those mathematicians of 50 years ago soon evolved into programmers as a consequence of their growing ambitions for computing applications. Just a decade after the Backus paper, Fortran was used to support list-processing applications, typeset- ting applications, compilers for other languages, and other applications whose abstractions required some considerable programming sophistication (and representational gerrymandering) to be repre- sented effectively as Fortran data structures—arrays and numeric values. (See further discussion in footnote 26 in Chapter 5.) 1 John W. Backus, November 1958, “Automatic Programming: Properties and Performance of FORTRAN Sys- tems I and II,” in Proceedings of the Symposium on the Mechanisation of Thought Processes, Teddington, Middlesex, England: The National Physical Laboratory. Available online at http://archive.computerhistory.org/resources/text/ Fortran/102663114.05.01.acc.pdf. Last accessed August 10, 2010. 2 An early example is the 1954 “Symposium on Automatic Programming for Digital Computers.” See John Backus, 1978, “The History of Fortran I, II, and III,” History of Programming Languages, New York: ACM. process of commoditization, in that many of the key architectural interfaces effectively define market structures for competitive supply of capabilities. But, at the same time, providers and clients continue to innovate above and around the infrastruc - ture, creating new kinds of capability and differentiation. Additionally, there is very often continued innovation within the infrastructure to add capability, create differentiation, or make other enhance - ments. This is certainly evident in the case of relational databases, for which there is a conventionalized set of abstractions (the concepts associated with relational tables, indexes, etc.) and also some standards related to access to those abstractions (SQL and enhancements such as ODBC and JDBC to support queries across software interfaces). But it is also the case that the particular vendors may add special - ized features to these standard interfaces to support new capabilities in response to the market. In other words, although there is a seemingly inevitable commoditization of software component capabilities, there is also a seemingly indefinite deferment of reaching the goal of fully predictable decision outcomes regarding innovative software-manifest capabilities in systems. This is a key point about the intrinsic lack of limits of software (except those we choose to impose, such as architecture—see Chapter 3), and indeed this is a principal characterizing feature of software as an engineering building material. Finding 1-5: It is dangerous to conclude that we are reaching a plateau in capability and technology for software producibility. To avoid loss of leadership, the DoD will need to become more fully engaged in the innovative processes related to software producibility.

OCR for page 17
4 RECOGNIZE THE PIVOTAL ROLE OF DOD SOFWARE INNOVATION The Continued Maturing of the Software Discipline Creates New Challenges A consequence of this phenomenon of continuous capability improvement is the challenge of inte - grating innovative software development into mainstream systems engineering processes. Despite the pervasiveness of software and its pivotal role in systems and infrastructures, the engineering of software has not yet matured into a fully rigorous discipline. For innovative software projects, fully measurable and predictable process flows and outcomes—a hallmark of conventional process maturity—will neces - sarily remain elusive. This is partly due to the fact that each new technical advance in software not only creates opportu - nities but also presents new difficulties of measurement and risk assessment. Engineering repeatability is achievable only for the more precedented systems. Additionally, a characteristic of many innovative projects is that the scope of the intended impact may be defined while specific details regarding func - tional and quality attributes emerge only in the course of development. Fully elaborated requirements against which predictions can be made often do not (and in many cases should not) exist.45 In other words, “predictability” may have more to do with success in addressing a need and less to do with how that need is specifically addressed. Of course, this is true of nearly all other engineering disciplines. A key difference with software is that development of particular software functionalities, once routinized, is then quickly automated. The result is that expensive custom development gives way to much-lower-cost component procurement. In turn this often gives way to open-source availability of the same functionality. Moreover, unlike other areas of engineering, the intrinsic cost of replicating or deploying software artifacts is near zero. We can conclude that the effect is that a relatively much larger portion of the overall engineering effort in a software enterprise is devoted to creating specifically innovative functionalities. In other words, once development of specific software functionalities is routinized, the cost can vanish relatively quickly, which means more of the overall hands-on engineering activity is in the realm of the innovative and unprecedented. Additionally, the risks and difficulties of software are growing in severity and diversity, and we continue to experience failures of all kinds—related to reliability, security, flexibility, and other attributes. Software-related problems are responsible for life-threatening failures in health devices, failures of space missions, failures in military systems, cascading failures in infrastructure for telecommunications and power utilities, and so on. This may create a perception that there is an unavoidable trade-off between precedent and proj - ect risk, and that the only way to avoid major project completion risk is to compromise on ambitions regarding system capability. The committee’s analysis suggests that this is not necessarily the case, and in Chapter 2, the committee considers the means by which the engineering risks associated with innova - tive projects can be mitigated incrementally, thus potentially reducing the cost and project completion risk without overly compromising functionality. Conclusions Regarding Software Innovation The committee draws several conclusions from these observations regarding software capability improvement. First, mere presence in the market as a software user requires keeping pace with ongoing software innovation and improvements to practices. This is true even for individual software compo - nents—indeed, commercial software managers recognize that software starts to “die” (in the sense of becoming less valuable to users) the moment it stops evolving. It is also true for practices—continuous improvement in practices and processes is essential for survival. Desktop computers now are almost 45 See, for example, the description of the double helix design methodology in BG Harry Greene, USA, Larry Stotts, Ryan Paterson, and Janet Greenberg, January 2010, “Command Post of the Future: Successful Transition of a Science and Technology Initiative to a Program of Record,” Defense AR Journal, Defense Acquisition University. Available online at http://www.dau. mil/pubscats/PubsCats/AR%20Journal/arj53/Greene53.pdf. Last accessed August 20, 2010.

OCR for page 17
44 CRITICAL CODE: SOFTWARE PRODUCIBILITY FOR DEFENSE 1 million times more powerful than those of 1980. The software capabilities have similar jumps, although these jumps are less apparent and not easy to measure. Second, leadership in the market, as a producer or consumer, requires an active organizational role in defining the architecture of systems, and doing so as a first mover or fast follower. Software economics are focused on externalities—where the technical manifestation of the system structure is the software architecture and internal framework and component interfaces. This requires sustained technological leadership and clear thinking about the significance of architectural control. This is particularly signifi - cant in the definition and leadership of the design of the major ecosystems. Some of these are wholly controlled by commercial vendors, but others involve complex community processes. Third, software technical challenges are broadening. These include, for example, software assurance, ultra-scale architecture, concurrency (multi-core and distributed), framework design, programming language improvements for assurance and scale, concepts for “big data” systems, and so on. These challenges are addressed in Chapters 2, 3, and 4 of this report. Fourth, risk management models need to be continually adjusted to accommodate the new reali - ties of software and of IT-enabled business practices, as noted above. This is the subject primarily of Chapters 2 and 4. Finally, the role of software leadership in the global economy is growing, and this is increasingly recognized, with the result that global competition is becoming more intense at every level of capabil - ity. Overseas competition is greatly facilitated by the low barrier to entry—costly physical facilities are not needed in the software economy, but education and technical currency are fundamental and ongo - ing challenges. This is significant for the DoD, which has counted on U.S. industry, including defense contractors and their supply-chain participants, to sustain technological leadership in software as a key driver of capability leadership in systems. Such leadership must be maintained through constant investment in innovation and in people. At the highest level of technical sophistication, this requires investment in university research to produce a sufficient pipeline of technical leaders. 46 46 U.S. PhD students in computer science and IT-related areas are almost universally supported with tuition and stipends covered by sponsored research. Universities rarely have funds to provide direct fellowships to PhD students in these areas, and few students have the resources or capacity to self-fund or to take on (often additional) loans to cover their costs.