National Academies Press: OpenBook

Critical Code: Software Producibility for Defense (2010)

Chapter: 1 Recognize the Pivotal Role of DoD Software Innovation

« Previous: Summary
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

1
Recognize the Pivotal Role of DoD Software Innovation

THE ROLE OF SOFTWARE IN DEFENSE

The pivotal role of information technology (IT) in defense has been noted in multiple studies.1,2,3,4 Software is increasingly used to embody the functionality of defense systems of all kinds,5 and IT is used pervasively in the Department of Defense (DoD) for a multitude of different purposes and in a multitude of different program types (Box 1.1). It is a key enabler of overall systems scale and complexity, of integration among systems (net-centricity and “ultra-scale”), and of agility in systems. Mission capability embodied in software has become a unique source of strategic and military advantage, and software producibility is emerging as a key component of military strength, capability, and readiness. The committee uses the term “software producibility” to refer to the capacity to design, produce, assure, and evolve software-intensive systems in a predictable manner while effectively managing risk, cost, schedule, and complexity.

The Defense Science Board’s (DSB’s) Task Force on Mission Impact of Foreign Influence on DoD Software, which explored the essential role of software in defense, released its report in September

1

Defense Science Board (DSB), September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

2

DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.

3

National Research Council (NRC), 2010, Achieving Effective Acquisition of Information Technology in the Department of Defense, Washington, DC: National Academies Press.

4

NRC, 1999, Realizing the Potential of C4I: Fundamental Challenges, Washington, DC: National Academy Press. Available online at http://www.nap.edu/catalog.php?record_id=6457. Last accessed August 20, 2010.

5

DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.1

A Taxonomy of Information Technology Program Types

Many taxonomies have emerged to assist managers in identifying common patterns among requirements and system types and then building on that knowledge to define and implement best practices and standards that are suited to particular categories of systems. Distinctions are made based on the function and role of the system, the types of risk to be addressed, the scale of systems and budgets, and other factors.

This report adopts the following taxonomy of IT programs:

  1. Business systems and office IT

  2. Command and control

  3. Computing and communications infrastructure

  4. Intelligence, Surveillance, and Reconnaissance (ISR), space, and weapons

The classification, which is loosely based on a classification scheme used within the DOD to track IT acquisition programs,1 is primarily functional. But the functional categories also correspond roughly to distinctions among programs based on the extent of innovation (more in categories 2 and 4) and those that are more likely to have precedented requirements and architectures and thus build on established ecosystems (categories 1 and 3).

These categories also separate IT that is embedded in weapons or weapons systems or similar platforms with potentially high systems risk (category 4), IT in which software and hardware are less tightly integrated (categories 1 and 2), and IT that provides the computing and communications infrastructure (category 3) that can be used by systems identified in the other categories.

Because modern larger-scale systems are interconnected and therefore more often integrate across these functionalities, greater numbers of systems may cross these boundaries. For example, many weapons systems incorporate command-and-control functionalities.

Finally, despite the differences among these categories, most systems rely on similar development practices, including design and architectural concepts, programming languages, process and measurement concepts, and tools.

  

1 Based on a taxonomy used by the Office of the Assistant Secretary of Defense for Networks and Information Integration to categorize major automatic information system programs.

2007.6 The report notes that “in the Department of Defense, the transformational effects of information technology (IT—defined here broadly to include all forms of computing and communications), joined with a culture of information sharing, called Net-Centricity, constitute a powerful force multiplier. The DoD has become increasingly dependent for mission-critical functionality upon highly interconnected, globally sourced, IT of dramatically varying quality, reliability and trustworthiness.”7 In other words, at the core of the ability to achieve integration and maintain agility is the ability of the DoD to produce and evolve software. This echoes a judgment expressed in many other studies that have considered the role of software in defense.8 The report further notes, however, that “each year the Department of Defense depends more on software for its administration and for the planning and execution of its missions,”

6

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

7

Ibid., p. vii.

8

See referenced studies above.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

and additionally “this growing dependency is a source of weakness exacerbated by the mounting size, complexity, and interconnectedness of its software programs.”9

The rapid growth of software in defense systems is especially significant and parallels the growing role of software in a broad range of application domains. This growth is a natural outcome of the special engineering characteristics of software: Software is uniquely unbounded and flexible, having relatively few intrinsic limits on the degree to which it can be scaled in complexity and capability. Software is an abstract and purely synthetic medium that, for the most part, lacks fundamental physical limits and natural constraints. For example, unlike physical hardware, software can be delivered and upgraded electronically and remotely, greatly facilitating rapid adaptation to changes in adversary threats, mission priorities, technology, and other aspects of the operating environment. The principal constraint is the human intellectual capacity to understand systems, to build tools to manage them, and to provide assurance—all at ever-greater levels of complexity.

The extent of the DoD code in service has been increasing by more than an order of magnitude every decade, and a similar growth pattern has been exhibited within individual, long-lived military systems. In addition to this growth in size (as well as growth in other system aspects such as resource usage), there is a corresponding growth in overall system capability and complexity.

This chapter addresses the first two of the four questions taken up by the committee:

  • To what extent is software capability significant for the DoD? Is it becoming more significant or less so?

  • Will the advances in software producibility needed by the DoD emerge unaided from industry at a pace sufficient to meet evolving defense requirements?

Growth in the Role and Significance of Software to Defense

The value that software contributes to major systems is increasing rapidly and becoming more fundamental to system capability. The DSB Task Force report on defense software (2000)10 illustrates this point in the case of combat aircraft. The percentage of system functions performed by software has risen from 8 percent of the F-4 in 1960, to 45 percent of the F-16 in 1982, to 80 percent of the F-22 in 2000.11 Software has become essential to all aspects of military system capabilities and operations, and software-specific investment is critical to them.12

Macroeconomic data show analogous growth in the role software plays in the commercial world. This is significant because commercial vendors are key contributors to the defense software supply chain—for Future Combat Systems,13 for example, 27 million source lines of code (more than 42 percent

9

DSB, September 2007, Report of the Defense Science Board Task Force on Defense Software, p. v.

10

DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.

11

Ibid., Table 3.3a. Available online at http://www.acq.osd.mil/dsb/reports/defensesoftware.pdf. Accessed February 25, 2008.

12

Boehm, Kind, and Turner quote an unidentified U.S. Air Force General, “About the only thing you can do with an F-22 without software is take a picture of it.” In Barry Boehm, Richard Turner, and Peter Kind, 2002, “Risky Business: Seven Myths About Software Engineering That Impact Defense Acquisitions,” Program Manager, May 1, 2002. The committee notes, however, that with modern cameras, even taking a picture cannot be done without software.

13

Future Combat Systems (FCS) was “the Army’s modernization program consisting of a family of manned and unmanned systems, connected by a common network, that enables the modular force, providing our Soldiers and leaders with leading-edge technologies and capabilities allowing them to dominate in complex environments.” U.S. Army, “Future Combat Systems.” Available online at http://www.army.mil/fcs/. Accessed March 3, 2008.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

of the total delivered executable source lines of code) were commercial off-the-shelf (COTS) or open source.14

It is also significant because software capability has become a strategic source of market differentiation in many industries, from financial services and health care to telecommunications and entertainment. A 2002 report by the National Research Council’s (NRC’s) Board on Science, Technology, and Economic Policy15 noted that since 1995 the IT and networking industries had accounted for 20 percent of the nation’s economic growth, even though they accounted for only 3 percent of gross domestic product (GDP). Comparable figures exist in the European Community—the information and communications technology (ICT) sector represents just above 5 percent of the European GDP, but reports show that ICT drives 25 percent of overall growth and about 40 percent of the increase in productivity.


Finding 1-1: Software has become essential to a vast range of military system capabilities and operations, and its role is continuing to deepen and broaden, including interlinking diverse system elements. This creates both benefits and risks.

Software in Systems

Military system capability is heavily dependent on software, which has become an enabler for much of the functionality and flexibility of our war-fighting systems. Software has proven to be a differentiator in system capability for a wide range of current systems such as the F-22, F-35 Lightning II, and the Aegis Combat System. Software to modify and integrate existing capabilities was a key enabler in the February 2008 successful shoot-down of an errant U.S. satellite as it tumbled back to Earth.

This critical role of software in defense is also noted in the more recent DSB Task Force report on foreign software, which states, “The DoD now relies upon networked, highly-interconnected systems for many mission-critical capabilities, and this reliance is projected to increase. The software in these systems is the key ingredient that provides much of the increased capability delivered to the warfighter, just as it represents the key factor in increased productivity and new capabilities for industry today. For the DoD, this advanced technology is a force multiplier.”16,17 A high level of software capability is also important in producing defense systems. For example, very-large-scale, highly networked, and crypto-secured software systems were needed for the robotic design used to construct the production line for F-35 manufacturing.

Given the importance of software to the DoD, a vital question is how the department can ensure that it will be able to meet its software needs now and into the future. The subsequent chapters of this report explore significant facets of this issue.

Software provides the means to manifest the modeling and simulation capability now essential in the design and testing of advanced military platforms and weapons in all branches of the DoD. These design-focused software capabilities can save millions of dollars—all before the first piece of metal is bent. But perhaps even more importantly, in these early stages this software capability enables the customer to focus on driving out risks related to the definition of weapons and systems functionality and

14

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, p. 77. Available at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 10, 2010. The FCS program was cancelled in 2009, but the experience of that program nonetheless provides valuable insight.

15

NRC, 2002, Measuring and Sustaining the New Economy: Report of a Workshop, Washington, DC: National Academies Press, p. 52. Available online at http://www.nap.edu/openbook.php?record_id=10282&page=52. Last accessed August 10, 2010.

16

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, p. 12. Available at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

17

DSB, April 2009, Creating a DoD Strategic Acquisition Platform, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA499566&Location=U2&doc=GetTRDoc.pdf. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

architecture. The savings are due to the elimination of the need to build and test numerous prototype designs—this is now done through software. For many systems, this modeling and simulation software implements high-fidelity, massively parallel computational fluid dynamics (CFD) simulations. These simulations enable experimentation with signatures of different missile and aircraft designs without having to resort to expensive physical tests. Wind tunnel testing alone, for example, can cost millions of dollars per week, with months of testing required to settle design issues.

Other examples of software-manifest functionality used pervasively in defense systems include onboard prognostic health systems and automated logistic support systems, both of which emerged from Defense Advanced Research Projects Agency (DARPA) research funding in the 1980s. These software-enabled systems are quietly saving millions of dollars.

Software in Organizations

In addition to deepening the extent of reliance on software in systems components and in the tooling associated with their development, there is a significant broadening of the role of software in systems and organizations. These changes reflect both a growing centrality in the role of software and also a growing portfolio of associated risks. This combination raises the significance of software-related decisions in systems development—software has become the medium of choice for innovative functionalities. These functionalities include not only component capabilities, but also functionalities that are enabled through the use of software to implement interconnections across a family of constituent systems. However, largely as a consequence of both of these roles—innovation and interconnection—software has emerged as the locus for a range of engineering challenges related to reliability, security, and development predictability.

The interconnection aspect of this shift has three principal elements. First, there is a shift in emphasis from the development of functionally focused systems to the development of systems that interconnect and integrate capabilities within and across enterprises. The “net-centric” and “ultra-scale” concepts are reflective of this shift. This has great strategic benefit, in that it leverages the value of dispersed assets and enables agile responses at a broad range of echelons in war-fighting situations when multiple systems elements are involved. This interconnection has associated risks, of course, primarily related to the magnitude of failures experienced as a consequence of internal errors, vulnerabilities exploited, etc. In civilian systems, for example, there are widely reported examples of cascading failures of interconnected systems in telecommunications, utilities, and supply chain systems.

Second, largely as a consequence, IT staffs are generally less involved in mediating between a system and its users. Much larger numbers of DoD personnel interact directly with systems, and indeed many systems can be (and need to be) accessed through public communication infrastructure. This is more efficient because it removes the delays and inaccuracies caused by intermediation. But it also means that many more individuals—usually inadvertently, but not always—can take actions with wide-reaching consequences, both positive and negative.

A third element of the interconnection aspect is that modern systems can support immediate electronic enactment of decisions. This enables agility and fast response in decisions and actions—getting inside the command loop of an adversary, but it also means that failures and compromises can happen very quickly, inside a human decision loop. An example in the civilian context is the recent discussion over the duration in milliseconds of the stock trading look-ahead window.

Fourth, interconnection introduces new information security challenges.

Software Supply Chains

The growth in the role of software, as described above, is enabled in part by a surprisingly recent phenomenon in IT, which is the diversification and enrichment of the supply-chain structure for IT systems. This enrichment is more than just systems outsourcing as experienced in the past half century.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

Supply chains for software systems are both broader and deeper today, and they include commercial as well as defense players and involve technically rich and complex architectures, with frameworks, libraries, services, and other roles contributed by multiple players. The value of outsourcing, which was initially primarily cost reduction and access to expertise, now includes greater agility and ability to respond to changes in the operating environment.

The supply-chain structure for modern defense software is evolving in a similar way, and is now significantly more complex and more international than it was even just a decade ago. Indeed, this combination of factors motivated the Defense Science Board study mentioned above to assess the impact of this internationalization on defense software systems, including their development and their assurance.18 The complexity—and the internationalization—are due to a combination of factors, including certain technical developments in software technology, the economic forces and technological enablers of globalization, the geographic dispersion of the trained workforce, the minimal capital investment required (not including education and training) for the workforce, and increasing demand for precedented (routinized) projects. The complexity is also enabled by the maturation and acceptance of a diverse set of commercial (COTS) ecosystems with their associated components and infrastructure.

From a technological perspective, this richness in the supply chain is enabled by advances in both organizational collaboration technologies and software technology. The collaboration technologies build on Internet infrastructure to provide, for example, messaging, process support, team information servers (document sharing and configuration management), issue databases, servers for software builds, wikis, automated test and analysis tools, and the like.


Finding 1-2: The growth in the role of software in systems is due to a combination of technological advances and a maturing of the supply chain structure associated with software systems development at all levels of scale.

PRECEDENT AND INNOVATION IN SOFTWARE

Precedented Software and Externalities

Software development today relies heavily on established architecture and infrastructure component configurations, which the committee calls software ecosystems (see Boxes 1.2, 1.3, and 1.4 for elaboration). The success of the ecosystem model derives from the natural convergence of component functionalities and the associated architectural elements, chiefly protocols and software/service interfaces, through which these functionalities are delivered within applications and systems. These functionalities are called precedented, in the sense that new users of these functionalities benefit, in terms of design costs and risks, from the experience of existing and prior users. Once ecosystems are established, the development processes associated with them are often characterized primarily by selection of an ecosystem and then, within that ecosystem, tailoring through configuration of settings and the authoring of a relatively very small amount of custom software code. Thus, custom development in these areas of convergence gives way to product selection and procurement. Indeed, because engineering risks are relatively modest, a straight-line sequential process may often be appropriate for development management in these precedented portions of larger systems.

The emergence of generally accepted ecosystem structures has in recent years become one of the enablers of the growth in richness in the supply-chain structure for software systems, which in the commercial world has promoted a diversity of suppliers, growth of a market for specialists in component-level innovation, and tools geared to development productivity for particular ecosystems. A

18

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

countervailing trend in recent years has been the consolidation at the top end of the software vendor market through mergers and acquisitions.

How does this ecosystem phenomenon influence the DoD? First, it should be clear that the DoD derives huge benefit from the ecosystem phenomenon, because so many of the business systems required by the DoD have requirements that are strongly analogous to commercial requirements. In these cases, only modest adaptation of commercial best practices, including choices and configuration of ecosystems, can yield major advantage to the DoD in the form of low costs, managed risk, and predictable outcomes. Additionally, the software elements of an ecosystem are complemented by a supplier ecosystem of expert integrators, consultants, add-in vendors, and the like. In other words, when the DoD’s requirements are similar to commercial requirements, the DoD benefits significantly by adopting commercial best practices where possible. (See Boxes 1.2 and 1.3 for elaboration on the concept and role of ecosystems.)

Not only are engineering risks reduced for precedented developments, but also there are benefits from the richness of the supply chain due to network effects—the positive externalities associated with systems adoption.19 When an ecosystem is successful, the commonalities of structures and interfaces enable larger numbers of organizations to participate efficiently in the development of large systems, providing software components, libraries, frameworks, plug-ins, custom elements, and so on. This makes it possible for more suppliers to participate and also reinforces the status of accepted frameworks, broadening the benefits for framework adopters and affording them the opportunity to make good choices at the component level while working within the safe conventions of an established ecosystem.

Indeed, for many of these categories of requirements, it is nearly intractable, from the standpoint of cost and risk, to develop “separate but equal” approaches. Not only are these expensive, time-consuming, and risky, but also the DoD would then need to bear the entire cost of advancing the idiosyncratic technology, whereas ecosystem participants would benefit from actions by other participants to incrementally advance the performance and capability of the ecosystem, its infrastructure, and its constituent components. (Of course there are also negative externalities associated with widely adopted ecosystems, such as their attractiveness to developers of security exploits and the consequent ease of access by adversaries to offensive capability.)

Innovative Requirements and the Activity of Innovative Software Development

The situation becomes more complex and challenging when the DoD requires functionality more specifically focused on the defense mission—weapon systems, command and control systems, intelligence analysis systems, and other systems more directly supportive of war-fighting and intelligence. They include high-performance embedded systems, large-scale systems with unprecedented architecture, and highly interconnected systems. They often require high degrees of software assurance. The functionalities of these systems are more specialized and also, due to the presence of ambitious adversaries, constantly evolving in response to changing threats. They require the management of complex and evolving requirements.

Many of these systems are less precedented in the sense that innovation is required in system architecture, design, infrastructure, linkages with hardware sensors and effectors, and other respects. Technological enablers for the development of these more innovative systems—either with respect to innovative functionality or innovative engineering or both—is the principal focus of this report. This does not exclude considerations regarding precedented systems, however, because larger systems almost always involve a mix of innovative and precedented functionalities and components. Indeed, there are few modern defense systems of scale that do not build on technologies extensively drawn from related defense systems and from the various mainstream software ecosystems. This means that development and acquisition practices must account for this mix of the innovative and the precedented. This mix

19

Carl Shapiro and Hal R. Varian, 1998, “Information Rules: A Strategic Guide to the Network Economy,” Journal of Technology Transfer 25(2):250-253.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.2

The Concept of Software Ecosystems

In web applications, there are conventional configurations of server operating systems, relational databases, web servers, application server frameworks, business rules, and other elements that are combined to create e-commerce servers. These servers rely, in turn, on the “rich client” ecosystem of a modern client-side web browser, which includes not just HTML and basic HTTP, but also technologies such as JavaScript, XML, DOM access, and asynchronous HTTP.

Analogous configurations, with very different sets of interfaces and components, are used to support mobile applications (Apple’s iPhone ecosystem and Google’s Android ecosystem are two recent examples), business intelligence (OLAP, etc.), enterprise resource planning, and other common functionalities. There are competing ecosystems in the commercial world—for example, web application servers can be developed using Java-based ecosystems such as the platform-independent Java EE (formerly known as J2EE) or using the Windows-based .NET Framework, which supports multiple programming languages sharing common runtime services related to memory management, security, etc. In web applications, there are also open source ecosystems, one of which is the so-called LAMP stack, which comprises the Linux operating system, the widely adopted Apache web server, the MySQL relational database, and scripting in a language starting with the letter “P,” most usually PHP, Python, or Perl. The open-source character of this ecosystem means effectively that it operates as a kind of quasi-consortium linking the various stakeholders that participate in the ongoing development of the overall architecture, the details of the interfaces, and the code base. Regardless of particulars, in most of these ecosystems, choice of programming language is often driven by the choice of ecosystem.

The committee defines a software ecosystem as a conventional structure consisting of a family of infrastructural elements that are intended to be combined in a patterned way. Ecosystems include software-architectural structure, but they can also include configurations of hardware and services platforms. Ecosystems generally provide a reuse of major elements and infrastructure, which can entail strong structural and semantic commitments.1 Ecosystems often also include documents, tools, practices, and even organizations to accompany these elements. The principal benefits include potentially significant reductions in cost, mitigation of engineering risk, and up-front agreement on representations and meanings for data that are shared within a system or across systems.

It is also significant to note that, if we broadly construct the idea of a software ecosystem, then the Internet family of protocols would also be an example. The ecosystem comprising these protocols and its evolution have been much studied—one of the results of this is the “hourglass” model. This model illustrates how there can be a diversity of means for provisioning the service associated with a particular interface or protocol, such as TCP, and a separate diversity of client applications that build on the service. For example, “TCP service” is generally provided as a “layer” above IP, which in turn can be provisioned over fiber, wireless, copper, and many other means. TCP, in turn, underlies the web protocol HTTP as well as file transfer FTP and many other higher-level services.2

  

1 Not all ecosystems involve direct reuse of components. A family of protocols, for example, defines a “means of exchange” among system components. Other examples include instruction set architectures (with multiple vendors providing chips) and agreed-upon XML or other data exchange representations for shared information.

  

2 See, e.g., NRC, 2001, The Internet’s Coming of Age, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=9823. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

Another example is the relatively small set of widely adopted ecosystem architectures for embedded and real-time control applications (such as the QNX, RTLnix, VxWorks, and Windows CE real-time operating systems), the automotive industry’s AUTOSAR (AUTomotive Open System Architecture), and the SCADA protocols and interfaces used in the electrical grid. Many government users of embedded and real-time capability have taken actions to work with researchers and vendors to develop more capable ecosystems that build on modern concepts and abstractions related to processors, languages, and tools.3,4 Ecosystems are also being adopted or are emerging in areas ranging from robotic systems to data-intensive supercomputing.5

The ecosystem phenomenon is now pervasively apparent in commercial industry, and it is actively promoted by leading vendors, in part due to the stability of market structure derived from the network effects. Ecosystem or framework “owners” (from the examples above: Apple, Microsoft, Google, Oracle, several open-source foundations, and many others) control the trajectory of the overall market for components and services associated with that ecosystem, but many firms participate in that “internal” market. Although the risks and costs associated with introducing a new ecosystem or framework may be very high, the risks and costs for niche providers within an established framework can be low. Additionally, once a community of suppliers is engaged within an ecosystem, the overall ecosystem can continue to evolve in response to the broad market trajectory and also to new technology developments. Thus, new languages can be added to .NET (e.g., functional programming with F#), new libraries and language features can be added within Java EE (closures to Java), and so on. This is one of the enablers and sustainers of the global supply chain.

The committee notes, in addition, that the structure of ecosystems may become more or less exposed to developers and users, and indeed entire ecosystems may split or merge. For example, a vendor could choose to expose a previously inaccessible internal interface to allow greater customization by customers and integrators. In the case of enterprise resource planning (ERP) systems, for example, vendors expose interfaces that allow “add-in” developers to provide a diverse set of functionalities tailored to particular market segments. This enables a broader diversity of client requirements to be met with less risk to both clients and the framework vendor. The ecosystem structure thus evolves according to changing opportunities and risks for the various stakeholders. Examples of the considerations include network effects (benefits of broader adoption of particular “hourglass necks”), vertical integration (reduction in risks associated with integrating separately developed components), and lock-in (greater friction in interoperation and choices available for component functionalities).

  

3 See, for example, the workshop convened by the NITRD High Confidence Software and Systems Coordinating Group, “National Workshop on High-Confidence Automotive Cyber-Physical Systems,” April 3-4, 2008, Troy, Michigan. Available online at http://varma.ece.cmu.edu/Auto-CPS/. Last accessed August 20, 2010.

  

4 See also, Jeanette Wing, 2008, “Cyber-Physical Systems Research Charge,” presented at the Cyber-Physical Systems Summit, April 24, 2008, St. Louis, MO. Available online at http://www.cra.org/ccc/docs/cps-summit.pdf. Last accessed August 20, 2010.

  

5 NRC, 2009, Assessing the Impacts of Changes in the Information Technology R&D Ecosystem: Retaining Leadership in an Increasingly Global Environment, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=12174. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.3

The Role of the Software Ecosystems for the DoD

There are a number of key advances in software technology that enable the emergence of ecosystems and other architecture-based enrichments in supply-chain structure, with consequent benefits to cost and agility and also the consequent risks of more diverse sourcing. These advances range from the design of software architectures, frameworks, and components to the technical properties of modern programming languages, including first-class encapsulation, advanced typing, interface and package structures, and framework architectures.

These and other software technology improvements have also enabled the development and successful implementation of a number of conventional structures of software components used for commercial and government applications. These conventional structures, here called ecosystems, are families of infrastructural elements that are combined in a patterned way to construct precedented applications such as many business and back-office systems, e-commerce and web applications, and mobile applications. Ecosystems include such software-architectural structure as stacks, hardware and software platforms, and software frameworks. They often also include the documents, tools, and practices that accompany these elements (e.g., Eclipse and ASP.NET). The ecosystems also include a diverse array of software and organizational services that support conventional architectural structures.

The ability to successfully define robust and broadly adoptable standards (de facto or ratified) and to stimulate a critical mass of compliant implementations is a significant enabler of ecosystem success. The economics of network externalities1 play a significant role in reinforcing successful ecosystems (often regardless of technical merit) and in guiding the initial stages of promotion of emerging new ecosystems. (This issue is elaborated below.)

A particular challenge for the DoD in defining its own ecosystems is to keep up with rapidly evolving technology and to select those interfaces and component capabilities for which certified components and/or trusted suppliers exist. Ecosystems in different categories may share elements and typically support diverse ranges of applications. Over the years, the DoD has attempted to codify its preferences regarding the components and interfaces in these conventional aggregates with efforts such as the Defense Information Infrastructure Common Operating Environment (DIICOE), the Joint Technical Architecture (JTA), and the System of Systems Common Operating Environment (SOSCOE). Generally speaking, these are broad suites of conventionalized standards and infrastructural elements that have been judged acceptable for adoption in systems generally. These suites may identify particular acceptable ecosystems, but they are not themselves ecosystems by the committee’s definition (or architectures, in the sense of Chapter 3).

  

1 See, e.g., Carl Shapiro and Hal R. Varian, 1998, Information Rules: A Strategic Guide to the Network Economy, Boston: Harvard Business Press.

often creates confusion regarding development processes and other systems engineering choices. “One size fits all” models—even for incremental iterative developments—can be dangerous.20

Because of the extent to which modern software builds on existing ecosystem and infrastructural elements, modern software development processes entail activities that go well beyond the design and authoring of new code. Modern software development is much more about identifying and defining appropriate and scalable architectures; selecting, using, and adapting infrastructure such as frameworks, components, and libraries; and deploying best practices and tools for collaboration, process support, and

20

With respect to process (Chapter 2), hybrid approaches may best be employed, for example, combining straight-line processes for precedented elements with iteration and prototyping for innovative elements. With respect to architecture (Chapter 3), modular designs that “concentrate” particular innovative or rapidly evolving functionalities in individual components can greatly reduce overall project risk.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

Modern e-commerce frameworks, for example, are ubiquitous in Internet-based commerce, but they have also been adopted as internal coordination frameworks for large-scale DoD systems.

The ecosystems contribute enormous value to software and systems projects that rely on them by allowing developers to leverage an enormous investment for which costs are spread across a wide base of users rather than taking on the full effort and expense of developing an entire software system from the ground up. When ecosystems are widely adopted, architectural risks are drastically reduced because principal architectural commitments are embodied in the successful ecosystem, and, additionally, the extent of value to the DoD can grow over time due to the positive network externalities. Thus, the DoD derives benefits from ecosystem adoption, but it must be attentive both to selection criteria and to its ability to participate in the overall evolution and development of the ecosystems within which it participates. One particular issue is the sustainability of ecosystems that are adopted into systems. In some instances, choices may depend more on appraisals of sustainability and network effects (to use economic terms) than on particular technical characteristics. These sustainability factors may influence engineering risk (see Box 2.2). The DoD may derive great benefits from investing in the evolution of the ecosystems in which it participates, which enhances both technical fit and sustainability.

As noted above, these ecosystems are enabled by a wide range of computer science and software engineering advances. The modern software application frameworks essential to Web-based systems, ecommerce, and graphical user interfaces of all kinds are enabled by the same advances in programming language design that led to languages such as Java, C#, and Ada95. Many of these “component” advances and, perhaps more importantly, the principal abstractions and architectural concepts underlying established ecosystems, are legacies of past DoD investment in computing technology R&D, primarily in the form of 6.1 and 6.2 extramural research funding.

Because of the rapid pace of infrastructural development, the competitive business environment, and the need to accommodate new functionality, the ecosystems are generally in a state of continuous evolution, carefully managed to stage out new increments of value while minimizing costs and risks for existing adopters—and thus to retain the benefits of the positive externalities. The evolutionary trajectory for some ecosystems is entirely driven by particular vendors, as is the case with Microsoft and .NET or Oracle and its E-Business Suite. Others are driven by complex community processes, as in the case of the open-source LAMP stack (see above), the Internet protocols themselves, and also some commercial ecosystems, as is the case with many of the ecosystems surrounding Java—following the Java Community Process.2 The evolution may include specific component capabilities, architectural and interfaces structures, and associated tooling (as in the case of Visual Studio and .NET).

  

2 For more information, see the Java Community Process at http://jcp.org/en/home/index. Last accessed August 20, 2010.

validation. Indeed, it is generally recognized that, for large systems in industry and aerospace, the most significant costs are generally associated with gathering functional and non-functional requirements, developing architecture and design, managing process, and achieving assurance—and somewhat less with the writing and evolution of code.21 (Issues related to innovative systems for defense are addressed in two chapters of this report—Chapter 2 focuses on requirements, and Chapter 3 focuses on architecture and agility at scale.)

21

See, for example, economics studies by Barry Boehm, studies of the IBM Rational Unified Process, and other work that shows that the proportion of coding in the overall process is diminishing. See, RTI, 2002, The Economic Impacts of Inadequate Infrastructure for Software Testing, Planning Report 02-3, RTI Project Number 7007.011. Available online at http://www.nist.gov/director/planning/upload/report02-3.pdf. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.4

The Concept of Alignment

As noted earlier, complex evolving software ecosystems and rich supply chains are enabled by relatively recent technological developments, although the market forces have been present for a much longer time. The benefit to government and firms alike is in how software is managed, with the breakthrough result of a better alignment of IT structures with operational structures in organizations. This affords firms greater flexibility in buy-versus-build decisions. But, more importantly, it enables many organizations to outsource common infrastructure such as databases, application servers, and software frameworks, and to in-source only those critical elements that provide unique capabilities and advantage over competitors. This amounts to an “escape” from the need to redundantly develop infrastructural capabilities.

The committee uses the term “alignment” to refer to this ability to in-source only key differentiators and organization-specific capabilities. Generally speaking, alignment is achieved incrementally. As capabilities that previously were innovative become commonplace across firms, the task of advancing them (from the standpoint of a technology user organization such as DoD back-office business functions) is shifted from internal resources to external ones (vendors and other outsource suppliers), enabling the firms to redirect their internal resources to new areas where they can differentiate themselves from their competitors. This is how, for example, the central database for many firms evolved from early network and hierarchical databases into relational transactional databases into virtualized application server capabilities wrapped around web servers and relational databases, with most of the functionality being provided by outside vendors or through open-source de facto consortia as in the LAMP stack.

The DoD also benefits from this when it can shift from expensive custom components (that it must maintain throughout an entire system lifecycle) to off-the-shelf components that are constantly being improved upon by their vendors in response to market forces. Thus, as technologies evolve and “commoditize,” there is a general trend to shift function from in-source to outsource.1 An issue for the DoD, however,

  

1 But this is not always the case, as new dimensions of capability and differentiation emerge for formerly commoditized infrastructural elements, as is happening now for data centers and their architectures.

As noted both in the workshop report issued by this committee22 and in the recent Software Engineering Institute (SEI) report on ultra-scale systems,23 these issues may be made more challenging for modern interconnected defense systems due to overall scale and complexity, and particularly in requirements and architecture. These systems experience a great deal of architectural risk due to the often long delay until the consequences of early engineering decisions are felt and understood. Additionally, the decentralized governance models that are typical for large-scale interlinked systems (ultra-scale, net-centric, system of systems) can have both positive and negative effects on risk. Also, overly conservative choices regarding how to measure progress and earned value can lead toward local optima but away from overall systems-scale success (see Chapter 2). Finally, over-commitment to particular requirements early in the process can result in lost opportunities for radical cost savings or capability improvements downstream. These risks could potentially be mitigated through innovation in both technological and process measures.

22

NRC, 2007, Summary of a Workshop on Software-Intensive Systems and Uncertainty at Scale, Washington, DC: National Academies Press. Available online at http://books.nap.edu/catalog.php?record_id=11936. Last accessed August 20, 2010.

23

Software Engineering Institute (SEI), 2006, Ultra-Large-Scale Systems: The Software Challenge of the Future, Pittsburgh, PA: Carnegie Mellon University. Available online at http://www.sei.cmu.edu/library/assets/ULS_Book20062.pdf. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

as for other entities seeking to maintain leadership in software use and development, is how to effectively track the evolution of the conventional interfaces and architectures and not fall behind. Another issue, more directly related to innovation, is how to work with the broad technology community to ensure that—where the DoD has leading demand—its requirements can be met as the infrastructure evolves. Historically, the DoD has accomplished this in many core IT areas, as noted in multiple studies. 2,3,4

Indeed, when infrastructures are traced historically, it is evident that many of the fundamental architectural concepts originated in or were stimulated by DoD-sponsored research. For example, many of the architectural elements of Linux can be traced back to BSD Unix and even Multics. This point relates to the discussion later in the report regarding the role of the DoD in architecture.

From the defense perspective, this yields both benefit and risk. Expertise in critical component and infrastructural technologies is concentrating, and the capability of those components is rapidly advancing. The DoD cannot so easily “build” when other players are all “buying,” even when there may be assurance challenges with respect to component providers. If the DoD makes too many “build” decisions for infrastructural capabilities, costs and risks escalate to the point of intractability. This is because the DoD would have to bear the entire cost and risk of developing and sustaining its own custom version of the technology. The established implementations of that technology may have been evolving in the larger commercial market over a period of years, with effective investments spread across a multitude of vendors and users.

  

2 NRC, 1997, Ada and Beyond: Software Policies for the Department of Defense, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=5463. Last accessed August 20, 2010.

  

3 NRC, 2000, Making IT Better: Expanding Information Technology Research to Meet Society’s Needs, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=9829. Last accessed August 20, 2010.

  

4 NRC, 1997, The Evolution of Untethered Communications, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=5968. Last accessed August 20, 2010.

Commercial Software Supply Chains

The DoD’s supply chain has become increasingly complex. One driver has been the commercial success of software ecosystems that provide a foundation on which to build defense systems. Another driver has been the increasing use of tools for collaboration at a distance. These are the same technologies, infrastructure, and practices that have enabled the globalization of diverse services. (Indeed because of this it is easy to erroneously conflate supply-chain richness with the globalization phenomenon.)

The factors driving the supply-chain structure include not only the direct costs of development, but also the resulting management agility (such as the ability to revisit choices in infrastructure, technology, and particular suppliers) and rapid access to specialized expertise (domain knowledge and requirements, code development, vendor components, testing and evaluation, process structuring, software architecture, and so on). For commercial applications, these factors combine to enable large firms to quickly adapt and enhance their business models to address competitive challenges. The DoD can and should realize similar advantages, but of course it also needs to address the risks, including both the sourcing risks intrinsic in this kind of supply structure and the particular requirements risks that derive from the defense mission.

The complexity of software supply-chain structures is evident in diverse sectors. A single firm in a sector such as financial services, health care, or manufacturing may develop software at dozens of

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

separate sites around the world. This software is likely to depend on infrastructural components from dozens of vendors, each of whom may also have global operations. This is how the advantages of cost, agility, and expertise are realized and fundamental new functionalities are achieved, for example, those involving cross-cutting capabilities related to business intelligence or enterprise process management (both of which are highly relevant to the defense mission). But the risks must also be addressed—when the software components are interconnected within a single system, all of these components may be “behind the firewall” and with direct access to mission data. (Issues related to assurance, including challenges related to supply chains, are addressed later in this report, in Chapter 4.)

The market forces that drive the complexity of the modern supply-chain structure for software systems have long been present, but the supply chain and ecosystem richness are only relatively recent phenomena, enabled by the richness of modern software technology. The enabling technologies include modern programming languages, system software components, network protocol development, object-oriented frameworks, emerging service-oriented concepts (e.g., cloud, SAAS, SOA), advanced tooling for team development and collaboration, and process support. Indeed, a number of these core ideas are results of DARPA and Service funding of research projects in prior decades.

An important element of the globalization phenomenon is the pace at which global suppliers outside the United States, in many countries, are moving up the value chain—that is accounting for an increasing share of the overall value embodied in a product or service. Global suppliers, which in the early days focused primarily on low-technology offerings such as providing black-box testing services for Web-based software systems developed in the United States and elsewhere or on provisioning remote first-tier technical support capability, are now developing the software for those systems directly, as well as engaging in requirements analysis, architecture, and design for those systems. This commercial trend, accelerated through direct strategic investment by governments, exacerbates the DoD concerns about the mission impact of foreign influence on DoD software—namely, the risk of unwanted functionality in delivered software (and hardware as well, where the assurance challenges can be greater).


Finding 1-3: The DoD relies fundamentally on mainstream commercial components, supply chains, and software ecosystems for both business systems and many mission systems. Nonetheless, the DoD has special needs in its mission systems driven by the growing role of software in systems. As a result, the DoD needs to address directly the challenge of building on and, where appropriate, contributing to the development of mainstream software that can contribute to its mission.

DoD Software Supply Chains

The DoD is aggressively applying these ideas for business applications. In the case of IT systems and components such as databases, operating systems, and business systems applications, the DoD can align well with commercial products being produced to support industry.

At the same time, however, software has become a critical differentiator in most mission-related systems and services—and (as noted above) it is growing in the extent and depth of its impact every year. It is safe to claim, in fact, that the largest opportunities for successful differentiation in new mission systems are very often derived from software-manifest capabilities. It is therefore not surprising that the largest risks in systems development are associated with the software production.

Almost always, DoD mission systems rely on a combination of innovative functionality and capabilities already present in established ecosystems. The DoD must obviously leverage the extensive commercial development of software processes, methods, tools, architectures, and products. But, as the committee notes below, the DoD must also take action, as it has done historically, to foster the capabilities of its supply chain, broadly construed, to enable it to stay ahead of its rapidly advancing adversaries. In particular, this new reality poses challenges not only for developing innovative functionalities, but also for assurance and ecosystems leadership.

As noted above, larger-scale innovative mission-focused applications generally include both innova-

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

tive custom software components and components based on existing ecosystems. As a result, the DoD necessarily relies to some extent on the extant supply-chain structures with their diversity of suppliers. This creates both technical and management risks related to assuring control over functionality and assurance.24 These challenges focus on understanding the extent and nature of supply-chain and ecosystem dependencies, on developing both incentives and technical means to mitigate assurance risks, and on understanding the leadership challenges with respect to sustaining some control (or at least awareness) over future trajectories where there are necessary ecosystem dependencies.

Governments are recognizing the difference between component-level participation in an ecosystem-associated supply chain and architectural leadership of that ecosystem.25 U.S. firms have developed and led the evolution of most of the key ecosystems on which the DoD and the entire industry rely. This is a consequence of technological and market leadership, and the technological aspects of this leadership, in turn, are in large measure consequences of the long record of R&D investment in core computer and information technologies by DARPA and the Services and a small number of other Networking and Information Technology Research and Development (NITRD) agencies, principally the National Science Foundation (NSF) and, historically, NASA. These investments are now diminished (Box 1.5). The committee considers priorities regarding this investment (Chapter 5) as well as arguments for and against a scaling up of this investment (this chapter) and means by which the investment can be evaluated and optimized (Chapter 5).

Summary—Software and the DoD

Software is highly significant for the DoD and becoming more so. The DoD depends not only on the ability to develop new code, but also on commercial software capability, particularly as manifested in established evolving software ecosystems. Software supply chains are growing in scale, complexity, and geography, and the influence of these shifts on DoD software must be considered.

Although the United States continues to retain innovation leadership in software areas important to the DoD, there are three proximate factors that could cause the loss of that leadership. First, as noted above and documented in Box 1.5, the DoD investment in software producibility has in recent years diminished considerably from its prior levels, which had been sustained for more than three decades. Second, concomitant with the diminishing of U.S. investment is a ramping up of investment by foreign governments in their national IT capabilities, including in software.26 The third factor, also as noted above, is the inexorable trend of globalization and rich supply chains.

Of course, very strong shifts overseas have happened in other sectors, such as consumer electronics, and there is still debate regarding the strategic impact of these shifts. It is the committee’s view, however, that the leveraged role of software and the particular special role of software in defense and national

24

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

25

“Internet—New Shot in the Arm for US Hegemony,” China Daily, January 22, 2010. Available online at http://www.china-daily.cn/china/2010-01/22/content_9364327_3.htm. Last accessed August 20, 2010.

26

See Organisation for Economic Co-Operation and Development (OECD), Information Technology Outlook 2006, Paris, France: OECD. Available online at http://www.oecd.org/document/10/0,3343,en_2649_37441_37486858_1_1_1_37441,00.html#TOCat. Accessed February 26, 2008. Also see NRC, 2009, Assessing the Impacts of Changes in the Information Technology R&D Ecosystem: Retaining Leadership in an Increasingly Global Environment, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=12174. Last accessed August 20, 2010. See also Ashish Arora and Alfonso Gambardella, eds., 2005, From Underdogs to Tigers: The Rise and Growth of the Software Industry in Brazil, China, India, Ireland, and Israel, Oxford, England: Oxford University Press, pp. 171-206. Rafiq Dossani and Martin Kenney, 2007, “The Evolving Indian Offshore Services Environment: Greater Scale, Scope and Sophistication,” Sloan Industry Studies Working Papers, Number WP-2007-34, 2007. Available at http://www.industry.sloan.org/industrystudies/workingpapers/index.php. Accessed February 26, 2008. OECD, 2006, “China Will Become World’s Second Highest Investor in R&D by End of 2006, Finds OECD,” OECD Online, http://www.oecd.org/document/26/0,2340,en_2649_201185_37770522_1_1_1_1,00.html. Accessed February 26, 2008.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.5

The Decline in Federal Investment in Software Producibility Research

The extent and structure of federal IT-related R&D investment are documented in a series of reports published by the National Coordination Office (NCO) for Networking and Information Technology Research and Development (NITRD). These reports have been issued annually since the first such report was included in the FY1992 federal budget submission, and they document research sponsorship from a diverse set of federal agencies related to IT and networking. NITRD is a multi-agency coordination activity, which operates under the auspices of the White House Office of Science and Technology Policy (OSTP) and the Office of Management and Budget (OMB), but the funding that is reported is in the individual agencies’ budgets and generally under their control. The reports include extensive narrative descriptions of research accomplishments and plans, as well as a budget matrix that shows investment levels by agency, organized into a set of eight categories.1 The budget matrix shows both proposed amounts for the forthcoming fiscal year and approximate actual amounts for the then-current fiscal year.

There are two categories that relate to software producibility—Software Design and Productivity (SDP) and High Confidence Software and Systems (HCSS). The committee analyzed the trends in these two categories over the past decade and related its findings to the overall NITRD-coordinated budget that totals investment in all eight categories. (Note: The analysis excluded National Institutes of Health (NIH) data. This was done for two reasons: First, NIH changed its reporting methodology in 2010, which creates non-commensurability for a longitudinal analysis. Second, NIH allocations among the NITRD topic categories were determined through the use of an automated text-based pattern-matching algorithm. The committee believes this approach, particularly in topics related to software production generally (rather than, for example, the production of software for particular applications), is likely to lead to significant over-reporting of application software development projects as SDP or HCSS research.)

The principal result of the analysis is that the SDP and HCSS investments, separately and combined and in absolute dollars and as a percentage of the NITRD budget, have dropped considerably in the past 5 years. At the same time, the overall NITRD-coordinated budget has grown. For example, from 2004 to 2010 the combined allocation to SDP and HCSS fell by 45 percent, while the overall NITRD budget more than doubled. On a percentage basis, the combined SDP and HCSS allocation fell by a factor of almost four, from 24.6 percent of the NITRD total in 2004 to just 6.5 percent of the total in 2010.

One of the challenges in this type of budget analysis is the breadth of the categories and the imprecision of category boundaries. This challenge is unavoidable in the analysis of research budgets, but it is particularly difficult in the analysis of NITRD budgets because the different agency staff may apply slightly different criteria when categorizing diverse and innovative research projects. When category labels change, for example, with the introduction of the category of Cybersecurity and Information Assurance (CSIA), it is very likely that some projects in HCSS were relabeled as CSIA. Although it would be desirable to assess each grant for its relevance to the categories or, better, to the particular technical disciplines that support “software producibility,” this would be infeasible because of the large number of research grants and contracts and also because agencies are reluctant to share detailed data regarding awards and category assignments. An analysis of the narrative descriptions associated with the categories in the NITRD report suggests that there is an acceptably close alignment of SDP and HCSS with the overall investment that might directly relate to software producibility. The narrative descriptions do reveal some areas included in SDP or HCSS that might not be included in a “software producibility” category, for example, due to application specificity or other attribute.

Taking all this into consideration, the committee judges that the combination of SDP and HCSS is sufficiently close to a notional category of software producibility that we accept it as a surrogate for overall investment across NITRD agencies (NIH excluded) in research that relates to the present report.

  

1 The categories have slowly evolved over the years, but categories related to software have remained unchanged for the past decade.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Figure 1.5.1 Investment in HCSS and SDP by agency and by year. NOTE: Office of the Secretary of Defense (OSD), Defense Information Systems Agency (DISA), and Service investments have been rolled up into a single category that covers defense agency investments other than those in DARPA and the National Security Agency (NSA). NIH amounts excluded for reasons noted in the text.

Figure 1.5.1 Investment in HCSS and SDP by agency and by year. NOTE: Office of the Secretary of Defense (OSD), Defense Information Systems Agency (DISA), and Service investments have been rolled up into a single category that covers defense agency investments other than those in DARPA and the National Security Agency (NSA). NIH amounts excluded for reasons noted in the text.

Figure 1.5.2 Total NITRD investment by agency and by year. NOTE: OSD, DISA, and Service investments have been rolled up into a single category that covers defense agency investments other than those in DARPA and NSA. NIH amounts excluded for reasons noted in the text.

Figure 1.5.2 Total NITRD investment by agency and by year. NOTE: OSD, DISA, and Service investments have been rolled up into a single category that covers defense agency investments other than those in DARPA and NSA. NIH amounts excluded for reasons noted in the text.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

More details regarding the state of research investment are shown in Figures 1.5.1 and 1.5.2. The former shows the investment by year and by agency for HCSS (left side) and SDP (right side). It is evident from Figure 1.5.1, for example, that NASA, DARPA, and the Department of Energy (DOE) have stepped almost completely away from engaging in research related to SDP. Figure 1.5.2 shows the overall extent of the NITRD investment in all eight categories of research related to networking and IT.

Two trends are immediately apparent from these charts: there was a surge of SDP and HCSS investment in the mid-2000s by several agencies—DARPA, DOE/National Nuclear Security Administration (NNSA), and NASA—followed by a precipitous drop in their investments and in the total investment. The result is that NSF is now the dominant source of investment in both categories. Figure 1.5.3 shows the percentage of total NITRD investment in SDP and HCSS. It illustrates that while the total NITRD investment more than doubled over the past decade, the percentage of investment in SDP and HCSS fell off sharply.

Figure 1.5.3 Percentage of total NITRD investment in either SDP or HCSS.

Figure 1.5.3 Percentage of total NITRD investment in either SDP or HCSS.

security systems of all kinds make this kind of shift much more consequential for defense software producibility and for U.S. ability to advance overall defense system capability.

In exploring the role of the DoD in advancing software producibility, which is the topic of Chapters 2, 3, and 4, the committee considers the interplay of four factors:

  1. Productivity. The DoD benefits from efficiencies gained in the development of innovative functionality through advances in mainstream software producibility. For mission systems, there are particular challenges relating to requirements and validation, architecture and modeling, process and measurement, and tools and language systems.

  2. Innovation. The DoD relies on technologically enabled advances in software producibility to enable the more effective creation of unprecedented systems and the interconnecting of existing and new systems to deliver advanced functionality with acceptable cost and risks.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
  1. Assurance. The DoD faces new challenges to addressing the risks associated with diverse and international supply chains, including the development of practices and technologies for software assurance.

  2. Ecosystems and infrastructure. The DoD unavoidably relies on mainstream software ecosystems in defense systems and therefore has a stake in the processes by which those ecosystems evolve and are led.

THE ROLE OF THE DOD IN ADDRESSING ITS SOFTWARE NEEDS

Given the importance of software to the DoD, and to its mission systems in particular, and given also the ongoing rapid advances in software capability worldwide, it is vital to ensure that the department can not only meet its software needs now, but also sustain its software leadership well into the future. A key question addressed by the committee is to what extent the DoD, without providing its own explicit R&D stimulus, can rely on industry—specifically the domestic defense industrial base and supporting vendors—to produce software innovations in areas of defense significance at a rate fast enough to allow the DoD to fully meet software requirements and remain ahead of potential adversaries. This leadership must be with respect to both the capability of systems and effective defense against attacks on those systems.

As noted above, the DoD has particular requirements that must be dealt with on systems that are both very large scale and have life-critical mission requirements. Although these areas may overlap with civil and commercial needs, very often the DoD requirements are more sophisticated and cutting-edge than those in the rest of the marketplace. Also, DoD adversaries may choose to “attack” software in the supply chain during development phases of a project—security is much more than about attacks staged over networks during system operations. Additionally, major DoD development projects are structured in a way that often keeps development teams at arm’s length from the key operational mission stakeholders and from overall project management. For these reasons, technological advancement would significantly benefit the DoD’s ability to produce the software it needs. The areas identified in this report, and particularly in Chapter 5, are areas where the committee sees the DoD as having leading demand. The committee notes that the issue is not areas where the DoD has “unique” requirements, but rather the much broader category of areas where it has leading demand with respect to particular kinds of requirements. One obvious example is software assurance, where DoD and national security needs may go well beyond even what is being developed for commercial financial services or health care devices. Another example is the risk-managed development of unprecedented architectural design of the kind required to create high-interconnectivity systems such as FCS, net-centric systems, and many other major defense platforms that have few commercial precedents or analogs.

The committee notes that even where industry is aggressively innovative, it may not have sufficient incentives to produce the technology and supporting tools necessary to generalize application-specific software innovations. Additionally, the technologies may manifest innovative concepts, but in a way that cannot be readily adopted by the DoD, for example due to the safety, reliability, and assurance considerations particular to defense applications that, to be addressed, require further technological innovation.

It would thus be overly optimistic to conclude that DoD needs such as these will somehow be adequately addressed through a combination of demand-pull from the DoD and technology-push from the defense sector (i.e., firms that primarily supply the DoD) or the broader commercial IT sector. In many other industries and infrastructures, this may be a legitimate conclusion, and in these areas the best policy may be for the DoD to follow the market. However, this is not generally true for software technology, particularly as needed for defense mission systems, where the DoD has leading demand in multiple critical areas, as detailed later in this report.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

Finding 1-4: The DoD’s needs will not be sufficiently met through a combination of demand-pull from the military and technology-push from the defense or commercial IT sectors. The DoD cannot rely on industry alone to address the long-term software challenges particular to defense.


The above finding is based on consideration of both the history of software innovation and the set of future needs identified in this study.

Commercial R&D Investment

Defense contractors do invest extensively in software research, but generally speaking it is focused on manifesting specific capabilities in supporting the competitiveness of bids through differentiated skills and products. Commercial IT firms also invest in software research and form an important part of the defense IT supply chain, but may not necessarily carry out research aimed at meeting the DoD’s needs. Both defense and commercial IT firms lack strong incentives to invest directly in broad-impact areas such as these, particularly when many of the advantages derived from the investment are non-appropriable, in that the associated intellectual property cannot be readily controlled and as a consequence the benefits may readily diffuse not only into the supply chain for the contractor but also to competitors (more on this point is given below).27 Indeed, it is important to note that there are certain technological improvements that may in fact not necessarily be good for business, even when the DoD derives capability and acquisition advantage.

Additionally, and perhaps most importantly, there is the issue of horizon. Prudent business decisions are generally informed by return-on-investment calculations, which depend on (1) appropriability, (2) timeliness, (3) investment risk, and (4) measurability/observability of return. Many improvements in practices, for example, come only after sustained commitments and much technical exploration (after which benefits may rapidly diffuse across the industry). Additionally, benefits, even when judged significant by technical leaders, may be difficult to quantify, due to the measurement challenges that persist in software and the software-related aspects of systems engineering (see Chapter 2). In other words, in investing in software producibility, all four of the elements above are problematic. And therefore, in a competitive market, individual companies generally have few market incentives for such investments. These factors tend to drive internal R&D investments in contractors toward a combination of addressing business needs expected 1 to 2 years in the future and avoiding technological surprise from competitors. Government research and development in software producibility, from a purely structural perspective, can be less focused on appropriability and investment risk. Additionally, government can invest directly in improving measurement capability, when that is a source of risk (see the discussion of measurement in Chapter 5).

The Challenge of ROI and Appropriability

It is an economic reality that it is difficult for a firm to make a compelling case of return on investment (ROI) to undertake innovations when those innovations have a non-appropriable character, which is to say that the intellectual property associated with the innovations diffuses broadly into the engineering discipline and the economy. This is a familiar issue to those involved in defining industry-wide best practices, standards, and other commonalities. Many of the most important and highly leveraged, government-originated innovations (undertaken by both academia and industry) are in the economic “commons.” This creates challenges in measuring value created, because the value has broad and dif-

27

See, e.g., NRC, 2002, Information Technology, Research, Innovation, and E-Government, Washington, DC: National Academies Press, p. 101. Available online at http://www.nap.edu/catalog.php?record_id=10355. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

fuse benefits.28 A recent NRC report on software and the economy29 notes that “the economic rationale for government investment is based on the non-appropriability of many significant IT innovations, including the most widely used idiomatic data structures and algorithms, as well as design and architectural patterns. Moreover, the IT industry relies on a number of technical and process commonalities or standards such as the suite of Internet protocols, programming languages, core design patterns, and architectural styles.” These innovations effectively “raise everyone’s boat” in the same way as do government investments in bioscience, health care, and other strategically important scientific disciplines. This includes many of the most highly leveraged areas of software research such as improvements in abstraction mechanisms, design notations, programming languages, software analysis and model checking, basic algorithms, design patterns and architecture concepts, and other core techniques. This is not to say that industry does not invest in non-appropriable results—it does so extensively in the area of standards and also through investment in university projects and other activities. But (for the reasons cited here) the incentives to sustain this investment are lower than for proprietary development efforts.

The committee notes, in addition, that when research results are not appropriable, researchers are less likely to be able to secure patents. (Thus, it can be safely hypothesized that direct revenues from licensing university-owned patents are likely to be significant underestimates of the value created by federally funded research, especially in the case of software-related university inventions.)

To complicate matters further, the manner in which software is protected as intellectual property is often distinct from what is done in other fields such as biomedicine. A software system is often a combination of differently protected elements, organized following architectural elements of several ecosystems along with custom elements. There is often considerable intellectual property embedded in the architectural designs themselves, even in the absence of components that populate those designs. All this combines to make the non-appropriable, yet valuable, aspects of the work hard to identify.

Finally, the committee notes that the software industry is shifting from the development of software products to an increased focus on integration, custom development, and other services.30 More than half the revenues of software product companies are now coming from services rather than from product sales, but product sales tend to be the most scalable and profitable part of the business.31 Additionally, the software-product sector is consolidating and thus shrinking in numbers, decreasing from more than 400 to less than 150 publicly listed software product companies on U.S. stock exchanges in the past 8 years.32 Another consequence of the shifts toward services and consolidation may be a reduction in

28

Indeed, fear of value diffusing to competitors that can assimilate it more efficiently into their practices can create a negative incentive to invest, although it can be dangerous to make this judgment when evidence is lacking, because innovation leadership may be less easy to recover once lost.

29

NRC, 2006, Measuring and Sustaining the New Economy, Software, Growth, and the Future of the U.S Economy: Report of a Symposium, Washington, DC: National Academies Press. Available online at http://www.nap.edu/openbook.php?record_id=11587. Last accessed August 20, 2010.

30

For quite a few years, about two-thirds of global revenues in the software industry have actually been from services (such as custom software development, maintenance, IT consulting, and technical support), and only one-third of revenues have come from the product companies. See Michael A. Cusumano, 2004, The Business of Software, New York: Free Press, p. 46, footnote 19, citing Standard & Poor’s annual data. An issue in these analyses is how the word “services” is defined—it is used both for custom development/integration by teams of people and also for “software-as-a-service” (SAAS) and cloud-based delivery of software value. Additionally, the distinction between “product” and “service” in software is becoming increasingly muddy as licensed product software delivered to customers is complemented by off-site SAAS and cloud-based software generally. For many end users, for example, there may be relatively little distinction in the experience, say, of Microsoft Office tools on a desktop computer (a “product”) and Microsoft Office Live or Google documents tools on a browser (a “service”). A similar statement could be made at the enterprise level, for example, regarding customer relationship management (CRM) software. This shift is the result of evolving business models enabled by technology and infrastructure developments.

31

Thus, only about one-sixth of global software industry revenues (half of the one-third of revenues from products) are from product sales. See Michael A. Cusumano, 2008, “The Changing Software Business; Moving from Products to Services,” IEEE Computer 40(1):20-27.

32

See p. 22 in Michael A. Cusumano, 2008, “The Changing Software Business; Moving from Products to Services,” IEEE Computer 40(1):20-27. This may also reflect post-bubble consolidation in both technology- and media-focused companies.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

support for new software research. De facto consortia, often but not always manifest as open-source projects, are often the centers for industry-wide innovation.33

Changes in the DoD Innovation Environment

The question regarding the extent to which the DoD can meet its ongoing software needs through innovation-push from the commercial sector has to be answered in the context of three additional factors, which follow three significant shifts in the environment of technology innovation.

The first is the growing globalization of the software industry, as noted earlier, with rapid gains in capability in India and China (and elsewhere in Asia) and Russia as well as steady gains in Europe.34,35 This shift creates competitive pressures with respect to the rapidly increasing proportion of defense mission capability embodied in software. It also amplifies the challenge of mission assurance, given the increasing extent to which DoD software will likely be developed in foreign countries. This topic was taken up by the committee, and Chapter 4 addresses practice and technology issues related to software assurance, including both preventive measures in the engineering process and evaluative measures appropriate for development and for test and evaluation. Other dimensions of cybersecurity are consequential but not within the scope of this report—these were discussed at length in the 2007 DSB Task Force report Mission Impact of Foreign Influence on DoD Software.36

The second shift has been the reduction over the past decade of direct DoD investment in advancing software capability within the defense industrial base and its supply chain (see Box 1.5). Although this shift may be under reconsideration, it nonetheless raises a key question that was considered extensively by this committee, which is whether industry, without explicit R&D stimulus from the DoD, will produce innovations in areas of interest to the DoD of the kind and extent that are needed to meet the ongoing rapid growth in DoD software requirements.

The third shift is a consequence of the second, which is the reduction in PhD output due to the drop in R&D investment in software producibility. Historically, the DoD has had a significant leadership role in creating and sustaining the innovation advantage of the United States in IT and in fostering new generations of innovators and technical leaders in computer science and IT. This role has been evident from the earliest days of computing during World War II. That this DoD investment has been a significant driver of U.S. capability and innovation in information technologies is documented in several national studies.37,38,39 This has had the salutary benefit of enabling the United States to develop and retain innovation and ecosystems leadership. In areas related to software producibility, including high-

33

Examples in the realm of open source include Apache, Linux, Eclipse, and other widely adopted open source. In addition, formal consortia are often created to address industry-wide issues, as in the case of W3C (HTTP, HTML, XML, CSS, and other web-related standards) and TCG (TPM, trusted storage, and other trust-related standards for hardware). Finally, expert groups are often convened by standards organizations to address common issues, as in the case of JPEG (ISO and ITU) and MPEG (ISO).

34

See Michael A. Cusumano, 2006, “Where Does Russia Fit into the Global Software Industry?” Communications of the ACM 49(2):31-34. See also, Michael A. Cusumano, 2006, “Envisioning the Future of India’s Software Services Business,” Communications of the ACM 49(10):15-17.

35

In China, there are private and state-connected companies under government sponsorship to develop ecosystems and infrastructure software (China versions of CDMA/GSM, embedded operating systems, and search engines, for example) to reduce dependence on firms such as Qualcomm, Nokia, Microsoft, Google, etc.

36

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

37

NRC, 1997, Ada and Beyond: Software Policies for the Department of Defense, Washingon, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=5463. Last accessed August 20, 2010.

38

NRC, 2000, Making IT Better: Expanding Information Technology Research to Meet Society’s Needs, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=9829. Last accessed August 20, 2010.

39

NRC, 1997, The Evolution of Untethered Communications, Washington, DC: National Academies Press. Available online at http://www.nap.edu/catalog.php?record_id=5968. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

confidence systems, data from the NITRD coordination reports show that the DoD has reduced its basic research investment (see Box 1.5).

In addition to these shifts, there may also be cases where industry has little economic incentive to acknowledge fundamental gaps in knowledge. This is not so unusual. One obvious example is ill-structured contracts, often from the government, that create perverse incentives—for example, where project difficulties (or inadequate tooling and practices) may accrue to a contractor’s economic benefit in the form of increased contract costs and profits, along with long-term revenue streams resulting from costly post-deployment repair and maintenance requirements. A second example is the consequence of poor measurement capability (as noted above), particularly relating to quality and security—“assurance metrics” in the terminology of the 2007 DSB Task Force report.40 When metrics and observables are lacking, it is difficult to construct a business case for improvement of the underlying phenomena of concern—quality and security in this case. A third example is inadequate best practices. When metrics are weak, we must rely disproportionately on folklore-derived best practices and processes and organizational maturity to achieve product-related goals. Process compliance is relatively easier to achieve and certify than quality, but in software it is not always strongly correlated. As noted, fundamental improvements in best practices to enhance what can be achieved in terms of systems capability, productivity, quality, agility, and other characteristics sometimes fall into the category of non-appropriable innovations, discussed above, and thus may not readily be the subject of industry investment. These factors combine to lower industry economic incentives to address the producibility challenge.


Recommendation 1-1: The DoD, through its Director of Research & Engineering (DDR&E), should regularly undertake an identification of areas of technological need related to software producibility where the DoD has “leading demand” and where accelerated progress is needed to support the defense mission.

THE NECESSITY OF INNOVATION IN SOFTWARE

Is There a Need to Innovate?

That global suppliers are moving up the value chain41 suggests the possibility that U.S. leadership may be eclipsed in many of the core technologies related to systems architecture, languages and tools, and software assurance, as well as with respect to key design elements of software ecosystems and infrastructure. This suggests a key question, particularly as we contemplate the commitment of resources to new R&D activity: Is there, in fact, strategic value in retaining U.S. leadership in software producibility? The committee argues strongly in the affirmative, based on the unique role and technological characteristics of software.

The DSB Task Force report on foreign software also asks this question, focused on the particular area of software assurance, and offers an affirmative response, noting the essential requirement that the United States maintain advanced capability for “test and evaluation” of IT products. In other words, reputation-based or trust-based credentialing of software (“provenance”) needs more and more to be augmented by direct, artifact-focused means to support acceptance evaluation. The DSB Task Force recommends more effective direct evaluation by consuming organizations throughout the software supply chain, including better ways for producers to create software for which direct evidence of critical quality

40

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

41

This is in the sense of moving from more routine activities that require minimal technological sophistication (such as first-tier call centers, system-level “black-box” testing focused on the user experience, and similar activities) to more value-enhancing activities (such as custom software design and development). For software, this shift is enabled primarily through increased sophistication in technology and business practices.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

and functionality attributes can be provided. It concluded that test and evaluation should be supported by a broad range of software engineering technologies and interventions, not just those employed at the late test phase of development. (These issues are explored more extensively in Chapter 4.)

Both the 2007 DSB Task Force report on foreign software42 and the DSB 2000 report on defense software43 also highlight the importance of commercial technology to the DoD, including the essential elements (operating systems, databases, application servers, and so on) of most of the predominant software ecosystems architectures. DoD’s historical investment in basic research has influenced these commercial technologies in ways that have facilitated DoD adoption (for example, early attention to scalability, process separation, interconnection, and survivability in operating systems and distributed systems infrastructure). Without continued research investment, that influence will diminish just when the off-the-shelf technologies are rising in importance to DoD systems.

Additionally—and looking beyond software assurance into the other critical dimension of software producibility—without continued research investment, the DoD will lose effectiveness in its ability to undertake custom software engineering to rapidly achieve high levels of capability and to adapt with maximum agility to changes in the operating environment. A significant loss of U.S. leadership in either area could threaten the DoD’s ability to produce and assure the software it requires.

Historically, the DoD investment has been an enabler, both directly and indirectly, of U.S. technological leadership in software innovation (Box 1.6). This has helped enable the United States to be a leader in the development of software ecosystems, which means that the DoD is able to rely on ecosystem architectures and components with greater confidence. It has afforded a high level of capability in software producibility for the necessary custom software that the DoD must develop or acquire.

Will Software Capability Reach a Plateau?

Some have contended that software capability may be reaching a plateau, and as a consequence there is reduced need for leadership and innovation, because the technologies are inevitably commoditizing and the engineering focus is shifting to optimization and routinization. This suggestion is sometimes offered as an analogy with other technology disciplines, ranging from many specialized materials to display panels and memory chips. The committee views this contention as dangerously incorrect.

The reality is that software is not at a plateau, despite the fact that suggestions of the possibility are made on a regular basis. Consider, for example, the ambitious aspirations of those developing Fortran (Box 1.7). A similar story can be told about the “Fourth Generation” database languages introduced a few decades later and, more recently, about languages for business rules. These are all major innovations with far-reaching impact, and in all cases they delivered considerable value. But they also inspired computer users to greater ambitions, and thus the limits of these innovations were reached.

One of the most significant special characteristics of software, as noted at the outset of this report, is its unboundedness—the lack of natural physical limits on its scale and complexity. As the sophistication of languages, models, tools, and practices increases, the ambitions of computer users continue to be realized. These characteristics also mean that while these developments move us forward, they do not actually get us closer to “being there” at some plateau of capability and emerging commodity status. New software-manifest capabilities are constantly emerging—for example, machine-learning technology is now used in applications ranging from data mining to robot design and quality-of-life enhancement for seniors. The profound fact is that software seems to be limitless. For software, “continuous improvement” in capability (as distinct from process) is less a matter of fine-tuning than an innovation

42

DSB, September 2007, Report of the Defense Science Board Task Force on Mission Impact of Foreign Influence on DoD Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA473661. Last accessed August 20, 2010.

43

DSB, November 2000, Report of the Defense Science Board Task Force on Defense Software, Washington, DC: Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. Available online at http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA385923. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.6

Lessons About the Nature of Research in IT

The following material is reprinted from National Research Council, 2003, Innovation in Information Technology, National Academies Press, Washington, DC, pp. 2-4.

The Results of Research

  • America’s international leadership in IT—leadership that is vital to the nation—springs from a deep tradition of research….

  • The unanticipated results of research are often as important as the anticipated results….

  • The interaction of research ideas multiplies their impact—for example, concurrent research programs targeted at integrated circuit design, computer graphics, networking, and workstation-based computing strongly reinforced and amplified one another….

Research as a Partnership

  • The success of the IT research enterprise reflects a complex partnership among government, industry, and universities….

  • The federal government has had and will continue to have an essential role in sponsoring fundamental research in IT—largely university-based—because it does what industry does not and cannot do…. Industrial and governmental investments in research reflect different motivations, resulting in differences in style, focus, and time horizon….

  • Companies have little incentive to invest significantly in activities whose benefits will spread quickly to their rivals…. Fundamental research often falls into this category…. the vast majority of corporate R&D addresses product and process development….

  • Government funding for research has leveraged the effective decision making of visionary program managers and program office directors from the research community, empowering them to take risks in designing programs and selecting grantees…. Government sponsorship of research especially in universities also helps to develop the IT talent used by industry, universities, and other parts of the economy….

The Economic Payoff of Research

  • Past returns on federal investments in IT research have been extraordinary for both U.S. society and the U.S. economy…. The transformative effects of IT grow as innovations build on one another and as user know-how compounds. Priming that pump for tomorrow is today’s challenge.

  • When companies create products using the ideas and workforce that result from federally sponsored research, they repay the nation in jobs, tax revenues, productivity increases, and world leadership….

process that regularly delivers order-of-magnitude jumps in capability. The improvements are not just outcomes from the Moore’s Law curves and other exponentials in the underlying physical infrastructure of processors, memory, and networks. Those improvements are significant enablers, but our ability to produce increasingly complex software artifacts has largely been enabled by a steady pace of technological breakthroughs in software practices, technology, languages, models, and tooling. (The performance of software has been enabled by the development of new algorithms, by advances in hardware, and by simplification and rescoping of the computational problems being solved.)

Similar observations apply to software ecosystems. These are a critical and essential development to provide conventionalized access to infrastructure and capability on which systems are built. They are, in fact, the success of software reuse.44 As technical progress is made, these structures evolve, with increasing levels of infrastructure capability, for example in operating systems, databases, application servers, frameworks of various kinds, data center services, and so on. In a sense, this is analogous to a

44

Butler W. Lampson, 1999, “Software Components: Only the Giants Survive,” 21st International Conference on Software Engineering (ICSE’99), Keynote Address. Available online at http://research.microsoft.com/en-us/um/people/blampson/70-softwarecomponents/70-softwarecomponents.doc. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

BOX 1.7

Fortran Was a Breakthough but Did Not Lead to a “Software Plateau”

It is instructive to consider the publication more than 50 years ago of the 1958 landmark paper by John Backus describing the first Fortran compiler.1 The title made reference to “automatic programming,” and indeed this phrase was widely used at the time, including in the titles of technical conferences.2 The point of this phrase, with respect to Backus’s great accomplishment, is that there was a much more direct correspondence between his high-level programming notation—the earliest Fortran code—and pure mathematical thinking than had been the case with the early machine-level code. One can construe that it was imagined that Fortran enabled mathematicians to express their thoughts directly to computers, seemingly without the intervention of programmers. This was an extraordinary and historical breakthrough. But we know that, in the end, those mathematicians of 50 years ago soon evolved into programmers as a consequence of their growing ambitions for computing applications. Just a decade after the Backus paper, Fortran was used to support list-processing applications, typesetting applications, compilers for other languages, and other applications whose abstractions required some considerable programming sophistication (and representational gerrymandering) to be represented effectively as Fortran data structures—arrays and numeric values. (See further discussion in footnote 26 in Chapter 5.)

  

1 John W. Backus, November 1958, “Automatic Programming: Properties and Performance of FORTRAN Systems I and II,” in Proceedings of the Symposium on the Mechanisation of Thought Processes, Teddington, Middlesex, England: The National Physical Laboratory. Available online at http://archive.computerhistory.org/resources/text/Fortran/102663114.05.01.acc.pdf. Last accessed August 10, 2010.

  

2 An early example is the 1954 “Symposium on Automatic Programming for Digital Computers.” See John Backus, 1978, “The History of Fortran I, II, and III,” History of Programming Languages, New York: ACM.

process of commoditization, in that many of the key architectural interfaces effectively define market structures for competitive supply of capabilities.

But, at the same time, providers and clients continue to innovate above and around the infrastructure, creating new kinds of capability and differentiation. Additionally, there is very often continued innovation within the infrastructure to add capability, create differentiation, or make other enhancements. This is certainly evident in the case of relational databases, for which there is a conventionalized set of abstractions (the concepts associated with relational tables, indexes, etc.) and also some standards related to access to those abstractions (SQL and enhancements such as ODBC and JDBC to support queries across software interfaces). But it is also the case that the particular vendors may add specialized features to these standard interfaces to support new capabilities in response to the market. In other words, although there is a seemingly inevitable commoditization of software component capabilities, there is also a seemingly indefinite deferment of reaching the goal of fully predictable decision outcomes regarding innovative software-manifest capabilities in systems. This is a key point about the intrinsic lack of limits of software (except those we choose to impose, such as architecture—see Chapter 3), and indeed this is a principal characterizing feature of software as an engineering building material.


Finding 1-5: It is dangerous to conclude that we are reaching a plateau in capability and technology for software producibility. To avoid loss of leadership, the DoD will need to become more fully engaged in the innovative processes related to software producibility.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

The Continued Maturing of the Software Discipline Creates New Challenges

A consequence of this phenomenon of continuous capability improvement is the challenge of integrating innovative software development into mainstream systems engineering processes. Despite the pervasiveness of software and its pivotal role in systems and infrastructures, the engineering of software has not yet matured into a fully rigorous discipline. For innovative software projects, fully measurable and predictable process flows and outcomes—a hallmark of conventional process maturity—will necessarily remain elusive.

This is partly due to the fact that each new technical advance in software not only creates opportunities but also presents new difficulties of measurement and risk assessment. Engineering repeatability is achievable only for the more precedented systems. Additionally, a characteristic of many innovative projects is that the scope of the intended impact may be defined while specific details regarding functional and quality attributes emerge only in the course of development. Fully elaborated requirements against which predictions can be made often do not (and in many cases should not) exist.45 In other words, “predictability” may have more to do with success in addressing a need and less to do with how that need is specifically addressed.

Of course, this is true of nearly all other engineering disciplines. A key difference with software is that development of particular software functionalities, once routinized, is then quickly automated. The result is that expensive custom development gives way to much-lower-cost component procurement. In turn this often gives way to open-source availability of the same functionality. Moreover, unlike other areas of engineering, the intrinsic cost of replicating or deploying software artifacts is near zero. We can conclude that the effect is that a relatively much larger portion of the overall engineering effort in a software enterprise is devoted to creating specifically innovative functionalities. In other words, once development of specific software functionalities is routinized, the cost can vanish relatively quickly, which means more of the overall hands-on engineering activity is in the realm of the innovative and unprecedented.

Additionally, the risks and difficulties of software are growing in severity and diversity, and we continue to experience failures of all kinds—related to reliability, security, flexibility, and other attributes. Software-related problems are responsible for life-threatening failures in health devices, failures of space missions, failures in military systems, cascading failures in infrastructure for telecommunications and power utilities, and so on.

This may create a perception that there is an unavoidable trade-off between precedent and project risk, and that the only way to avoid major project completion risk is to compromise on ambitions regarding system capability. The committee’s analysis suggests that this is not necessarily the case, and in Chapter 2, the committee considers the means by which the engineering risks associated with innovative projects can be mitigated incrementally, thus potentially reducing the cost and project completion risk without overly compromising functionality.

Conclusions Regarding Software Innovation

The committee draws several conclusions from these observations regarding software capability improvement. First, mere presence in the market as a software user requires keeping pace with ongoing software innovation and improvements to practices. This is true even for individual software components—indeed, commercial software managers recognize that software starts to “die” (in the sense of becoming less valuable to users) the moment it stops evolving. It is also true for practices—continuous improvement in practices and processes is essential for survival. Desktop computers now are almost

45

See, for example, the description of the double helix design methodology in BG Harry Greene, USA, Larry Stotts, Ryan Paterson, and Janet Greenberg, January 2010, “Command Post of the Future: Successful Transition of a Science and Technology Initiative to a Program of Record,” Defense AR Journal, Defense Acquisition University. Available online at http://www.dau.mil/pubscats/PubsCats/AR%20Journal/arj53/Greene53.pdf. Last accessed August 20, 2010.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×

1 million times more powerful than those of 1980. The software capabilities have similar jumps, although these jumps are less apparent and not easy to measure.

Second, leadership in the market, as a producer or consumer, requires an active organizational role in defining the architecture of systems, and doing so as a first mover or fast follower. Software economics are focused on externalities—where the technical manifestation of the system structure is the software architecture and internal framework and component interfaces. This requires sustained technological leadership and clear thinking about the significance of architectural control. This is particularly significant in the definition and leadership of the design of the major ecosystems. Some of these are wholly controlled by commercial vendors, but others involve complex community processes.

Third, software technical challenges are broadening. These include, for example, software assurance, ultra-scale architecture, concurrency (multi-core and distributed), framework design, programming language improvements for assurance and scale, concepts for “big data” systems, and so on. These challenges are addressed in Chapters 2, 3, and 4 of this report.

Fourth, risk management models need to be continually adjusted to accommodate the new realities of software and of IT-enabled business practices, as noted above. This is the subject primarily of Chapters 2 and 4.

Finally, the role of software leadership in the global economy is growing, and this is increasingly recognized, with the result that global competition is becoming more intense at every level of capability. Overseas competition is greatly facilitated by the low barrier to entry—costly physical facilities are not needed in the software economy, but education and technical currency are fundamental and ongoing challenges. This is significant for the DoD, which has counted on U.S. industry, including defense contractors and their supply-chain participants, to sustain technological leadership in software as a key driver of capability leadership in systems. Such leadership must be maintained through constant investment in innovation and in people. At the highest level of technical sophistication, this requires investment in university research to produce a sufficient pipeline of technical leaders.46

46

U.S. PhD students in computer science and IT-related areas are almost universally supported with tuition and stipends covered by sponsored research. Universities rarely have funds to provide direct fellowships to PhD students in these areas, and few students have the resources or capacity to self-fund or to take on (often additional) loans to cover their costs.

Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 17
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 18
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 19
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 20
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 21
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 22
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 23
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 24
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 25
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 26
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 27
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 28
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 29
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 30
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 31
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 32
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 33
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 34
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 35
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 36
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 37
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 38
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 39
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 40
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 41
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 42
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 43
Suggested Citation:"1 Recognize the Pivotal Role of DoD Software Innovation." National Research Council. 2010. Critical Code: Software Producibility for Defense. Washington, DC: The National Academies Press. doi: 10.17226/12979.
×
Page 44
Next: 2 Accept Uncertainty: Attack Risks and Exploit Opportunities »
Critical Code: Software Producibility for Defense Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Critical Code contemplates Department of Defense (DoD) needs and priorities for software research and suggests a research agenda and related actions. Building on two prior books—Summary of a Workshop on Software Intensive Systems and Uncertainty at Scale and Preliminary Observations on DoD Software Research Needs and Priorities—the present volume assesses the nature of the national investment in software research and, in particular, considers ways to revitalize the knowledge base needed to design, produce, and employ software-intensive systems for tomorrow's defense needs.

Critical Code discusses four sets of questions:

  • To what extent is software capability significant for the DoD? Is it becoming more or less significant and strategic in systems development?
  • Will the advances in software producibility needed by the DoD emerge unaided from industry at a pace sufficient to meet evolving defense requirements?
  • What are the opportunities for the DoD to make more effective use of emerging technology to improve software capability and software producibility?
  • In which technology areas should the DoD invest in research to advance defense software capability and producibility?
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!