Strategic Management of Information and Communication Technology: The United States Air Force Experience with Y2K (2007)

Y2K was a massive meta-experiment that touched on the core processes of the new digital millennium that was dawning. It offers us valuable perspectives on the nature of software, our vulnerabilities in a computer-dependent world, the future evolution of information technology, and the relationship of these with people and organizations. If Y2K was a threat to the trustworthiness of critical infrastructure, what lessons from Y2K are relevant to other threats, such as blackouts, terrorism, or software reliability? Can the Air Force’s Y2K experience help us understand those vulnerabilities and better appreciate their differences and the effectiveness of potential responses? Going even further, does the Y2K experience provide lessons that will better enable us to take advantage of the increased capabilities of networked information and communication systems while minimizing the inherent risks in this increasingly connected world?

The “decision” (however complex its evolution) to represent calendar years with two digits was human and organizational, not technical—just as the mismatch between metric and English measurements that destroyed the Mars Climate Orbiter in 1999 was a human and organizational error, not a technical or a mathematical one (or a terrorist attack). Thus, a key perspective reinforced by this study is that technology is socially embedded. It exists in the context of people and organizations. Ineffective organizations with great technology will usually produce ineffective results, while effective organizations with less than state-of-the-art technology can get by just fine.

As in many other organizations, Y2K instigated the Air Force’s first enterprise-wide, formal effort to integrate IT management with organizational missions and functions. This large-scale alignment of operational and strategic management (see Section 2.4) reinforced the importance of recognizing the social nature of technology. For example, continuity planning was an element of the buildup to Y2K and is an essential part of preparations for natural disasters, terrorist attacks, and other threats and disruptive events. This planning requires deciding what functions are vital and identifying who depends most crucially on what systems, decisions that necessarily involve social, organizational, and political issues.

History repeatedly shows us the necessity of incorporating a human, social, and organizational perspective on technology security and reliability. For example, even a mathematically perfect encryption system (the “one-time pad”) is vulnerable to the human element when people decide to reuse pages. (Benson). Similarly, the widespread electricity grid failure in August 2003 has been attributed in part to an analyst who fixed a data error for an automated tool assessing the health of the grid, then forgot to reset it to run automatically and left for lunch (U.S.-Canada Power System Outage Task Force). This set of conclusions is actually a point of view: large-scale complex IT systems must be viewed through the lens of a social system, giving priority to management and other “people” issues.

The Air Force’s Y2K experience teaches us about software as a social system. It highlights the limitations and pathologies that typically grow out of social organization, training, and group complexity. It also illustrates that some technical approaches are better adapted than others to the social systems currently in use. Y2K thus enables us to ask what alternative approaches to software development, deployment, maintenance,

testing, and security may be more successful, recognizing the enduring impact of people on technical systems.

This report rejects the idea that the Y2K problem was simply one of fixing the technology, recognizing that it was driven instead by a concatenation of institutional, leadership, economic, and political factors, as well as technical ones. As the introduction to Chapter 2 observes, “the problem… taught those who worked on it more about their overall organizational operation than about their technology.”

A key organizational issue identified by this report is that no single unit “owned” the problem, that “no single group could fully control the issues. Enterprise-wide perspectives…had to be considered.” This meant, for example, that “efforts to decompose the Y2K problem and organizational responses into discrete components were largely unsuccessful.”¹ Cross-unit interdependencies were the single biggest challenge to remediation.

As the report observes, these problems were not unique to Y2K. They just became more obvious under its intense spotlight. We should therefore understand the Air Force’s Y2K experience not as a freestanding phenomenon but as typical of large-scale software systems embedded in a complex institutional setting. As the Air Force gradually discovered (see Section 2.3), this meant shifting the focus from hardware and software to organizational issues.

The extent to which remediation efforts were successful can be attributed in part to a shift in perspective—an evolution from techno-determinism to a broader social understanding. A “technical” problem like Y2K may initially be seen as sui generis, to be described only in its own terms. Its effects are seen as direct, widespread, and determinative of other social outcomes. Eventually, these initial erroneous enthusiasms are subdued and put into perspective when traditional paradigms of analysis in the social sciences, engineering, or even the humanities reveal this “unique” thing to be a member of broader social categories with their own determinants and well-known laws of motion.

This general trajectory has been true for the Information Revolution as a whole.² The arc of understanding starts with an unhealthy dose of techno-determinism. As the report notes, “Over the course of Y2K it became clear that changes made to hardware and software generally did not address the central Y2K … issues.”³ The shift to a broader contextual focus on knowledge, management, and institutions, away from a more narrow hardware and software focus, is imperative for successful action, whether in Y2K or beyond. This experience underscores that more research is needed to understand and support individuals and organizations shifting from a techno-perspective to a strategic and managerial one.

Related examples from the safety field reinforce the hazards of treating large technological systems as consisting purely of technology. Consider the most notorious technology-heavy accidents of the past quarter-century: Three Mile Island and Chernobyl, the Challenger and Columbia shuttles, the USS Vincennes, and Bhopal. In

each case the accidents occurred despite the presence of sophisticated safety systems and devices, whose effectiveness in each case was cancelled out by social, managerial, and organizational issues. Similarly, computer-based automation of key safety tasks can paradoxically increase the risk of failure. An expert system introduced for aircraft maintenance at a leading airline saw a rise in mechanical problems, apparently because maintenance staff came to depend more on the system and less on their own experience, powers of observation, and personal initiative. When the software was changed to provide just information, not decisions—and even that only on request—quality again rose (Leveson).

The inextricably intertwined nature of software and organizational issues is not new, yet it is still not well understood, even after many decades. Thomas Hughes describes the development of the first large-scale real-time general purpose digital computer (SAGE, the first machine that we would recognize today as being a computer at all), observing that “system builders in the 1950s were learning that the managerial problems of large-scale projects loomed as large as engineering ones.” Even in this early system, software development and project management became a prominent issue, as the number of programmers grew from a handful to more than 800, and as the programming group at the RAND almost outnumbered all other RAND employees. One striking recollection that Hughes relates is that “When we all began to work on SAGE, we believed our own myths about software—that one can do anything with software on a general-purpose computer, that software is easy to write, test, and maintain. …We had a lot to learn” (Bennington, as quoted in Hughes).

Cross-organizational issues during real-time operations played a key role in the buildup to the August 2003 blackout (see Section 3.7). Several IT problems triggered automated pages to the IT staff at the key Ohio utility (FirstEnergy). While the staff responded and thought they had restored full functionality, there was no communication between the IT staff and the control room operators. The IT personnel therefore did not know that some functionality had been restored to the immediately previous defective state, and control room personnel did not know that they were relying on out-of-date data or frozen systems (U.S.-Canada Power System Outage Task Force).

Y2K reminded organizations that the ultimate goal of IT is not the continued functioning of local clusters of technology but, rather, the effective use of information in support of strategic goals. Why does this horrific gap continue to persist, even as Y2K is viewed as a temporary blip? Will the lessons learned from Y2K have lasting effects (see Sec 2.7)? A critical issue was that temporary organizations and money were used to guide the Y2K effort, and as a result, no permanent homes were established for the arduously developed policy and practice. Nevertheless, there has been a historical trend in the direction of better alignment between mission objectives and IT, and Y2K helped this transition. The CIO offices that grew out of Y2K have expanded in their scope and mission, while new data standards make it far easier to share information (that is, data integration instead of application integration, as discussed in Section 2.3). SOAP, XML, message-oriented middleware, Enterprise JavaBeans, and similar standards focus on mission-critical data, both technically and managerially, helping to address integration and organizational issues. Still, considerable follow-up research is needed.

Y2K tells us that the Air Force, or any other IT-dependent organization, can mitigate risk by becoming more process based and less technology based. Process-based

tools focus on training people to do their jobs by providing procedures that mitigates risks. While the procedures are adopted at the enterprise-wide level, they can guide the creation of more specific, site-variant processes and desktop procedures that would mitigate the risk that uniformity might introduce. (As described earlier by an information warfare defense officer,⁴ relying on one system could be a risk in itself.) If the right metrics are selected, this approach can also enable better tracking of the health of the organization as well as the IT infrastructure.. However, this approach must consider organizational and technology changes and the ongoing need to modify procedures and to continue learning.

The perspective that “built” or self-consciously designed systems need to fit harmoniously with their full context is not new (Alexander). Its transition from traditional fields such as architecture into supposedly modern fields like software engineering has been inconsistent: ironically, the more technologically sophisticated endeavor is undertaken with a more sociologically naive approach. The Air Force Y2K experience can be compared with current business writing on the important “people questions” to consider when executing a big software project. Representative of this category is Tom Demarco and Timothy Lister’s Peopleware: Productive Projects and Teams (1999). The primary thesis in Peopleware is expressed as: “The major problems of [implementing large-scale IT systems] are not so much technological as sociological in nature.” Demarco and his team have been studying failed software development since 1979, and “for the overwhelming majority of the bankrupt projects we studied, there was not a single technological issue to explain the failure.” Rather, managerial, organizational, and other “people” issues are usually the underlying cause (Demarco and Lister).⁵

Clearly, the issues described in this report apply well beyond Y2K. Anyone who thinks they can carry out an IT project without thinking about organizational and social systems is heading for failure. As this report shows, it is essential to ask these questions, among others:

• What are the relationships with management and other organizations?

• What incentives are people responding to as the software is developed, deployed, and maintained?

• What are the different skill levels, needs, and assumptions of users, implementers, and supervisors?

• Who decides what’s important?

• Who decides what gets done—when, and how?

• Who gets to contest those decisions?

• Who controls the resources needed to get it done?

• Who else is competing for those same resources?

These are the same questions facing a company planning to launch a new product, a local government planning an airport expansion, or a federal government planning a national incident management system.

Certainly there is a need to understand specialized design and implementation issues and a need for skilled workers and expensive tools. But any IT-dependent organization also needs to understand its users (whether war fighters, customers, or random people impacted by a disaster) and to choose the right executives and management structure. They need to get all the relevant stakeholders on board before starting a major project. Just as a real estate developer needs to know what kinds of newly built communities will attract home buyers and sustain property values before they can succeed in the technical task of building houses, so the Air Force, faced with the Y2K threat, needed to look at its information and communication systems from the perspective of their use and evolution in an organizational context.

The Y2K experience helped introduce the Air Force and other technology-based organizations to a human, organizational, and social perspective on technology risk. The degree to which these organizations understand its repercussions and choose to act on that understanding is a key question for the future.

This page intentionally left blank.