The Economics of Software: Technology, Processes, and Policy Issues
William J. Raduchel
Software is the medium through which information technology (IT) expresses itself on the economy. Software harnesses and directs the power of hardware, enabling modern data analysis and a variety of application domains. While software is becoming increasingly central to our productive lives, it is also becoming increasingly hard to create, increasingly hard to maintain, and increasingly hard to understand.
Software represents the core of most modern organizations, most products and most services. The U.S. military believes that the majority of cost in future weapons systems will be software. Software embodies the knowledge and practice by which the organization performs it mission and consists of a design and an implementation.
Software today may well be the largest single class of assets in the world, but it is intermingled with many other components, so distinguishing its contribution is not simple. So, for reasons discussed below, it is not measured that way. It is also undoubtedly the largest single class of liabilities. Operating, maintaining and creating software is surely the largest single class of expenses other than direct labor. Enterprise Resource Planning systems determine the way that factories run. Software and the business practices it enables are the biggest single driver of
productivity growth.1 According to Dale Jorgenson, IT accounted for more than half of the one percentage point increase in the Gross Domestic Product (GDP) growth rate between 1990-1995 and 1995-1999.2 And, as we learn more every week, software by far creates the greatest vulnerabilities to our economy.3 The primary driver of the August 14, 2003, northeastern U.S. blackout, which cut off electricity to 50 million people in eight states and Canada, was a deeply embedded software flaw in a General Electric energy management system.4 The U.S. economy is so dependent on software in ways that we currently do not understand.
Software drove the great bubble of the 1990s, which created more wealth than anything else in history.5 Software created millions of new high-paying
positions6 and redefined the way hundreds of millions of others do their jobs. Over the next decade, eight of the ten fastest growing occupations will be computer related, with software engineers comprising the fastest growing group.7 This prediction may be at risk, as we see software emerging as a key way India and China are upgrading their economies by outsourcing these jobs from the United States, where computer science continues to decline in popularity as a field of study.8 Immigration policy finds it hard to distinguish in favor of the truly talented developers, so firms now outsource whole projects instead of sponsoring a few immigrants.
There is currently no good economic model for software and, consequently, we have no good understanding of software. This paper is an attempt to start a search for what is a good economic model for software. This is an attempt to state the problem, in the belief that all good work begins with a correct problem statement. It does not attempt to answer the questions raised. It does distill over 40 years of personal experience creating, maintaining and using software and managing others doing these tasks from multiple perspectives over a varied career.
The future will only increase the importance of software to our society and economy. The costs of rendering, processing, storing, and transmitting digital information—whether data or pictures or voice or anything—continues to decline in all dimensions and likely will continue to do so for decades.9 We are not at the end of the computer revolution at all. No industry is safe from reengineering as cheaper computing enables ever more sophisticated software to redefine business practices.
A firm is an information system—one composed of highly decentralized computing by fallible agents called people. In many cases, the value of a firm is entirely tied up in the value of its software systems. Software defines the informa-
In the United States alone, more than 1.2 million jobs were created in the software sector from 1990-2001. Furthermore, “jobs in the software sector alone are only part of the contribution of software activities to total employment. In the United States, all employment in computer and related services accounted for only 1 percent of total employment in 1998 (including the public sector), whereas the share of computer-related occupations in the economy was around 2 percent. As a result, as much as 76 percent of computer-related occupations were in other industries.” Organisation for Economic Co-operation and Development, OECD Information Technology Outlook 2002: The Software Sector—Employment, Paris: Organisation for Economic Co-operation and Development, pg. 106-107.
“Fastest growing occupations, 2000-2010,” Office of Occupational Statistics and Employment Projections, U.S. Bureau of Labor Statistics, accessed at <http://www.bls.gov/news.release/ecopro.t06.htm>.
Michelle Kessler, “Computer Majors Down Amid Tech Bust,” USA Today, October 8, 2002.
Trends recently reviewed in the STEP Workshop on Deconstructing the Computer. See National Research Council, Deconstructing the Computer, Dale W. Jorgenson and Charles W. Wessner, eds., Washington, D.C.: National Academies Press, 2005.
tion system, and increasingly the value of a firm is embedded in its brands, its contracts and its systems, all intangible. Tangible assets increasingly are rented on an as-needed basis because software enables this. However, these software systems, the most key assets for companies, are not reported on balance sheets because to the accounting profession software is intangible and hard to measure. The result is that investors are left in the dark about a company’s single most valuable asset. I for one fail to understand how that is in the best interest of investors.
The growing importance of software can be seen in the way that it is redefining the way consumers communicate, entertain, and inform themselves. Voice telephony, music, and movies are all rapidly becoming just another application. The number one device for managing and playing music in the world is the personal computer, followed by devices like iPods that connect to it. The very definition of music has been changed in the past 5 years, and the music industry has to totally reinvent its business model as a result.10 Personal computers, mobile telephones, and consumer entertainment devices are morphing together in ways that will wrench the communications, entertainment, and publishing industries for decades to come. The size of the gaming industry, valued around $35 billion, is rapidly approaching the $38 billion music industry and has already surpassed the motion picture industry and continues to grow its hold on the leisure time worldwide.11 The growth of gaming is so large that eventually we will rewrite almost all software that interacts with consumers into a gaming paradigm.
Piracy is going to remain a recurring issue because the entertainment industry makes a profit by delivering content to consumers in ways that consumers do not want it. The reason that piracy exists is because consumers want music organized by a mix of tracks on a single playlist. The music industry has refused to provide music in this way because it does not suit any of their business models.
NATURE OF SOFTWARE
Software, on the one hand, looks remarkably easy and straightforward, and to some extent it is. The challenge is making software that is failure-free, hazard-free, robust against change, and capable of scaling reliably to very high volumes, while integrating seamlessly and reliably with many other software systems in real time. I recently was the Castle Lecturer on Computer Science at West Point
Academy and many of the cadets were pretty smug at finishing their first programming exercise of 50 to 100 lines. That example is writ large, as the problem with software is not creating a piece of code, but rather, creating one that will work well over time. The robustness of software is not only an issue of sloppy coding. Many software flaws are deeply rooted and substantive vulnerabilities can come from misunderstandings about the true requirements for a system, its environment of use, and other factors. The errors allowing most recent security breaches are relatively simple and easily eradicated.12
Real-world software is hundreds of millions of lines of code that make up the applications that run big companies. Those applications are resting on middleware and operating systems that, in turn, are tens of millions of lines of code. The complexity is astounding. The software stack on a modern personal computer is probably the most complex good ever created by man.13 The average corporate IT system is far more complicated than the Space Shuttles or the Apollo project, but engineered with none of the rigor or disciplines or cost of those programs. Bill Joy calls the spiraling complexity putting “Star Wars-scale” software on the desktop.14
The costs of maintaining and modifying software increase over time and increase with the amount of accumulated change because as modifications begin, increasing complexity is introduced and eventually, changes cannot be made at all. The biggest issues we face lie in how we administer and update our software, and we have a situation where software is sold without liability and where the markets are heavily dominated by one or two vendors. Competition is unlikely to work its usual magic to solve this problem.
Software has a useful life. Because the underlying technology changes, eventually the design needs to change as well. Modifications to the software make it harder to further modify. Seven years is a good starting point for software life, but few organizations build reserves to decommission or replace software. Equally few design in for this reality. We will begin to see this as all the systems created for the so-called “Year 2000 Problem” require replacement. U.S. productivity growth could fall off a cliff.
“There’s no other major item most of us own that is as confusing, unpredictable and unreliable as our personal computers.”15 It is not of course the computer—the hardware usually works well—it is the software. In general, anything really is possible with software. That is the problem. I have seen managers ask countless times if something can be done. The truthful answer is almost always
yes. The problem is with the question: Yes it can be done, but will the software work reliably at scale over time at what expense and at what impact on further changes and replacement? It is easy to be a hero by making quick and dirty changes that leave your successor with enormous problems that are untraceable back to the “hero.” Software is seldom managed well in my experience because the people managing it seldom have experience doing so.
Software is a stack composed of multiple layers of software created by different teams over time for different purposes and brought together for the unique task at hand. Individual elements of software do not matter—what matters is the entire stack. The stack of software runs the computer and begins with a small piece of code called the kernel. The kernel allocates resources on the computer among the several claims for those resources. Simple computers and all the early computers did not have a kernel because they ran only one program at a time. The next layer is the operating system, which includes the kernel and other basic software on which all other software operates and to which all other software is written. For desktop computers, the windows manager presents the display and implements the user interface. For Windows and Mac it is tightly bound to the operating system, but for UNIX variants and Linux it is not. The window manager today can be larger and more complicated than the operating system. The next layer, middleware, hides the window manager and operating system from the application and provides a higher level of abstraction in which to program. Finally, there are the applications, which perform a useful purpose and consist of business logic, a user interface, and interactions with other systems.
When something goes right or something goes wrong with a computer, it is the entire software stack that is operating. If one piece of the hundred million lines of code is not working correctly, it affects the entire stack, so one of the biggest challenges is to view the stack as a whole, rather than the individual components. Except for some embedded systems, all running software stacks are unique. No two are alike except by pure, random chance.
Researchers have known for years that there are powerful heuristics that play on the software development process. The very best software developers are orders of magnitude better than the average, and the software stack leverages these wizards so that mortals can write effective software. There are only a very limited number of software developers worldwide at this high level that end up writing the basic kernel of the software we use.
The late Maurice Halstead in his book, Elements of Software Science,16 explored at length how one attribute of the human brain, modestly called Halstead
length, drove programming ability.17 No one can solve a problem he or she cannot understand, and Halstead length defines how complicated a problem a person can understand. He found an average of about 250, which in his metrics was about one page of FORTRAN, but wizard programmers appear to have Halstead lengths over 65,000.
Any software stack is defined not only by its specifications, but also by its embedded errors and by its undocumented features. Many software developers are forced to build new software that is “bug-for-bug” compatible, as preexisting software bugs must be recognized and replicated for the new software to work. Even the best software contains errors and “even experienced engineers on average will inject a defect every nine to ten lines of code.”
Applications knowledge converts business rules and practice into algorithms and processes that can be implemented in software. People who do this must understand both the application as well as how to create software. Gaining and sustaining this application knowledge, therefore, can be a monumental challenge. Many large organizations today, for example, face a challenge in re-implementing their original software because many of the software developers who originally wrote the software have retired. Since rules for doing business are encoded into software, a firm may find that it no longer understands the details of how it operates.
Systems knowledge is the ability to create a working system to operate the software reliably and at scale. “Computer” knowledge is the ability to make the movement of ones and zeros a system. Few people ever have all this knowledge, so software is a uniquely team activity. At the same time, this also means that just adding additional software developers will not necessarily create the complementarities needed to combine both systems knowledge with computer knowledge successfully. Making the task of writing applications software uniquely complicated is the reality that software is a stack composed of multiple layers of software, usually created by different teams over times for different purposes and brought together for the unique task at hand.
The way the industry has dealt with the complexity of software problems in the past 30 years is by introducing increasing levels of abstraction to the software
An annotated bibliography of the psychology of programming compiled by Tim Matteson can be accessed at <http://www.cise.ufl.edu/research/ParallelPatterns/PatternLanguage/Background/Psychology/Psych-bibliography.htm>.
stack as you move up it. The increasing abstraction allows for easier portability across systems and expands the pool of people able to create software. The higher the level of abstraction generally the less efficient is the software; much of the gain in computing power historically has been used to create additional abstraction levels.
The labor to create each of the layers in the software stack is highly specialized (except that application builders usually also handle the user interface and the application error handling) and becomes increasingly rare as you go down the software stack. The reason that developers introduce abstraction is to allow for an increase in the number of workers that can work on a particular software project. While millions can write applications in Microsoft Visual Basic, the global labor pool that can write at the operating system kernel level is measured in the hundreds.
The major costs in making a system operational are in configuration, testing, and tuning, a process inherently messy and difficult to manage. Packaged software almost never represents more than 5 percent of the total project cost.18 While this 5 percent is tracked in the budget, 95 percent is expensed to G&A or divided into fractions and into different expenses that make the project extremely difficult to measure. This process also takes many more people (1 designer, 10 coders, and 100 testers would be a good starting point), but managers tend to allow too few resources and time for this step. The vast majority of needed resources for any corporate information system are non-system. Furthermore, when software is tested, the complete software stack is being tested and errors anywhere may show up for the first time.
Following the initial configuration, test and tune, any system has to be maintained over time. The lower portions of the software stack change almost daily, and so-called “patch management” becomes a significant task. Most readers see this process in something called Microsoft Windows Update. There are costs just in managing the computers and networks that operate the system. The trade-offs between operation and maintenance and development and replacement are huge, with many opportunities to save near-term expense for long-term costs. Thus, from the engineering perspective, the immense difficulty of measuring or predicting “flexibility” or “maintainability” or “reusability” during development means that development managers are often unable to steer a prudent course between over-investment, which creates unused flexibility and slows down development,
and under-investment, which can lead to unhappy “time-bomb” failures when small changes are made during maintenance.
SOFTWARE IS OFTEN MISCAST
Because of the lack of understanding about software discussed above, software is often miscast. Most of the existing models account for software in commercial and national accounts in ways that rely on flawed analogies to machines. Forty years ago, software was given away for free, but it has now evolved from a minor part of the value to nearly all the value in the system and value is often totally unrelated to cost.
To begin to understand the economics of software, we must understand that software affects production not as a factor of production like labor or capital, although software professionals and computing resources may be, but by defining the production function. While in general software improves the efficiency of the production function (i.e., you can produce more from the same inputs), it does so by increasing complexity, creating a probabilistic risk of catastrophic loss and raising the cost and risk of further change.
Software takes on economic value in the context of a system. In the case of desktop computers, lower levels of the software stack are bought and sold in the market and can be valued by their market price, although the natural monopoly aspects of these layers induces distortion. Because software is a large business, these metrics are interesting, but they do not capture at all the overall productivity implications for the economy.
Economists are not alone at all in miscasting software. The accounting and legal professions have both cast software in ways that fit their preexisting mental models. There is nothing malicious here at all by anyone. Software remains a mysterious black box to nearly all, and those who do understand its black arts are often enigmatic, unusual, even difficult people. Software is a unique human activity in which knowledge is congealed and given external form processable by both humans and machines, and it is completely dependent on a small labor pool of highly-talented designers and subject to little-understood but very powerful constraints. This knowledge must be, but never is, represented exactly, and errors cause damage but are today inevitable.
Fine software is like fine wine: there is no known way to produce error-free software except time. IBM mainframes now often run years without failure, but that software is more than 30 years old. The man who led the effort to create the operating system for IBM mainframes was Fred Brooks, and his book on software engineering, The Mythical Man Month,19 remains seminal today. Among
his key tenets was the Plan to Throw One Away rule: the only way to design a system is to build it. In my personal experience, the wisdom of this adage is manifestly obvious, but not to most executives who see it as wasteful.
The only complete specification for any software stack is the stack itself, so the only way to design a system is to build it. Software designers understand that a specification without a reference implementation is meaningless. This is why compatibility and portability have been so elusive. The published specifications for Java 2 Enterprise Edition, the standard today for building corporate information systems, measure slightly over a meter high.
ECONOMICS OF SOFTWARE
Because of a lack of a good economic model of software, measurement remains enigmatic. At the root, the declining costs of silicon have enabled massive change, but it is software that converts this change into productivity and quality improvements. Unfortunately, we manage and account for software largely as a period expense, when in fact it is a balance sheet transaction adding both assets and liabilities. The real data needed to understand what is happening simply do not exist in firm records.
Some of the complications of measuring the value of software start with the fact that most people think of software as the programming phase while, in reality, programming costs are usually less than 10 percent of the total system cost. The systems design is an intangible asset whose value grows and shrinks with the operations and prospects for the business. The implementation is an intangible asset of potentially enormous value, as finding even one way to make a design work reliably at scale is not assured, but it also is a liability, for it must be maintained and evolved as the business changes over time.20
The costs of maintaining and modifying commercial software increase over time and increase with the amount of accumulated change.21 Change is required by the rapid change in the foundation platform as well as by the evolution of the environment within which the software operates, and the requirements and needs it fulfills. As mentioned above, software on average has a useful life of 7 to 10 years, after which time it must be replaced. This comes from two forces: accumulated change and platform evolution.
Every software veteran understands that a manager can choose only two of schedule, cost, and features, where features includes quality and performance. In
turn, quality, performance and capabilities all trade off. The most important way to cut costs and time and increase features is to improve the quality of the developers (which can be done to some degree by increasing the quality and power of their tools). Adding more people is the surest way to make a late software project later. An old adage is that software is 90 percent complete for 90 percent of the development cycle.
Putnam studied these tradeoffs at GE decades ago and found that halving the schedule required a 16-fold increase in total staffing, everything else being the same.22 He also found that increasing complexity had massive impact on total cost: 10 percent increase in complexity required a threefold increase in the number of developers working simultaneously. Software design is at least 90 percent of whether software is good or bad, and the quality of the designer is at least 90 percent of the quality of the design.
PUBLIC POLICY ISSUES
Although, to my knowledge at least, no major politician has yet been asked for a software agenda, public policy issues abound:
Are we as a nation investing adequately in the systems that improve productivity for the economy? Indeed, how much are we investing in such systems and how is that amount trending? Does the United States face a productivity cliff in the next few years when the systems installed for the year 2000 age and prove difficult to replace or upgrade? What public policies encourage the creation of such productivity-enhancing systems? No data currently exist to answer these questions.
Do public corporations properly report their investments in software and their resulting expenses? Can investors accurately understand corporate performance and outlooks without better knowledge of the software systems that operate modern corporations and products? Telecommunications companies’ main challenge today is implementing billing systems whose software can effectively communicate with other systems.
Do traditional public policies on competition work when applied to software-related industries? Is software really so different that traditional economic regulations and remedies no longer work? Software changes so fast that by the time traditional public policies work, the issues have already changed.
Do we educate properly in our schools and universities given the current and growing importance of software? What should an educated person
know about software? Corporate reporting and control systems are software systems. What should the professionals charged with operating and inspecting those systems know about software?
What should be our policy on software and business methods patents? Recently, the European Union adopted a fundamentally different course from the United States.23 One recurring issue in the United States is that new patents are coming out on ideas from 30 years ago, but the prior art is not readily available.
What is the proper level of security for public and private systems? How do we achieve that? What are the externalities of security? Are special incentives required? There is a proposal in some U.S. government agencies that would require companies to certify in their public reporting the security level of all their key systems, particularly those that are part of the critical infrastructure. These companies would then have to take responsibility for the security of these systems. The worms that have been circulating lately have the potential to infiltrate home computers and create an unstoppable attack on vital infrastructure as 10-20 million broadband connected homes would be simultaneously attacking infrastructure.
What is happening to software jobs? Should our policies change? What are the implications of the apparent migration to India and China? Is the United States simply getting lower-cost labor that will enable further advancements in the United States or are these jobs the tip of the value chain that will eventually make sustaining a software system difficult?
What export controls make sense for software? Most software is dual use in the sense that it can be used for military and non-military purposes.
Should we encourage or enable open-source software? Is this the right economic model given the nature of talented software people? What is the proper legal environment for open-source software? Should governments mandate its use on the fundamental principle that the data of the people cannot be captive to any proprietary vendor ever, a principle increasingly voiced and occasionally enacted, most recently by Munich?24 Is open source really the best economic model for software? Is open-source software more or less reliable and secure? Source code, instead of being the most valuable part of software, is the least valuable part of software and giving it away may be a good strategy to promote new innovation.
Should strict liability apply at all to software? Should software professionals be licensed? Software traditionally has been sold in the United States on an as-is basis, although that is now being tested in the courts. While process, programming language, frameworks, architecture, tools and infrastructure have all evolved considerably in the last 5 years, there is still no fundamentally new way of writing software. Bill Joy has highlighted this issue by observing that almost all software today is created using variants of what he would call antique technologies. 25 If the U.S. could find a new to way to write software, it would have an enormous positive impact on the value equation for the U.S. economy.
How do we get the right investment into fundamental software technology? We have little visibility into how much we are investing.
Although this list is not exhaustive by any means, it is apparent that the public policy agenda is rich. Software pervades our economy and our society and should, not surprisingly, also pervade our policy agenda. As we move forward, the awareness of software’s importance must be raised and a dialogue must be started in the hopes of better understanding the economics of software.
APPENDIX: THE SOFTWARE PROCESS
Designing the software and specifying its architecture is the first and most important task.
Rule one is to hire great people and make one of them the chief architect. Everyone else has to play a supporting role. Paranoia is a desirable attribute for the chief architect. Experience proves that the only people capable of hiring well are great software developers themselves.
Rule two is to encapsulate complexity as much as possible. The costs of spreading complexity are enormous. This explains why changing software in ways not anticipated by the design is so costly because doing so inherently spreads complexity. Complexity is viral.
Rule three is to write at the highest abstraction layer possible.
Rule four is to use the best tools you can find. This is harder than it seems. Study after study showed that writing Java code could be up to ten times more efficient than writing C code,26 but many developers did not like the constraints
that Java imposed. In addition, the cost-plus economics of software development sometimes take away the productivity gains provided by good tools.
Properly encapsulating complexity can greatly reduce the cost and increase the life of software. This is why object-oriented programming and modern programming languages such as Java are so important.27 An error should affect only the item with the error: memory leaks being the counterexample. When your personal computer suddenly and inexplicably ceases to work, the vast majority of the time you are probably the victim of a memory leak: some application somewhere, maybe one you did not even know was running, failed to manage its use of system memory properly leading to system failure.28
The phase most people think of as software is the implementation or purchase of programming. A bad implementation of a great design is probably mediocre software, but a great implementation of a bad design is probably miserable software. The costs here are usually less than 10 percent of the total system cost. “Technical feasibility assured,” the accounting standard for capitalization, occurs in this phase.29
This phase often appears to be mechanical, like constructing a house. I was once told that my economics colleagues considered software to be like the glass blower was to the chemist: that is, of no intellectual value. Industry slang often refers to something as SMOP, a simple matter of programming, but programming in general is never simple. The phrase in fact is meant as a pun.
It is usually unclear when design ends and implementation begins. The worse the software the more unclear you can be sure it was when it was created. More importantly, it is also unclear when implementation ends. Software always has errors. Developers do not measure absolute errors. Instead, they measure the rate at which new errors are discovered.
Brooks, Frederick. 1975. The Mythical Man Month: Essays on Software Engineering. Reading, MA: Addison-Wesley Publishing Company.
Computerworld. 1999. “Generally Accepted Accounting Principles.” November 29.
Executive Office of the President. 2003. The National Strategy to Secure Cyberspace: Cyberspace Threats and Vulnerabilities. February.
Hagel, J. and A.G. Armstrong. 1997. Net Gain. Cambridge, MA: Harvard Business School Press.
Halstead, Maurice H. 1977. Elements of Software Science. New York: Elsevier North Holland.
Jorgenson, Dale W. 2002. “The Promise of Growth in the Information Age.” The Conference Board, Annual Essay.
Kessler, Michelle. 2002. “Computer Majors Down Amid Tech Bust.” USA Today October 8.
Lam, Monica. 2003. Private communication. November 7.
Maney, Kevin. 2003. “Music Industry Doesn’t Know What Else To Do As It Lashes Out at File-sharing.” USA Today September 9.
Meller, Paul. 2003. “European Parliament Votes to Limit Software Patents.” New York Times September 25.
Mossberg, Walter. 2004. “Mossberg’s Mailbox.” Wall Street Journal June 24.
National Research Council. 2005. Deconstructing the Computer. Dale W. Jorgenson and Charles W. Wessner, eds. Washington, D.C.: The National Academies Press.
Oliner, Stephen and Daniel Sichel. 2000. “The Resurgence of Growth in the Late 1990s: Is Information Technology the Story?” Journal of Economic Perspectives 14(4) Fall.
Organisation for Economic Co-operation and Development. 2002. Information Technology Outlook 2002—The Software Sector. Paris: Organisation for Economic Co-operation and Development.
Organisation for Economic Co-operation and Development. 2002. Strengthening the Knowledge-based Economy. Paris: Organisation for Economic Co-operation and Development.
Organisation for Economic Co-operation and Development. 2003. ICT and Economic Growth: Evidence from OECD Countries, Industries and Firms. Paris: Organisation for Economic Co-operation and Development.
Poulsen, Kevin. 2004. “Software Bug Contributed to Blackout.” Security Focus February 11.
Pritchard, Stephen. 2003. “Munich Makes the Move.” Financial Times October 15.
Schlender, Brent. 1999. “The Edison of the Internet.” Fortune February 15.
Shuster, Loren. 2002. “Global Gaming Industry Now a Whopping $35 Billion Market.” Compiler July 2002.