The United States—indeed much of the world—is in the midst of a great transformation wrought by information technology (IT). Fueled by continuing advances in computing and networking capabilities, IT has moved out of the laboratories and back rooms of large organizations and now touches people everywhere. The indicators are almost pedestrian: computing and communications devices have entered the mass market, and the language of the Internet has become part of the business and popular vernacular. These changes are often considered to be the outcomes of technology development—the second half of the familiar term “research and development” (R&D)—whose role is to create specific IT systems and products. What is sometimes overlooked is the critical role of the first half of the R&D process: the research that uncovers underlying principles, fundamental knowledge, and key concepts that fuel the development of numerous products, processes, and services. Research has been an important enabler of IT innovations—from the graphical user interface to the Internet itself—and it will continue to enable the more capable systems of the future, the forms of which have yet to be determined. It has another role as well: in universities especially, it serves to educate and to build a knowledgeable IT workforce.
The future of IT, and of the society it increasingly powers, depends on continued investments in research. Despite the incredible progress made over the past five decades, IT is anything but a mature, stable technology. Revolutionary new technologies based on quantum physics, molecular chemistry, and biological processes are being examined as replacements
for or complements to the silicon-based chips that perform basic computing functions. Computing and communications capabilities are being embedded in a widening range of existing and novel devices, presaging an age of ubiquitous or pervasive computing, when IT is absorbed almost invisibly into the world around us. IT systems are being deployed to support countless tasks, from monitoring the health of patients with chronic diseases to controlling the flight paths of aircraft to analyzing mountains of data for private corporations and government agencies. Yet, the potential of IT will not be harnessed to meet society's needs automatically; it is not simply a matter of producing IT products and distributing them more widely. Research is needed to enable progress along all these fronts and to ensure that IT systems can operate dependably and reliably, meeting the needs of society and complementing the capabilities of their users. The question becomes, Can the nation's research establishment generate the advances that will enable tomorrow's IT systems? Are the right kinds of research being conducted? Is there sufficient funding for the needed research? And are the existing structures for funding and conducting research appropriate to the challenges IT researchers must address?
This report by the Committee on Information Technology Research in a Competitive World, convened by the Computer Science and Telecommunications Board (CSTB) of the National Research Council, attempts to answer these questions. It examines the overall funding levels for IT research from industry and government, the scope of ongoing research efforts, and the structures and mechanisms that support research. It advances the argument that the nation's needs for IT systems have changed in ways that demand a much broader agenda for such research—one that includes more explicit support for research on large-scale IT systems and the social applications they support (see Box ES.1)—and mechanisms for funding and conducting research that are better attuned to this broadened agenda. The report was written with an awareness of the legacy of reports about IT research and recognizes that some of the research it covers is not new. What distinguishes this report is that it considers the big picture emerging from research programs that have been cataloged and recommended in other reports and uses this perspective to assess the sufficiency of today's research efforts. The report recognizes that long-standing problems cannot be solved instantly, and it acknowledges the institutional, cultural, and resource factors that will make the recommended changes difficult to achieve. But after lengthy analysis and deliberation, the authoring committee concluded, with conviction, that a reorientation of IT research is vital to the well-being of the technology base.
Defining Large-Scale Systems and Social Applications of IT
Large-scale systems are IT systems that contain many (thousands, millions, billions, or trillions or more) interacting hardware and software components. They tend to be heterogeneous—in that they are composed of many different types of components—and highly complex because the interactions among the components are numerous, varied, and complicated. They also tend to span multiple organizations (or elements of organizations) and have changing configurations. Over time, the largest IT systems have become ever larger and more complex, and, at any given point in time, systems of a certain scale and complexity are not feasible or economical to design with existing methodologies.
Social applications of IT serve groups of people in shared activities. The most straightforward of these applications improve the effectiveness of geographically dispersed groups of people who are collaborating on some task in a shared context. More sophisticated applications may support the operations of a business or the functioning of an entire economy; systems for e-commerce are an example. Characteristic of social applications of IT is the embedding of IT into a large organizational or social system to form a “sociotechnical” system in which people and technology interact to achieve a common purpose—even if that purpose is not obviously social, such as efficient operation of a manufacturing line (which is a conjunction of technological automation and human workers) or rapid and decisive battlefield management (which is a conjunction of command-and-control technology and the judgment and expertise of commanders). Social applications of IT—especially those supporting organizational and societal missions—tend to be large-scale and complex, mixing technical and nontechnical design and operational elements and involving often-difficult social and policy issues such as those related to privacy and access.
TOWARD AN EXPANDED RESEARCH AGENDA
Overall, the nation's IT research base appears to be thriving. Federal funding for IT research rose steadily throughout the 1990s, from approximately $1.4 billion in 1990 to $2.0 billion in 1998 (the most recent year for which consistent data are available),1 and the Clinton Administration's budget for fiscal year 2001 proposes to increase funding for IT R&D by $1 billion above 1999 levels. Industrial support for R&D also appears to be increasing dramatically. The combined R&D expenditures of companies in the six industry sectors most closely associated with IT totaled $52 billion in 1998, of which approximately $14 billion was classified as research.2 These figures compare to $39 billion and $8.5 billion, respectively, in 1995.3 Over the past decade, a number of large IT firms, including Microsoft Corporation, Motorola, Inc., and Intel Corporation, have
established central research laboratories, signaling their increased commitment to long-term research.
Nevertheless, current investments in IT research are insufficient to support an important expansion of the IT research agenda. Work needs to continue in ongoing areas of research, but society's growing reliance on IT also demands greater attention to problems associated with the design, deployment, and operation of large-scale systems and social applications. The proliferation of the Internet has greatly accelerated the deployment of large-scale IT systems to serve a variety of personal, social, and business needs. Yet large-scale system efforts in both government and industry are often characterized by cost overruns, development failures, and operational problems, ranging from limited adaptability to breakdowns of various types.4 As more and more people, activities, and organizations come to depend on such systems—that is, as the systems become critical societal infrastructures—their impact, and the cost to society of their failure, grow. But the necessity of addressing systems problems is only one of the reasons for renewing and reorienting the focus on fundamental research. There is also the promise of much greater societal good from IT systems, a promise hinted at by the systems that have already been successfully deployed and used and by the connectedness achieved through the Internet and the exuberant experimentation with new types of businesses, services, and social and nonprofit activity it has fostered. This is a future that can be attained only with significant improvements in the science and technology base.
Research is needed on the science and engineering issues associated with large-scale systems, to devise ways to make IT systems better —more scalable, flexible, predictable, and reliable (see Box ES.2). Work is also needed to better understand the technical and nontechnical issues that arise when such systems are integrated into social applications. Social applications of IT are expected to motivate technical research that will develop new capabilities to satisfy a growing set of societal needs, and they will demand that technological needs be considered in the social and organizational context in which they will be applied. Such work is by its very nature interdisciplinary, demanding insight into both the technical capabilities of IT and the ways in which people engage or are affected by IT systems in a variety of operational settings. It requires computer science and engineering, but it also requires economists and other social scientists and business-school researchers who understand how IT systems are selected, used, and integrated into organizational processes (see Box ES.3). Such work also benefits from practical perspectives—the knowledge of people working in systems development and end-user organizations—to ground thinking about systems in the contexts of their development, deployment, and use.
What Makes Large-Scale IT Systems So Difficult to Design, Build, and Operate?
Neither large-scale systems nor social applications of IT are adequately addressed by the IT research community today. Most IT research is directed toward the components of IT systems: the microprocessors, computers, and networking technologies that are assembled into large systems, as well as the software that enables the components to work together.5 This research nurtures the essence of IT, and continued work is needed in all these areas. But component research needs to be viewed as part of a much larger portfolio, in which it is complemented by research aimed directly at improving large-scale systems and the social applications of IT. The last of these includes some work (such as computer-supported cooperative work and human-computer interaction) traditionally viewed as within the purview of computer science. Research in all
Research on the Social Applications of Information Technology
Research on the social applications of information technology (IT) combines work in technical disciplines, such as computing and communications, with research in the social sciences to understand how people, organizations, and IT systems can be combined to most effectively perform a set of tasks. Such research can address a range of issues related to IT systems, as demonstrated by the examples below (more detailed discussion is contained in Chapter 3 andChapter 4):
three areas—components, systems, and social applications—will make IT systems better able to meet society's needs, just as in the medical domain work is needed in biology, physiology, clinical medicine, and epidemiology to make the nation 's population healthier.
Research on large-scale systems and the social applications of IT will require new modes of funding and performing research that can bring together a broad set of IT researchers, end users, system integrators, and social scientists to enhance the understanding of operational systems. Research in these areas demands that researchers have access to operational large-scale systems or to testbeds that can mimic the performance of much larger systems. It requires additional funding to support sizable projects that allow multiple investigators to experiment with large IT systems and develop suitable testbeds and simulations for evaluating new approaches and that engage an unusually diverse range of parties. Research by individual investigators will not, by itself, suffice to make progress on these difficult problems.
Today, most IT research fails to incorporate the diversity of perspectives needed to ensure advances on large-scale systems and social applications. Within industry, it is conducted largely by vendors of IT components: companies like IBM, Microsoft, and Lucent Technologies. Few of the companies that are engaged in providing IT services, in integrating large-scale systems (e.g., Andersen Consulting, EDS, or Lockheed Martin), or in developing enterprise software (e.g., Oracle, SAP, PeopleSoft) have significant research programs.6 Nor do end-user organizations (e.g., users in banking, commerce, education, health care, and manufacturing) tend to support research on IT, despite their increasing reliance on IT and their stake in the way IT systems are molded. Likewise, there is little academic research on large-scale systems or social applications. Within the IT sector, systems research has tended to focus on improving the performance and lowering the costs of IT systems rather than on improving their reliability, flexibility, or scalability (although systems research is slated to receive more attention in new funding programs). Social applications present an even greater opportunity and have the potential to leverage research in human-computer interaction, using it to better understand how IT can support the work of individuals, groups, and organizations. Success in this area hinges on interdisciplinary research, which is already being carried out on a small scale.
One reason more work has not been undertaken in these areas is lack of sufficient funding. More fundamentally, the problems evident today did not reach critical proportions until recently. There has been no crisis to motivate the research community or to compel a broader set of companies to fund research, no compelling set of visions to inspire broad-based interest. From a practical perspective, conducting the types of research
advocated here is difficult. Significant cultural gaps exist between researchers in different disciplines and between IT researchers and the end users of IT systems. These groups tend to have different sets of motivations, interests, and even perspectives on what constitutes research.7 But if IT is to meet society's growing needs, then the challenges of collaborative research will have to be overcome. Luckily, a few seeds have been planted that, if nurtured, may sprout and blossom in ways that can support and encourage a larger, more diverse range of efforts. Government, industry, and universities need to ensure that this happens. This report provides guidance that they can follow in doing so.
Now is the time to adjust and expand the IT research portfolio, both to overcome problems that have become urgent and to better meet society 's needs. Doing so will require efforts on many fronts. Increased funding will be needed to extend the scope of IT research more fully into large-scale systems and social applications. At the same time, existing mechanisms for funding IT research will have to be strengthened to ensure that fundamental research continues to be supported in a way that will prove most productive for the IT industry and, ultimately, the nation. New mechanisms will be needed to fund and conduct research on large-scale systems and social applications of IT—the nature of which differs from that of traditional components research. In many cases, mechanisms for research on large-scale systems and social applications can build on existing programs and initiatives, expanding their scale and scope.
Changes like these have been suggested before. The less-than-satisfactory outlook today simply reflects insufficient follow-through on those earlier suggestions. All of the relevant recommendations made by CSTB committees, past and present, draw on the expertise of the IT research community and other relevant experts; engaging the community effectively requires listening to its advice. The committee's recommendations in each of these areas are presented below, organized according to the group that would carry them out, and elaborated on in Chapter 5.
Recommendations for Government
Recommendation 1. The federal government should continue to boost funding levels for fundamental information technology research, commensurate with the growing scope of research challenges.
The first step toward strengthening the nation's IT research base is to ensure that sufficient investments are made in IT research. Increased federal funding for such research is necessary to support continued growth in existing component-oriented research (e.g., microprocessors, computing systems, networking equipment, and software) while expanding the research base to look at the problems of large-scale systems and the social applications of IT, to help the nation harness IT's potential for a range of public and private-sector missions. Although it is not possible to specify precisely how much additional funding is needed, the committee believes that the increases proposed in recent years by the President's Information Technology Advisory Committee ($1 billion over 5 years) and the Clinton Administration ($1 billion between 1999 and 2001) are the right order of magnitude and would allow the IT research community to grow larger while providing adequate resources for each investigator.
Funding increases need to be aimed primarily at fundamental research, not applied research. Not only is such work important to the long-term evolution of the field, but it would also shift some of the responsibility from industry, which faces a number of strong disincentives to investment in fundamental, long-term research. The results of such work cannot be anticipated and their most important implications often lie far in the future, even if some benefits can be gained more immediately. As experience demonstrates—and economic theory supports—companies that make fundamental breakthroughs often have difficulty capturing the benefits of these advances while preventing competitors from doing so. Hence, they tend to underinvest in such research.8 Only the largest, most profitable, and most dominant IT firms tend to be able to invest in long-term research (in part because they are better positioned to capture its benefits), and even this source of funding has been in short supply. Increasing competition, which can erode the market share of leading firms in an industry, and the need to introduce new products and services rapidly into the marketplace have forced even the most forward-thinking companies to shift more of their resources to applied research efforts. The federal government is much better able to provide sustained funding for research with long-term potential, but it, too, has increased funding for applied research more quickly than funding for fundamental research in recent years. IT researchers corroborate this trend, noting that federally funded projects have become more focused on near-term objectives and demonstrations of capability—precisely the same types of things that industry is likely to do—rather than on fundamental advances in the technology. As shown in earlier CSTB reports, federal funding for fundamental research laid the groundwork for many of today's common commercial innovations, from graphical user interfaces and relational databases to computer graphics and even the Internet itself (CSTB, 1995, 1999).
The need for it has not diminished. Although the IT industry has grown and is highly profitable, the barriers to investments in long-term research persist, and the government continues to have a role to play.
Recommendation 2. The National Science Foundation and the Defense Advanced Research Projects Agency should establish significant programs of fundamental research in large-scale information technology systems.
The National Science Foundation (NSF) and the Defense Advanced Research Projects Agency (DARPA) each have a number of efforts under way that address aspects of large-scale systems. As yet, these efforts have not been integrated into a larger programmatic thrust that attempts to gain a more fundamental understanding of large-scale systems (as opposed to pursuing development of specific applications). The NSF and DARPA should exercise the leadership they have demonstrated in the past and create more comprehensive, cohesive programs in this area that would allow a vibrant research community to coalesce around the problems of large-scale systems. The programs run by the organizations should complement one another and should together have the following characteristics:
Support both theoretical and experimental work;
Offer awards in a variety of sizes (small, medium, and large) to support individual investigators, small teams of researchers, and larger collaborations;
Investigate a range of approaches to large-scale systems problems, such as improved software design methodologies, system architecture, reusable code, and biological and economic models (see Chapter 3);
Attempt to address the full scope of large-scale systems issues, including scalability, heterogeneity, trustworthiness, flexibility, and predictability;9 and
Give academic researchers some form of access to large-scale systems for studying and demonstrating new approaches.
Given the wide circle of agencies interested in and involved with IT research and the even wider circle coming to depend on large-scale IT systems, the NSF and DARPA should attempt to involve in their research other federal agencies, such as the Department of Health and Human Services and the Federal Aviation Administration, that operate large-scale IT systems and would benefit from advances in their design. Such involvement could provide a means for researchers to gain access to operational systems for analytical and experimental purposes.
Recommendation 3. Federal agencies should increase support for interdisciplinary work on social applications of information tech nology that draws on the expertise of researchers from IT and other disciplines and includes end users of IT systems.
Research on the social applications of IT demands the perspectives of IT researchers, researchers in other academic disciplines, and end users of IT systems who are familiar with the particular challenges faced and the viability of different solutions. A number of programs are in place, such as the Digital Government program, the Digital Libraries Initiative, and the NSF's Computing and Social System program, that combine these perspectives and apply them to problem areas, but an initiative is needed that has a larger scale and scope and that possesses the following characteristics:
Support provided through a variety of research mechanisms, including single-investigator grants, small teams of researchers, and larger research centers that bring together researchers from several disciplines and different industries for an extended period of interaction;
Explicit participation in the research process of end users and systems integrators who understand the problems faced in using large-scale systems and social applications;
Participation of the federal agencies that are major users of IT systems and that invest considerable resources in the development of IT systems (such as the Internal Revenue Service, the Social Security Administration, and the Federal Aviation Administration), not just the traditional funders of IT research;
Access to large systems or testbeds so that researchers can gain insight into operational problems and appreciate the relationships between an IT system and the larger social or organizational context in which it operates; and
Management and oversight by traditional funders of IT research to ensure that the work retains a research focus and does not become linked too closely to development efforts at particular end-user organizations.
The NSF has already allocated some funding for IT research centers focusing on social, economic, and workforce issues associated with IT. These centers could make valuable contributions to research and education in this area if they incorporate end-user perspectives as well as the perspectives of disciplines such as business, law, economics, and other social sciences. They should also be sure to complement their attention to the effects of IT on society and the economy with parallel efforts to develop the scientific and engineering knowledge needed to improve the design
of IT systems. Additional effort will be needed to review proposals for interdisciplinary work related to IT and for assuring its quality. Quality controls are especially important in fields with growing research budgets, and interdisciplinary research can be especially difficult to evaluate. Review and evaluation processes will need to reflect the full range of perspectives involved in the research.
Recommendation 4. The Bureau of the Census should work with the National Science Foundation to develop more effective procedures for classifying data on federal and industry investments in information technology R&D that better account for the dynamic nature of the industry.
Before they can make better decisions on IT research, policy makers need better data on current expenditures by the federal government and industry on such research. Existing data fluctuate from year to year, largely because of reclassifications of companies among sectors. These reclassifications occur as a result of changes within the individual companies (e.g., new lines of business, mergers, outsourcing of production) as well as the rules for the classification process itself, which are based on the composition of the payrolls of individual firms. The result is inconsistent data that make it difficult to discern trends, even very general ones. The NSF and the Census Bureau need to develop ways to collect and disseminate more-consistent data describing past, present, and future investments in IT R&D. Additional efforts will be needed to develop more consistent time-series data for the IT industries in particular and to develop robust procedures for classifying firms within industrial sectors.
Recommendations for Universities
Recommendation 5. Universities should take steps to increase the ability of faculty members and students to participate in interdisciplinary research related to information technology and research on large-scale systems.
Universities have the potential to make significant strides in large-scale systems and social applications research because they have all the key ingredients: researchers in a broad range of related areas, from computer science and electrical engineering to business, law, economics, and other social sciences. They are also able to complement this research with educational initiatives that can teach students about large-scale systems and social applications, thereby helping create a future workforce capable of researching, developing, and using them. For the most part, universities
are not currently set up properly to make progress on these issues. With some degree of change, they could contribute to the research base at the same time as they educate students and imbue them with an appreciation of the issues. Because getting diverse researchers to work together has never been easy, incentives must be provided and barriers to collaboration removed. The availability of funding for work on large-scale IT systems and the social applications of IT would motivate academic researchers to pursue the sorts of interdisciplinary research needed to make progress in these fields, but additional efforts will also be needed to create a more suitable environment for interdisciplinary research and one that ensures the quality of such research.
The first step should be to ensure that hiring, reviewing, and tenure processes are aligned to suit the interdisciplinary nature of the research that this report recommends and to ensure its quality. This can be accomplished through a variety of mechanisms, including the creation of interdisciplinary schools or departments that have their own hiring and promotion processes10 or the establishment of guidelines for evaluating faculty in traditional academic departments who pursue interdisciplinary work. The sharing of information should be encouraged between university researchers (both faculty and students) and their counterparts in industry, especially in companies that urgently need to resolve problems of large-scale systems and social applications of IT. The purpose of industry involvement should not be to facilitate the commercialization of university research (although this is a welcome outcome) but to provide researchers with the knowledge they will need to make progress in large-scale systems and social applications. In particular, universities should work with industry to establish more internship opportunities for students and sabbatical opportunities for faculty, especially in end-user organizations that do not have established programs aimed at technologists. Although such activities may take students away from their faculty supervisors for a time, the committee believes the experience will ultimately prove valuable to the work that these students perform at the university. Universities should also bring industry leaders into the classroom and the research lab.
Recommendation 6. Senior faculty members should take the lead in pioneering research on large-scale systems and social applications of information technology.
Even though they may have innovative ideas, junior faculty members are at a distinct disadvantage when they set off in new research directions that are not considered part of the intellectual core of their disciplines. Concerns about tenure can limit their willingness to work on topics such
as large-scale systems and social applications of IT that do not fall neatly within established research areas. Senior faculty members, by contrast, sometimes seek a refreshing change in emphasis. They should be encouraged to establish interdisciplinary research projects, to attract funding and people to them, and to articulate a vision for such work. Their involvement would help legitimize these areas of inquiry and provide an umbrella under which junior faculty could join them, bringing new ideas and insights.
Recommendations for Industry
Recommendation 7. Organizations that are significant end users of information technology systems should actively seek opportunities to engage in IT research.
Large end-user organizations in industries ranging from banking to health care to manufacturing face significant challenges in designing, developing, and operating the IT systems on which they rely and in whose development they invest large sums of money. Ideally, they too should support IT research that would address problems of large-scale systems and the social applications of IT. These organizations could benefit handsomely from research in these areas. They also have knowledge of the application and its operation (as well as its failures) that will prove vital in these areas of research. Involving end users in IT research will not be easy, because few have much interest in research, let alone experience in conducting or managing it, and the benefits of such engagement may not be immediately apparent to them. Nevertheless, the time is right to overcome these obstacles and experiment with ways to bring end users more effectively into the research process. As a first step, end-user representatives should be engaged to serve on advisory boards to IT research programs, labs, or academic departments. Over time, they should become more directly involved, and some end-user organizations could even fund research for groups or centers whose capabilities match their needs. Recent research support by leading financial services firms shows it is possible to meaningfully engage end-user organizations, albeit on a limited scale.
Recommendation 8. Information technology companies with established R&D organizations should develop mechanisms for engaging end users more actively in the research process.
To help end users become better engaged in IT research, IT companies with a tradition of research (as opposed to development) should develop mechanisms for involving end users more extensively in inter-
disciplinary and large-scale systems-related work. Such IT companies have track records in research and an interest in better understanding customer needs. Companies such as IBM Corporation and Microsoft Corporation have demonstrated the utility of working more closely with customers and researchers from different disciplines. Other companies may be able to build on these examples and develop other mechanisms for achieving similar results.
A FINAL WORD
The committee believes that these recommendations will strengthen the nation's IT research base sufficiently to help meet society's growing need for, and dependence on, IT systems. By strengthening the existing mechanisms for IT research and experimenting with new mechanisms to expand IT research into large-scale systems and social applications of IT, the nation will be able to ease its transition to an information economy. A strong research base will provide the industry and the nation with the knowledge resources needed to harness IT for the common good.
Arrow, Kenneth . 1962. “Economic Welfare and the Allocation of Resources for Invention,” in Richard Nelson, ed., The Rate and Direction of Innovative Activity. Princeton University Press, Princeton, N.J.
Barr, Avron, and Shirley Tessler. 1998. “How Will the Software Talent Shortage End?” American Programmer 11(1). Available online at <http://www.cutter.com/itjournal/itjtoc.htm#jan98>.
Campbell, Donald T. 1969. “Ethnocentrism of Disciplines and the Fish-Scale Model of Omniscience, ” pp. 328-348 in Muzafer Sherif and Carolyn W. Sherif, eds., Interdisciplinary Relationships in the Social Sciences. Aldine Publishing Co., Chicago.
Computer Science and Telecommunications Board (CSTB), National Research Council. 1995. Evolving the High Performance Computing and Communications Initiative to Support the Nation's Information Infrastructure. National Academy Press, Washington, D.C.
Computer Science and Telecommunications Board (CSTB), National Research Council. 1999. Funding a Revolution: Government Support for Computing Research. National Academy Press, Washington, D.C.
Gibbs, W.W. 1994. “Software's Chronic Crisis,”Scientific American (September):86-95.
Johnson, Jim. 1999. “Turning Chaos into Success,” Software Magazine, December. Available online at <http://www.softwaremag.com/archives/1999dec/Success.html>.
Jones, C. 1996. Applied Software Measurement. McGraw-Hill, New York.
Nelson, Richard. 1959. “The Simple Economics of Basic Research,”Journal of Political Economy 67(2):297-306.
Standish Group International, Inc. 1995. The Chaos. Available online at <http://www.standishgroup.com/chaos.html>.
1. These figures represent combined federal obligations for research (basic and applied) in computer science and electrical engineering, the two academic disciplines most closely associated with information technology (IT). Some work in electrical engineering, such as research on power systems, is not applicable to IT, and some work in other disciplines is applicable but is not captured in these statistics.
2. These sectors, as defined by the Standard Industrial Classification (SIC) system, are SIC 357, office, computing, and accounting machines; SIC 366, communications equipment; SIC 367, electronic components (including semiconductor devices); SIC 48, communications (services); SIC 504, professional and commercial equipment and supplies; and SIC 737, computer and data processing services (including prepackaged software, custom programming, systems integration, and other services).
3. As described in greater detail in Chapter 2, federal statistics on IT industries' R&D investment are not compiled in a consistent manner from year to year because of the frequent reclassification of firms from one IT sector to another as well as into and out of the IT industries. The aggregate figures used in this report account for reclassifications among IT sectors but not into or out of the IT industry.
4. It is estimated that between 70 and 80 percent of all major system development efforts are never completed, are late, or overrun cost projections by a wide margin. Estimates of failure rates in large-scale system development efforts are contained in several studies. See Johnson (1999), Standish Group (1995), Gibbs (1994), Jones (1996), and Barr and Tessler (1998).
5. This definition of a component is much broader than the definition typically used in the research community.
6. The most notable exception to this general rule is IBM, which derives a significant portion of its revenues from IT-related services and systems work and which maintains a substantial research program. About one-quarter of the work conducted by IBM Research supports its systems and services businesses.
7. A discussion of the long-standing challenges inherent in interdisciplinary work in the social sciences can be found in Campbell (1969).
8. Most notably, Xerox failed to capture much of the benefit of its pioneering work in personal computing. IBM's work in relational databases and reduced-instruction-set computing seeded not only its own product development efforts but also those of numerous competitors. Economics provides a theoretical verification of this phenomenon. See CSTB (1999), Nelson (1959), and Arrow (1962).
9. See Chapter 3 for a more in-depth discussion of each of these topics.
10. In recent years, a number of universities have created interdisciplinary schools that examine issues at the intersection of IT, business, and the social sciences. Examples include the School of Information Management and Systems at the University of California at Berkeley and the School of Information at the University of Michigan. Carnegie Mellon University and the Massachusetts Institute of Technology also have a number of interdisciplinary departments and divisions in this general area. Chapter 4 contains a more complete list of such programs.