4
Measuring Innovation in Business and Industry

Innovation encompasses but is more than research and development. It is becoming apparent that much technological innovation does not result from traditional research and development, particularly in the service industries (Guellec and Pattinson, 2000). The main activities involved in innovation are, in addition to R&D, other acquisition of knowledge (patents, licenses, and technical services); acquisition of machinery and equipment (both incorporating new technology and for standard use when producing a new product); various other preparations for production and delivery, including tooling up and staff training; and internal and external marketing aimed at the introduction of the innovation (Organisation for Economic Co-operation and Development, 1992). Although many of these activities extend well beyond the subject of this report, nonetheless it is useful to discuss the measurement of innovation because of the close interaction between traditional R&D and the process of innovation.

Innovation measures must cover five activities: the introduction to the market of new products; the development of new processes to produce, or deliver, products for the market; the development of new markets; the finding of new sources of supply of raw materials; and changes in the organization of firms. Introducing new products to the market has implications for economic growth, and new processes provide opportunities for improvements in productivity, quality, or other desired objectives, such as reduced environmental emissions or a happier labor force.

Moreover, there is a general feeling that the role of technological innovation in improving productivity and increasing economic growth is increasing, as is international economic competition for domestic and inter-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy 4 Measuring Innovation in Business and Industry Innovation encompasses but is more than research and development. It is becoming apparent that much technological innovation does not result from traditional research and development, particularly in the service industries (Guellec and Pattinson, 2000). The main activities involved in innovation are, in addition to R&D, other acquisition of knowledge (patents, licenses, and technical services); acquisition of machinery and equipment (both incorporating new technology and for standard use when producing a new product); various other preparations for production and delivery, including tooling up and staff training; and internal and external marketing aimed at the introduction of the innovation (Organisation for Economic Co-operation and Development, 1992). Although many of these activities extend well beyond the subject of this report, nonetheless it is useful to discuss the measurement of innovation because of the close interaction between traditional R&D and the process of innovation. Innovation measures must cover five activities: the introduction to the market of new products; the development of new processes to produce, or deliver, products for the market; the development of new markets; the finding of new sources of supply of raw materials; and changes in the organization of firms. Introducing new products to the market has implications for economic growth, and new processes provide opportunities for improvements in productivity, quality, or other desired objectives, such as reduced environmental emissions or a happier labor force. Moreover, there is a general feeling that the role of technological innovation in improving productivity and increasing economic growth is increasing, as is international economic competition for domestic and inter-

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy national markets. Therefore, it is important to understand the innovation process and why some companies, industries, and countries are apparently more innovative and enjoy a greater gross domestic product (GDP) per capita than others. The panel observed that one reason for the strong interest in measuring innovation activity in Canada is that the GDP per capita for Canada is $28,000 (United States; adjusted for purchasing power1) but in the United States it is $36,000 US. This disparity is in part attributed to an evident “productivity gap” that is not being resolved by the market. This, in turn, has given rise to policy interventions to close the gap. In Canada, this takes the form of a 10-year innovation strategy, which aims to make Canada one of the world’s most innovative countries. European countries face a similar gap. Emphasis on innovation policy in Europe was affirmed at a special meeting of the European Council held in Lisbon in 2000, which set a strategic goal for the coming decade of making the European Union “the most competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth with more and better jobs and greater social cohesion” (European Council, 2000). The “Lisbon strategy” was reaffirmed and strengthened by the European Commission in March 2003 (European Commission, 2003a). Developing countries in Latin America, including Argentina, Chile, and Mexico, among others, have also conducted innovation surveys. If this perceived shortfall in productivity attributed to a relative lack of innovation is the driving force for interest in innovation studies abroad, is there a similar rationale for the gathering of information on innovation by NSF in the United States? Although interest in systematic innovation measurement has been less intense in the United States, it is widely recognized that innovation information is valuable for offering insights about best practices and where they can be applied with greatest effect. It is also recognized that there are intraregional and structural dimensions to innovation in the United States. The GDP per capita of the state of Georgia, for example, is less than that of California, and presumably less than that of several European countries, and the propensity to innovate in pharmaceuticals is greater than that in wood products industries. Although economic statistics can identify a productivity gap, understanding why the gap exists requires information about firms. This can 1   Purchasing power parities (PPPs) are the rate of currency conversion that eliminates the differences in price levels between countries. They are used to compare the volume of GDP in different countries. PPPs are obtained by evaluating the costs of a basket of goods and services among countries for all components of GDP (Organisation for Economic Co-operation and Development, 2002b).

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy come from case studies, surveys, or a combination of both. Surveys that identify the introduction of new or significantly changed products or processes by firms can relate the activity of innovation to the broader economic context that led the firm to innovate. They can also identify the barriers to innovation, the sources of information and technologies used to innovate, and the impacts of the activity, such as a change in the level of employment in the firm or in the skill levels required by the workforce as a result of the change. All of this can be related to productivity measures. SURVEYS OF INNOVATION IN INDUSTRIALIZED COUNTRIES Innovation surveys are not as well developed as those for industrial research and development expenditures. R&D surveys emerged after World War II and their concepts and definitions were eventually codified by the Organisation for Economic Co-operation and Development (OECD) in 1963 with the first Frascati Manual (Organisation for Economic Co-operation and Development, 1963). Innovation surveys have gone through a long experimental period in many industrialized countries. Hansen and Hill (Hill et al., 1982, 1983) led this work in the United States in the 1980s. There were surveys in the Nordic countries at the same time, as well as in Canada and Germany, where the first annual survey of innovation was put in place. Some of these early surveys were quite ambitious. In his review of technology innovation surveys, Hansen (2001) identified the largest effort as one conducted in Italy in the 1980s (see Table 4-1 for major innovation surveys, 1979-2001). The Italian National Research Council and Central Statistical Office sent a survey to the 35,000 firms in Italy with more than 20 employees, which revealed that many more firms were engaged in innovation activities than just those conducting in-house R&D. They followed up with a detailed questionnaire to the 25,000 firms that reported some innovation activity in response to the first survey, which asked about number and types of innovations, types of underlying technologies, sources of information, obstacles to and costs of innovation, and impact on sales (Archibugi et al., 1991; Cesaratto et al., 1991). These initiatives were rather uncoordinated and experimental, so concepts and definitions differed and comparison was difficult. As with the Frascati Manual, innovation surveys came into their own only when the OECD provided a forum for developing a common set of concepts and definitions codified in the first Oslo Manual (Organisation for Economic Co-operation and Development, 1992). The Oslo Manual was used to guide the first Community Innovation Survey (CIS.1) in the European Community in 1992. The manual dealt only with technological innovation and with manufacturing. Experience

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy TABLE 4-1 Surveys of Innovation at the Firm Level, 1979-2001 Survey Year Country Responses Method Focus Ifo Institute 1979 Germany Sources of innovation Italian National Research Council-Institute of Information Science and Technologies 1986 Italy 24,700 firms Two-stage survey Sources of innovation Yale 1982 United States 650 respondents 42% response rate R&D manager survey Appropriability role of science French: Piatier Observatory of Science and Technology Late 1980s France 5,300 respondents 15,000 respondents Questionnaires Sources of innovation Nordic Innovation Surveys 1989 Nordic countries 650 respondents Questionnaires Sources of innovation Community Innovation Survey 1 (CIS-1) 1992 12 European Union countries and Norway Circa 40,000 Differences in questionnaires Sources of innovation Canadian Innovation Survey 1 1993 Canada Oslo Manual 5,000 responses Questionnaire Sources of innovation Policies, Appropriation, and Competitiveness in Europe (PACE) Survey 1993 United Kingdom, Ireland, Benelux, Italy, Germany, France, Denmark R&D managers 706/1,200 return Yale type questionnaires Appropriability role of science

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy Carnegie Mellon Survey 1 and 2 1994 United States Japan 1,478 (54% response rate) 643 (53% response rate) Yale II - same questionnaire Appropriability role of science Canadian Innovation Survey 2 1996 Canada 5,000 responses 100% return Questionnaire Sources of innovation Community Innovation Survey 2 (CIS-2) 1997 15 European Union countries, Norway, and Iceland 16,950 manufacturing firms (38% response rate overall—median: 65%) 11,932 service enterprises (7 response rate) Questionnaire Sources of innovation Canadian Innovation Survey 3 1999 Canada 5,700 95% return Computer assisted telephone interviewing Sources of innovation Survey 3 (CIS-3) Community Innovation 2001 15 European Union countries, Norway, and Iceland Not yet released Questionnaire Sources of innovation SOURCE: Adapted from Science and Technology Policy Research (expanded and revised) (http://www.sussex.ac.uk/Users/prff0/RM1/Autum_2002/SurveyII%20week%20VIII%202002.doc).

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy gained from surveys led to the first revision of the manual, jointly with the statistical office of the European Union, Eurostat. The second edition (Organisation of Economic Co-operation and Development and Eurostat, 1997) still dealt with technological innovation, but its industry coverage was expanded to include all of the private sector. It guided CIS.2, conducted in 1997, and CIS.3 in 2001. By 2003 it was clear that the manual needed further revision to take account of nontechnological innovation, such as the adoption of new business practices and changes in organization of the firm. The revision was launched by the OECD and Eurostat in June 2003, with completion anticipated in 2005.2 Not all surveys of technological change have focused on the introduction of new products or processes. In the United States, for example, there have been surveys that concentrated on the ability of firms to capture, or appropriate, the results of their inventions and their interactions with universities and national laboratories (Levin et al., 1987). The Yale I survey, as it became known, was conducted in 1982 and was aimed at high-level R&D managers who had knowledge of both the technological and market conditions facing their firms. This was followed by the so-called Carnegie Mellon survey in 1994 (Cohen et al., 2000, 2002a). The Carnegie Mellon survey built on but went well beyond the Yale I survey by collecting measures on the source and channels of knowledge affecting industrial R&D, the regional sources of that knowledge, a large number of measures of R&D activity, the patenting behavior of firms, the actual uses of patents, the intensity of technological rivalry, the impacts of public research on industrial R&D, the actual uses of patents, the management of innovation in the firm, and R&D performance. This design was intended to capture measures of the determinants of industrial R&D and R&D performance. 2   The Oslo Manual provides definitions and methodologies for collecting data on corporate strategies, the role of diffusion, sources of innovative ideas and obstacles to innovation, inputs to innovation, the role of public policy in industrial innovation, the outputs of innovation, and the impacts of innovation (Archibugi and Sirilli, 2001). The categories of information to be collected were: prevalence of the innovation (number of firms, industries); types of innovation (products, processes); goals of innovation (improved or radically new product, new market, higher quality, better performance); internal sources of innovation (in-house R&D, sales and marketing, management); external sources of innovation (suppliers, clients, university or government laboratories, technical literature); practices for protecting innovations (patents, trademarks, trade secrets, complexity of industrial design); intensity of innovation (ongoing, occasional); obstacles to innovation (lack of skilled personnel, high risk, lack of information, regulatory barriers); impact on workers (number of employees, productivity, skills); and impact on economic performance (percentage of sales attributable to new or improved products or processes) (Hansen, 2001).

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy Cohen was also central to a collaboration with the National Institute of Science and Technology Policy in Japan that resulted in surveys of R&D directors and laboratory managers across major innovation-intensive industries in the United States and in Japan and Europe in 1994.3 The purpose of the survey was to improve the understanding of the factors affecting innovative activity across nations in order to provide a more informed basis for designing national and multilateral policies that promote technological progress leading to economic growth (Cohen et al., 2002b). In Europe, the Policies, Appropriability and Competitiveness for European Enterprises (PACE) project included a survey of managers of the 500 largest manufacturing firms in the European Union (Arundel et al., 1995). The survey collected data on the goals of innovation, external sources of knowledge, methods to protect intellectual property, and legal or regulatory impediments to innovation. The survey was similar to the Yale I and II surveys. While there was an interest in how knowledge was acquired and protected, there was also an interest in what technologies were being used in industry and how they were diffusing. This led to surveys conducted by the U.S. Census Bureau, the first of which was done in 1988 (U.S. Census Bureau, 1989). Similar surveys were done in Australia and Canada (Ducharme and Gault, 1992), and, as a result, the first edition of the Oslo Manual included a section on the importance of measuring the use and planned use of manufacturing technologies, which remained in the second edition (Organisation for Economic Co-operation and Development and Eurostat, 1997). As with the case for the industry R&D survey, there is much to learn by turning attention to initiatives north of the border. Statistics Canada has conducted several surveys of innovation, beginning in 1993, to better understand innovation in Canada: The 1993 Survey of Innovation and Advanced Technology, which surveyed manufacturing firms. The 1996 Survey of Innovation, which surveyed the communications, financial services, and technical business services industries. The 1999 Survey of Innovation, Advanced Technologies and Practices in the Construction and Related Industries, which was the first survey of advanced technologies and practices in the construction sector. The 1999 Survey of Innovation surveyed manufacturing and was the first innovation survey of industries in the natural resource sector. 3   See www.cgp.org/cgplink/vol04/researchvol04html.

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy The surveys were designed in accordance with the guidelines contained in the Oslo Manual, although they were not the same as (harmonized with) the CIS surveys. In fact, for various reasons, the Canadian survey findings cannot be compared completely with the results of CIS-2 (see Mohnen and Thierrien, 2003, and Therrien and Mohnen, 2003, who compare Canada’s Survey of Innovation 1999 with CIS-2 data from France, Germany, Ireland, and Spain). For one thing, because it is the country’s only statistical agency and because completion of its surveys is a legal requirement, Statistics Canada is able to field fairly long surveys and also link easily with its other datasets (for example, its industrial R&D survey data). This means it can collect a broader range of data and go into greater detail than the CIS surveys. For example, the Canadian survey asks about employee development and training programs, employee access to the Internet, and use of the Internet by the firm to sell its products. It collects much more detail on a firm’s objectives in undertaking innovation, the firm’s sources of knowledge relevant to innovation, the barriers to innovation, and methods of protecting the results of innovation (these examples are all from Hansen, 2001). The experience in Australia is also instructive. The Australian Bureau of Statistics (ABS) conducted two innovation surveys in the 1990s using the conceptual framework of the Oslo Manual. A third was to be conducted in 2004. Interest in innovation during this period shifted from a singular focus on measurement issues, to a broader interest in understanding the roots of innovation. In March 2000, ABS announced it was moving away from the idea of directly measuring innovation. “Instead, we are investigating whether we can produce a range of statistical indicators on the knowledge-based economy and society.” In 2001, the prime minister launched “Backing Australia’s Ability: An Innovation Action Plan for the Future.” ABS had already begun developing statistics to measure a knowledge-based economy and society and released a framework for discussion in 2002. The discussion paper proposed approximately 125 statistical indicators grouped in broad dimensions to enable assessment of the degree to which Australia is a knowledge-based economy and society. The dimensions are innovation, and entrepreneurship; human capital; information, and communications technology; context; and economic and social impacts. Within each dimension there are a number of proposed characteristics, and within each characteristic are potential statistical indicators that provide quantitative measures of that characteristic. For instance, one proposed characteristic of the context dimension is social and cultural factors; indicators for this characteristic might include the age structure of the population, the income level and distribution of the population, and participation in community activities (Australia Bureau of Statistics, 2002).

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy INNOVATION AND NSF NSF was involved in work related to innovation in industry in the 1960s (National Science Foundation, 1967; Myers and Marquis, 1969). In 1967, the Charpie Report called attention to the inadequacies of government policies and management practices concerning innovation (U.S. Department of Commerce, 1967), which resulted in the establishment of the National R&D Assessment program at NSF in 1972. During the 1970s, the program supported research on the impact of industrial management practices and government policies on industrial innovation. Another period of interest began in the 1970s, in response to concern that the United States was losing its technological edge, that there was little productivity growth, that government regulations were slowing innovation, and that industry was not doing enough basic research. The White House responded by conducting a Domestic Policy Review of Industrial Innovation in 1978, which was followed by the President’s Commission on Industrial Competitiveness in 1983 and the creation of the Council on Competitiveness in 1986. During this period, NSF supported special efforts to develop indicators and a survey of industrial innovation conducted at the Massachusetts Institute of Technology and Boston University. NSF’s interest in innovation has slowly evolved from experimental studies to piloting official surveys. In 1994 NSF and the Census Bureau conducted a pilot survey of 1,000 respondents in manufacturing and one service-sector industry (computer programming, data processing, and other computer-related services) that followed the Oslo Manual (National Science Foundation, 1996). The survey found that innovation was fairly widespread, with one-third of firms reporting recent introduction of new products and processes or plans to introduce a new product or process in the near future. Not surprisingly, computer hardware, precision instruments, pharmaceuticals, and chemicals firms reported levels of innovation well above the average. Although the survey yielded interesting results, it suffered from a relatively low response rate (57 percent), underscoring the difficulty in collecting this kind of innovation information. The problem of low response has plagued innovation surveys both here and abroad over their developmental cycle.4 In 2003, NSF conducted a second innovation survey. This time the focus was on the information technology sector and a sample of companies 4   The preliminary response rates for countries in the CIS.3 round of surveys ranged from 22 to 80 percent (Larsson, 2004).

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy outside information technology that were extensive purchasers of information technology products and services. Again, the concepts and definitions were drawn from the Oslo Manual. The final response rate for this survey was again about 57 percent (2,005 of 3,504 firms completed the survey). The results are now being analyzed. LESSONS LEARNED What has been learned from more than 20 years of measuring innovation in many countries is, first, that innovation can be measured, along with its linkages and outcomes and its economic and social environment. However, it is necessary to learn the second lesson before the information resulting from the many measurements can be widely used. The second lesson is that common concepts and definitions are necessary to provide guidance to those conducting the surveys and interpreting the results. The forum for the development of these concepts, definitions, and guidelines for the interpretation of the results was provided by the OECD in the late 1980s, leading to the first edition of the Oslo Manual in 1992. The Oslo Manual guided the first European Community Innovation Survey (CIS.1) and other surveys and provided boundaries and direction for the work—boundaries that survey teams began to challenge from the moment the manual was published. The third lesson is that the formal structure of the manual and the ongoing measurement activity combine to provide a dynamic learning environment for survey practitioners and users of the new information. Fourth, measurements that can be made, codified, and developed in a learning environment can be used. The results of innovation surveys influenced policy development in many OECD countries and especially in Europe (European Commission, 2003b). Other lessons have been learned about the feasibility of measuring the means used by firms to acquire and protect intellectual property in the United States, Japan, and Europe and to make comparisons across these countries (Arundel, 2003). In summary, the panel concludes that innovation, linked activities, and outcomes can be measured (Conclusion 4.1). They are measured successfully in other countries, and, except for survey management issues here in the United States, have been shown to produce potentially useful measures of U.S. innovative activities. At a basic level, information on innovation in U.S. firms could provide contextual material for the work of the Economic Development Administration as it guides “innovation-led development” (Economic Development Administration, 2001).

OCR for page 91
Measuring Research and Development Expenditures in the U.S. Economy While this may suggest support for a program of innovation measurement and analysis, there are still lessons being learned (Gault, 2003) that argue for a more cautious approach. These lessons deal with the period of observation (which is currently 3 years in the Oslo Manual but is clearly industry dependent) and the appropriate unit of observation (Are some questions better put at the level of the firm, or should they go to the plant or the establishment?). The entry level for the novelty of innovation in the Oslo Manual is “new to the firm,” which gives quite high rates of innovation in some industries. An additional issue has to do with the measurement of world-first and market-first innovations, which are more interesting from the competitiveness perspective. Finally, there is the question of nontechnological innovation and how that should be measured. RECOMMENDATIONS The need to understand the process of innovation is of critical importance to answering key questions about the source of growth of the U.S. economy. There have been several successful efforts in other countries to develop such measures, and, although lessons are still being learned, there is a growing science to support innovation measurement. The Science Resources Statistics Division (SRS) has dipped into these waters twice in the past decade, with mixed success. The division needs resources and the capacity to explore the impact of innovation on the U.S. economy. This can be done by commissioning surveys and analyzing and publishing the results, as well as by supporting academic research. After the analytical capacity has developed in SRS and its network of experts has been established, the SRS may wish to propose, based on its findings, a more comprehensive set of measures of technological change comparable to those that now exist for research and development. The panel recommends that resources be provided to SRS to build an internal capacity to resolve the methodological issues related to collecting innovation-related data. The panel recommends that this collection be integrated with or supplemental to the Survey of Industrial Research and Development. We also encourage SRS to work with experts in universities and public institutions who have expertise in a broad spectrum of related issues. In some cases, it may be judicious to commission case studies. In all instances, SRS is strongly encouraged to support the analysis and publication of the findings (Recommendation 4.1). The panel recommends that SRS, within a reasonable amount of time after receiving the resources, should initiate a regular and comprehensive program of measurement and research related to innovation (Recommendation 4.2).