Capturing Change in Science,
Technology, and Innovation

IMPROVING INDICATORS TO INFORM POLICY

Panel on Developing Science, Technology, and Innovation Indicators for the Future

Robert E. Litan, Andrew W. Wyckoff, and Kaye Husbands Fealing, Editors


Committee on National Statistics
Division of Behavioral and Social Sciences and Education

and

Board on Science, Technology, and Economic Policy
Division of Policy and Global Affairs

NATIONAL RESEARCH COUNCIL
                              OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.

www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Capturing Change in Science, Technology, and Innovation I M P R O V I N G I N D I C AT O R S T O I N F O R M P O L I C Y Panel on Developing Science, Technology, and Innovation Indicators for the Future Robert E. Litan, Andrew W. Wyckoff, and Kaye Husbands Fealing, Editors Committee on National Statistics Division of Behavioral and Social Sciences and Education and Board on Science, Technology, and Economic Policy Division of Policy and Global Affairs

OCR for page R1
THE NATIONAL ACADEMIES PRESS • 500 Fifth Street, NW • Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. The project that is the subject of this report was supported by grant number SES-0453930 between the National Science Foundation and the National Academy of Sciences. Support for the Commit- tee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (grant number SES-1024012). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number-13:  978-0-309-29744-8 International Standard Book Number-10:  0-309-29744-3 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2014 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Cover credit: Image of U.S. Capitol from Architect of the Capitol. All other images from iStock cover images. Cover designed by Jacqui Sovde of the National Research Council. Suggested citation: National Research Council. (2014). Capturing Change in Science, Technology, and Innovation: Improving Indicators to Inform Policy. Panel on Developing Science, Technology, and Innovation Indicators for the Future, R.E. Litan, A.W. Wyckoff, and K.H. Fealing, Editors. Com- mittee on National Statistics, Division of Behavioral and Social Sciences and Education. Board on Science, Technology, and Economic Policy, Division of Policy and Global Affairs. Washington, DC: The National Academies Press.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1

OCR for page R1
PANEL ON DEVELOPING SCIENCE, TECHNOLOGY, AND INNOVATION INDICATORS FOR THE FUTURE ROBERT E. LITAN (Cochair), Bloomberg Government, Washington, DC ANDREW W. WYCKOFF (Cochair), OECD, Paris, France CARTER BLOCH, Danish Center for Studies in Research and Research Policy, University of Aarhus, Denmark NICHOLAS R. CHRISMAN, Geospatial Science, RMIT University, Melbourne, Australia CARL J. DAHLMAN, Edmund A. Walsh School of Foreign Service, Georgetown University, Washington, DC GEOFF M. DAVIS, Quantitative Insights Group, Google, Inc., Mountain View, CA KATHARINE G. FRASE, Industries Research, IBM, Yorktown Heights, NY BARBARA M. FRAUMENI, Muskie School of Public Service, University of Southern Maine RICHARD B. FREEMAN, Department of Economics, Harvard University FREDERICK D. GAULT, Economic and Social Research and Training Centre on Innovation and Technology, United Nations University, Maastricht, The Netherlands DAVID GOLDSTON, Government Affairs, Natural Resources Defense Council, Washington, DC MICHAEL MANDEL, Wharton School, University of Pennsylvania JOHN E. ROLPH, Marshall School of Business, University of Southern California LELAND WILKINSON, Data Visualization, Skytree, Inc., San Jose, CA KAYE HUSBANDS FEALING, Study Director ESHA SINHA, Associate Program Officer ANTHONY S. MANN, Program Coordinator v

OCR for page R1
COMMITTEE ON NATIONAL STATISTICS 2013-2014 LAWRENCE D. BROWN (Chair), Department of Statistics, Wharton School, University of Pennsylvania JOHN M. ABOWD, School of Industrial and Labor Relations, Cornell University MARY ELLEN BOCK, Department of Statistics, Purdue University DAVID CARD, Department of Economics, University of California, Berkeley ALICIA CARRIQUIRY, Department of Statistics, Iowa State University MICHAEL E. CHERNEW, Department of Health Care Policy, Harvard Medical School CONSTANTINE GATSONIS, Center for Statistical Sciences, Brown University JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan MICHAEL HOUT, Department of Sociology, New York University SALLIE ANN KELLER, Virginia Bioinformatics Institute, Virginia Polytechnic Institute and State University LISA LYNCH, The Heller School for Social Policy and Management, Brandeis University COLM A. O’MUIRCHEARTAIGH, Harris Graduate School of Public Policy Studies, University of Chicago RUTH D. PETERSON, Department of Sociology (emerita) and Criminal Justice Research Center, Ohio State University EDWARD H. SHORTLIFFE, Departments of Biomedical Informatics, Columbia University and Arizona State University HAL S. STERN, Donald Bren School of Information and Computer Sciences, University of California, Irvine CONSTANCE F. CITRO, Director JACQUELINE R. SOVDE, Program Associate vi

OCR for page R1
BOARD ON SCIENCE, TECHNOLOGY, AND ECONOMIC POLICY PAUL L. JOSKOW (Chair), Alfred P. Sloan Foundation, New York ERNST R. BERNDT, Sloan School of Management, Massachusetts Institute of Technology JEFF BINGAMAN, U.S. Senate (former senator), New Mexico ELLEN R. DULBERGER, Ellen Dulberger Enterprises, LLC ALAN M. GARBER, Office of the Provost, Harvard University RALPH E. GOMORY, Stern School of Business, New York University JOHN L. HENNESSY, Office of the President, Stanford University WILLIAM H. JANEWAY, Warburg Pincus, LLC, New York RICHARD K. LESTER, Department of Nuclear Science and Engineering, Massachusetts Institute of Technology DAVID T. MORGENTHALER, Morgenthaler Ventures, Palo Alto, CA LUIS M. PROENZA, Office of the President, University of Akron KATHRYN L. SHAW, Graduate School of Business, Stanford University LAURA D’ANDREA TYSON, Haas School of Business, University of California, Berkeley HAL R. VARIAN, Google, Inc., Mountain View, CA ALAN WM. WOLFF, McKenna, Long, & Aldridge, LLP, Washington, DC RALPH J. CICERONE, Ex Officio, President, National Academy of Sciences, Washington, DC VICTOR J. DZAU, Ex Officio, President, Institute of Medicine, Washington, DC C. D. MOTE, Jr., Ex Officio, President, National Academy of Engineering, Washington, DC RICHARD E. BISSELL, Acting Director vii

OCR for page R1

OCR for page R1
Preface Indicators are a scaffolding of statistics to which decision international policy makers and government administra- makers can relate other elements needed to make decisions. tors; academic researchers and administrators; and business Indicators often are used to tell an end-to-end story on a managers and consultants. Users want indicators that relay policy-relevant topic. Science, technology, and innovation timely information about drivers, trends, advances, vulner- (STI) indicators are at a transition point in the formation abilities, culture/climate, and distributions related to the of mainstream policy, especially in the current economic STI system. Users also are eager to explore nontraditional environment, in which a premium is placed on sustainable sources of data from which indicators are derived and want growth (e.g., growth that is not dependent on speculative to be informed about the reliability of those data. Lastly, bubbles). The United States has always played a leader- qualitative or descriptive information should accompany ship role internationally in the development of standards quantitative measures to explain the economic tenets, social for statistical measurement of STI activities. This report is norms, regulatory constructs, and political atmosphere with intended to offer the National Center for Science and Engi- which the STI system engages. Qualitative information and neering Statistics (NCSES) at the U.S. National Science even case studies allow for deeper insights into not only what Foundation guidance that will help keep it at the forefront happened but also why. of this endeavor. The user community relies on STI indicators for answer- The primary audience for this report is the sponsor— ing key questions regarding the global science and tech- NCSES—as well as similar statistical agencies that produce nology enterprise, including: What are direct measures of data and statistics on innovative activities worldwide. The innovation, and what drives innovation? Where is leadership report assesses the demand for STI indicators from dif- in science and engineering knowledge generation trending? ferent perspectives: national, international, subnational, What is the status of STEM talent around the world? What is and sectoral. Although STI indicators often are retrospec- the portfolio of spending and other support by governments tive—measuring stocks, flows, and networks within the and private firms and organizations for STI activities, par- system—some bellwethers can show prospective trends in ticularly those at universities? What institutions, networks, science, technology, engineering, and mathematics (STEM) and regulations facilitate or impede advances in science, talent; new areas of scientific exploration; potentially vibrant technology, and entrepreneurship? What is the trend in online regions of innovation activity; and arcs toward the interna- learning in the United States and abroad, and what impact tional rise or decline of countries based on their technological will that have on university finance, operations and recruit- capabilities. STI indicators, therefore, can be used to report ment? What are important subnational collaborative activi- macro-level STI activities, outputs, outcomes, and linkages, ties that promote innovation and economic growth? What are as well as micro-level metrics related to actors and intermedi- the general perceptions about science and the public value ate outputs in the system. New indicators can be developed of science in the general population in the United States and by layering, linking, or blending existing data and indicators. abroad? Dozens more such questions are enumerated in this The users of NCSES data and statistics also are expected report. to be an important audience for this report. The user com- Given its broad disciplinary scope, the study was con- munity for NCSES’s STI indicators is diverse and includes ducted by a panel of experts that collectively represent more the National Science Board; staff at other statistical agencies than a dozen fields, including computer science, economics, in the United States and abroad; local, state, federal, and education, engineering, finance, geography, mathematics, ix

OCR for page R1
x PREFACE physics, political science, psychology, statistics, and visual indicators but also addressed processes for prioritizing data analytics. The panel also reflects the international nature of development and the production of indicators in the future, the topic, with members from Canada, Denmark, France, because it was clear that the changing environment in which and the Netherlands. NCSES operates is a key determinant of the agency’s priori- In undertaking this study, the panel first relied on users, ties from year to year. Internal processes that are observant, experts, and written reports and peer-reviewed articles to networked, and statistically and analytically balanced are establish current and anticipated user needs for STI indica- important for NCSES’s indicators program. tors. Second, the panel recognized that no one model informs On request of the sponsor, an interim report was published the types of indicators NCSES needs to produce. Policy in February 2012, summarizing the panel’s early findings questions served as an important guide to the panel’s review, and recommendations. The recommendations offered in but the study was also informed by systems approaches this report expand on those of the interim report. They are and international comparability. Third, it was important to intended to serve as the basis for a strategic program of work identify data resources and tools NCSES could exploit to that will enhance NCSES’s ability to produce indicators that develop its indicators program. Understanding the network capture change in STI to inform policy and optimally meet of inputs—including data from NCSES surveys, other fed- the needs of its user community. eral agencies, international organizations, and the private sector—that can be tapped in the production of indicators Robert E. Litan and Andrew W. Wyckoff, Cochairs gave rise to a set of recommendations for working with Panel on Developing Science, Technology, other federal agencies and public and private organizations. and Innovation Indicators for the Future Fourth, the panel did not limit its recommendations to

OCR for page R1
Acknowledgments* It is with extreme gratitude that the panel thanks the many Borga, and Carol Robbins (Bureau of Economic Analysis); people who made contributions to this study on science, Laurie Salmon, Jim Spletzer, and David Talan (Bureau of technology, and innovation (STI) indicators. The staff of Labor Statistics); David McGranahan and Tim Wojan (U.S. the National Center for Science and Engineering Statistics Department of Agriculture); Daniel McGrath, Jessica Shedd, (NCSES) at the National Science Foundation, under the Matthew Soldner, and Tom Weko (National Center for Edu- directorship of John Gawalt and formerly Lynda Carlson, cation Statistics); Stuart Graham (U.S. Patent and Trademark gracefully provided invaluable input and insights, including Office); and George Chacko and Walter Schaffer (National clear direction on what they wanted to learn from the study, Institutes of Health). We also thank Rochelle (Shelly) as well as useful sources of information from their division Martinez and her colleagues and Katherine Wallman at the and other resources. Robert Bell, Lawrence Burton, John Office of Management and Budget for an engaging discus- Jankowski, Nirmala Kannankutty, Beethika Khan, Rolf sion regarding synergies in the federal statistical system with Lehming, Francisco Moris, Jeri Mulrow, Christopher Pece, respect to measures of STI activities. and Emilda Rivers all contributed their knowledge and exper- Because international comparability is an important tise to answer our questions. aspect of this study, the panel convened two workshops of The panel’s work benefited greatly from presenters and international researchers and practitioners who use STI indi- attendees at our open meetings. The insights of the following cators. The first workshop, in July 2011, covered STI mea- individuals were critical for the framing of policy issues that sures and described opportunities and obstacles that NCSES are relevant to this study: Jeff Alexander (SRI International), should anticipate as it further develops its STI indicators Patrick Clemins (formerly of the American Association for program. We thank the workshop presenters: Shinichi Akaike the Advancement of Science), Mark Doms (U.S. Department (Hitotsubashi University, Tokyo), Howard Alper (Canada’s of Commerce), Matthew Gerdin (U.S. State Department), Science Technology and Innovation Council), Jayanta Kei Koizumi (U.S. Office of Science and Technology Pol- Chatterjee (Indian Institute of Technology), Gustavo Crespi icy), Christine Matthews (Congressional Research Service), (Inter-American Development Bank), Matthieu Delescluse Amber Hartman Scholz (President’s Council of Advisors (European Commission), Changlin Gao (Chinese Academy on Science and Technology), Dahlia Sokolov (U.S. House of Science and Technology), Jonathan Haskel (Imperial Col- of Representatives), and D. Greg Tassey (formerly of the lege Business School), Hugo Hollanders (United Nations National Institute of Standards and Technology). Conceptual University-Maastricht Economic and Social Research frameworks for STI indicators were presented by Michelle Institute on Innovation and Technology), Brian MacAulay Alexopoulos (University of Toronto), Bronwyn Hall (Uni- (National Endowment for Science, Technology and the Arts, versity of California, Berkeley), and Adam Jaffe (Brandeis London), and Philippe Mawoko (The New Partnership for University). Africa’s Development). Opportunities for advances in STI data collections and The second workshop, in June 2012, included par- statistics, particularly among U.S. federal agencies, were ticipants from the OECD-National Experts on Science discussed with panel members by B.K. Atrostic, Cheryl and Technology Indicators (NESTI) representing nations, Grim, Richard Hough, Dave Kinyon, Erika McEntarfer, and economic regions, and international institutions, includ- Mary Potter (U.S. Census Bureau); Ana Aizcorbe, Maria ing the African Union, Australia, Austria, Belgium, Brazil, Chile, the Czech Republic, Estonia, the European Union, *All listed affiliations are as of February 2014. Germany, Hungary, India, Israel, Italy, Japan, Korea, La Red xi

OCR for page R1
xii ACKNOWLEDGMENTS Iberoamericana e Interamericana de Indicadores de Ciencia bers. We thank Ray Bowen, Kelvin K. Droegemeier, José- y Tecnología (RICYT), Luxembourg, the Netherlands, Marie Griffiths, Arthur Reilly, and Arnold F. Stancell for Norway, Poland, Portugal, Russia, the Slovak Republic, in-depth and insightful responses to our questions, as well as South Africa, Spain, Sweden, Switzerland, Turkey, the the board’s staff members Jennie Moehlmann, Michael Van United Kingdom, and the United States. The panel chair, Woert, and Matthew Wilson, for facilitating the meetings. study director, and panel member Fred Gault, who is also This report has been reviewed in draft form by individuals chair of the NESTI advisory board, discussed the interim chosen for their diverse perspectives and technical expertise, report from the panel and solicited comments from attend- in accordance with procedures approved by the National ees, primarily on which indicators would be most useful Research Council’s Report Review Committee. The purpose to policy makers in their nations or regions. We thank the of this independent review is to provide candid and criti- presenters, who discussed prioritization methods and specific cal comments that will assist the institution in making its indicators that are important for international comparisons: published report as sound as possible and to ensure that the Alessandra Colecchia, Dominique Guellec, and Joaquim report meets institutional standards for objectivity, evidence, Oliveria Martins (OECD); Matthieu Delescluse (European and responsiveness to the study’s charge. The review com- Commission); Almamy Konté (African Union); Leonid ments and draft manuscript remain confidential to protect the Gokhberg (National Research University, Russia); and Veijo integrity of the deliberative process. We wish to thank the Ritola (Eurostat). following individuals for their review of this report: Henry During the course of its work, the panel also obtained Brady, Office of the Dean, Goldman School of Public Policy, input from several other science and technology policy University of California, Berkeley; Michael Conlon, Clini- experts, including Aaron Chatterji (Duke University, for- cal and Translation Science Institute, University of Florida; merly of the Council of Economic Advisers); Bhavya Lal Martin Fleming, chief economist and vice president, Busi- and Stephanie Shipp (Institute for Defense Analyses–Science ness Performance Services, International Business Machines and Technology Policy Institute); Donna Ginther (University Corporation; Jacques S. Gansler, Center for Public Policy of Kansas); and Alessandra Colecchia, Gili Greenberg, and and Private Enterprise, School of Public Policy, University Fernando Galindo-Rueda (OECD). The panel explored the of Maryland; Christopher T. Hill, Emeritus Department of use of microdata, particularly administrative records and Public Policy and Technology, School of Public Policy, web tools, to create STI statistics. We heard from several George Mason University; Graham G. Kalton, Westat, Inc., experts in this diverse field of study at the July 2011 work- Rockville, Maryland; Jason Owen-Smith, Barger Leadership shop, including Carl Bergstrom (University of Washington), Institute, University of Michigan; Georgine M. Pion, Depart- Stefano Bertuzzi (National Institutes of Health and the STAR ment of Psychology and Human Development, Vanderbilt METRICS Program), Erik Brynjolfsson (Massachusetts University; Hal Salzman, Bloustein School of Planning and Institute of Technology), Lee Giles (Pennsylvania State Public Policy and Heldrich Center for Workforce Develop- University), John Haltiwanger (University of Maryland), ment, Rutgers University; Phillip Swagel, School of Public Richard Price (Academia.edu), and Alicia Robb (Kauffman Policy, University of Maryland; Albert H. Teich, director, Foundation). Science and Policy Programs, American Association for the The development of STI indicators at subnational levels Advancement of Science; and Ward Ziarko, Scientific and was also an important topic for this study. At the July 2011 Technical Information Service, Belgian Federal Science workshop, Rob Atkinson (Information Technology and Inno- Policy. vation Foundation), Maryann Feldman (University of North Although the reviewers listed above provided many Carolina), Andrew Reamer (George Washington University), constructive comments and suggestions, they were not and Robert Samors and David Winwood (Association of asked to endorse the report’s conclusions or recommenda- Public and Land-grant Universities) presented options for tions, nor did they see the final draft of the report before its measuring STI activities at a variety of geographic scales. release. The review of this report was overseen by Chuck Nicholas Donofrio (IBM) participated in a roundtable Phelps, provost emeritus, University of Rochester, and discussion with panel members during the workshop. We John Haltiwanger, Department of Economics, University greatly appreciate his insights from a business perspective of Maryland. Appointed by the National Research Council, on measuring research and development and innovation. His they were responsible for making certain that an independent comments reminded us that the role of multinational corpora- examination of this report was carried out in accordance with tions in the global STI system should be examined carefully institutional procedures and that all review comments were and that entrepreneurial activities at firms of various sizes carefully considered. Responsibility for the final content of deserve careful measure. this report rests entirely with the authoring panel and the Because the National Science Board (NSB) is a primary institution. user of NCSES’s STI indicators—the biennial Science and We extend special thanks to staff of the Committee on Engineering Indicators volumes are published by NSB—the National Statistics. The study director, Kaye Husbands panel conducted two rounds of interviews with board mem- Fealing, provided invaluable assistance to the panel in orga-

OCR for page R1
ACKNOWLEDGMENTS xiii nizing the meetings and preparing this report. Connie Citro, map exercises for this study and cowrote the accompanying Tom Plewes, and Michael Cohen gave excellent guidance data appendix (Appendix F). Anthony Mann provided out- to the panel. Stephen Merrill, director of the Board on Sci- standing administrative and logistical support to the panel. ence, Technology, and Economic Policy, contributed to panel Our Mirzayan fellow, Daniel Grady, drew on his expertise in meetings. Esha Sinha assembled and developed a public use systems dynamics and web tools to benefit the panel’s work. databank of STI indicators from several international sources (see http://sites.nationalacademies.org/DBASSE/CNSTAT/ Robert E. Litan and Andrew W. Wyckoff, Cochairs Science_Technology_and_Innovation_Indicators/index.htm Panel on Developing Science, Technology, [May 2014]. In collaboration with panel member Leland and Innovation Indicators for the Future Wilkinson, she also conducted the cluster analysis and heat

OCR for page R1

OCR for page R1
Acronyms, Abbreviations, and Descriptions ACRONYMS AND ABBREVIATIONS America COMPETES America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, Act and Science APLU Association of Public and Land-grant Universities BEA Bureau of Economic Analysis, U.S. Department of Commerce BERD business enterprise expenditure on research and development BLS Bureau of Labor Statistics, U.S. Department of Labor BOP balance of payments BRDIS Business Research and Development and Innovation Survey CICEP Commission on Innovation, Competitiveness, and Economic Prosperity CIS Community Innovation Survey ERS Economic Research Service, U.S. Department of Agriculture ETA Employment and Training Administration, U.S. Department of Labor EU European Union FTE full-time equivalent GBAORD government budget appropriations or outlays for research and development GDP gross domestic product GERD gross domestic expenditure on research and development GO→SPIN Global Observatory on Science, Technology and Innovation Policy Instruments GOVERD government intramural expenditure on research and development HERD higher education expenditure on research and development or Higher Education Research and Development Survey ICT information and communication technology IMF International Monetary Fund IPEDS Integrated Postsecondary Education Data System MEP Manufacturing Extension Partnership MIST Microbusiness, Innovation Science, and Technology MOOC massive open online course xv

OCR for page R1
xvi CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION NAICS North American Industry Classification System NCES National Center for Education Statistics, U.S. Department of Education NCSES National Center for Science and Engineering Statistics, U.S. National Science Foundation NEPAD New Partnership for Africa’s Development NESTI National Experts on Science and Technology Indicators NIH National Institutes of Health, U.S. Department of Health and Human Services NIST National Institute of Standards and Technology, U.S. Department of Commerce NLM National Library of Medicine, U.S. Department of Health and Human Services NSCG National Survey of College Graduates NSF U.S. National Science Foundation NSRCG National Survey of Recent College Graduates OECD Organisation for Economic Co-operation and Development OMB U.S. Office of Management and Budget OSTP U.S. Office of Science and Technology Policy PISA Programme for International Student Assessment R&D research and development S&E science and engineering S&T science and technology SBIR Small Business Innovation Research SDR Survey of Doctorate Recipients SED Survey of Earned Doctorates SEI Science and Engineering Indicators SESTAT Science and Engineering Statistical Data System SIBS Survey of Innovation and Business Strategy STAR METRICS Science and Technology for America’s Reinvestment: Measuring the Effect of Research on Innovation, Competitiveness and Science STEM science, technology, engineering, and mathematics STI science, technology, and innovation STTR Small Business Technology Transfer UNESCO United Nations Educational, Scientific and Cultural Organization UNU-MERIT United Nations University’s Maastricht Economic and Social Research Institute on Innovation and Technology USDA U.S. Department of Agriculture DESCRIPTIONS BERD: Research and development expenditure in the business enterprise sector in a given year at the regional level. GBAORD: Government budget appropriations or outlays for research and development, a way of measuring government sup- port for research and development activities. GERD: Gross domestic expenditure on research and development, defined as total intramural expenditure on research and development performed on the national territory during a given period. GO→SPIN: Global Observatory on Science, Technology and Innovation Policy Instruments. Contains not only indicators but also an inventory of science, technology, and innovation (STI) national system descriptions; STI legal frameworks (with access to text of the acts and decrees); and an inventory of operational policy instruments that promote STI activities in a country. Developed by a group at the United Nations Educational, Scientific and Cultural Organization (UNESCO) Regional Bureau for Science for Latin America and the Caribbean.

OCR for page R1
ACRONYMS, ABBREVIATIONS, AND DESCRIPTIONS xvii HERD: Research and development expenditure in the higher education sector in a given year at the national and different subnational geographic scales. Human capital: The ability, knowledge, and skill base that are typically acquired or enhanced by an individual through edu- cation and training. Innovation: The implementation of a new or significantly improved product (good or service) or process; a new marketing method; or a new organizational method in business practices, workplace organization, or external relations (OECD-Eurostat, 2005, p. 46). A common feature of an innovation is that it must have been implemented. A new or improved product is imple- mented when it is introduced on the market. New processes, marketing methods, or organizational methods are implemented when they are brought into actual use in a firm’s operations (OECD-Eurostat, 2005, p. 47). NLSY: National Longitudinal Survey of Youth, conducted by the Bureau of Labor Statistics. Research and development (R&D): Comprises creative work undertaken on a systematic basis to increase the stock of knowledge, including knowledge of man, culture, and society, and the use of this stock of knowledge to devise new applica- tions (OECD, 2002a, p. 30). Science and technology (S&T): A broad concept that includes science and technology activities, defined as follows: “For statistical purposes, Scientific and Technological Activities (STA) can be defined as all systematic activities which are closely concerned with the generation, advancement, dissemination, and application of scientific and technical knowledge in all fields of science and technology, that is the natural sciences, engineering and technology, the medical and the agricultural sciences (NS), as well as the social sciences and humanities (SSH)” (United Nations Educational, Scientific and Cultural Organization, 1984, p. 17). Also included are Scientific and Technological Services (STS) and Scientific and Technological Education and Training (STET), the definitions of which are found in UNESCO (1978). Research and development is included in STA. The OECD Frascati Manual (OECD, 2002a, p. 19) notes that “R&D (defined similarly by UNESCO and the OECD) is thus to be distinguished from both STET and STS.” In the Frascati definition, R&D includes basic research, applied research, and experimental development. SIBS: Survey of Innovation and Business Strategy, conducted by Statistics Canada. Statistic: A numerical fact or datum, especially one computed from a sample. Statistical data: Data from a survey or administrative source used to produce statistics (OECD, 2002b, pp. 205-230). Statistical indicator: A statistic, or combination of statistics, providing information on some aspect of the state of a system or of its change over time. For example, gross domestic product (GDP) provides information on the level of value added in the economy, and its change over time is an indicator of the economic state of the nation. The decline of GDP for two quarters is indicative of a recession. The ratio of gross domestic expenditure on research and development (GERD) to GDP is an indicator of the formal generation of new knowledge and is used both for international comparisons and for the setting of targets, such as the Lisbon target for the European Union of 3 percent. There are also composite indicators involving many component indica- tors. A single indicator is indicative but not definitive in its description of the system. As an example, GDP per capita provides one piece of information about an economy and may be indicative of wealth or productivity, but the income distribution for the country, another indicator summarized in a Gini coefficient, provides complementary information on income inequality. Employment is yet another indicator of the state of the economy. Statistical information: Statistical data, or a statistic, placed in a context. As an example, the number of people making less than a dollar a day in a country is a statistic populated by statistical data that may result from estimation based on a sample. The context is the analysis of poverty, and in that context, the statistic provides information on poverty, but it is only one dimension.

OCR for page R1

OCR for page R1
Contents SUMMARY 1 1 INTRODUCTION 5 Challenges, 6 Charge to the Panel, 7 Organizing Framework, 7 Study Approach, 8 Report Organization, 10 2 CONCEPTS AND USES OF INDICATORS 11 Desirable Attributes of Indicators, 11 In Search of a Framework, 12 A Policy-Driven Framework, 13 User Priorities, 18 Cautions, Possibilities, and Limitations, 27 Summary, 28 3 DATA RESOURCES FOR INDICATORS 29 NCSES Databases, 29 NCSES’s STI Indicators, 31 Heat Map Exercise, 32 Gaps in STI Indicators That NCSES Should Fill, 35 Prioritization, 38 Summary, 39 4 MEASURING INNOVATION 41 Definitions, 41 Why Measure Innovation?, 43 Policy Relevance of Innovation Measures, 44 The Role of Innovation Surveys, 45 Improvements to BRDIS, 50 Use of Nontraditional Methodologies, 56 Summary, 58 5 MEASURING THE THREE K’S: KNOWLEDGE GENERATION, KNOWLEDGE NETWORKS, AND KNOWLEDGE FLOWS 59 Codified Definitions, 60 Measuring Science and Technology: Major Gaps in International Comparability, 61 xix

OCR for page R1
xx CONTENTS Traditional Indicators of the Three K’s, 63 Business R&D: R&D Services and Intangible Assets, 65 Summary, 71 6 MEASURING HUMAN CAPITAL 73 What Users Want, 73 NCSES’s Existing Human Capital Indicators, 73 The Survey Problem, 76 Potential for New Data Sources, 78 Revised and New Human Capital Indicators, 79 Key Opportunities, 85 Summary, 85 7 A PARADIGM SHIFT IN DATA COLLECTION AND ANALYSIS 87 New Methods, New Data Sources, 87 Implications for NCSES, 88 Indicators from Frontier Tools: Example of the Data Science Discipline, 89 A New Direction for NCSES, 93 Next Steps, 96 Summary, 98 Annex 7-1: Potential Data Sources to Explore, 98 8 INFORMING THE STRATEGIC PLANNING PROCESS 101 NCSES’s Accomplishments, 101 Processes for Change, 102 A Program of Work, 105 Panel Recommendations: Chapters 2-7, 107 REFERENCES 111 APPENDIXES A Biographical Sketches of Panel Members and Staff 119 B  Users of Science, Technology, and Innovation (STI) Data and Indicators and Their Questions and Requests for STI Indicators 123 C  Workshop on Developing Science, Technology, and Innovation Indicators for the Future Agenda and Participants 129 D OECD-National Experts on Science and Technology Indicators (NESTI) Workshop and Attendees 135 E National Center for Science and Engineering Statistics Research Abstracts 2012 139 F Science, Technology, and Innovation Databases and Heat Map Analysis 143 G 2011 BRDIS Results 187 H 2011 BRDIS Table 46 233 I 2011 BRDIS Table 47 241 J 2011 BRDIS Table 48 249 K 2011 BRDIS Table 49 251