Click for next page ( R2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
PREPUBLICATION COPY: UNCORRECTED PROOFS Capturing Change in Science, Technology, and Innovation: Improving Indicators to Inform Policy Panel on Developing Science, Technology, and Innovation Indicators for the Future Robert E. Litan, Andrew W. Wyckoff, and Kaye Husbands Fealing, Editors Committee on National Statistics Division of Behavioral and Social Sciences and Education Board on Science, Technology, and Economic Policy Division of Policy and Global Affairs ADVANCE COPY NOT FOR PUBLIC RELEASE BEFORE Monday, November 25, 2013 8:00 A.M. EST PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. The project that is the subject of this report was supported by grant no. SES-0453930 between the National Science Foundation and the National Academy of Sciences. Support for the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SES- 1024012). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number-??? International Standard Book Number-??? Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2014 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Research Council. (2014). Capturing Change in Science, Technology, and Innovation: Improving Indicators to Inform Policy, Panel on Developing Science, Technology, and Innovation Indicators for the Future, R.E. Litan, A.W. Wyckoff, and K.H. Fealing, Editors. Committee on National Statistics, Division of Behavioral and Social Sciences and Education, and Board on Science, Technology, and Economic Policy, Division of Policy and Global Affairs. Washington, DC: The National Academies Press. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council. www.national-academies.org PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page R1
PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page R1
PANEL ON DEVELOPING SCIENCE, TECHNOLOGY, AND INNOVATION INDICATORS FOR THE FUTURE ROBERT E. LITAN (Cochair), Bloomberg Government, Washington, DC ANDREW W. WYCKOFF (Cochair), OECD, Paris, France CARTER BLOCH, Danish Center for Studies in Research and Research Policy, University of Aarhus, Denmark NICHOLAS R. CHRISMAN, Geospatial Science, RMIT University, Melbourne, Australia CARL J. DAHLMAN, Edmund A. Walsh School of Foreign Service, Georgetown University, Washington, DC GEOFF M. DAVIS, Quantitative Insights Group, Google, Inc., Mountain View, CA KATHARINE G. FRASE, Industry Research, IBM, Yorktown Heights, NY BARBARA M. FRAUMENI, Muskie School of Public Service, University of Southern Maine RICHARD B. FREEMAN, Department of Economics, Harvard University FREDERICK D. GAULT, Economic and Social Research and Training Centre on Innovation and Technology, United Nations University, Maastricht, The Netherlands DAVID GOLDSTON, Natural Resources Defense Council, Washington, DC MICHAEL MANDEL, The Wharton School, University of Pennsylvania JOHN E. ROLPH, Marshall School of Business, University of Southern California LELAND WILKINSON, Data Visualization, Skytree, Inc., San Jose, CA KAYE HUSBANDS FEALING, Study Director ESHA SINHA, Associate Program Officer ANTHONY S. MANN, Program Coordinator PREPUBLICATION COPY: UNCORRECTED PROOFS v

OCR for page R1
COMMITTEE ON NATIONAL STATISTICS LAWRENCE D. BROWN (Chair), Department of Statistics, The Wharton School, University of Pennsylvania JOHN M. ABOWD, School of Industrial and Labor Relations, Cornell University MARY ELLEN BOCK, Department of Statistics, Purdue University DAVID CARD, Department of Economics, University of California, Berkeley ALICIA CARRIQUIRY, Department of Statistics, Iowa State University MICHAEL E. CHERNEW, Department of Health Care Policy, Harvard Medical School CONSTANTINE GATSONIS, Center for Statistical Sciences, Brown University JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan MICHAEL HOUT, Survey Research Center, University of California, Berkeley SALLIE ANN KELLER, Virginia Bioinformatics Institute, Virginia Polytechnic Institute and State University LISA LYNCH, The Heller School for Social Policy and Management, Brandeis University COLM A. O’MUIRCHEARTAIGH, Harris Graduate School of Public Policy Studies, The University of Chicago RUTH D. PETERSON, Criminal Justice Research Center, The Ohio State University EDWARD H. SHORTLIFFE, Columbia University and Arizona State University HAL STERN, Donald Bren School of Information and Computer Sciences, University of California, Irvine CONSTANCE F. CITRO, Director PREPUBLICATION COPY: UNCORRECTED PROOFS vi

OCR for page R1
BOARD ON SCIENCE, TECHNOLOGY, AND ECONOMIC POLICY PAUL L. JOSKOW (Chair), President, Alfred P. Sloan Foundation, New York, NY ERNEST R. BERNDT, Sloan School of Management, Massachusetts Institute of Technology JOHN DONOVAN, Chief Technology Officer, AT&T Labs, Dallas, TX MARY L. GOOD, Donaghey College of Information Science and Systems Engineering, University of Arkansas at Little Rock RICHARD K. LESTER, Department of Nuclear Science and Engineering, Massachusetts Institute of Technology WILLIAM F. MEEHAN, Stanford Graduate School of Business, Stanford University DAVID T. MORGENTHALER, Founding Partner, Morgenthaler Ventures, Cleveland, OH ARATI PRABHAKAR, U.S. Venture Partners, Menlo Park, California WILLIAM J. RADUCHEL, Opera Software ASA, Great Falls, VA KATHRYN L. SHAW, Stanford Graduate School of Business, Stanford University LAURA D’ANDREA TYSON, Haas School of Business, University of California, Berkeley HAROLD R. VARIAN, Chief Economist, Google, Inc., Mountain View, CA ALAN WM. WOLFF, Dewey & LeBoeuf, LLP, Washington, DC RALPH J. CICERONE, Ex Officio, President, National Academy of Sciences, Washington, DC HARVEY V. FINEBERG, Ex Officio, President, Institute of Medicine, Washington, DC CHARLES M. VEST, Ex Officio, President, National Academy of Engineering, Washington, DC STEPHEN A. MERRILL, Executive Director PREPUBLICATION COPY: UNCORRECTED PROOFS vii

OCR for page R1
PREPUBLICATION COPY: UNCORRECTED PROOFS viii

OCR for page R1
Contents Preface xiii Acronyms, Abbreviations and Descriptions xix Summary S-1 1 Introduction 1-1 Challenges 1-3 Charge to the Panel 1-4 Organizing Framework 1-5 Study Approach 1-7 Report Organization 1-8 2 Concepts and Uses of Indicators 2-1 Desirable Attributes of Indicators 2-1 In Search of a Framework 2-3 User Priorities 2-11 Cautions, Possibilities, and Limitations 2-23 Summary 2-26 3 Data Resources for Indicators 3-1 NCSES Databases 3-1 NCSES’s STI Indicators 3-7 Heat Map Exercise 3-9 Gaps in STI Indicators that NCSES Should Fill 3-13 Prioritization 3-17 Summary 3-17 4 Measuring Innovation 4-1 Definitions 4-2 Why Measure Innovation? 4-3 Policy Relevance of Innovation Measures 4-6 The Role of Innovation Surveys 4-8 Improvements to the BRDIS 4-14 Use of Nontraditional Methodologies 4-23 Summary 4-27 PREPUBLICATION COPY: UNCORRECTED PROOFS ix

OCR for page R1
5 Measuring the Three K’s: Knowledge Generation, Knowledge Networks, 5-1 and Knowledge Flows Codified Definitions 5-3 Measuring Science and Technology: Major Gaps in International Comparability 5-4 Traditional Indicators of the Three K’s 5-8 Business R&D: R&D Services and Intangible Assets 5-11 Summary 5-20 6 Measuring Human Capital 6-1 What Users Want 6-1 NCSES’s Existing Human Capital Indicators 6-3 The Survey Problem 6-6 Potential for New Data Sources 6-9 Revised and New Human Capital Indicators 6-10 Key Opportunities 6-19 Summary 6-20 7 A Paradigm Shift in Data Collection and Analysis 7-1 New Methods, New Data Sources 7-1 Implications for NCSES 7-3 Indicators from Frontier Tools: Example of the Data Science Discipline 7-5 A New Direction for NCSES 7-10 Next Steps 7-16 Summary 7-19 Annex 7-1: Potential Data Sources To Explore 7-20 8 Informing the Strategic Planning Process 8-1 NCSES’s Accomplishments 8-1 Processes for Change 8-3 A Program of Work 8-8 References RE-1 Appendixes A Biographical Sketches of Panel Members and Staff A-1 B Users of Science, Technology, and Innovation (STI) Data and Indicators and B-1 Their Questions and Requests for STI Indicators C Workshop on Developing Science, Technology, and Innovation Indicators C-1 for the Future Agenda and Participants D OECD-National Experts on Science and Technology Indicators (NESTI) D-1 Workshop and Attendees E National Center for Science and Engineering Statistics Research Abstracts 2012 E-1 F Science, Technology, and Innovation Databases and Heatmap Analysis F-1 G 2009 BRDIS Results G-1 H 2009 BRDIS Table 40 H-1 I 2009 BRDIS Table 41 I-1 PREPUBLICATION COPY: UNCORRECTED PROOFS x

OCR for page R1
J 2009 BRDIS Table 42 J-1 K 2009 BRDIS Table 43 K-1 PREPUBLICATION COPY: UNCORRECTED PROOFS xi

OCR for page R1
PREPUBLICATION COPY: UNCORRECTED PROOFS xii

OCR for page R1
Preface Indicators are a scaffolding of statistics to which decision makers can relate other elements needed to make decisions. Indicators often are used to tell an end-to-end story on a policy-relevant topic. Science, technology, and innovation (STI) indicators are at a transition point in the formation of mainstream policy, especially in the current economic environment, in which a premium is placed on sustainable growth (e.g., growth that is not dependent on speculative bubbles). The United States has always played a leadership role internationally in the development of standards for statistical measurement of STI activities. This report is intended to offer the National Center for Science and Engineering Statistics (NCSES) at the U.S. National Science Foundation guidance that will help keep it at the forefront of this endeavor. The primary audience for this report is the sponsor—NCSES—as well as similar statistical agencies that produce data and statistics on innovative activities worldwide. The report assesses the demand for STI indicators from different perspectives: national, international, subnational, and sectoral. Although STI indicators often are retrospective—measuring stocks, flows, and networks within the system—some bellwethers can show prospective trends in science, technology, engineering, and mathematics (STEM) talent; new areas of scientific exploration; potentially vibrant regions of innovation activity; and arcs toward the international rise or decline of countries based on their technological capabilities. STI indicators, therefore, can be used to report macro-level STI activities, outputs, outcomes, and linkages, as well as micro-level metrics related to actors and intermediate outputs in the system. New indicators can be developed by layering, linking, or blending existing data and indicators The users of NCSES data and statistics also are expected to be an important audience for this report. The user community for NCSES’s STI indicators is diverse, and includes the National Science Board; staff at other statistical agencies in the United States and abroad; local, state, federal, and international policy makers and government administrators; academic researchers and administrators; and business managers and consultants. Users want indicators that relay timely information about drivers, trends, advances, vulnerabilities, culture/climate, and distributions related to the STI system. Users also are eager to explore nontraditional sources of data from which indicators are derived, and want to be informed about the reliability of those data. Lastly, qualitative or descriptive information should accompany quantitative measures to explain the economic tenets, social norms, regulatory constructs, and political atmosphere with which the STI system engages. Qualitative information and even case studies allow for deeper insights into not just what happened but also why. The user community relies on STI indicators for answering key questions regarding the global science and technology enterprise, including: What are direct measures of innovation, and what drives innovation? Where is leadership in science and engineering knowledge generation trending? What is the status of STEM talent around the world? What is the portfolio of spending and other support by governments and private firms and organizations for STI activities, particularly those at universities? What institutions, networks, and regulations facilitate or PREPUBLICATION COPY: UNCORRECTED PROOFS xiii

OCR for page R1
impede advances in science, technology, and entrepreneurship? What is the trend in online learning in the U.S. and abroad, and what impact will that have on university finance, operations and recruitment? What are important subnational collaborative activities that promote innovation and economic growth? What are the general perceptions about science and the public value of science in the general population in the U.S. and abroad? Dozens more such questions are enumerated in this report. Given its broad disciplinary scope, the study was conducted by a panel of experts that collectively represent more than a dozen fields, including computer science, economics, education, engineering, finance, geography, mathematics, physics, political science, psychology, statistics, and visual analytics. The panel also reflects the international nature of the topic, with members from Canada, Denmark, France, and the Netherlands. In undertaking this study, the panel first relied on users, experts, and written reports and peer-reviewed articles to establish current and anticipated user needs for STI indicators. Second, the panel recognized that no one model informs the types of indicators NCSES needs to produce. Policy questions served as an important guide to the panel’s review, but the study was also informed by systems approaches and international comparability. Third, it was important to identify data resources and tools NCSES could exploit to develop its indicators program. Understanding the network of inputs—including data from NCSES surveys, other federal agencies, international organizations, and the private sector—that can be tapped in the production of indicators gave rise to a set of recommendations for working with other federal agencies and public and private organizations. Fourth, the panel did not limit its recommendations to indicators, but also addressed processes for prioritizing data development and the production of indicators in the future, since it was clear that the changing environment in which NCSES operates is a key determinant of the agency’s priorities from year to year. Internal processes that are observant, networked, and statistically and analytically balanced are important for NCSES’s indicators program. On request of the sponsor, an interim report was published in February 2012, summarizing the panel’s early findings and recommendations. The recommendations offered in this report expand on those of the interim report. They are intended to serve as the basis for a strategic program of work that will enhance NCSES’s ability to produce indicators that capture change in science, technology, and innovation to inform policy and optimally meet the needs of its user community. Robert E. Litan and Andrew W. Wyckoff, Cochairs Panel on Developing Science, Technology, and Innovation Indicators for the Future PREPUBLICATION COPY: UNCORRECTED PROOFS xiv

OCR for page R1
Acknowledgments It is with extreme gratitude that the panel thanks the many people who made contributions to this study on science, technology, and innovation (STI) indicators. The staff of the National Center for Science and Engineering Statistics (NCSES) at the National Science Foundation, under the directorship of John Gawalt and formerly Lynda Carlson, gracefully provided invaluable input and insights, including clear direction on what they wanted to learn from the study, as well as useful sources of information from their division and other resources. Robert Bell, Lawrence Burton, John Jankowski, Nirmala Kannankutty, Beethika Khan, Rolf Lehming, Francisco Moris, Jeri Mulrow, Christopher Pece, and Emilda Rivers all contributed their knowledge and expertise to answer our questions. The panel’s work benefited greatly from presenters and attendees at our open meetings. The insights of the following individuals were critical for the framing of policy issues that are relevant to this study: Jeff Alexander (SRI International), Patrick Clemins (formerly of the American Association for the Advancement of Science), Mark Doms (U.S. Department of Commerce), Matthew Gerdin (U.S. State Department), Kei Koizumi (U.S. Office of Science and Technology Policy), Christine Matthews (Congressional Research Service), Amber Hartman Scholz (President’s Council of Advisors on Science and Technology), Dahlia Sokolov (U.S. House of Representatives), and D. Greg Tassey (formerly of the National Institute of Standards and Technology). Conceptual frameworks for science, technology, and innovation (STI) indicators were presented by Michelle Alexopoulos (University of Toronto), Bronwyn Hall (University of California, Berkeley), and Adam Jaffe (Brandeis University). Opportunities for advances in STI data collections and statistics, particularly among U.S. federal agencies, were discussed with panel members by B.K. Atrostic, Cheryl Grim, Richard Hough, Dave Kinyon, Erika McEntarfer, and Mary Potter (U.S. Census Bureau); Ana Aizcorbe, Maria Borga, and Carol Robbins (Bureau of Economic Analysis); Laurie Salmon, Jim Spletzer, and David Talan (Bureau of Labor Statistics); David McGranahan and Tim Wojan (U.S. Department of Agriculture); Daniel McGrath, Jessica Shedd, Matthew Soldner, and Tom Weko (National Center for Education Statistics); Stuart Graham (U.S. Patent and Trademark Office); and George Chacko and Walter Schaffer (National Institutes of Health). We also thank Rochelle (Shelly) Martinez and her colleagues and Katherine Wallman at the Office of Management and Budget for an engaging discussion regarding synergies in the federal statistical system with respect to measures of STI activities. Since international comparability is an important aspect of this study, the panel convened two workshops of international researchers and practitioners who use STI indicators. The first workshop, in July 2011, included covered STI measures and described opportunities and obstacles that NCSES should anticipate as it further develops its STI indicators program. We thank the workshop presenters: Shinichi Akaike (Hitotsubashi University), Howard Alper (Canada’s Science Technology and Innovation Council), Jayanta Chatterjee (Indian Institute of Technology), Gustavo Crespi (Inter-American Development Bank), Matthieu Delescluse PREPUBLICATION COPY: UNCORRECTED PROOFS xv

OCR for page R1
(European Commission), Changlin Gao (Chinese Academy of Science and Technology), Jonathan Haskel (Imperial College Business School), Hugo Hollanders (United Nations University-Maastricht Economic and Social Research Institute on Innovation and Technology), Brian MacAulay (National Endowment for Science, Technology and the Arts), and Philippe Mawoko (The New Partnership for Africa’s Development). The second workshop, in June 2012, included participants from the OECD-National Experts on Science and Technology Indicators (NESTI) members representing nations, economic regions, and international institutions, including the African Union, Australia, Austria, Belgium, Brazil, Chile, Czech Republic, Estonia, the European Union, Germany, Hungary, India, Israel, Italy, Japan, Korea, La Red Iberoamericana e Interamericana de Indicadores de Ciencia y Tecnología (RICYT), Luxembourg, the Netherlands, Norway, Poland, Portugal, Russia, the Slovak Republic, South Africa, Spain, Sweden, Switzerland, Turkey, the United Kingdom, and the United States. The panel chair, study director, and panel member Fred Gault, who is also chair of the NESTI advisory board discussed the interim report from the panel and solicited comments from attendees, primarily on which indicators would be most useful to policy makers in their nations or regions. We thank the presenters, who discussed prioritization methods and specific indicators that are important for international comparisons: Alessandra Colecchia, Dominique Guellec, and Joaquim Oliveria Martins (OECD); Matthieu Delescluse (European Commission); Almamy Konté (African Union); Leonid Gokhberg (National Research University, Russia); and Veijo Ritola (Eurostat). During the course of its work, the panel also obtained input from several other science and technology policy experts, including Aaron Chatterji (Duke University, formerly of the Council of Economic Advisers); Bhavya Lal and Stephanie Shipp (Institute for Defense Analyses–Science and Technology Policy Institute); Donna Ginther (University of Kansas); and Alessandra Colecchia, Gili Greenberg, and Fernando Galindo-Rueda (OECD). The panel explored the use of microdata, particularly administrative records and web tools, to create STI statistics. We heard from several experts in this diverse field of study at the July 2011 workshop, including Carl Bergstrom (University of Washington), Stefano Bertuzzi (National Institutes of Health and the STAR METRICS program), Erik Brynjolfsson (Massachusetts Institute of Technology), Lee Giles (Pennsylvania State University), John Haltiwanger (University of Maryland), Richard Price (Academia.edu), and Alicia Robb (Kauffman Foundation). The development of STI indicators at subnational levels was also an important topic for this study. At the July 2011 workshop, Rob Atkinson (Information Technology and Innovation Foundation), Maryann Feldman (University of North Carolina), Andrew Reamer (George Washington University), and Robert Samors and David Winwood (Association of Public and Land-grant Universities) presented options for measuring STI activities at a variety of geographic scales. Nicholas Donofrio (IBM) participated in a roundtable discussion with panel members during the workshop. We greatly appreciate his insights from a business perspective on measuring research and development and innovation. His comments reminded us that the role of multinational corporations in the global STI system should be examined carefully and that entrepreneurial activities at firms of various sizes deserve careful measure. Since the National Science Board (NSB) is a primary user of NCSES’s STI indicators— the biennial Science and Engineering Indicators volumes are published by NSB—the panel conducted two rounds of interviews with board members. We thank Ray Bowen, Kelvin K. Droegemeier, José-Marie Griffiths, Arthur Reilly, and Arnold F. Stancell for in-depth and PREPUBLICATION COPY: UNCORRECTED PROOFS xvi

OCR for page R1
insightful responses to our questions, as well as the board’s staff members Matthew Wilson, Michael Van Woert, and Jennie Moehlmann, for facilitating the meetings. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Henry Brady, Office of the Dean, Goldman School of Public Policy, University of California, Berkeley; Michael Conlon, Clinical and Translation Science Institute, University of Florida; Martin Fleming, Chief Economist and Vice President, Business Performance Services, International Business Machines Corporation; Jacques S. Gansler, Center for Public Policy and Private Enterprise, School of Public Policy, University of Maryland; Christopher T. Hill, Emeritus Department of Public Policy and Technology, School of Public Policy, George Mason University; Graham G. Kalton, Westat, Inc., Rockville. MD; Jason Owen-Smith, Barger Leadership Institute, University of Michigan; Georgine M. Pion, Department of Psychology and Human Development, Vanderbilt University Hal Salzman, Bloustein School of Planning and Public Policy and Heldrich Center for Workforce Development, Rutgers University; Phillip Swagel, School of Public Policy, University of Maryland; Albert H. Teich, Director, Science and Policy Programs, American Association for the Advancement of Science; and Ward Ziarko, Scientific and Technical Information Service, Belgian Federal Science Policy. Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the report’s conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Chuck Phelps, provost emeritus, University of Rochester, and John Haltiwanger, Department of Economics, University of Maryland. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring panel and the institution. We extend special thanks to staff of the Committee on National Statistics. The study director, Kaye Husbands Fealing, provided invaluable assistance to the panel in organizing the meetings and preparing this report. Connie Citro, Tom Plewes, and Michael Cohen gave excellent guidance to the panel. Stephen Merrill, director of the Board on Science, Technology, and Economic Policy, contributed to panel meetings. Esha Sinha assembled and developed a public use databank of STI indicators from several international sources. In collaboration with panel member Leland Wilkinson, she also conducted the cluster analysis and heat map exercises for this study and wrote the accompanying data appendix (Appendix F). Anthony Mann provided outstanding administrative and logistical support to the panel. Our Mirzayan fellow, Daniel Grady drew on his expertise in systems dynamics and web tools to benefit the panel’s work. Robert E. Litan and Andrew W. Wyckoff, Cochairs Panel on Developing Science, Technology, and Innovation Indicators for the Future PREPUBLICATION COPY: UNCORRECTED PROOFS xvii

OCR for page R1
PREPUBLICATION COPY: UNCORRECTED PROOFS xviii

OCR for page R1
Acronyms, Abbreviations, and Descriptions ACRONYMS AND ABBREVIATIONS AAAS American Association for the Advancement of Science America COMPETES America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science APLU Association of Public and Land Grant Universities BEA Bureau of Economic Analysis, U.S. Department of Commerce BERD business enterprise expenditure on research and development BLS Bureau of Labor Statistics, U.S. Department of Labor BOP balance of payments BRDIS Business Research and Development and Innovation Survey CICEP Commission on Innovation, Competitiveness, and Economic Prosperity CIS Community Innovation Survey COSSA Consortium of Social Science Associations CRS Congressional Research Service ERS Economic Research Service, U.S. Department of Agriculture ETA Employment and Training Administration, U.S. Department of Labor EU European Union FTE full-time equivalent GBAORD government budget appropriations or outlays for research and development GDP gross domestic product GERD gross domestic expenditure on research and development GO-SPIN Global Observatory on Science, Technology and Innovation Policy Instruments GOVERD government intramural expenditure on research and development HERD higher education expenditure on research and development or Higher Education Research and Development Survey ICT information and communication technology IMF International Monetary Fund IPEDS Integrated Postsecondary Education Data System IRS Internal Revenue Service, U.S. Department of the Treasury MEP Manufacturing Extension Partnership MIST Microbusiness, Innovation Science, and Technology MOOC massive open online course NAICS North American Industry Classification System NASSCOM National Association of Software and Services Companies NCES National Center for Education Statistics, U.S. Department of Education NCSES National Center for Science and Engineering Statistics, U.S. National Science Foundation PREPUBLICATION COPY: UNCORRECTED PROOFS xix

OCR for page R1
NEPAD New Partnership for Africa’s Development Nesta National Endowment for Science, Technology, and the Arts NESTI National Experts on Science and Technology Indicators NIH National Institutes of Health, U.S. Department of Health and Human Services NIST National Institute of Standards and Technology, U.S. Department of Commerce NLM National Library of Medicine, U.S. Department of Health and Human Services NLSY National Longitudinal Survey of Youth NSCG National Survey of College Graduates NSF U.S. National Science Foundation NSRCG National Survey of Recent College Graduates OMB U.S. Office of Management and Budget OSTP U.S. Office of Science and Technology Policy PISA Programme for International Student Assessment R&D research and development S&E science and engineering S&T science and technology SBIR Small Business Innovation Research SDR Survey of Doctorate Recipients SED Survey of Earned Doctorates SEI Science and Engineering Indicators SESTAT Science and Engineering Statistical Data System SIBS Survey of Innovation and Business Strategy STAR METRICS Science and Technology for America’s Reinvestment: Measuring the EffecT of Research on Innovation, Competitiveness and Science STEM science, technology, engineering, and mathematics STI science, technology, and innovation STTR Small Business Technology Transfer UNESCO United Nations Educational, Scientific and Cultural Organization UNU-MERIT United Nations University’s Maastricht Economic and Social Research Institute on Innovation and Technology USDA U.S. Department of Agriculture DESCRIPTIONS BERD: Research and development expenditure in the business enterprise sector in a given year at the regional level. GBAORD: Government budget appropriations or outlays for research and development, a way of measuring government support for research and development activities. GERD: Gross domestic expenditure on research and development, defined as total intramural expenditure on research and development performed on the national territory during a given period. PREPUBLICATION COPY: UNCORRECTED PROOFS xx

OCR for page R1
GO-SPIN: Global Observatory on Science, Technology and Innovation Policy Instruments. Contains not only indicators but also an inventory of science, technology, and innovation (STI) national system descriptions; STI legal frameworks (with access to text of the acts and decrees); and an inventory of operational policy instruments that promote STI activities in a country. Developed by a group at the United Nations Educational, Scientific and Cultural Organization (UNESCO) Regional Bureau for Science for Latin America and the Caribbean. HERD: Research and development expenditure in the higher education sector in a given year at the national and different subnational geographic scales. Human capital: The ability, knowledge, and skill base that are typically acquired or enhanced by an individual through education and training. Innovation: The implementation of a new or significantly improved product (good or service) or process; a new marketing method; or a new organizational method in business practices, workplace organization, or external relations (OECD-Eurostat, 2005, p. 46). A common feature of an innovation is that it must have been implemented. A new or improved product is implemented when it is introduced on the market. New processes, marketing methods, or organizational methods are implemented when they are brought into actual use in a firm’s operations (OECD-Eurostat, 2005, p. 47). NLSY: National Longitudinal Survey of Youth, conducted by the Bureau of Labor Statistics. Research and development (R&D): Comprises creative work undertaken on a systematic basis to increase the stock of knowledge, including knowledge of man, culture, and society, and the use of this stock of knowledge to devise new applications (OECD, 2002a, p. 30). Science and technology (S&T): A broad concept that includes science and technology activities, defined as follows: “For statistical purposes, Scientific and Technological Activities (STA) can be defined as all systematic activities which are closely concerned with the generation, advancement, dissemination, and application of scientific and technical knowledge in all fields of science and technology, that is the natural sciences, engineering and technology, the medical and the agricultural sciences (NS), as well as the social sciences and humanities (SSH)” (United Nations Educational, Scientific and Cultural Organization, 1984, p. 17). Also included are Scientific and Technological Services (STS) and Scientific and Technological Education and Training (STET), the definitions of which are found in UNESCO (1978). Research and development is included in STA. The OECD Frascati Manual (OECD, 2002a, p. 19) notes that “R&D (defined similarly by UNESCO and the OECD) is thus to be distinguished from both STET and STS.” In the Frascati definition, R&D includes basic research, applied research, and experimental development. SIBS: Survey of Innovation and Business Strategy, conducted by Statistics Canada. Statistic: A numerical fact or datum, especially one computed from a sample. Statistical data: Data from a survey or administrative source used to produce statistics (OECD, 2002b, pp. 205-230). Statistical indicator: A statistic, or combinations of statistics, providing information on some aspect of the state of a system or of its change over time. For example, gross domestic product (GDP) provides information on the level of value added in the economy, and its change over time is an indicator of the economic state of the nation. The decline of GDP for two quarters is PREPUBLICATION COPY: UNCORRECTED PROOFS xxi

OCR for page R1
indicative of a recession. The ratio of gross domestic expenditure on research and development (GERD) to GDP is an indicator of the formal generation of new knowledge and is used both for international comparisons and for the setting of targets, such as the Lisbon target for the European Union of 3 percent. There are also composite indicators involving many component indicators. A single indicator is indicative but not definitive in its description of the system. As an example, GDP per capita provides one piece of information about an economy and may be indicative of wealth or productivity, but the income distribution for the country, another indicator summarized in a Gini coefficient, provides complementary information on income inequality. Employment is yet another indicator of the state of the economy. Statistical information: Statistical data, or a statistic, placed in a context. As an example, the number of people making less than a dollar a day in a country is a statistic populated by statistical data that may result from estimation based on a sample. The context is the analysis of poverty, and in that context, the statistic provides information on poverty, but it is only one dimension. PREPUBLICATION COPY: UNCORRECTED PROOFS xxii