the global STI system. Also, frontier methods are emerging for monitoring phenomena such as the number of new product introductions through sophisticated web-scraping algorithms; for tracking innovation activities through help-wanted ads; and for tracing networks of scientists engaged in research through textual analysis of grant abstracts, patent applications, and online working papers and publications. At present, such data sources, although promising, are largely untested and hence of highly uncertain quality, which means they will require careful evaluation to determine those that may be suited for statistical use.
These new challenges and opportunities raise questions about whether NCSES’s current STI statistical activities are properly focused to produce the information needed by policy makers, researchers, and businesses. Such questions have become especially acute given tightening fiscal measures and the importance of innovation for economic growth and other aspects of social well-being.
STUDY PURPOSE AND METHODOLOGY
In response to a request from NCSES, the Committee on National Statistics and the Board on Science, Technology, and Economic Policy of the National Research Council convened the Panel on Developing Science, Technology, and Innovation Indicators for the Future. The panel was charged to assess and provide recommendations regarding the need for revised, refocused, and newly developed indicators of STI activities that would enable NCSES to respond to changing policy concerns. In carrying out its charge, the panel reviewed STI indicators from the United States as well as Australia, Canada, China, India, Japan, Russia, South Korea, and several countries in Africa, Europe, and Latin America. It consulted with a broad range of users of STI indicators in the academic, public, and private sectors and with statistical agencies and organizations that produce relevant information in the United States and abroad. Although its focus was on indicators, the panel also believed it important to identify and assess both existing and potential data resources and tools that NCSES could exploit to further develop its indicators program. Finally, the panel considered strategic pathways for NCSES to move forward with an improved STI indicators program.
The panel recognized that no one model informs the types of indicators NCSES needs to produce. Policy questions for the United States served as an important guide to the panel’s review, but the study was also informed by considerations of international comparability. In addition, the panel recognized the need to balance the introduction of new indicator series against the need to maintain long-standing bellwether indicators that require continual monitoring over long periods of time—for example, to ascertain the rise or decline of countries based on their technological capabilities.
The United States has historically played a leadership role internationally in the development of STI indicators, particularly in the areas of human capital and R&D expenditures. NCSES specifically has displayed areas of strength and innovation in its data collection and indicator development, outreach to the user community, and collaboration with statistical agencies in the United States and abroad. This report is intended to offer NCSES guidance that will help keep it at the forefront of providing accurate, relevant, timely, and objective STI indicators that can inform policy makers about the health of the STI enterprise, signal trends of policy concern, and suggest questions for in-depth research.
In reviewing STI indicators around the globe, the panel found a depth and breadth of indicator programs that is truly remarkable—many countries are placing a high priority on collecting information on innovation and related activities, and they are gathering high-quality data. Nevertheless, after hearing presentations from different countries ranging across the African, Asian, and European continents, the panel was unable to identify any proven STI indicators or methodologies used by other countries that NCSES lacks and could easily and inexpensively adopt for its own program. Given the global nature of STI, however, it is essential for NCSES to remain aware of the experimentation currently under way and to benefit from the lessons learned from these experiments.
The panel also identified a number of ways in which NCSES could improve its current STI indicators program with relatively little new investment in original data collection. Examples include increasing comparability with international classifications and concepts and improving the usefulness of the agency’s Business Research and Development and Innovation Survey (BRDIS) to policy makers and researchers.
Finally, changes in the economy have made it necessary to develop new concepts and measures of STI and its economic and social impacts. Economic changes also have given rise to new methodologies for data collection and analysis that have the potential to lower costs and expand the usefulness of STI indicators while enabling NCSES to maintain and enhance its global leadership. NCSES may find it difficult to fund and supervise the development of new STI measures and methodologies, especially while continuing its current program of STI indicators. Nonetheless, continued production of only the traditional STI measures will provide an incomplete and possibly misleading indication of how well or poorly the economies of the United States and other countries are performing in generating the innovations in products, services, and production and delivery chains that lead to improved living standards.