found in Gault (2010), who outlines four ways that indicators are used for policy purposes: monitoring, benchmarking, evaluating, and “foresighting.”3
At the panel’s workshop, several presenters described attributes of indicators that NCSES should keep in mind as it develops new STI indicators. One important desirable attribute that was emphasized is a low sensitivity to manipulation. In addition, STI indicators are like baseball statistics—it is unlikely that one single statistic tells the whole story. Instead, users will need to rely on a collection or suite of indicators. Mindfully, during the workshop, Hugo Hollanders, of UNU-MERIT,4 stated that there is both political and media appeal of composite indices.5 Other ideal characteristics of indicators that workshop participants mentioned included scientifically derived/evidence based; comparable across regions; powerful for communication; affordable; accessible; scalable; sustainable; and policy and analytically relevant. STI indicators need to be policy neutral, even though the particular ones selected may reflect the preferences of the stakeholders who request them.
Although the production of indicators across many fields has an established history, there are at least three major cautions regarding their use that are important to note.
3For example, at the panel’s workshop Changlin Gao reported that China is targeting its STI indicators’ program on the four broad measures: (1) monitoring (international innovation system, linkages within and between national innovation systems; regional innovation systems and industrial clusters; firms; innovation; the implementation of national S&T projects; the selected quantitative indicators in the S&T development goals); (2) evaluating (performance of public investment on S&T; performance of government research institutes and national labs; national S&T programs; specialization of S&T fields; advantages versus disadvantages; new emerging industries, such as information technology, biotechnology industry, energy, health, knowledge based services, and etc.); (3) benchmarking (international benchmarking; interprovincial benchmarking); and (4) forecasting (the latest data not available in gathered statistics).
4UNU-MERIT—the U.N. University Maastricht Economic and Social Research Institute on Innovation and Technology—is a research and training center of the United Nations University and works in close collaboration with the University of Maastricht.
5To clarify, the panel is not advocating that NCSES develop a “headline indicator.” A suite of key STI indicators should be more informative for users of the statistics.