Appendix A
Workshop Agenda
NCSES/CNSTAT Workshop on Advancing Concepts and Models of Innovative Activity and STI Indicator Systems
National Academy of Sciences
2101 Constitution Ave. NW, Washington, DC 20418
Conference Room 120
Thursday-Friday, May 19-20, 2016
Workshop Goals
The workshop will bring together academic researchers, private- and public-sector experts, and public policy agencies to develop strategies for broadening and modernizing science, technology, and innovation (STI) information systems. The workshop will be oriented toward helping the NSF’s National Center for Science and Engineering Statistics (NCSES) refine and prioritize its work on innovation metrics to maximize the relevance and utility of its data collection programs and statistical products to users. The focus will be on conceptualizing innovation—its inputs, outputs and consequences—in a way that reveals elements that are being measured well and those that are being measured inaccurately or not at all. Presentations and discussion will take into account the role of innovation, not only as it affects economic growth and productivity directly, but also as a mechanism for creating greater public good and meeting social challenges such as those associated economic mobility, health, civic engagement, population aging, or climate change. Workshop participants should also imagine how new kinds of information—e.g., naturally occurring, unstructured, digital data—may be used to complement more traditional survey and administrative data in the construction of innovation metrics. Participants should seek to identify questions that cannot be answered now but could be with additional data that have a reasonable chance of being collected.
Thursday, May 19 (9 am-5 pm)
9:00 am | Welcome; Project Objectives |
(breakfast will be available outside conference room 120) | |
Key issues pertaining to the changing face of innovation and potential approaches to its measurement will be identified. NCSES will provide an overview of the agency’s goals and future directions, recent work, and the variety of studies initiated on innovation and human capital data. The focus on concepts and models in this workshop is meant to generate input to data development strategies that take into account both user needs and emerging opportunities. | |
|
|
9:15 | Assessing Innovation Measurement: How Accurately Do We Measure Innovation Processes and the Resultant Outcomes Delivered to Society and the Economy? |
Traditional measures of innovation, based largely on data reflecting expenditures on inputs (e.g., investment in R&D), capture the impact of innovation on economic outputs and outcomes in only a partial way. This has led for calls to collect new kinds of data and create new metrics. In this session, participants will identify aspects of the process and its impacts that are currently being measured well as well as those that are being measured incorrectly, incompletely, or not at all. Such assessment encompasses inputs to innovation (R&D and other), outputs and outcomes of innovation (economic growth, better functioning society), and the utility of metrics to stakeholders ranging from government to businesses. The takeaway from this discussion should be guidance to NSF about the extent to which the right data are being collected, and what additional kinds of information should be pursued. [15 minutes each] | |
|
|
Open Discussion [15 minutes] | |
10:45 | Break (refreshments available outside conference room 120) |
11:00 | Innovation Beyond R&D |
Data on narrow aspects of innovation—such as R&D expenditures or patent applications, assignments, and citations—are often used to proxy levels of innovation activity. However, in the measurement community, it has long been understood that activities both technical and nontechnical, including those that are not a directly measurable function of R&D and science, also play an important role in innovation. | |
NCSES’s Census Business R&D and Innovation Survey (BRDIS) asks firms for information about newly introduced or improved products and processes, and about purchased R&D inflows and outflows, revenue from the sale of patents and patent licensing, and a number of IP-related activities. Some questions about innovative activities can be adequately addressed with current data sources such as BRDIS, while others may require new or different kinds of data. For example, less is known about how innovation takes place in organizations that are not R&D intensive, as is often the case in the service sectors. Questions to be addressed here include: What models exist for assessing different kinds of (e.g., non- |
R&D-based) innovation? What do we know about innovators acquiring inventions from external sources? How would understanding the division of innovative labor affect our understanding of the propensity to innovate? [15 minutes each] | |
|
|
Open Discussion [15 minutes] | |
12:30 pm | Lunch (available outside conference room 120) |
1:30 | The Role of Individuals (and networks of individuals) in Innovation |
Compared with institutions and organizations, less attention has been given to the role of individuals and teams in the innovative process. How people’s education, entrepreneurial talents, and other human capital characteristics result in innovation are vast and largely unresolved research areas. One example of how innovation relates to individuals, with direct and high-profile policy content, is in the area of immigration and human capital formation. Linkages between individuals and institutions also play an important role in innovation. To what extent do we understand how these linkages to work? What opportunities to understand the role of human capital in innovation within a firm could be gained from human |
resource surveys and by linking existing data to other data sets? [15 minutes each] | |
|
|
Open Discussion [15 minutes] | |
3:00 | Break (refreshments available outside conference Room 120) |
3:15 | Measuring Public-Sector Innovation and Social Progress |
Measuring the full range of benefits generated through innovation requires capturing data on a range of activities that goes beyond those traditionally tracked. Innovation that takes place in the public sector is one area to be explored and is a high-priority topic within the international community. Innovation that advances the public good in ways that take place beyond the market (in noneconomic ways) is another related topic of interest. Both have featured in discussions about revision of the Oslo Manual and in planning for September’s Blue Sky Conference. | |
Questions to be addressed include the following: What approaches to innovation measurement can be developed for capturing activities within the public sector? Can innovation metrics be developed that bring into scope how the successful exploitation of ideas affects the provision of public goods and the well-being of society more broadly, beyond the contribution to efficiency, effectiveness, or quality in the production of market |
goods and services? Are there negative social impacts of innovation that should be included in our understanding of the outcomes of innovation? [15 minutes each] | |
|
|
Open Discussion [15 minutes]. Overarching questions for presenters during this session and for discussion afterward: | |
|
|
5:00 pm | Planned Adjournment |
Friday, May 20 (9 am-2 pm)
9:00 am | Welcome Back(breakfast will be available outside conference room 120) |
Scott Stern (MIT; workshop chair): Key questions, goals for the day | |
9:15 | Regional Innovation Models and Data Needs |
Many of the spillovers and complementarities of innovative activity take place at the local level. What models help policy makers and other data users at the sub-national level? [15 minutes each] | |
|
investments, recruiting new companies, and laying the groundwork for new industries. [15 minutes each] |
|
Open Discussion [15 minutes] | |
10:45 | Break (refreshments available outside conference room 120) |
11:00 | Innovation Measurement Agendas of the Future |
Going forward, new measures of science and technology describing inputs and outcomes associated with innovative activity will be needed, and multiple data modes will be called upon. Commercial data, some recoverable from Web scraping and other computer science methods, may shape future measurement in a range of areas—e.g., new product introduction, quality change, prices and productivity, product diffusion—where large datasets provide advantage in terms of granularity, timeliness, geographic specificity, etc. To what extent can the digital revolution transform metrics in the area of innovation measurement? What are the roles of specialized surveys in innovative data collection? Administrative records (e.g., linking patent information to other data sources)? [15 minutes each] | |
|
|
Open Discussion [15 minutes] | |
12:30 pm | Lunch (available outside conference room 120) |
Synthesis and Directions: Shaping Innovation Data and Indicators for the Future
Scott and Bronwyn will facilitate discussion summarizing guidance to NCSES as it prepares for the OECD Blue Sky III Conference. Insights aimed at broadening and modernizing innovation indicators—through advancement of measurement frameworks, identification of data gaps, and exploitation of new data sources—will be distilled. The goal is to emerge from the meeting envisioning priorities for NCSES data collection programs as the agency seeks to make its statistics relevant to data users and producers across measurement, academic, and practitioner communities spanning both the public and private sectors. | |
2:00 pm | Planned Adjournment |
This page intentionally left blank.