1

Introduction

BACKGROUND

This Proceedings of a Workshop summarizes the presentations and discussions at the Workshop on the Implications of Convergence for How the National Center for Science and Engineering Statistics (NCSES) Measures the Science and Engineering Workforce, which was held virtually and live-streamed on October 22–23, 2020. The workshop was convened by the Committee on National Statistics (CNSTAT) to help NCSES, a division of the National Science Foundation (NSF), set an agenda to inform its methodological research and better measure and assess the implications of convergence for the science and engineering (S&E) workforce and enterprise. While various definitions exist, the NSF website describes convergence research as having two primary characteristics: It is driven by a specific and compelling problem, and it requires deep integration across disciplines. The convergence paradigm “intentionally brings together intellectually diverse researchers to develop effective ways of communicating across disciplines by adopting common frameworks and a new scientific language. This may in turn help solve the problem that necessitated the collaboration, resulting in novel ways of framing research questions and opening new research vistas.”¹

The workshop was structured to bring together scientists and researchers from multiple disciplines, along with experts in science policy, university administration, and other stakeholders to review and provide input on

___________________

¹ See Convergence Research at NSF at https://www.nsf.gov/od/oia/convergence/index.jsp.

defining and measuring convergence and its impact on science and scientists. The workshop agenda is provided in Appendix A. Biographical sketches of the steering committee members and workshop presenters are included in Appendix B.

CNSTAT has a long history of providing advice on matters of science to the federal government and conducting independent reviews of federal statistical activities to improve the quality of statistical information, contribute to statistical policies, and coordinate the federal government’s activities aimed at providing a forward-looking vision for the federal statistical system. CNSTAT’s director, Brian Harris-Kojetin, described measurement as a frequent focus of CNSTAT’s work and as being fundamental to efforts by statistical agencies to provide reliable statistics on various topics. It requires statistical agencies to develop a deep understanding of their topic areas and clear definitions for what should be measured. This knowledge includes understanding what information exists, whether this information is well organized, how it can be accessed, and how agencies interpret the data, particularly if they are asked to provide information about them. Statistical agencies excel at the details that must be considered to accurately measure the nuances of a particular topic. The challenge to accurate measurement is that our society and economy are constantly evolving, requiring innovation in methods of measurement. Harris-Kojetin thanked NCSES for the opportunity to convene this workshop.

WORKSHOP FOCUS

The purpose of the workshop was to help NSF and NCSES “set an agenda to inform its methodological research and improve its data collection programs to better measure and assess the implications of convergence for the S&E workforce and enterprise.” Entwisle explained how convergence has expanded beyond the term coined nearly 10 years ago when MIT professors Phil Sharp and Robert Langer published their work on convergence and biomedical science in 2011.² Initially, convergence focused on joining engineering, physical science, and life science within academic institutions. It now has expanded to encompass novel disciplines with new outcomes. The workshop sought to explore the status of the definitions of convergence, its practice, its outputs, its effects, and promising paths forward for NCSES in this area. A central objective of the workshop was to help NCSES refine and prioritize its data collection and methodological research on convergence. Box 1-1 shows the workshop’s full statement of task.

___________________

² Sharp, P.A., and R. Langer. (2011). Research agenda. Promoting convergence in biomedical science. Science 333(6042):527. doi:10.1126/science.1205008.

THE NATIONAL SCIENCE FOUNDATION’S INTEREST IN MEASURING CONVERGENCE

Arthur (Skip) Lupia (assistant director, NSF) described 2020 as an ideal time to consider convergence because of the enormous challenges facing not only the United States but also countries across the globe. The COVID-19 pandemic has impacted lives everywhere, for example, in terms of health, education, business, and social supports. As one way of improving the quality of life in these circumstances, Lupia called attention to NSF’s Convergence Accelerator Program, which accelerates use-inspired convergence research in areas of national importance.³

Quoting American chemist Linus Pauling, he proposed “the best way to have good ideas is to have a lot of ideas.” Lupia added that the types of good ideas that can have the largest social impact are almost always novel combinations of existing concepts. The challenge for an agency such as NSF is that many “novel combinations” fall between the cracks of existing disciplinary categories. This is one reason that NSF seeks to develop new programs like the Convergence Accelerator.

Lupia identified interdisciplinarity, multidisciplinarity, and convergence as concepts that can energize and leverage ideas to effect new ways of thinking. These terms are often used interchangeably, but they have different

___________________

³ National Science Foundation. (October 9, 2020). Dear Colleague Letter: Request for Information on Future Topics for the NSF Convergence Accelerator (NSF 21-012). Available: https://www.nsf.gov/pubs/2021/nsf21012/nsf21012.jsp.

meanings (their definitions will be discussed throughout the workshop). Understanding that workshop participants would likely disagree about the meanings of these terms and the best ways to describe them, he tried to set expectations by providing an example of challenges associated with defining the term “implicit bias.” He and Jon Krosnick convened a workshop of leading experts on this topic at NSF. During that workshop, the experts spent 2 days engaged in rigorous and illuminating discussions. On day 3, they asked the experts to define implicit bias. It took more than 2 hours and was much more difficult than anyone expected.

Lupia then turned to the topic of measurement. NSF’s Social, Behavioral and Economic Sciences Directorate (SBE) seeks to support “rigorous, precise research that empowers people and improves quality of life.” SBE does this by supporting groundbreaking research, the creation of fundamental data and analytic infrastructure, and essential training. He also outlined SBE principles of rigor (the ability to explain how we know what we know), ethics (empowering people through awareness of the whole scientific work-flow), precision in measurement and conceptualization, and causality.

NCSES is part of SBE and is tasked with measuring key aspects of the nation’s science and engineering ecosystem. It must deliver measures that meet SBE and other federal standards. Moreover, the value of NCSES measures depends on their being broadly viewed as valid and legitimate by the research community.

With these criteria in mind, Lupia indicated that the concept of convergence is potentially ambiguous, and wondered whether it might be better to treat it as an aspirational idea rather than something to measure explicitly. To assess this possibility, Lupia posed five critical questions for the group:

1. Is a valid and broadly legitimate convergence measure possible, that is, a measure that signals standardization and can be reliably and consistently derived from the activities of diverse institutions?
2. Is it possible to create mutual exclusivity and collective exhaustion in critical aspects of the underlying concept? Is neuroscience, for instance, an example of convergent work, or is it its own discipline?
3. If we can answer the first two questions, who does the measure empower to effectively accomplish high-value goals to a greater degree?
4. If we answer all the preceding questions, what opportunity costs exist for NCSES to develop such a measure?
5. Having answered these questions, is convergence more effective as a motivational ideal than a measured phenomenon?

After Lupia’s presentation, Khargonekar asked whether Lupia is considering single or multiple measures of convergence when posing these

questions. Lupia responded that his view of convergence is a highly dimensional concept, and if other questions about the measure can be answered, he has no preconceived notion as to whether convergence should have a single measure or multiple measures. Holbrook followed by asking whether it is possible to answer both the first and fifth questions in the affirmative. Lupia expressed his belief that it is logically possible to answer both questions in the affirmative.

WORKSHOP GOALS FOR THE NATIONAL CENTER FOR SCIENCE AND ENGINEERING STATISTICS

Emilda Rivers (director, NCSES) described the needs, goals, and challenges for NCSES relative to the measurement of convergence. She agreed with Lupia that one or several concepts could be related to convergence and that NCSES’ goal would be to convey the reliability, precision, and transparency of the measurement process. This goal could become increasingly difficult to achieve if individual communities hold different definitions of convergence. Rivers organized her presentation as a set of questions directed to the steering committee. First, she asked whether any definitional issues could be overcome. Goals are sometimes framed with a policy emphasis that may differ significantly from an emphasis on measurement, and this difference can result in the loss of nuance and lead to more confusion rather than clarity. She then inquired about (1) the unintended consequences of measuring convergence and the gaps that may exist when providing a framework at the national level, (2) the communication of gaps where they exist, and (3) the means for accurately reflecting the impact of convergence on the scientific community.

As a federal agency, NCSES is obligated to provide information that is influential across the nation. NCSES requires high-quality data and must be able to communicate both the level of expertise involved in the production of data used and the impact of the data to people unfamiliar with convergence. After providing these questions to the committee, Rivers wondered about the timing of this measurement goal. As Entwisle mentioned, it may be too early to try to derive a national measure, given that the concept of convergence was originally defined only a little more than a decade ago.

Erin Leahey (University of Arizona) asked whether a national measure is one that seeks consistency on a national scale (or in other words, uses the same measure used throughout the United States) or an international scale (that is, a measure that allows for comparison across different countries). Rivers responded that a national measure would consist of data that are representative of convergence across the United States. She expressed a reluctance toward thinking too broadly as well as a hope in the adoption of a national estimate of convergence across the United States as an initial

goal. Bethany Laursen (Michigan State University) asked whether NCSES is willing to consider qualitative as well as quantitative measures of convergence. Rivers responded that NCSES plans to focus on a quantitative measure at this time. She clarified that multiple phases of qualitative research to improve understanding often occur before a quantitative measure is reached. Qualitative research can thus play a very important role.

ORGANIZATION OF THE REPORT

This summary describes the workshop presentations and discussions that followed each topic. The chapters are organized around the sessions of the workshop, which included overviews of the prior work of NCSES in this area and of the definitions of convergence, discussions of how convergence is practiced in a variety of settings, the outcomes and impacts of convergence, and existing efforts to measure convergence using a variety of approaches. The final sessions of the workshop focused on questions related to the feasibility of collecting national-level data on convergence, implementation considerations, and discussions among workshop participants about how NCSES could move the work in this area forward.

The full meeting agenda and biographical sketches of steering committee members and workshop presenters appear in the appendixes. This proceedings has been prepared by the workshop rapporteur as a factual summary of what occurred at the workshop. The planning committee’s role was limited to planning and convening the workshop. The views contained in the proceedings are those of individual workshop participants and do not necessarily represent the views of all workshop participants, the planning committee, or the National Academies of Sciences, Engineering, and Medicine.