Transparency in Statistical Information for the National Center for Science and Engineering Statistics and All Federal Statistical Agencies - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2022)
Widely available, trustworthy government statistics are essential for policy makers and program administrators at all levels of government, for private sector decision makers, for researchers, and for the media and the public. In the United States, principal statistical agencies as well as units and programs in many other agencies produce various key statistics in areas ranging from the science and engineering enterprise to education and economic welfare. Official statistics are often the result of complex data collection, processing, and estimation methods. These methods can be challenging for agencies to document and for users to understand. At the request of the National Center for Science and Engineering Statistics (NCSES), this report studies issues of documentation and archiving of NCSES statistical data products in order to enable NCSES to enhance the transparency and reproducibility of the agency's statistics and facilitate improvement of the statistical program workflow processes of the agency and its contractors. Transparency in Statistical Information for the National Center for Science and Engineering Statistics and All Federal Statistical Agencies also explores how NCSES could work with other federal statistical agencies to facilitate the adoption of currently available documentation and archiving standards and tools.
Keywords: Surveys and Statistics

Reproducibility and Replicability in Science - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2019)
One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science.
Keywords: Surveys and Statistics; Policy for Science and Technology

The Mathematical Sciences in 2025 - National Research Council (The National Academies Press, 2013)
The mathematical sciences are part of nearly all aspects of everyday life—the discipline has underpinned such beneficial modern capabilities as Internet search, medical imaging, computer animation, numerical weather predictions, and all types of digital communications. The Mathematical Sciences in 2025 examines the current state of the mathematical sciences and explores the changes needed for the discipline to be in a strong position and able to maximize its contribution to the nation in 2025. It finds the vitality of the discipline excellent and that it contributes in expanding ways to most areas of science and engineering, as well as to the nation as a whole, and recommends that training for future generations of mathematical scientists should be re-assessed in light of the increasingly cross-disciplinary nature of the mathematical sciences. In addition, because of the valuable interplay between ideas and people from all parts of the mathematical sciences, the report emphasizes that universities and the government need to continue to invest in the full spectrum of the mathematical sciences in order for the whole enterprise to continue to flourish long-term.
Keywords: Math, Chemistry, and Physics; Education; Surveys and Statistics

Communicating National Science Foundation Science and Engineering Information to Data Users: Letter Report - National Research Council (The National Academies Press, 2011)
This report from the Panel on Communicating National Science Foundation (NSF) Science and Engineering Information to Data Users recommends action by the National Center for Science and Engineering Statistics (SRS) on four key issues: data content and presentation; meeting changing storage and retrieval standards; understanding data users and their emerging needs; and data accessibility. This report also includes a summary of the workshop that focused on the several aspects of the NCSES's current approaches to communicating and disseminating statistical information -- including NCSES's information products, website, and database systems. It included presentations from NCSES staff and representatives of key use groups -- including the academic research, private nonprofit research, and federal government policy making communities.
Keywords: Computers and Information Technology; Behavioral and Social Sciences; Surveys and Statistics

Nonresponse in Social Science Surveys: A Research Agenda - National Research Council (The National Academies Press, 2013)
For many household surveys in the United States, responses rates have been steadily declining for at least the past two decades. A similar decline in survey response can be observed in all wealthy countries. Efforts to raise response rates have used such strategies as monetary incentives or repeated attempts to contact sample members and obtain completed interviews, but these strategies increase the costs of surveys. This review addresses the core issues regarding survey nonresponse. It considers why response rates are declining and what that means for the accuracy of survey results. These trends are of particular concern for the social science community, which is heavily invested in obtaining information from household surveys. The evidence to date makes it apparent that current trends in nonresponse, if not arrested, threaten to undermine the potential of household surveys to elicit information that assists in understanding social and economic issues. The trends also threaten to weaken the validity of inferences drawn from estimates based on those surveys. High nonresponse rates create the potential or risk for bias in estimates and affect survey design, data collection, estimation, and analysis. The survey community is painfully aware of these trends and has responded aggressively to these threats. The interview modes employed by surveys in the public and private sectors have proliferated as new technologies and methods have emerged and matured. To the traditional trio of mail, telephone, and face-to-face surveys have been added interactive voice response (IVR), audio computer-assisted self-interviewing (ACASI), web surveys, and a number of hybrid methods. Similarly, a growing research agenda has emerged in the past decade or so focused on seeking solutions to various aspects of the problem of survey nonresponse; the potential solutions that have been considered range from better training and deployment of interviewers to more use of incentives, better use of the information collected in the data collection, and increased use of auxiliary information from other sources in survey design and data collection. Nonresponse in Social Science Surveys: A Research Agenda also documents the increased use of information collected in the survey process in nonresponse adjustment.
Keywords: Behavioral and Social Sciences; Surveys and Statistics

Data Science for Undergraduates: Opportunities and Options - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2018)
Data science is emerging as a field that is revolutionizing science and industries alike. Work across nearly all domains is becoming more data driven, affecting both the jobs that are available and the skills that are required. As more data and ways of analyzing them become available, more aspects of the economy, society, and daily life will become dependent on data. It is imperative that educators, administrators, and students begin today to consider how to best prepare for and keep pace with this data-driven era of tomorrow. Undergraduate teaching, in particular, offers a critical link in offering more data science exposure to students and expanding the supply of data science talent.
Keywords: Computers and Information Technology; Math, Chemistry, and Physics; Education; Surveys and Statistics

Communicating Science and Engineering Data in the Information Age - National Research Council (The National Academies Press, 2012)
The National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation (NSF) communicates its science and engineering (S&E) information to data users in a very fluid environment that is undergoing modernization at a pace at which data producer dissemination practices, protocols, and technologies, on one hand, and user demands and capabilities, on the other, are changing faster than the agency has been able to accommodate. NCSES asked the Committee on National Statistics and the Computer Science and Telecommunications Board of the National Research Council to form a panel to review the NCSES communication and dissemination program that is concerned with the collection and distribution of information on science and engineering and to recommend future directions for the program. Communicating Science and Engineering Data in the Information Age includes recommendations to improve NCSES's dissemination program and improve data user engagement. This report includes recommendations such as NCSES's transition to a dissemination framework that emphasizes database management rather than data presentation, and that NCSES analyze the results of its initial online consumer survey and refine it over time. The implementation of the report's recommendations should be undertaken within an overall framework that accords priority to the basic quality of the data and the fundamentals of dissemination, then to significant enhancements that are achievable in the short term, while laying the groundwork for other long-term improvements.
Keywords: Computers and Information Technology; Math, Chemistry, and Physics; Policy for Science and Technology; Surveys and Statistics

Strengthening the Linkages Between the Sciences and the Mathematical Sciences - National Research Council (The National Academies Press, 2000)
Over three hundred years ago, Galileo is reported to have said, "The laws of nature are written in the language of mathematics." Often mathematics and science go hand in hand, with one helping develop and improve the other. Discoveries in science, for example, open up new advances in statistics, computer science, operations research, and pure and applied mathematics which in turn enabled new practical technologies and advanced entirely new frontiers of science. Despite the interdependency that exists between these two disciplines, cooperation and collaboration between mathematical scientists and scientists have only occurred by chance. To encourage new collaboration between the mathematical sciences and other fields and to sustain present collaboration, the National Research Council (NRC) formed a committee representing a broad cross-section of scientists from academia, federal government laboratories, and industry. The goal of the committee was to examine the mechanisms for strengthening interdisciplinary research between mathematical sciences and the sciences, with a strong focus on suggesting the most effective mechanisms of collaboration. Strengthening the Linkages Between the Sciences and the Mathematical Sciences provides the findings and recommendations of the committee as well as case studies of cross-discipline collaboration, the workshop agenda, and federal agencies that provide funding for such collaboration.
Keywords: Math, Chemistry, and Physics; Surveys and Statistics

The Interface of Three Areas of Computer Science with the Mathematical Sciences: Summary of a Workshop - National Research Council (The National Academies Press, 2000)
Keywords: Computers and Information Technology; Math, Chemistry, and Physics; Surveys and Statistics

Measuring the 21st Century Science and Engineering Workforce Population: Evolving Needs - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2018)
The National Science Foundation’s National Center for Science and Engineering Statistics (NCSES), one of the nation’s principal statistical agencies, is charged to collect, acquire, analyze, report, and disseminate statistical data related to the science and engineering enterprise in the United States and other nations that is relevant and useful to practitioners, researchers, policymakers, and to the public. NCSES data, based primarily on several flagship surveys, have become the major evidence base for American science and technology policy, and the agency is well respected globally for these data.
Keywords: Behavioral and Social Sciences; Surveys and Statistics

Capturing Change in Science, Technology, and Innovation: Improving Indicators to Inform Policy - National Research Council (The National Academies Press, 2014)
Since the 1950s, under congressional mandate, the U.S. National Science Foundation (NSF) - through its National Center for Science and Engineering Statistics (NCSES) and predecessor agencies - has produced regularly updated measures of research and development expenditures, employment and training in science and engineering, and other indicators of the state of U.S. science and technology. A more recent focus has been on measuring innovation in the corporate sector. NCSES collects its own data on science, technology, and innovation (STI) activities and also incorporates data from other agencies to produce indicators that are used for monitoring purposes - including comparisons among sectors, regions, and with other countries - and for identifying trends that may require policy attention and generate research needs. NCSES also provides extensive tabulations and microdata files for in-depth analysis. Capturing Change in Science, Technology, and Innovation assesses and provides recommendations regarding the need for revised, refocused, and newly developed indicators of STI activities that would enable NCSES to respond to changing policy concerns. This report also identifies and assesses both existing and potential data resources and tools that NCSES could exploit to further develop its indicators program. Finally, the report considers strategic pathways for NCSES to move forward with an improved STI indicators program. The recommendations offered in Capturing Change in Science, Technology, and Innovation are intended to serve as the basis for a strategic program of work that will enhance NCSES's ability to produce indicators that capture change in science, technology, and innovation to inform policy and optimally meet the needs of its user community.
Keywords: Behavioral and Social Sciences; Policy for Science and Technology; Surveys and Statistics

Strengthening Data Science Methods for Department of Defense Personnel and Readiness Missions - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2017)
The Office of the Under Secretary of Defense (Personnel & Readiness), referred to throughout this report as P&R, is responsible for the total force management of all Department of Defense (DoD) components including the recruitment, readiness, and retention of personnel. Its work and policies are supported by a number of organizations both within DoD, including the Defense Manpower Data Center (DMDC), and externally, including the federally funded research and development centers (FFRDCs) that work for DoD. P&R must be able to answer questions for the Secretary of Defense such as how to recruit people with an aptitude for and interest in various specialties and along particular career tracks and how to assess on an ongoing basis service members' career satisfaction and their ability to meet new challenges. P&R must also address larger-scale questions, such as how the current realignment of forces to the Asia-Pacific area and other regions will affect recruitment, readiness, and retention.
Keywords: Math, Chemistry, and Physics; Conflict and Security Issues; Surveys and Statistics

Envisioning the Data Science Discipline: The Undergraduate Perspective: Interim Report - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2018)
The need to manage, analyze, and extract knowledge from data is pervasive across industry, government, and academia. Scientists, engineers, and executives routinely encounter enormous volumes of data, and new techniques and tools are emerging to create knowledge out of these data, some of them capable of working with real-time streams of data. The nation’s ability to make use of these data depends on the availability of an educated workforce with necessary expertise. With these new capabilities have come novel ethical challenges regarding the effectiveness and appropriateness of broad applications of data analyses.
Keywords: Education; Math, Chemistry, and Physics; Computers and Information Technology; Surveys and Statistics

Improving Measures of Science, Technology, and Innovation: Interim Report - National Research Council (The National Academies Press, 2012)
The National Center for Science and Engineering Statistics (NCSES), at the U.S. National Foundation, is 1 of 14 major statistical agencies in the federal government, of which at least 5 collect relevant information on science, technology, and innovation activities in the United States and abroad. The America COMPETES Reauthorization Act of 2010 expanded and codified NCSES's role as a U.S. federal statistical agency. Important aspects of the agency's mandate include collection, acquisition, analysis, and reporting and dissemination of data on research and development trends, on U.S. competitiveness in science, technology, and research and development, and on the condition and progress of U.S. science, technology, engineering, and mathematics (STEM) education.
Keywords: Behavioral and Social Sciences; Industry and Labor; Policy for Science and Technology; Surveys and Statistics

The Mathematical Sciences in 2025/Fueling Innovation and Discovery Two Volume Set - National Research Council (The National Academies Press, 2013)
Keywords: Math, Chemistry, and Physics; Policy for Science and Technology; Surveys and Statistics

Fueling Innovation and Discovery: The Mathematical Sciences in the 21st Century - National Research Council (The National Academies Press, 2012)
The mathematical sciences are part of everyday life. Modern communication, transportation, science, engineering, technology, medicine, manufacturing, security, and finance all depend on the mathematical sciences. Fueling Innovation and Discovery describes recent advances in the mathematical sciences and advances enabled by mathematical sciences research. It is geared toward general readers who would like to know more about ongoing advances in the mathematical sciences and how these advances are changing our understanding of the world, creating new technologies, and transforming industries. Although the mathematical sciences are pervasive, they are often invoked without an explicit awareness of their presence. Prepared as part of the study on the Mathematical Sciences in 2025, a broad assessment of the current state of the mathematical sciences in the United States, Fueling Innovation and Discovery presents mathematical sciences advances in an engaging way. The report describes the contributions that mathematical sciences research has made to advance our understanding of the universe and the human genome. It also explores how the mathematical sciences are contributing to healthcare and national security, and the importance of mathematical knowledge and training to a range of industries, such as information technology and entertainment. Fueling Innovation and Discovery will be of use to policy makers, researchers, business leaders, students, and others interested in learning more about the deep connections between the mathematical sciences and every other aspect of the modern world. To function well in a technologically advanced society, every educated person should be familiar with multiple aspects of the mathematical sciences.
Keywords: Math, Chemistry, and Physics; Surveys and Statistics

Mathematical Sciences Research Challenges for the Next-Generation Electric Grid: Summary of a Workshop - National Academies of Sciences, Engineering, and Medicine (The National Academies Press, 2015)
If the United States is to sustain its economic prosperity, quality of life, and global competitiveness, it must continue to have an abundance of secure, reliable, and affordable energy resources. There have been many improvements in the technology and capability of the electric grid over the past several decades. Many of these advances to the grid depend on complex mathematical algorithms and techniques, and as the complexity of the grid has increased, the analytical demands have also increased.
Keywords: Math, Chemistry, and Physics; Energy and Energy Conservation; Surveys and Statistics

Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies - National Research Council (The National Academies Press, 2000)
The science and engineering enterprise has continued to evolve, responding over the last decade to increased economic globalization, a post-cold war military, federal budget fluctuations, and structural changes in the way science and engineering are conducted and innovations are adopted. This report suggests ways to revise the data collection activities of the Science Resources Studies Division (SRS) of the National Science Foundation to better capture the current realities of R&D funding and S&E human resources. The report's recommendations would improve the relevance of the data on graduate education, the labor market for scientists and engineers, and the funding and conduct of research and development, and thus better meet the data needs of policymakers, managers, and researchers.
Keywords: Industry and Labor; Math, Chemistry, and Physics; Surveys and Statistics

Using the American Community Survey for the National Science Foundation's Science and Engineering Workforce Statistics Programs - National Research Council (The National Academies Press, 2008)
The National Science Foundation (NSF) has long collected information on the number and characteristics of individuals with education or employment in science and engineering and related fields in the United States. An important motivation for this effort is to fulfill a congressional mandate to monitor the status of women and minorities in the science and engineering workforce. Consequently, many statistics are calculated by race or ethnicity, gender, and disability status. For more than 25 years, NSF obtained a sample frame for identifying the target population for information it gathered from the list of respondents to the decennial census long-form who indicated that they had earned a bachelors or higher degree. The probability that an individual was sampled from this list was dependent on both demographic and employment characteristics. But, the source for the sample frame will no longer be available because the census long-form is being replaced as of the 2010 census with the continuous collection of detailed demographic and other information in the new American Community Survey (ACS). At the request of NSF's Science Resources Statistics Division, the Committee on National Statistics of the National Research Council formed a panel to conduct a workshop and study the issues involved in replacing the decennial census long-form sample with a sample from the ACS to serve as the frame for the information the NSF gathers. The workshop had the specific objective of identifying issues for the collection of field of degree information on the ACS with regard to goals, content, statistical methodology, data quality, and data products.
Keywords: Behavioral and Social Sciences; Industry and Labor; Surveys and Statistics

The Mathematical Sciences' Role in Homeland Security: Proceedings of a Workshop - National Research Council (The National Academies Press, 2004)
Mathematical sciences play a key role in many important areas of Homeland Security including data mining and image analysis and voice recognition for intelligence analysis, encryption and decryption for intelligence gathering and computer security, detection and epidemiology of bioterriost attacks to determine their scope, and data fusion to analyze information coming from simultaneously from several sources.
Keywords: Computers and Information Technology; Math, Chemistry, and Physics; Surveys and Statistics