Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
1 Introduction1 Sharing knowledge is what drives scientific progressâeach new advance or innovation in biomedical research builds on previous observa- tions. However, for experimental findings to be broadly accepted as cred- ible by the scientific community, they must be verified by other research- ers. An essential step is for researchers to report their findings in a manner that is understandable to others in the scientific community and provide sufficient information for others to validate the original results and build on them. In recent years, concern has been growing over a number of studies that have failed to replicate previous results and evidence from larger meta-analyses, which have pointed to the lack of reproducibility in bio- medical research (e.g., Bik et al., 2016, 2018; Ioannidis, 2005; Landis et al., 2012; Sena et al., 2010). In response, funders, publishers, and other key stakeholders have recognized the need to encourage and enhance transparent reporting of preclinical research findings across the biomedi- cal research life cycle (e.g., Kiley et al., 2017; McNutt, 2014; Nosek et al., 1 This workshop was organized by an independent planning committee whose role was limited to identification of topics and speakers. This Proceedings of a Workshop was pre- pared by the rapporteurs as a factual summary of the presentations and discussion that took place at the workshop. Statements, recommendations, and opinions expressed are those of individual presenters and participants and are not endorsed or verified by the National Academies of Sciences, Engineering, and Medicine, and they should not be construed as reflecting any group consensus. 1 PREPUBLICATION COPYâUncorrected Proofs
2 ENHANCING SCIENTIFIC REPRODUCIBILITY 2015; the Center for Open Science2; policies at the National Institutes of Health (NIH),3 the Gates Foundation,4 or the Wellcome Trust,5 or broadly accepted policy statements such as in the case of public health emergencies6). On September 25 and 26, 2019, the National Academies of Science, Engineering, and Medicine (the National Academies) hosted a public workshop in Washington, DC, to discuss the current state of transparency in the reporting of preclinical biomedical research and to explore opportunities for harmonizing reporting guidelines across jour- nals and funding agencies. The workshop primarily focused on transpar- ent reporting in preclinical research, but also considered lessons learned and best practices from clinical research reporting. The agenda for the workshop, titled Enhancing Scientific Reproducibility in Biomedical Research through Transparent Reporting, was developed by an inde- pendent planning committee; the workshop statement of task is available in Box 1-1.7 The workshop was convened jointly by the Forum on Drug Discovery, Development, and Translation; the Forum on Neuroscience and Nervous System Disorders; the National Cancer Policy Forum; and the Roundtable on Genomics and Precision Health, and was sponsored by the National Institutes of Health, Cell Press, The Lancet, and Nature Research. This workshop builds on recent consensus reports by the National Academies, including Reproducibility and Replicability in Science (NASEM, 2019, discussed further in Chapter 2), Open Science by Design: Realizing a Vision for 21st Century Research (NASEM, 2018), and Fostering Integrity in Research (NASEM, 2017). Selected guidelines and checklists for trans- parent reporting were also discussed and brief background information was provided to participants to facilitate the discussions (available in Appendix B). 2 See https://cos.io (accessed January 13, 2020). 3 See https://grants.nih.gov/policy/reproducibility/index.htm (accessed January 10, 2020). 4 Open access policy available at https://www.gatesfoundation.org/How-We-Work/ General-Information/Open-Access-Policy (accessed January 13, 2020). 5 Open research policies available at https://wellcome.ac.uk/what-we-do/our-work/ open-research (accessed January 13, 2020). 6 For more information, see https://wellcome.ac.uk/press-release/statement-data- sharing-public-health-emergencies (accessed February 10, 2020). 7 The workshop agenda can be found in Appendix C. Archived webcast videos and speakersâ presentations are available on the National Academies website. See http:// nationalacademies.org/hmd/Activities/Research/DrugForum/2019-Sept-25.aspx (accessed November 20, 2019). PREPUBLICATION COPYâUncorrected Proofs
INTRODUCTION 3 BOX 1-1 Workshop Statement of Task An ad hoc committee of the National Academies of Sciences, Engineering, and Medicine will plan and conduct a public workshop to discuss the current state of transparency in reporting biomedical research (e.g., disclosure of the availability and location of data, materials, analysis, and methodology) and to explore the possibility of improving the harmonization of guidelines across jour- nals and funding agencies so that biomedical researchers propose and report data in a consistent manner. Workshop objectives: â¢ Highlight current efforts by researchers, institutions, funders, and jour- nals to increase transparency in proposing and reporting preclinical biomedical research; â¢ Discuss journal and funder assessments of researchersâ adherence to reporting guidelines, including a discussion of the effectiveness of checklists; â¢ onsider lessons learned from field-specific best practices for increased C transparency in reporting rigor elements (research design, methodology, analysis, interpretation, reporting of results) that are generalizable across biomedical research domains; â¢ Discuss opportunities for improving the consistency of reporting guide- lines and requirements for rigor and transparency by journals, funders, and institutions across the biomedical research life cycle; and â¢ Consider approaches to compare reporting of rigor elements proposed in grant applications to those included in publications. The committee will plan and organize the workshop, develop the agenda, select and invite speakers and discussants, and moderate or identify moderators for the discussions. The agenda will include a panel discussion on facilitating the development of consistent guidelines (e.g., a common set of minimal reporting standards) that could be applied across journals and funders to increase trans- parency in proposing and reporting biomedical research. Proceedings of the presentations and discussions at the workshop will be prepared by a designated rapporteur in accordance with institutional guidelines. ORGANIZATION OF THE PROCEEDINGS This Proceedings of a Workshop summarizes the presentations and discussions that took place at the workshop. As background for the work- shop discussions, an overview of the findings and recommendations of the most recent National Academies consensus study related to reproduc- ibility in scientific research, Reproducibility and Replicability in Science, was presented by Harvey Fineberg, president of the Gordon and Betty Moore Foundation and chair of both the workshop and the consensus study PREPUBLICATION COPYâUncorrected Proofs
4 ENHANCING SCIENTIFIC REPRODUCIBILITY (Chapter 2). Participants were also addressed by National Academy of Sciences (NAS) President Marcia McNutt, who delivered a keynote talk on sustaining public trust in science (Chapter 2). The first workshop panel focused on the current culture of science as it relates to transparency in proposing and reporting preclinical biomedical research. Researchers rep- resenting different roles in the science ecosystem shared their perspectives on the incentives, disincentives, challenges, and opportunities associated with transparent reporting and replicability in science (Chapter 3). The second panel considered lessons learned and best practices for increased transparency from the field of clinical research that could be applied to improving transparent reporting of preclinical studies (Chapter 4). In the third panel session, speakers explored the practical application and effec- tiveness of checklists and guidelines for enhancing transparent reporting of biomedical research (Chapter 5). In the final panel session, speakers representing different stakeholders discussed opportunities for action to harmonize guidelines and develop a common set of minimal report- ing standards (Chapter 6). The last session of the workshop provided an opportunity for all participants to engage in small group discussions to apply what they had learned throughout the workshop and consider what actions researchers, publishers, institutions, and funders could take to improve transparent reporting of biomedical research (Chapter 7). Points of interest were shared on Twitter throughout the workshop by participants using the hashtag #reproducibilityinscience.8 8 The Twitter discussion that took place on September 25 and 26 in association with the workshop can be viewed at https://twitter.com/hashtag/ReproducibilityInScience (ac- cessed November 20, 2019). PREPUBLICATION COPYâUncorrected Proofs