National Academies Press: OpenBook
« Previous: Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 111
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 112
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 113
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 114
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 115
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 116
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 117
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 118
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 119
Suggested Citation:"Appendix C: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 120

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix C Workshop Agenda Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting A Workshop September 25–26, 2019 National Academy of Sciences Building, Lecture Room 2101 Constitution Avenue, NW, Washington, DC 20418 An ad hoc committee of the National Academies of Sciences, Engineering, and Medicine is convening a public workshop to discuss the current state of transparency in reporting preclini- cal biomedical research (e.g., disclosure of the availability and location of data, materials, analysis, and methodology) and to explore the possibility of improving the harmonization of guide- lines across journals and funding agencies so that biomedical researchers propose and report data in a consistent manner. This workshop is sponsored by the National Institutes of Health, Cell Press, The Lancet, and Nature Research. 111 PREPUBLICATION COPY—Uncorrected Proofs

112 ENHANCING SCIENTIFIC REPRODUCIBILITY WORKSHOP OBJECTIVES: • Highlight current efforts by researchers, institutions, funders, and journals to increase transparency in proposing and report- ing preclinical biomedical research; • Consider lessons learned from field-specific best practices for increased transparency in reporting rigor elements (i.e., research design, methodology, analysis, interpretation, and reporting of results) that are generalizable across biomedical research domains; • Discuss journal and funder assessments of researchers’ adher- ence to transparent reporting guidelines, including a discus- sion of the effectiveness of checklists; • Discuss opportunities for improving the consistency of report- ing guidelines and requirements for rigor and transparency by journals, funders, and institutions across the biomedical research life cycle; and • Consider approaches to compare reporting of rigor ele- ments proposed in grant applications with those included in publications. DAY 1: September 25, 2019 8:00 a.m. Breakfast available outside the Lecture Room 8:30 a.m. Welcome and opening remarks Harvey Fineberg, Workshop Chair President Gordon and Betty Moore Foundation Highlights and related recommendations from the National Academies report on Reproducibility and Replicability in Science 9:15 a.m. Q&A with audience SESSION I CULTIVATING TRANSPARENT REPORTING IN BIOMEDICAL RESEARCH Session Objectives: • Highlight current efforts by researchers, institutions, funders, and journals to increase transparency in proposing and reporting preclinical biomedical research. PREPUBLICATION COPY—Uncorrected Proofs

APPENDIX C 113 • Discuss the incentives, disincentives, challenges, and opportuni- ties for researchers when it comes to transparent reporting of preclinical biomedical research (e.g., pressure to publish, institu- tional resources, training, funding). • Discuss experience with implementation of policies to encourage transparent reporting across the biomedical research life cycle. • Consider the role of stakeholders in supporting a cultural shift toward transparent reporting in preclinical biomedical research. For more information on cultural barriers as sources of non-reproduc- ibility, see p. 58, p. 97, and p. 104 of the National Academies’ Repro- ducibility and Replicability in Science report. 9:30 a.m. Opening remarks by session moderator Alexa McCray Professor of Medicine Harvard Medical School 9:40 a.m. A researcher (early career) perspective Yarimar Carrasquillo Investigator National Center for Complementary and Integrative Health, National Institutes of Health 9:55 a.m. A researcher/researcher support perspective Brian Nosek Co-founder Center for Open Science 10:10 a.m. A researcher (later career)/society publisher perspective Arturo Casadevall Professor, Molecular Microbiology and Immunology, Johns Hopkins University Editor-in-Chief, mBio 10:25 a.m. An NIH perspective Carrie Wolinetz Acting Chief of Staff and Associate Director for Science Policy Office of the Director, National Institutes of Health (NIH) 10:40 a.m. Audience Q&A with the panel PREPUBLICATION COPY—Uncorrected Proofs

114 ENHANCING SCIENTIFIC REPRODUCIBILITY Discussion questions: • What forces are influencing the culture of bio- medical research, and how is it changing? • What actions could influence practice and sup- port a cultural shift toward more transparent reporting? • What influence might transparent reporting or required reporting of rigor elements have on grant applications? Is there a role for preregistra- tion of preclinical studies? 11:10 a.m. BREAK SESSION II ANSWERING THE CALL FOR CHANGE: LESSONS LEARNED AND BEST PRACTICES Session Objectives: • Consider lessons learned from institutional and/or field-specific best practices for increased transparency in reporting rigor elements (i.e., research design, methodology, analysis, interpre- tation, and reporting of results) that are generalizable across biomedical research domains. • Consider available tools and best practices for increased trans- parent reporting that support researchers and are generalizable across biomedical research domains. • Discuss the roles of educational institutions, professional so- cieties, researchers, and funders in improving computational reproducibility (Reproducibility and Replicability in Science Recom- mendation 6-6). • Discuss how funding agencies and organizations could invest in research and development of open source, usable tools and infrastructure that support reproducibility for a broad range of studies across different domains in a seamless fashion, as well as in outreach to inform and train researchers on best practices (Reproducibility and Replicability in Science Recommendation 6-1). 11:30 a.m. Opening remarks by session moderator Veronique Kiermer Executive Editor Public Library of Science (PLOS) PREPUBLICATION COPY—Uncorrected Proofs

APPENDIX C 115 11:40 a.m. A clinical researcher perspective: Lessons from the SPIRIT initiative An-Wen Chan Phelan Scientist, Women’s College Research Institute Associate Professor, University of Toronto 11:50 a.m. An institution perspective Geeta Swamy Vice Dean for Scientific Integrity Associate Vice President for Research Duke University 12:00 p.m. A funder perspective Magali Haas Chief Executive Officer and President Cohen Veterans Bioscience 12:10 p.m. Moderated panel discussion among speakers 12:30 p.m. Audience Q&A with the panel Discussion questions: • How can challenges with preregistration, im- age analysis, cell line authentication, statistical analysis, or other rigor elements be addressed? • What actions can institutions or professional societies take to educate and support their constituents on best practices? How could this information be best provided? • How might funding agencies and organizations invest in development of open source reusable tools and infrastructure to support transparent reporting seamlessly across different domains? • What actions could funding agencies and organizations take to inform, train, and sup- port researchers on best practices in transparent reporting? • What has been learned from open access man- dates and from implementing policies around sharing data in preclinical research? How could those lessons inform transparent reporting guid- ance and adoption? 1:00 p.m. BREAK (Lunch available outside the Lecture Room) PREPUBLICATION COPY—Uncorrected Proofs

116 ENHANCING SCIENTIFIC REPRODUCIBILITY SESSION III STAKEHOLDER PERSPECTIVES ON CHECKLISTS AND GUIDELINES Session Objectives: • Discuss journal and funder assessments of researchers’ adher- ence to transparent reporting guidelines, including discussion of the effectiveness of checklists. o Highlight empirical assessments of checklist application from funders, journals, and researchers; and o Consider practical application and effectiveness of checklists and guidelines to encourage or require transparent reporting of preclinical biomedical research. • Discuss how funders could require thoughtful discussion in grant applications of how uncertainties will be evaluated, along with any relevant issues regarding replicability and computa- tional reproducibility (Reproducibility and Replicability in Science Recommendation 6-9). • Discuss how journals and scientific societies could disclose their policies relevant to achieving reproducibility and replicability, and how journals could be encouraged to set and implement desired standards of reproducibility and replicability and adopt policies to reduce the likelihood of non-replicability (Reproduc- ibility and Replicability in Science Recommendation 6-7). 2:00 p.m. Opening remarks by session moderator Barry Coller Physician-in-Chief, Vice President for Medical Affairs, and David Rockefeller Professor The Rockefeller University 2:10 p.m. The checklist approach at life science journals— challenges and opportunities Sowmya Swaminathan Head of Editorial Policy Nature Research Malcolm Macleod Professor University of Edinburgh PREPUBLICATION COPY—Uncorrected Proofs

APPENDIX C 117 2:30 p.m. An NIH funder perspective Shai Silberberg Director, Research Quality National Institute of Neurological Disorders and Stroke, NIH 2:40 p.m. Moderated panel discussion among speakers 3:10 p.m. Audience Q&A with the panel Discussion questions: • How valuable are checklists? How valuable is guidance such as the CONSORT statement? What are observed challenges to adherence, and how could they be addressed? • How could checklists be improved and/or complemented to further encourage transparent reporting? • What resources do researchers need to be able to submit proposals, publish, or otherwise report on specific rigor elements? • How might funders require thoughtful discus- sion in grant applications of how uncertainties (e.g., in measurement, computation, knowledge, modeling, or methods of analysis) will be evalu- ated by researchers? • Should scientific societies be encouraged to de- velop policies relevant to transparent reporting? SESSION IV TOWARD MINIMAL REPORTING STANDARDS FOR PART 1 PRECLINICAL BIOMEDICAL RESEARCH Session Objective: • Discuss opportunities for improving the consistency of report- ing guidelines and requirements for rigor and transparency by journals, funders, and institutions across the biomedical research life cycle. PREPUBLICATION COPY—Uncorrected Proofs

118 ENHANCING SCIENTIFIC REPRODUCIBILITY 4:00 p.m. Discussion with audience on potential steps stake- holders could take to support harmonizing reporting guidelines Harvey Fineberg, Workshop Chair and session moderator President Gordon and Betty Moore Foundation Benedict Kolber Associate Professor Duquesne University Richard Nakamura Former Director (Retired) Center for Scientific Review, National Institutes of Health Franklin Sayre STEM Librarian Thompson Rivers University Valda Vinson Editor, Research Science 5:00 p.m. ADJOURN WORKSHOP DAY 1 DAY 2: September 26, 2019 8:00 a.m. Breakfast available outside the Lecture Room 8:30 a.m. Welcome and overview of day 1 Harvey Fineberg, Workshop Chair President Gordon and Betty Moore Foundation 9:00 a.m. Keynote address Marcia McNutt President National Academy of Sciences 9:20 a.m. Q&A session 9:30 a.m. BREAK PREPUBLICATION COPY—Uncorrected Proofs

APPENDIX C 119 SESSION IV TOWARD MINIMAL REPORTING STANDARDS PART 2 FOR PRECLINICAL BIOMEDICAL RESEARCH Session Objectives: • Consider approaches to compare reporting of rigor elements pro- posed in grant applications to those included in publications. • Suggest stakeholder actions to encourage transparent reporting and practical next steps toward establishing minimal reporting standards for preclinical biomedical research. 10:00 a.m. Opening remarks by session moderator Harvey Fineberg, Workshop Chair President Gordon and Betty Moore Foundation 10:10 a.m. An early career researcher perspective Michael Keiser Assistant Professor University of California, San Francisco 10:20 a.m. An institution perspective Melissa Rethlefsen Associate Dean, George A. Smathers Libraries Fackler Director, Health Science Center Libraries University of Florida 10:30 a.m. A research educator perspective Steven Goodman Professor of Medicine and Health Research and Policy Co-director, Meta-Research Innovation Center at Stanford Stanford University 10:40 a.m. Moderated panel discussion among speakers 11:10 a.m. Small group table discussion and reporting Discussion questions: • What actions should funders, researchers, insti- tutions, and journals take to drive widespread adoption of minimal reporting standards? PREPUBLICATION COPY—Uncorrected Proofs

120 ENHANCING SCIENTIFIC REPRODUCIBILITY • Are reporting categories in guidelines for pub- lishing (e.g., materials, design, analysis, and reporting) relevant for funders? For institutions? For small publishers/professional societies? • What other information or reporting categories would be relevant? • How should funders instruct reviewers of grant applications to reinforce transparent report- ing? How much information should funders request, that is, to what level of detail, in grant applications? Is it possible to obtain sufficient information about transparent reporting in grant applications without dramatic expansion of the application? 12:25 p.m. W  orkshop wrap-up and concluding discussion with audience 12:30 p.m. ADJOURN WORKSHOP DAY 2 PREPUBLICATION COPY—Uncorrected Proofs

Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Sharing knowledge is what drives scientific progress - each new advance or innovation in biomedical research builds on previous observations. However, for experimental findings to be broadly accepted as credible by the scientific community, they must be verified by other researchers. An essential step is for researchers to report their findings in a manner that is understandable to others in the scientific community and provide sufficient information for others to validate the original results and build on them. In recent years, concern has been growing over a number of studies that have failed to replicate previous results and evidence from larger meta-analyses, which have pointed to the lack of reproducibility in biomedical research.

On September 25 and 26, 2019, the National Academies of Science, Engineering, and Medicine hosted a public workshop in Washington, DC, to discuss the current state of transparency in the reporting of preclinical biomedical research and to explore opportunities for harmonizing reporting guidelines across journals and funding agencies. Convened jointly by the Forum on Drug Discovery, Development, and Translation; the Forum on Neuroscience and Nervous System Disorders; the National Cancer Policy Forum; and the Roundtable on Genomics and Precision Health, the workshop primarily focused on transparent reporting in preclinical research, but also considered lessons learned and best practices from clinical research reporting. This publication summarizes the presentation and discussion of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!