National Academies Press: OpenBook

Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop (2020)

Chapter: Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting

« Previous: Appendix A: References
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 105
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 106
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 107
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 108
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 109
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 110

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix B Background Discussion Document: Selected Guidelines for Transparent Reporting SELECTED GUIDELINES FOR TRANSPARENT REPORTING The purpose of this document is to help inform workshop discussions on improving the harmonization of guidelines for transparent reporting across journals and funding agencies so that biomedical researchers pro- pose and report data in a consistent manner. This discussion document was compiled based on criteria described in the Transparency and Open- ness Promotion (TOP) guidelines;1 the National Academies of Sciences, Engineering, and Medicine report on Reproducibility and Replicability in Science;2 and the National Institutes of Health (NIH) policy on Enhancing Reproducibility through Rigor and Transparency.3 Each of the guidelines summarized below has a different scope and purpose. Rather than a comprehensive comparison of the several guidelines, this document indicates the criteria related to transparent reporting that are covered by the various guidelines. It is intended as a background for the workshop discussion. 1 Available at https://cos.io/top (accessed January 12, 2020). 2 Available at https://www.nap.edu/catalog/25303/reproducibility-and-replicability-in- science (accessed January 12, 2020). 3 Available at https://grants.nih.gov/policy/reproducibility/index.htm (accessed Janu- ary 12, 2020). 105 PREPUBLICATION COPY—Uncorrected Proofs

106 ENHANCING SCIENTIFIC REPRODUCIBILITY BRIEF DESCRIPTIONS OF GUIDELINES SUMMARIZED • The Animal Research: Reporting In Vivo Experiments (ARRIVE) guidelines recommend criteria for the reporting of primary research using animals. The guidelines were based on the Consoli- dated Standards of Reporting Trials (CONSORT) guidelines, but cover diverse study types. They were developed by the National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), and were first published in PLOS Biology in 2010. The ARRIVE guidelines are currently being revised; a preprint of the revised ARRIVE guidelines and the accompanying Explana- tion and Elaboration document are available on BioRxiv, respec- tively at https://www.biorxiv.org/content/10.1101/703181v1 and at https://www.biorxiv.org/content/10.1101/703355v1. • The CONSORT statement recommends information to include when reporting a randomized trial; was developed by an inter- national group of trialists, methodologists, and medical journal editors; and was first published in JAMA in 1996 and last revised in 2010 and published in multiple journals. For more information, see http://www.consort-statement.org. • The DRAFT materials, design, analysis, and reporting (MDAR) checklist for authors represents a generic set of minimum require- ments applicable to all reporting studies in the life sciences for the explicit purpose of increasing transparent reporting and reproduc- ibility, developed by the MDAR working group specifically seek- ing common reporting points from across multiple journals. The checklist is being pilot tested by volunteer journals, and therefore has not been published at the time of this writing. The statement of task is available at https://osf.io/preprints/metaarxiv/9sm4x. • NIH policies: °  “Data Sharing” policy, developed by NIH, to encourage data generated with NIH funding to be “made as widely and freely available as possible while safeguarding the privacy of partici- pants, and protecting confidential and proprietary data.” The policy requires a data sharing plan for final research data gen- erated on grants of $500,000 or more, and was implemented in 2003. For more information, see https://grants.nih.gov/grants/ policy/data_sharing. ° Dissemination of NIH-Funded Clinical Trial Information” “ policy, developed by NIH and implemented in 2017, mandates that all clinical trials funded in part or in whole by NIH must be registered at ClinicalTrials.gov. The policy also requires that summary results be posted to ClinicalTrials.gov. For more infor- PREPUBLICATION COPY—Uncorrected Proofs

APPENDIX B 107 mation, see https://grants.nih.gov/grants/guide/notice-files/ not-od-16-149.html. °  “Enhancing Reproducibility through Rigor and Transparency” policy, developed by NIH, to clarify expectations for grant- ees and reviewers in describing or assessing proposed studies in applications and progress reports, announced in 2015 and implemented in 2016. For more information, see https://grants. nih.gov/policy/reproducibility/index.htm. • The Preferred Reporting Items for Systematic Reviews and Meta- Analyses (PRISMA) statement recommends an evidence-based minimum set of reporting elements for systematic reviews and meta-analyses; was developed by an international group; and was first published in 2009 in multiple journals (PLOS Medicine, Annals of Internal Medicine, BMJ, Journal of Clinical Epidemiology, and Open Medicine). For more information, see http://www.prisma- statement.org. • The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement, developed by an international collabo- ration of trialists, methodologists, journal editors, and ethicists, recommends minimum content to include in clinical trial protocols, from study enrollment through closeout, first published in the Annals of Internal Medicine and BMJ in 2013. For more information, see: https://www.spirit-statement.org. • The TOP guidelines describe eight modular standards of transpar- ency that journals can select from to introduce policy and best practices at their publication; was developed by the Center for Open Science with input from journals, funders, and professional societies; and was first published in Science in 2015. See https:// cos.io/top for information about TOP, including a summary table and links to complete policy language. PREPUBLICATION COPY—Uncorrected Proofs

108 ENHANCING SCIENTIFIC REPRODUCIBILITY Legend ARRIVE Criterion covered: Dark boxes Primary CONSORT Criterion partially covered: Gray boxes Research in Randomized Criterion not covered: White boxes Animals Trials STUDY CHARACTERISTICS Code availability2 Data availability2,3 Data citation2,3 STUDY METHODS Analytical methods: attrition, statistical precision, statistical power2,3 Plan for analytical decisions/preregistration2,3 Animal use/sex as a biologic variable4 Details of in-laboratory study replication3 Details of study methods, computation, and associated parameters2,3 Ethics2,5 Information on computational environment (e.g., operating system, library dependencies)3 Materials availability discussed2,3 Material authentication required4 Methods and protocols2,3,4 Sample definition2 RESULTS AND DISCUSSION Adherence to community standards2 Discussion of uncertainty3 Discussion on generality constraints3 Discuss/assess rigor of prior research4 Dual use research of concern* 1. NIH policies represented include Data Sharing Policy, Dissemination of NIH-Funded Clinical Trial Information, and Rigor and Transparency. 2. Based on criteria described in the Transparency and Openness Promotion (TOP) guide- lines, https://cos.io/top. 3. Based on criteria described in the National Academies report Reproducibility and Replicabil- ity in Science. PREPUBLICATION COPY—Uncorrected Proofs

APPENDIX B 109 DRAFT MDAR NIH Policies1 TOP Guidelines Compilation of Preclinical Modular Multiple Life and Clinical PRISMA SPIRIT Clinical Publishing Science Journals Research Meta-analyses Trial Protocols Standards 4. Based on criteria described in the NIH policy on Enhancing Reproducibility through Rigor and Transparency. 5. Covered by multiple NIH policies and offices, including Human Subjects Research poli- cies and the NIH Office of Laboratory Animal Welfare. * Covered by U.S. government Policy for Institutional Oversight of Life Sciences Dual Use Research of Concern. PREPUBLICATION COPY—Uncorrected Proofs

PREPUBLICATION COPY—Uncorrected Proofs

Next: Appendix C: Workshop Agenda »
Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Sharing knowledge is what drives scientific progress - each new advance or innovation in biomedical research builds on previous observations. However, for experimental findings to be broadly accepted as credible by the scientific community, they must be verified by other researchers. An essential step is for researchers to report their findings in a manner that is understandable to others in the scientific community and provide sufficient information for others to validate the original results and build on them. In recent years, concern has been growing over a number of studies that have failed to replicate previous results and evidence from larger meta-analyses, which have pointed to the lack of reproducibility in biomedical research.

On September 25 and 26, 2019, the National Academies of Science, Engineering, and Medicine hosted a public workshop in Washington, DC, to discuss the current state of transparency in the reporting of preclinical biomedical research and to explore opportunities for harmonizing reporting guidelines across journals and funding agencies. Convened jointly by the Forum on Drug Discovery, Development, and Translation; the Forum on Neuroscience and Nervous System Disorders; the National Cancer Policy Forum; and the Roundtable on Genomics and Precision Health, the workshop primarily focused on transparent reporting in preclinical research, but also considered lessons learned and best practices from clinical research reporting. This publication summarizes the presentation and discussion of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!