APPENDIX B
Community Standards for Sharing Publication-Related Data and Materials
The Lecture Room
National Academy of Sciences
Washington, D.C. 20418
February 25, 2002
AGENDA
8:30 am |
Welcome, Orientation for the day. Dr. Thomas Cech, Study Chairman |
8:45 |
Keynote Presentation: Dr. Eric Lander, The Whitehead Institute for Biomedical Research |
9:15 |
Questions and Answers from audience and webcast listeners. |
9:30 |
Introduction of Situation 1: The Rare Resource. Dr. Cech |
9:35 |
Panelists Response: |
|
Maria Freire, CEO, Global Alliance for TB Drug Development (former director of NIH Office of Technology Transfer) |
|
Elizabeth Neufeld, Professor and Chair, Department of Biological Chemistry, UCLA School of Medicine |
|
Ira Mellman, Editor, Journal of Cell Biology, and Professor, Department of Cell Biology, Yale University |
|
Michael Hayden, Professor of Medical Genetics, University of British Columbia, Director, UBC Center for Molecular Medicine and Therapeutics; and Chief Scientific Officer, Xenon Genetics, Inc. |
10:30 |
Plenary Discussion of Situation 1 Do the positions take into account all the realities of the situation and the pressures on an investigator? Can a consensus be reached? How might it be implemented? How to deal with noncompliance? |
11:15 |
Working Groups adjourn to discuss Situations 2a 2b 2c |
1:10 pm |
Adjourn working groups, Reconvene in Plenary. Webcast continues. |
1:15 |
Reporting by working group rapporteurs. |
|
2a (10 min) Publishing Primary Brain Imaging Data Rapporteur: Fintan Steele, Editor, Molecular Therapy, and Executive Editor, Genomics |
|
2b (10 min) Genome Sequences of Quantitative Trait Loci Rapporteur: Robert Hazelkorn, Fanny L.Pritzker Distinguished Service Professor, Department of Molecular Genetics and Cell Biology, University of Chicago |
|
2c (10 min) Obligations in Exchange for Materials Received Rapporteur: Corey S.Goodman, Evan Rauch Professor of |
|
Neuroscience, University of California, Berkeley; and President and CEO, Renovis, Inc. |
|
30 min plenary discussion |
2:15 |
Introduction of Situation 3: The Virtual Heart. Dr. Cech |
2:20 |
Panelists Response: |
|
Ari Patrinos, Associate Director, Office of Biological and Environmental Research Office of Science, Department of Energy |
|
James Wells, President and Chief Scientific Officer, Sunesis Pharmaceuticals, Inc. |
|
Bob Waterston, James S.McDonnell Professor and Head, Department of Genetics,Washington University School of Medicine |
|
Barbara Cohen, Editor, The Journal of Clinical Investigation |
3:00 |
Questions and Comments from audience and webcast listeners. |
4:15 |
Consensus building: Where is there agreement in principle, and how can the community deal with noncompliance? Dr. Cech |
5:00 |
Adjourn |
Workshop Situations
Situation 1. The Rare Resource
Issues raised:
-
What are an author’s obligations to share a reagent on which he or she has published, especially given a rare resource?
-
Who is responsible for enforcing the requirements for investigators to share published materials?
A well-known senior investigator publishes a paper with others in his lab, including two graduate students, in which knockout mice are generated and characterized, in part, with a polyclonal antibody.
The mice reproduce poorly and the investigator plans on using the antibody for many further experiments and only has a limited amount of the reagent. After getting requests for the materials associated with the paper, he decides to write back and suggest he is willing to distribute the reagents as part of a collaboration that would include coauthorship with some of the requesters.
A young investigator also requested the materials but hasn’t gotten an answer, even though she has attempted to make contact several times in the last four months. She believes that there is no way the journal would ever sanction this famous investigator and wonders if she should complain to the agency that funded his work. The young investigator is concerned that her career may be damaged by these actions.
Questions for consideration by panelists:
-
What are the pressures faced by the young scientist?
-
What is the best way for the young scientist to get satisfaction?
-
What is a reasonable time to allow someone to turn around a request for a reagent?
-
Under what circumstances is it fair for the senior investigator to request collaboration in exchange for a published material?
-
How would the situation be different if a company had published the paper?
●○●○●
Situation 2a. Primary Brain Imaging Data
Issues raised:
-
What should journal editors do when authors refuse or are hesitant to supply all their primary data to reviewers?
-
To what extent is “data mining” a privilege of the team who originally collects the primary data?
-
To what extent should primary data involving expensive facilities and human or higher animal subjects be made available for evaluation and alternative interpretation? Within what time limits, and what constraints?
-
Is the cost and time it takes to collect data and materials a factor? If so, in which direction?
-
Is the source of funding (private or public) a factor?
-
What principles should guide investigators in this area?
A team funded by NIH submits a short article for publication to a leading journal Neural Hieroglyphica which comprises an analysis of functional and physical changes in the brains of schizophrenics, based on the results of several detailed fMRI-imaging procedures. Note: fMRI studies are always based on taking differences between images; only the ‘difference image’ [the subtraction of an image taken under one circumstance from an image taken in another circumstance] is normally published, and its appearance depends notoriously on how thresholds are taken.
Point 1: One of the reviewers approached by the editor asks the journal staff to obtain the complete set of fMRI data from the authors, as she
wants to check through some of the findings reported in the paper. The authors refuse. Should the journal press the authors (and state that if peer review cannot be completed on this paper as and by reviewers selected by the journal then there is no interest in publishing it)? OR find another reviewer?
After publication of the short article, a colleague asks to see the primary data which the authors had used to construct the images (only one ‘typical image’ was published) on which measurements were taken. The published data consisted of a summary of the quantitative measurements.
The colleague is in a department of psychology at an institution without fMRI facilities and without a medical school, and is in no position to acquire similar primary data. He finds that the conclusions to the short article do not fit with his anatomical studies of the brains of deceased schizophrenics, and wonders if there is an alternate interpretation of the primary data.
The team claims that the data is being used in a follow-on paper they are writing and declines to share the data at this moment. They also note that it took them 2 years to identify the subjects and collect the data, and think that they should be able to “mine” it before others do. Several postdocs are said to be pinning their futures on getting papers out of this data. (However, the paper was published in Neural Hieroglyphica and on average, only half of the short papers published in this journal have timely longer articles written to back them up.)
Point 2: Is the primary investigator acting within his/her rights?
●○●○●
Situation 2b. Gene Sequences for Quantitative Trait Loci
Issues raised:
-
Does withholding data compromise the quality of the results the company wants to publish?
-
Is the confidential information essential to support and understand the data?
-
Can the results without the confidential information benefit and accelerate public research?
AgriGenome, a plant genomics company, is studying the quantitative trait loci (QTL) governing drought resistance in wheat. They have identified a QTL down to 100 kb. They have done sample sequencing and have fragments of genes that appear promising with respect to function, but they haven’t pinned down a single gene responsible for drought resistance yet. Importantly, they have data showing that this QTL alone is able to increase drought resistance.
They submit a manuscript in which they show that this single QTL is able to increase drought resistance. They report detailed biological studies, including field tests under conditions of abundant water, limiting water, and extreme drought that show a strong correlation of drought resistance with the presence of markers for the QTL in different germplasm. Because they haven’t pinned down a gene yet, they do not want to publish any of the sample sequencing data. They have just filed for a patent on the markers, but because those could easily be used to find linked markers that would not be covered under the patent, they wish not to publish the sequence of the markers.
One reviewer wants to require that both the marker sequences and the DNA sequence of at least one gene be described in the paper and deposited in the database upon publication, while the second reviewer requires only the sequences of the markers. The journal would like to publish the paper, because they would be the first journal with a paper showing that this important trait is at a single QTL, and the journal is trying to establish itself in plant genomics. As written, it would be difficult, but not impossible, for others to reproduce the work; they would have to obtain the original lines (which would be in the public domain) and do several cycles of crosses to recreate germplasm from which, following the procedures in the paper, the markers could be isolated. It might take 3 to 4 years, versus one field season if they had the marker sequences. What should the editor decide?
●○●○●
Situation 2c. Obligations in Exchange for Materials Received
Issues raised:
-
Is it appropriate for materials providers to require a right to review publications? If so, at what point should the review take place? Prior to submission for publication, prior to actual publication? Should the materials provider have the right to require changes to the publication?
-
Should the right of review be enforceable? What recourse is appropriate if publication review obligations are ignored by the investigator?
-
What are the pros and cons a company should consider before releasing materials for use in research that will be published? Does the investigator have a responsibility to consider the provider’s interests in its materials?
The Gendefex company is a small biotech company that wants to allow its investigators to publish, and is willing to give academic investigators access to materials referred to in its investigators’ publications with a material transfer agreement (MTA). It recently received requests for two different reagents that were described in recent articles published by Gendefex biologists.
In the case of the first reagent, the proposed MTA includes a provision that would allow Gendefex the right to use improvements to the reagent made by the requesting investigator, and give the company the “freedom to operate” should the requesting investigator find new, potentially patentable, uses for the material.
In the second case, the MTA has no special provisions about the rights of Gendefex related to the investigators’ use of the reagent.
However, in both cases, the MTA stipulates that Gendefex should receive copies of manuscripts that include information related to the use of its materials 60 days prior to submission for publication. The university officials of one investigator who wants the materials are unwilling to sign the MTA with this requirement because it delays the publication process.
The president of the company, although a former academic, himself, is frustrated that some investigators who use the materials do not honor the request for the manuscript even though they sign the agreement and is disinclined to relent on the requirement. It makes him angry that he feels he has no recourse against noncompliant investigators and feels his company is being used.
Lawyers in the company are against permitting company investigators to publish if it means handing over materials for academic research. They point out that publications they have reviewed prior to publication describe a plethora of discoveries surrounding the company’s materials and reveal information the company does not want its competitors to have. In addition, some discoveries are in areas the company was working on and intended to patent itself. Because of materials made available to the university investigators, the company will have to bargain for licenses to intellectual property it needs but does not hold.
Questions for consideration by panelists:
-
What is the motivation for Gendefex to publish?
-
In each case, is the request to review copies of manuscripts prior to submission for publication reasonable or necessary? Is the request enforceable?
-
What obligations do the academic community have to the for-profit sector if they acquire commercial materials for research? Are there reasonable mechanisms that can be utilized to protect the company’s interests?
Situation 3. The Virtual Heart
Issues raised:
-
Review: How should increasingly complex software and databases be evaluated for publication? What depth of materials must be provided to reviewers?
-
Repeatability: Science relies on open disclosure of results in publications, and the possibility of replication (or refutation) of the results by other scientists. What information must be disclosed to enable repeatability? Does repeatability of science that involves complicated software or databases require complete access to computer source code and raw data, or only the possibility of running the code and interrogating the database?
-
Responsibility of the authors to enable future work: The end purpose of a scientific publication is to describe a result that others in the scientific community can build on. A paper describing a new fact or observation is generally sufficient to suggest new ideas and experiments to other investigators. However, an increasing number of papers describe large resources that are difficult to reproduce, such as a database or a software package. Should such resources be made freely available to the community as a condition of publication? If not, what is the societal purpose of scientific papers describing resources that are proprietary products? Certain types of archival data, such as genome sequences or protein structure coordinates, maybe generated in the course of a “standard” result-oriented paper, but these data, if available, may be mined in ways entirely unexpected by the original investigators; when should such data be deposited in public repositories, and when, if ever, should deposition be mandatory?
-
Enforcement: Who enforces openness? Is this the job of funding agencies; if so, should private companies be allowed to play by different rules? Is it the job of journals; if so, what mechanism prevents backsliding in the case of a “hot” paper that a journal has a strong desire to publish for its own reasons of profitability and publicity?
In 2015, the Cardiomics company announces that its proprietary model of the human heart (Virtual Heart) is complete, including heart models reflecting various stages of cardiovascular disease, genetic disorders, and the consequences of infarction. Virtual Heart incorporates an extensive database of experimental data collected by Cardiomics. Cardiomics is basing its impending IPO on Virtual Heart and is trying to gain publicity about it in the media. As part of this effort Cardiomics submits two papers on Virtual Heart to high-profile, for-profit journals.
The proprietary Cardiomics database is 500 terabytes of Oracle relational database tables. Among its many components are data on genetic polymorphism profiles in families with heart disease, MRI images, and electrocardiograms. In 2015, widely recognized public data repositories exist for human genetic profiles (PolymorphismDB) and MRI images (MRI-DB) but not for ECGs. The Virtual Heart program itself is 500,000 lines of code.
Paper A gives an overview of the entire Virtual Heart system, including the software and the database. The central point of this paper is that Virtual Heart is a useful system for virtual cardiac experimentation and diagnosis. Neither the database nor the software are available from Virtual Heart; they are closely held proprietary assets.
Paper B describes a specific result in which the Virtual Heart system is used to predict that thrombospondin variants are probably associated with early heart attacks. This computational prediction is validated by experimental results that are fully described in the paper; one SNP in a
thrombospondin gene is shown to be associated with heart disease in family pedigrees. The central point of the paper is that this thrombospondin variant will be useful for diagnostics and screening.
Questions for consideration by Panelists:
-
What materials should the reviewers receive in order to evaluate these manuscripts? Does it make a difference whether the point of the paper is Virtual Heart itself, or an independently verified result obtained from the system?
-
What data should be made available to the scientific community upon publication? Does this differ for the two papers? Does it matter whether public data repositories exist or not? What restrictions should Cardiomics be allowed to put on any data release to protect its proprietary interests? Is it acceptable if the data are made available, but only under expensive proprietary licensing terms?
-
What are the obligations of the journals to the scientific community? Would the obligations differ for journals that are society-based as opposed to for-profit? If a journal’s editorial decisions are inconsistent with scientific community standards, how should the scientific community respond?
-
What restrictions should Cardiomics be allowed to put on any data release to protect its proprietary interests? Would this be any different for a large pharmaceutical company with less at risk than a small bioinformation company like Cardiomics—or for an academic lab trying to protect itself from academic competition?
●○●○●