Summary

“Omics” is a term encompassing multiple molecular disciplines that involve the characterization of global sets of biological molecules such as DNAs, RNAs, proteins, and metabolites. For example, genomics investigates thousands of DNA sequences, transcriptomics investigates all or many gene transcripts, proteomics investigates large numbers of proteins, and metabolomics investigates large sets of metabolites.

Omics research generates complex high-dimensional data; these data are often generated through measurement of many more variables per sample than the total number of biological samples used to generate the dataset. These data can be used to produce a computational model1 that potentially distinguishes a health-related characteristic of clinical significance and is intended for eventual analysis of individual patient specimens in a clinical setting. High-dimensional data are particularly prone to overfitting; as a result, a computational model emerging from the research and discovery phase may function well on the samples used for the discovery research, but is inaccurate on any other sample. A carefully designed and executed series of studies is necessary to develop a clinically useful omics-based test for patient management and care, with the goal of improving patient outcomes.

Several characteristics distinguish omics-based tests from other medical technologies, including a different regulatory oversight process, the difficulty in defining the biological rationale behind a test based on multiple

______________

1 The series of computational steps performed in processing the raw data, as well as the mathematical formula or formulas used to convert the data into a prediction of the phenotype of interest, all precisely defined in written form.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary “Omics” is a term encompassing multiple molecular disciplines that involve the characterization of global sets of biological molecules such as DNAs, RNAs, proteins, and metabolites. For example, genomics investi- gates thousands of DNA sequences, transcriptomics investigates all or many gene transcripts, proteomics investigates large numbers of proteins, and metabolomics investigates large sets of metabolites. Omics research generates complex high-dimensional data; these data are often generated through measurement of many more variables per sam- ple than the total number of biological samples used to generate the dataset. These data can be used to produce a computational model1 that potentially distinguishes a health-related characteristic of clinical significance and is intended for eventual analysis of individual patient specimens in a clinical setting. High-dimensional data are particularly prone to overfitting; as a result, a computational model emerging from the research and discovery phase may function well on the samples used for the discovery research, but is inaccurate on any other sample. A carefully designed and executed series of studies is necessary to develop a clinically useful omics-based test for patient management and care, with the goal of improving patient outcomes. Several characteristics distinguish omics-based tests from other medical technologies, including a different regulatory oversight process, the dif- ficulty in defining the biological rationale behind a test based on multiple 1 The series of computational steps performed in processing the raw data, as well as the mathematical formula or formulas used to convert the data into a prediction of the phenotype of interest, all precisely defined in written form. 1

OCR for page 1
2 EVOLUTION OF TRANSLATIONAL OMICS individual biomarkers, the complexity of data sharing with other scientists, and the high degree of hope placed in the promise of omics-enabled tech- nologies and medical care. Omics-based tests, and indeed all clinical laboratory tests, are subject to a different regulatory framework than drugs. Specifically, there are more pathways for regulation of in vitro diagnostic test devices—the category under which omics-based tests fall—than there are for drugs. Tests can be developed, validated, and placed into clinical use either through review by the Food and Drug Administration (FDA) or through validation and performance in a specific laboratory, also called laboratory-developed tests (LDTs). Any clinical laboratory that reports tests for clinical management of patients falls under the purview of the Clinical Laboratory Improvement Act (a CLIA-certified clinical laboratory) that provides a baseline level of oversight with respect to test development and the quality of laboratory operations. While FDA has the authority for regulatory oversight of all tests used in patient care, it has not defined a regulatory framework that includes oversight of LDTs and has only reviewed LDT tests that it has determined to be of high complexity and therefore high risk to patients. This alternate LDT pathway is not possible for drug development, and all drugs must be approved by FDA. It is precisely this LDT pathway that allows academic medical centers to move omics-based tests from discovery to clinical use without external regulatory review of the new test, and places a new and mostly unrecognized demand on academic institutions to provide proper oversight for omics-based test development, validation, and clinical imple- mentation. While pharmaceutical and medical device companies follow well-established medical product development pathways and have many process controls in place for strong oversight of development, clinical vali- dation, and manufacturing, academic institutions are not as accustomed to overseeing the development of medical products. The frequent lack of a clear biological rationale further distinguishes omics-based tests from most other clinical laboratory tests based on a single analyte. The biological rationale behind a single-analyte test is fre- quently quite evident: The test is useful because the gene, RNA, protein, or metabolite plays an understood role in the disease pathology or other biological process under investigation. Examples of single-analyte tests include human epidermal growth factor receptor 2 (HER2) testing of breast cancers or measuring low-density lipoprotein (LDL) cholesterol level in the blood for cardiac risk assessment. In contrast, the biological rationale for the set of biomarkers in an omics-based test frequently is not well-defined scientifically. This difference puts an additional burden on the statisticians and bioinformatics experts involved in test validation to ensure that the biological data and computational model are scientifically sound. Because of the increased risk of overfitting large datasets in the development of the

OCR for page 1
3 SUMMARY computational model, the need for rigor, validation, and accountability is even higher than for other single biomarker-based tests. The complexity of omics research also makes data provenance more challenging and makes sharing of the complex datasets and computational models difficult, which limits the ability of other scientists to replicate and verify the findings and conclusions of omics research studies. Database repositories for genomic datasets are available, but data sharing is not routine, and without access to the datasets or a precisely defined computa- tional model, replication and verification are more difficult than for single biomarker tests. While independent confirmation studies are expensive, the need for replication is beneficial in the omics field given the data com- plexities that can lead to errors, from simple data management errors to incorrectly designed computational models. This level of complexity does not exist for single-biomarker test research, development, and validation. Despite the nearly complete identification of the human genome sequence in 2001, development of omics-based products that influence or improve patient health has been slower than expected. One possible reason for this limited progress is that there has not been a widely agreed-upon process for translation of omics discoveries into clinical omics-based tests intended to improve patient outcomes and care. Many hope that the prom- ise that omics science holds for medicine and public health will be realized. With the creation of high-throughput measurement technologies, it is now feasible to take a snapshot of a patient’s molecular profile at specific stages in the progression of disease pathology or at a given location in the body. However, the complexity of these technologies and of the resulting high- dimensional data introduces major challenges for the scientific community, as rigorous statistical, bioinformatics, laboratory, and clinical procedures are required to develop and validate these tests and evaluate their clinical usefulness. The failure of scientific collaboration, review processes by journals, regu- latory oversight, institutional systems for protection of patient-participants, and institutional systems for management of conflicts of interest in a recent case involving the premature use of gene expression-based tests in clinical trials at Duke University led the National Cancer Institute (NCI) to request establishment of this Institute of Medicine (IOM) committee. The commit- tee’s charge was to develop recommendations to clarify and improve the pathway from discovery to first use of omics-based tests in a clinical trial, to assess the potential for new omics-based tests to benefit patients. STUDY SCOPE Recent events have highlighted the lack of clarity about best practices for omics-based test validations and the failure of current oversight sys-

OCR for page 1
4 EVOLUTION OF TRANSLATIONAL OMICS tems. In July 2010, NCI Director Harold Varmus received a letter from more than 30 statisticians and bioinformatics scientists expressing concerns over several gene expression–based tests already in use in clinical trials at Duke University to predict the type of chemotherapy that individual cancer patients were most likely to benefit from. As a result, an IOM committee was convened to help clarify questions about how to effectively develop omics-based tests to enable progress toward improving patient outcomes. The IOM study was focused on making recommendations useful to inves- tigators in the broader field of omics-based test development, rather than simply examining what went wrong in the test development process at Duke University. With support from NCI, FDA, the Centers for Disease Control and Prevention, the U.S. Department of Veterans Affairs, the Amer- ican Society for Clinical Pathology, and the College of American Patholo- gists, the IOM committee’s charge was to recommend sound principles for appropriate development and evaluation in translating omics-based tests from the research laboratory into clinical trials, with the ultimate goal of guiding therapeutic decisions to improve patient outcomes. The complete charge to the committee can be found in Chapter 1. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS The committee considered its task in the context of the scientific pro- cesses of discovery, confirmation, validation, and evaluation for clinical use of candidate omics-based tests and in relation to the many parties respon- sible for the discovery and development of omics-based tests. The primary investigators, who often work in interdisciplinary teams, bear the great- est responsibility and accountability for the scientific rigor of the discov- ery research and test development. Academic institutions, other nonprofit research organizations, and for-profit companies that support the develop- ment of omics-based tests also bear responsibility for proper oversight of the discovery, translational, and clinical research conducted and reported by their faculty or research staff seeking to generate successful omics-based tests. Although these institutions depend on the rigor and integrity with which individual investigators perform and defend their work, they also have a significant role to play in providing necessary infrastructure, sup- porting scientific integrity, and organizing and conducting investigations of allegations of improper or incorrect research and reporting practices. The evaluation process recommended in this report (Figure S-1) defines the best practices for translation of an omics-based discovery into a validated omics-based test for use in a clinical trial, and focuses on the responsibili- ties of the investigators (Recommendations 1-3; Box S-1), with additional recommendations for other responsible parties, particularly institutions, but also funding agencies, journals, and FDA (Recommendations 4-7; Box S-2).

OCR for page 1
5 SUMMARY Throughout its recommendations, the committee emphasized the impor- tance of transparency in reporting—making data, metadata (information about a dataset and how it was generated), prespecified analysis plans, computer code, and fully specified computational procedures for external evaluation or confirmation. This reinforces recommendations made in sev- eral National Research Council reports (NRC, 2003, 2005, 2006). Development and Evaluation Process The committee’s recommended development and evaluation process for omics-based tests is summarized in Figure S-1. The two major stages of test development and evaluation entail (1) discovery and test validation phases and (2) evaluation of clinical utility and use. The discovery phase includes complete definition of the computational model to be used for data analysis in a clinical test and independent confirmation of that model. At this point, the fully specified computational procedures should be locked down— recorded and no longer changed. The candidate omics-based test from the research laboratory is then transferred to a CLIA-certified clinical labora- tory for development of the clinical testing methods followed by analytical validation and clinical/biological validation. The final stage is assessment of the clinical utility and use of the validated omics-based test within a clinical trial, with multiple design options depending on the intended clinical use of the test and availability of specimens from previous clinical trials. Statistics and bioinformatics validation occurs throughout both development stages. Overfitting of statistical models derived from omics data is common and many published gene expression results have been difficult to replicate. The committee was charged with the task of recommending an evalu- ation process for determining when predictive tests utilizing omics-based technologies are fit for use in clinical trials, especially those in which the assay is used to determine patient management. Review of published work and several case studies of tests—including some that have been adopted into clinical use and some that did not achieve that goal—helped the com- mittee outline and recommend a process for developing and evaluating omics-based tests. The committee’s recommendations aim to reinforce, clar- ify, and build on the current regulations and processes that apply to medi- cal devices such as in vitro diagnostic multivariate index assays and LDTs. The quality and quantity of data and information available for making this determination depends in part on the test discovery process. Because omics- based tests rely on interpretation of high-dimensional datasets, methods that avoid overfitting the data in development of the computational model should be used throughout the test development process. Overfitting due to use of improper statistical methods leads to a computational model that fits the training samples well, but will perform poorly on independent samples

OCR for page 1
ssons Learned and the Path Forward E: Omics-Based Test Development Process 6 EVOLUTION OF TRANSLATIONAL OMICS Discovery and Test Validation Stage Discovery Phase Test Validation Phase IRB Approval and Candidate Test Developed Consultation with the FDA on Training Set, Followed by Lock-Down of All Computational Procedures Define Clinical/ Clinical Analytical Biological Test Validation Confirmation of Candidate Validation Method Omics-Based Test Using: Using 1. An Independent Blinded Sample Set If Sample Set Available (Preferred); OR 2. A Subset of the Defined, Validated, and Locked-Down Test Training Set NOT (Intended Use, Assay, Computational Used During Training Procedures, and Interpretation Criteria) (Less Preferred) BRIGHT LINE Evaluation for Clinical Utility and Use Stage Three Potential Pathways (IRB Approval and FDA Consultation) Prospective/ Prospective Prospective Retrospective Clinical Trial; Clinical Trial; Study with Test Does NOT Test Directs Direct Patient Patient Archived Management Management Specimens IDE Needed? No No Yes FDA Approval/Clearance or LDT Process for Clinical Test Additional High-Quality Evidence to Evaluate Clinical Utility of the Test Practice Guidelines and Reimbursement Clinical Use TE: The computational procedures are locked down in the discovery phase, meaning they are recorded and longer changed in the subsequent development steps. A clinical test should be fully defined, validated, and ked down before crossing the bright line to enter the stage in which the test undergoes evaluation for its ended clinical use. (FDA is the Food and Drug Administration, IDE is an investigational device exemption, IR an Institutional Review Board, and LDT is a laboratory-developed test.)

OCR for page 1
7 SUMMARY not used in the discovery phase. Confirmation of all computational models and candidate omics-based tests on an independent sample set appropriate for the intended use of the test, blinded to assay results and clinical infor- mation linked to the specimens, is the “gold standard” method to assess test validity and to avoid taking an overfit computational model into subsequent development steps. In addition, the complexities of the data management and analyses of the large omics datasets highlight the need for availability of the data and computer code used in the discovery phase of omics-based test development for external evaluation and confirmation. The result of the discovery process, outlined in Figure S-1 and Recommendation 1 (see Box S-1) and fully described in Chapter 2, is a candidate omics-based test with fully specified and locked-down computational procedures. The candidate omics-based test is then transitioned into the test valida- tion phase to assess analytical and clinical/biological validation. There is a wealth of existing work on best practices for the development of clinical laboratory tests, and much of this can be applied to omics-based tests. A candidate omics-based test method should be optimized for performance FIGURE S-1 Omics-based test development process. In the first stage of omics- based test development, there are two phases: discovery and test validation. In the discovery phase, a candidate test is developed and confirmed. The fully specified computational procedures are locked down in the discovery phase and should re- main unchanged in all subsequent development steps. Ideally, confirmation should take place on an independent sample set. Under exceptional circumstances it may be necessary to move into the test validation phase without first confirming the candidate test on an independent sample set if using an independent test set in the discovery phase is not possible, but this increases the risk of test failure in the vali- dation phase. In the test validation phase, the omics-based test undergoes analytical and clinical/biological validation. The bright line signifies the point in test develop- ment where a fully defined, validated, and locked-down clinical test (analytical and clinical/biological validation) is necessary. Changes to the test after the bright line is crossed require a return to the test validation phase, approval by the Institutional Review Board, and possibly consultation with the Food and Drug Administration. In the second stage of test development, the fully defined, validated, and locked- down omics-based test undergoes evaluation for its intended clinical use. Evaluation of clinical utility and use is a process that often continues after initial adoption into clinical use. Statistics and bioinformatics validation occurs throughout the discovery and test validation stage as well as the stage of evaluation for clinical utility and use. NOTE: FDA = Food and Drug Administration, IDE = investigational device exemp- tion, IRB = Institutional Review Board, LDT = laboratory-developed test.

OCR for page 1
8 EVOLUTION OF TRANSLATIONAL OMICS BOX S-1 Development and Evaluation Process Recommendations Discovery and Test Validation Stage: Discovery Phase 1. When candidate omics-based tests from the discovery phase are intended for further clinical development, the following criteria should be satisfied and fully disclosed (for example, through publication or patent application) to enable independent verification of the findings: a. Candidate omics-based tests should be confirmed using an independent set of samples, not used in the generation of the computational model and, when feasible, blinded to any outcome or other phenotypic data until after the computational procedures have been locked down and the candidate omics-based test has been applied to the samples; b. The data and metadata used for development of the candidate omics-based test should be made available in an independently managed database (e.g., the database of Genotypes and Phenotypes [dbGaP]) in standard format; c. The computer code and fully specified computational procedures used for the development of the candidate omics-based test should be made sustainably available;a and d. The candidate omics-based test should be defined precisely, including the molecular measurements, the computational procedures, and the intended clinical use of the test, in anticipation of the test validation phase. Discovery and Test Validation Stage: Test Validation Phase 2. An omics-based test consists of both the data-generating assay and the fully specified computational procedures used for analysis of the assay data. Both components of omics-based tests used to direct patient management in a clinical trial setting should be validated during the test validation phase using the following steps: in a clinical laboratory, and then should undergo analytical validation and clinical/biological validation. The clinical test methods can be optimized based on feedback from the analytical validation performance charac- teristics, but must be fully defined, completely validated analytically and biologically/clinically with acceptable performance of the test, and locked down prior to assessment of the clinical utility and use of the test in a clini- cal trial. The optimal process for validating a candidate omics-based test is the same process used for validating any clinical test to be performed in clinical laboratories. This process is outlined in Figure S-1 and Recommen- dation 2 (see Box S-1) and fully described in Chapter 3. Once the analytical and clinical/biological performance of the defined omics-based test is acceptable based on intended clinical use, the test is

OCR for page 1
9 SUMMARY a. The candidate omics-based test and its intended use should be discussed with the Food and Drug Administration (FDA) prior to the initiation of valida- tion studies. b. Test validation should be performed in a CLIA-certified clinical laboratory, beginning with a defined candidate omics-based test from the discovery phase. c. The CLIA-certified laboratory should design, optimize, validate, and imple- ment the omics-based test under current clinical laboratory standards. d. If the omics-based test will be performed in more than one CLIA-certified laboratory for a clinical trial, analytical validation and CLIA requirements for the same omics-based test should be met by each laboratory, working with the primary laboratory. Evaluation for Clinical Utility and Use Stage 3. For investigators conducting a clinical trial to assess the clinical utility and use of an omics-based test that has been confirmed and validated as described above (Recommendations 1-2), the committee recommends that: a. Investigators should communicate early with FDA regarding the investiga- tional device exemption (IDE) process and validation requirements. b. Omics-based tests should not be changed during a clinical trial without a protocol amendment and discussion with FDA. A substantive change to the omics-based test may require that the study be restarted. aFor publicly funded research, code and fully specified computational procedures should be made publicly available either at the time of publication or at the end of funding. For commercially developed tests, this information would be submitted for FDA review if seeking approval or clearance, or would be described in a publication in the case of a laboratory- developed test. ready for a clinical study to assess clinical utility and use. Ideally, deter- mination of clinical utility should be derived from a prospective random- ized clinical trial, but in some circumstances that may not be feasible. Figure S-1 shows several possible pathways to evaluate clinical utility and use. Depending on the choice of study design, investigators may need to obtain an investigational device exemption (IDE) from FDA. Regardless of which pathway is chosen, however, the committee strongly recommends consulting with FDA prior to initiation of a clinical trial. In the case of a trial where patient management will be influenced by the omics-based test findings, this is a legal requirement. In other cases, the committee recom- mends consultation with FDA because the requirement for an IDE based on the trial design is not always clear. In addition, if the test will later need

OCR for page 1
10 EVOLUTION OF TRANSLATIONAL OMICS BOX S-2 Recommendations on Appropriate Actions to Ensure Adoption and Adherence Institutions 4.a. Institutions are responsible for establishing, supporting, and overseeing the infrastructure and research processes for omics-based test development and evaluation as well as best practices for clinical trials and observational research, including those incorporating omics technologies, and should assure that the evaluation process outlined in this report is followed for omics-based test development and evaluation at their institution. 4.b. Given the complexity of omics research and omics-based tests, the multi- disciplinary nature of omics research, and the potential for conflicts of interest in developing and evaluating tests for clinical use, institutional leaders should pay heightened attention to providing appropriate oversight and promoting a culture of scientific integrity and transparency. They should designate: i. A specific Institutional Review Board (IRB) member(s) to be responsible for considering investigational device exemption (IDE) and investigational new drug (IND) requirements as a component of ensuring the proper conduct of clinical omics research. ii. An institutional official who is responsible for comprehensive and timely documentation, disclosure, and management of financial and non-financial conflicts of interest, both individual and institutional. iii. An institutional official who is responsible for establishing and managing a safe system for preventing, reporting, and adjudicating lapses in scientific integrity, to enhance patient safety. iv. An institutional official who is responsible for establishing clear proce- dures for response to inquiries and/or serious criticism about the science being conducted at the institution. For example, this individual would be the responsible official for journals to contact with a serious con- cern about a manuscript, ensure that relevant information is provided to external scientists to help resolve issues of transparency of methods and data, and inform funders when an investigation of potential scientific misconduct is initiated. 4.c. Institutions that conduct biomedical omics research, including test develop- ment and clinical trials, should train, recognize, and support the faculty-level careers of individuals from the multiple collaborating disciplines, including biostatistics, bioinformatics, pathology, omics technologies, and clinical trial- ists, and ensure that they are: i. Treated as equal co-investigators and co-owners of responsibility ii. Represented on all relevant review and oversight bodies within the institutions

OCR for page 1
11 SUMMARY iii Intellectually independent, preferably reporting to an independent mentor and/or department chair as well as to the project leader Funders 5.a. All funders of omics-based translational research should: i. Require investigators to make all data, metadata, prespecified analysis plans, computer code, and fully specified computational procedures pub- licly available and readily interpretable either at the time of publication or, if not published, at the end of funding, and funders should financially support this requirement; ii. Provide continuing support for independent repositories to guarantee ongoing access to relevant omics and clinical data; iii. Support test validation in a CLIA-certified clinical laboratory and consider the usefulness of an independent confirmation of a candidate omics- based test prior to evaluation for clinical use; iv. Designate an official to alert the institutional leadership when serious alle- gations or questions have been raised that may warrant an institutional investigation; if the funder (e.g., the National Institutes of Health) has initiated that question, then the funder and institution should communicate during the investigation; and v. Establish lines of communication with other funders to be used when serious problems appear to involve interdependent research sponsored by another funder along the omics-based test development process. 5.b. Federal funders of omics-based translational research should have authority to exercise the option of investigating any research being conducted by a funding recipient after requesting an investigation by the institution. FDA 6.a. In order to enable investigators and institutions to have a clear understanding of their regulatory responsibilities, the Food and Drug Administration (FDA) should develop and finalize a risk-based guidance or a regulation on: i. Bringing omics-based tests to FDA for review; and ii. Oversight of laboratory-developed tests (LDTs). 6.b. FDA should communicate the IDE requirements for use of omics-based tests in clinical trials to the Office of Human Research Protections (OHRP), IRBs, and other relevant institutional leadership. continued

OCR for page 1
12 EVOLUTION OF TRANSLATIONAL OMICS BOX S-2 Continued Journals 7. Journal editors should: a. Require authors who submit manuscripts describing clinical evaluations of omics-based tests to: i. Register all clinical trials at www.clinicaltrials.gov or another trial reg- istry acceptable to the journal. ii. Make data, metadata, prespecified analysis plans, computer code, and fully specified computational procedures publicly available in an independently managed database (e.g., the database of Genotypes and Phenotypes [dbGAP]) in standard format. iii. Provide the journal with the sections of the research protocol relevant to their manuscript. iv. Identify each author’s role in the development, conduct, analysis, writing, and editing of the manuscript. Require the lead and senior authors to attest to the integrity of the study and the coauthors to confirm shared responsibility for study integrity. v. Use appropriate guidelines (e.g., the Consolidated Standards of Report- ing Trials [CONSORT] and the REporting recommendations for tumor MARKer prognostic studies [REMARK]) and submit checklists to certify guideline use. b. Develop mechanisms to resolve possible serious errors in published data, metadata, code, and/or computational models and establish clear procedures for management of error reports. c. Alert the institutional leadership and all authors when a serious question of accuracy or integrity has been raised. clearance or approval from FDA before clinical use, the study design and analysis will be subject to FDA review. A pre-IDE consultation can assist both the test developer and FDA in agreeing on the appropriate pathway for FDA review and the data necessary for FDA approval or clearance. Critical considerations for moving a fully defined omics-based test into clinical trials for assessing clinical utility and use are outlined in Figure S-1 and Recommendation 3 (see Box S-1) and fully described in Chapter 4.

OCR for page 1
13 SUMMARY Ensuring Adoption and Adherence to the Development and Evaluation Process The committee’s recommendations to ensure adoption of appropriate research and development practices (Recommendations 4-7) are put forth in the spirit of “best practices.” There could be numerous approaches to ensuring adherence, and the optimal approach will need to be determined by each of the stakeholders involved in omics research. Investigators and institutions that conduct omics-based research with the goal of improving patient care have responsibilities for supporting that research and test development. Both contribute to the scientific research culture in which omics-based research is conducted; investigators control the culture of individual laboratories, while institutions put policies and procedures in place that support scientific integrity and ensure sound and ethical practices for clinical research. To avoid adding new barriers to innovation in this promising field, the committee’s recommendations aim to emphasize and enhance institutional awareness of existing respon- sibilities to ensure the integrity of the scientific process. The commit- tee recognized that these recommendations might increase the oversight requirements for omics research in some institutions, but agreed that these potential costs were offset by the added safeguards for the integrity of this research. If an institution does not have the infrastructure or capability to follow the recommended Test Development and Evaluation Process defined in this report, then the committee believes that institution should consider not engaging in the translation of omics-based discoveries into validated tests intended for clinical use. As the Duke case study clearly demonstrates, existing procedures in some institutions may not adequately ensure the scientific integrity of translational omics. For example, although most institutions have clear policies and procedures for financial conflicts of interest for individuals, there is often less clarity when handling institutional conflicts, both finan- cial and non-financial. An institution might appear so conflicted in certain situations that an outside body should be asked to take responsibility for an investigation. The committee also addressed responsibilities of funders, FDA, and journals in ensuring rigorous development of omics-based tests (Box S-2). Funders play a leadership role in encouraging a culture of integrity and transparency in science, while they seek to accelerate progress through discovery, translation, and clinical applications. The committee highlighted the importance of funders supporting independent confirmation as well as validation in a CLIA-certified clinical laboratory of candidate omics- based tests. Funders have generally not supported such work in the past because they do not consider it to be original, innovative science. Without

OCR for page 1
14 EVOLUTION OF TRANSLATIONAL OMICS this support, confirmation and validation studies will be difficult, and the field will be left with promising ideas published in journals that may be used prematurely in clinical trials. The responsibilities of journal editors with respect to the adoption and adherence to the recommended omics- based test development and evaluation process are complicated by the wide spectrum of adopted policies and resources available to individual journals, but the committee recommends several actions—all specified in Box S-2—to improve the transparency and reproducibility of published research. Finally, there are several steps that FDA should take to improve understanding of the regulatory requirements for omics-based tests, by directly communicating with investigators and academic institutions and by developing a guidance or regulation that clarifies the relevant requirements in this dynamic field. Case Studies The committee examined several case studies of tests whose develop- ment histories provide lessons learned and illustrate the committee’s recom- mendations. These include the series of gene expression–based tests used in clinical trials at Duke University; the commercial tests Oncotype Dx, MammaPrint, Ova1, AlloMap Testing, CorusCAD, and the Tissue of Ori- gin Test; and the first OvaCheck test, which did not reach clinical use and for which errors were found in the methods used in the discovery phase. HER2 testing also is included as a case study to illustrate the challenges associated with a single-biomarker test, which could be magnified in omics- based tests. The committee was charged with presenting findings related to the gene expression–based tests used in the three Duke University clini- cal trials named in the statement of task. Published papers describing the development of those tests have been retracted, and, thus, it is now widely accepted that the clinical trials should not have used the omics-based tests for patient management decisions. The events at Duke University captured the attention of biological and quantitative scientists around the world. The committee gathered informa- tion about the series of events leading to the inappropriate use of the gene expression–based tests for patient management decisions in clinical trials at Duke University. Unfortunately, multiple systems put in place by Duke University to ensure the integrity and rigor of the scientific process failed. However, Duke University is not unique. Many of these failures stemmed from problems that may exist at other institutions: unclear lines of account- ability, lack of consistently strong data management, lack of confirmation of the omics discovery using an independent sample set, lack of definition or lock-down of the specific assay and computational analysis methods, lack of analytical and clinical/biological validation of the omics-based test

OCR for page 1
15 SUMMARY prior to commencing clinical trials, and individual and institutional con- flicts of interest, both financial and nonfinancial. As a result, public trust in the scientific and medical systems and patient-participant safety have been put at risk. During the 10 years since the research leading to the erroneous predic - tive tests was initiated, omics technologies and the regulation of omics- based tests have evolved. Institutions are better equipped now to answer investigator questions about appropriate development processes. None- theless, the committee identified needs for improvement. The committee believes the problems at Duke University could have been prevented had its recommendations been available and followed. Furthermore, the committee believes that scientific progress in omics-based test develop- ment will improve if these recommendations are broadly adopted because they ensure wide availability of data and computational models for the scientific community to explore, clarify the regulatory steps that must be followed along the process, and clarify responsibilities for the parties involved in this process. The committee hopes this report will provide a guide to the entire path- way for the development of omics-based tests, from discovery to clinical trials, to assist the many parties contributing to this translational research in understanding the complete pathway and not just their focused contribu- tions. This broader perspective may help the whole investigative team to understand the entire pathway and the pitfalls of each stage, with the hope of avoiding future problems in translating omics-based discoveries into clinical tests for the benefit of improved patient care. The committee’s recommendations are forward-looking and directed toward the whole field of translational omics research, with the goal of improving research and accelerating progress. Envisioning the improve- ment of omics-based test development through the implementation of its recommendations, the committee joins patients, clinicians, and scientists in seeking revolutionary new omics-based tools for improving patient care. REFERENCES NRC (National Research Council). 2003. Sharing Publication-Related Data and Materials: Responsibilities of Authorship in the Life Sciences. Washington, DC: The National Academies Press. NRC. 2005. Catalyzing Inquiry at the Interface of Computing and Biology. Washington, DC: The National Academies Press. NRC. 2006. Reaping the Benefits of Genomic and Proteomic Research: Intellectual Property Rights, Innovation, and Public Health. Washington, DC: The National Academies Press.

OCR for page 1