4

Supporting Data Infrastructure for Biomarker Tests for Molecularly Targeted Therapies

The rapid accumulation of disease-relevant molecular data following the complete sequencing of the human genome led the National Research Council (NRC) to explore the creation of a new taxonomy of human disease (NRC, 2011). The report outlined a framework based on broad access to patient data (through an “Information Commons”), to facilitate observational studies of emerging connections to evolving biological research (through a “Knowledge Network”). The hypotheses that would emerge about disease in various subpopulations could be validated in subsequent studies, ultimately establishing new taxonomies of disease that would lead to improved patient outcomes through more accurate diagnosis, clinical decision making, and treatment (NRC, 2011).

Although the NRC committee refrained from overspecifying the infrastructure necessary to create this new taxonomy, the collection, storage, and sharing of data were central to their vision. In a related report, Best Care at Lower Cost, the Institute of Medicine (IOM) recommended leveraging technology and policy mechanisms to address health care delivery’s rapidly mounting complexity, while maximizing value (IOM, 2013a). The report stated that a focus on clinicians alone was insufficient, and that health care tools, resources, and systems were all needed to create a “coordinated system of care” (IOM, 2013a, p. 135).

As discussed in Chapter 2, numerous experts have described an unmet need for a robust data infrastructure and the associated challenges with implementing a rapid learning health care system (Chute and Kohane, 2013; Etheredge, 2014; French and Kampfrath, 2014; Friedman et al., 2015;

Ginsberg, 2014; IOM, 2013a; Miriovsky et al., 2012). This chapter explores the ways in which overcoming current obstacles in health information technology and data sharing will provide the necessary infrastructure to support a rapid learning system for the clinical use of biomarker tests for molecularly targeted therapies.

ELECTRONIC HEALTH RECORDS

The goals of documentation in electronic health records (EHRs) and laboratory information systems (LISs) are complex, and reflect functions apart from clinical care, including billing and quality improvement; ensuring usefulness for these various functions requires properly structured data fields (Hripcsak and Vawdrey, 2013). The Office of the National Coordinator for Health Information Technology (ONC) reported in 2014 that more than half of office-based health care professionals, and more than 80 percent of hospitals in the United States, meaningfully use EHRs (ONC, 2014). Meaningful use in this context is defined by a set of criteria established by the Centers for Medicare & Medicaid Services (CMS) and divided into progressive implementation stages (i.e., Stage 1 criteria should be satisfied before advancing to Stage 2), which encompass broad data-related objectives that range from recording patient health information in structured formats, to sharing summary records, to protecting electronic health information (CMS, 2015). The adoption of meaningful use-compliant EHRs has been motivated, in part, by incentive payments by CMS to those providers who meet these criteria.

However, the widespread adoption of EHRs reported by ONC belies a critical gap relevant to precision medicine: structured -omics data fields are not currently required under criteria for meaningful use (CMS, 2014). Currently, omics test results and subsequent treatment and outcomes data are stored primarily in report or open-text formats (e.g., physician notes), and this precludes full integration into EHRs (IOM, 2012; Miriovsky et al., 2012). This is despite the need for structured data¹ to support a rapid learning system that integrates genetics into the clinic (Manolio et al., 2013). This gap is at least partially explained by the increasingly data-intensive nature of omics-based testing, requiring storage capacities that are orders of magnitude larger (Pelak et al., 2010) than for other ancillary EHR functions (e.g., radiology), and also by the persistent difficulty in

___________________

¹ Structured data, further discussed below, are part of CMS Meaningful Use criteria and help to improve data quality and to support data sharing for clinical care and research. Structured data are sometimes defined as a set of Common Data Elements (CDEs) that can standardize the names, definitions, and possible values of clinical information (Fridsma, 2013; NIH, 2013).

applying these data to improve health outcomes, as outlined in Chapters 3 and 5 of this report. Determining the optimal method of integrating these data into EHRs will be challenging. Distinct clinical and research needs must be considered; efficient decision support, based on actionable biomarker test results, must be balanced against the need to preserve access to the complete molecular test results that will enable research and discovery (Masys et al., 2012).

EHRs as Clinical Support

The increasing number of omics tests and the growing complexity of test results require tools to support health care providers in the evaluation of these relevant clinical data (IOM, 2015a; Masys, 2002; Stead et al., 2011; Welch and Kawamoto, 2013). One potential approach, explored in Chapter 5 of this report, is collaboration between community physicians and larger health care centers to evaluate complex clinical cases. However, a complementary role exists for decision support related to well-established clinical uses of biomarker tests and targeted therapies. This section of the chapter explores the data infrastructure needed for the use of such systems to reinforce evidence-based precision medicine within a rapid learning system.

The Role of Information Technology and Structured Data

Information technology has radically changed the way health care is practiced and documented. Currently, health care practices generate, exchange, and store vast amounts of patient-specific information. Apart from traditional clinical narratives, data generated in modern health care centers related to laboratory test results, diagnoses, imaging, and treatments are automatically captured in structured databases (Jensen et al., 2012). The interdisciplinary fields of bioinformatics and biomedical informatics, which involve managing, analyzing, and interpreting information from biological structures and sequences, have enabled the continual consolidation of systems that bear large-scale biomedical data (Sethi and Theodos, 2009). Crosscutting bioinformatics tools and techniques have led to exponential growth in the analysis of DNA and protein sequence databases, which in turn leads to a better understanding of disease mechanisms through not only genetics and proteomics, but by associating them with clinical data (Sethi and Theodos, 2009). When existing structured health care data are linked with biobanks and genetic data, the assimilation of biomedical informatics in a centralized EHR format can facilitate the exploration of genotype-phenotype associations, ultimately informing the use of genomic test results to improve patient care.

The American Medical Informatics Association (AMIA) defines bioinformatics as “the development of storage, analytic, and interpretive methods to optimize the transformation of increasingly voluminous biomedical data into proactive, predictive, preventative, and participatory health” (AMIA, 2006). Indeed, the objective of bioinformatics is to design and implement novel methodologies and tools that monitor and predict future health through identifying genetic mutations and protein interactions. More relevant to the implementation of precision medicine is the broader AMIA term biomedical informatics, defined as “the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving, and decision making, driven by efforts to improve human health” (Kulikowski et al., 2012, p. 931). Although progress has been achieved at the level of genomic data exploration, incorporation into health care records has yet to be fully realized (Sethi and Theodos, 2009). This is in part due to the nature of the data implementation. Health care providers using EHR systems have two primary methods for converting their observations into machine-computable and reusable data. The first method, qualitatively known as expressivity, includes unstructured data in which clinical narratives provide the medical reasoning behind various treatments and/or medications. This free-text form is convenient for recording impressions and concepts, but it is difficult for searching, summarization, decision support, and statistical analysis (Meystre et al., 2008). The second method consists of structured data entry that conforms to predefined or conventional syntactic organization and supports data reuse and comparison across multiple groups and timeframes.

The process of extracting structured patient data from clinical narratives “generally requires named entities or concepts in the text to be recognized and mapped to codes in a relevant controlled vocabulary, such as the Systematized Nomenclature of Medicine’s Clinical Terms (SNOMED CT) or 1 of the 100-plus vocabularies in the Unified Medical Language System” (Jensen et al., 2012, p. 398). Typically this is carried out through natural language-processing tools, which combine a range of linguistic, statistical, and heuristic methods to analyze free text and extract structured data (Jensen et al., 2012). The use of genomic data in standard clinical practice has commonly consisted of tests such as those for sickle cell anemia, cystic fibrosis, or cancer genomics (Green et al., 2011). The representation of actual genetic sequence information, generally stored in the LIS or related bioinformatics tools, is not widely implemented in EHRs. However, with increasing efforts toward integration of genomic and clinical data, semantic interoperability will be necessary for mapping between both platforms (i.e., the EHR and the LIS) (Sethi and Theodos, 2009). The Observational Health Data Sciences and Informatics

Initiative is one example of a project developing a common data model for interoperability that intends to be particularly well suited to scalable, distributed databases (OHDSI, 2016). As genetic research continues to pave the way for personalized medicine, researchers will be able to apply tools and technologies to better understand disease mechanisms that progress from molecular, cellular, tissue, and organ levels to the personal, and, ultimately, population level (Frueh et al., 2008).

Data sources with repeated and structured measurements are an attractive resource for assessing the relationship between changes in biological markers and risks of a clinical event. Improving patient outcomes through the application of genomic data will depend upon data structures that can easily integrate into EHRs, as well as provide a linking mechanism for genotype-phenotype data (Sethi and Theodos, 2009). The combination of detailed EHR-based patient phenotyping and genetic data has resulted in the emergence of a novel reversal of the genome-wide association study (GWAS) approach to gene-disease association (Jensen et al., 2012). A phenome-wide association study (PheWAS) instead starts with the individual variant² and checks for statistical association against hundreds of disease phenotypes of patients that have been genotyped for that variant, and has demonstrated usefulness as a tool to explore associations between genetic biomarkers and disease (Denny et al., 2013). Pharmacogenomics is an additional field that has recently embraced the assimilation of EHR data and genetic data (Wilke et al., 2011). The patient profile that can be constructed from an EHR, consisting of clinical data over a period of time, allows drug exposure profiles to be correlated with treatment outcome measures, such as efficacy and toxicity. Linked biobank and genetic data, properly structured, can then find associations of such correlations within the underlying genotype (Jensen et al., 2012). Dynamical modeling approaches, iteratively corrected and refined by translational science over time, can be derived from such data to develop predictive paradigms of disease evolution and drug response across the lifespan (Iyengar et al., 2015). The intersection of novel methods in translational biomedical informatics has the potential to greatly enhance clinical decision support.

Clinical Decision Support Systems

Clinical decision support systems (CDSSs), alternatively called clinical decision support tools, are “any electronic system designed to aid directly in clinical decision making, in which characteristics of individual patients are used to generate patient-specific assessments or recommendations

___________________

² In this example, the variant is a single nucleotide polymorphism (SNP).

that are then presented to clinicians for consideration” (Kawamoto et al., 2005, pp. 1-2). Biomarker test results that are structured to interact with decision-support algorithms, which in turn are based on an evolving synthesis of evidence, are critical for CDSSs (Pulley et al., 2012). Pilot projects to integrate and represent primarily pharmacogenomic biomarker data into EHRs for clinical use are underway. These examples illustrate the complex, interdisciplinary process of operationalizing CDSSs for guiding molecularly targeted therapy (see Box 4-1).

Varying approaches to implementation of CDSSs within EHRs exist, and can be combined as appropriate on an institutional or practice level. Passive CDSSs, for example, rely on clinicians to identify a knowledge gap, and seek educational resources embedded or linked within EHRs; active CDSSs can either be presented outside the clinical workflow (e.g.,

monthly reporting with relevant education) or at defined points within the clinical workflow (e.g., computerized order entry for a targeted therapy that requires a biomarker test result) (Williams, 2015). Implementing active CDSSs requires close collaboration with experts who have domain expertise with the target patient population, as well as careful and continuous consideration of the evidence to support specific drug/genotype associations (Pulley et al., 2012). This collaboration, inclusive of pathologists, clinicians, and bioinformaticians, can also be used to streamline and standardize test ordering and result reporting, while collecting feedback data to ensure the system provides value (e.g., lower unnecessary testing or higher necessary testing) (Stead, 2015). Given the potentially increasing role of CDSSs in routine clinical practice, implications for Continuing Medical Education (CME) and Specialty Board Maintenance of Certifica-

tion (MOC) may arise (see Chapter 5 for further discussion of CME and MOC).

In addition to ensuring routine testing for well-established clinical uses, CDSSs within a rapid learning system for biomarker tests for molecularly targeted therapies would also facilitate the matching of patients to clinical trials, where appropriate. In oncology, for example, strong evidence linking certain biomarker test results with effective therapy is growing, but currently limited; where no treatment alternatives exist, CDSSs could be leveraged to match patients to clinical trials evaluating the effectiveness of targeted agents (Meric-Bernstam et al., 2015). This functionality could enroll patients from increasingly small subpopulations, often studied together due to shared molecular characteristics across many cancer types (IOM, 2015b) (see Box 4-2).

Research to examine improvements in health care processes and health outcomes as a result of using CDSSs has produced mixed results. A health care process is defined as “a health care-related activity per-

formed for, on behalf of, or by a patient” (AHRQ, 2015b). Evidence exists for improvements in process measures, provided the CDSS operates well within the existing workflow, is computer based and provided at the time of clinical decision making, is based on good evidence, is applied to demonstrated inappropriate variability, and provides recommendations rather than assessments (Kawamoto et al., 2005; Williams, 2015). Health outcomes are more difficult to measure than processes, and although initial data are consistent with the potential for improvement through the use of CDSSs (Bright et al., 2012; Jaspers et al., 2011), further research is necessary to ensure optimal support of clinical care within rapid learning systems.

The recent Improving Diagnosis in Health Care report from the National Academies of Sciences, Engineering, and Medicine emphasized that CDSSs, while promising, must be used strategically in order to assist clinicians during their existing workflows and environments, and most importantly to ensure the CDSSs themselves do not introduce novel sources

of error into the decision-making process (NASEM, 2015). Of particular concern for the rapidly moving field of molecularly targeted therapies is ensuring that CDSSs are based on reliable and up-to-date evidence to avoid delivering incorrect recommendations to clinicians (AHLA, 2013).

Patient-Friendly EHRs

Potential challenges to patient-shared decision making in the context of omics-based tests and targeted therapies are discussed in Chapter 5 of this report; telemedicine in particular is explored as a mechanism to overcome barriers to patient access and clinician expertise. Similarly, patient-friendly EHR portals are well positioned to provide timely access to educational materials supportive of shared decision making. Research has demonstrated that patient attitudes are generally positive regarding the move toward precision medicine, but this is accompanied by a lack of knowledge about the nature of biomarker testing and implications for treatment (Blanchette et al., 2014; Issa et al., 2009). Additionally, health care providers and researchers are faced with ethical issues concerning the return to patients of incidental findings uncovered during clinical biomarker testing (see Box 4-3). Beyond reporting relevant test results in patient-readable formats, a number of online resources could be accessed from within the EHR portal to assist patients in understanding their test results and relevant treatment decisions (see Table 4-1).

Clinically Relevant Reporting

Certain considerations to ensure clinical relevance must be taken into account for biomarker test results to be used effectively to guide selection of molecularly targeted therapy; structured data fields should be reported for variants with an established effect on genes or protein variants associated with phenotypes, and implications for treatment (IOM, 2012). Historically, clinical laboratories have had primary responsibility for interpreting and reporting actionable test results, but the growing complexity of molecular testing, exacerbated by the lack of clinical data provided to clinical laboratories, is diffusing some of this responsibility to physicians and other health care providers (IOM, 2012). At the institutional level, multidisciplinary tumor boards or other interdisciplinary clinical conferences can help interpret test results and recommend treatment decisions for complex cases (Schilsky, 2014). Genetic counselors and other members of the health care team with relevant expertise will be essential to help health care providers and patients navigate this transition, and are discussed in Chapter 5.

More broadly, the task of defining the clinical utility of biomarkers for

the purposes of directing molecularly targeted therapy will be a continuous research endeavor. Larger collaborative efforts such as the American Society of Clinical Oncology’s (ASCO’s) Targeted Agents Profiling and Utilization Registry (TAPUR), the Actionable Genome Consortium,³ and

___________________

³ The Actionable Genome Consortium includes Dana-Farber Cancer Institute, Memorial Sloan Kettering Cancer Center, MD Anderson Cancer Center, and Fred Hutchinson Cancer Research Center, in a partnership with the sequencing company Illumina.

TABLE 4-1 Examples of Web-Based Resources to Assist Patients with Shared Decision Making

NOTE: URL = Uniform Resource Locator, an address to a resource on the Internet.

SOURCES: AHRQ, 2015a; ASCO and CancerNet, 2015a,b; Healthwise, 2015; Leukemia & Lymphoma Society, 2015; NCI, 2015b; NLM, 2015. Adapted from IOM, 2013b.

the Molecular Evidence Development Consortium (MED-C) are seeking to expand the availability of omics-based testing and treatment through demonstration of clinical utility for certain biomarker tests and molecularly targeted therapies (Dickson, 2015; Schilsky, 2014; Solit, 2014). The clinical utility of a biomarker test and corresponding targeted therapy is dependent on the interaction of treatment on phenotype and various outcomes, including survival and quality of life. Many of these data, including valuable patient experiences and reported outcomes, are only known by the patient and not always captured in the EHR; nevertheless, they are essential to the implementation of a value-driven rapid learning health system (Berenson et al., 2013; Millenson and Berenson, 2015). Methods to capture phenotype and exposure information from patients, as well as patient-reported outcomes, are therefore necessary to facilitate ongoing learning about the effectiveness of molecularly targeted therapies (see Box 4-4). EHR functionality will need to go beyond the support of clinical decision making, and enable rapid learning based on continuously aggregated data from “real-world” clinical practice and patient experience to facilitate the evaluation of outcomes associated with the use of biomarker tests and molecularly targeted therapies.

EHRs as Research Tools

With an increasing trend toward precision and personalized medicine, the omics world (including genomics, proteomics, metabolomics, lipidomics, transcriptomics, epigenetics, microbiomics, fluxomics, phenomics, etc.) has the potential to become a source of data-driven hypotheses and evidenced-based medicine. Clinical data, when properly captured in EHRs and correlated with data from biobanks or clinical laboratories, can be used as a tool to confirm genetic associations (Ritchie et al., 2010). Once established, such integrated health informatics systems can serve as “inexhaustible sources” of data for rapid learning (Krumholz, 2014, p. 1163). These rapid learning systems can leverage a wide variety of information, including individual patient data, clinical trials and other population-level data, and operational data (Yu, 2015). For molecularly targeted therapies in particular, generating evidence to support the use of a biomarker or associated therapy will require structured data fields

beyond the test results necessary for a CDSS as described above. Moreover, clinical use of potentially beneficial molecularly targeted therapies (through compassionate use programs, or general “off-label” use) do not currently produce usable outcomes data, which represents a tremendous loss of information that could be used to further assess the utility of the therapy (Schilsky, 2014). Thus, data on the treatments prescribed and used as well as longitudinal clinical patient data will need to be more rigorously captured, in a manner that is minimally burdensome to health care providers (Kullo et al., 2013).

Ensuring the ease of use of EHRs for this purpose is as important as data functionality. Results from a survey of physicians by the RAND Corporation further emphasize this point: Despite a general preference for EHRs over paper records, current EHR implementations decrease professional satisfaction due to a combination of perceived poor usability, disruption to workflow, and diminished quality of clinical documentation (Friedberg et al., 2013). The Improving Diagnosis in Health Care report stated that technologies such as speech recognition and natural language processing (which extracts data into structured formats from free text) may serve to bridge the gap between the need for structured data and clinician preference for more free-form documentation (NASEM, 2015). The diversity of clinical workflow environments may be best served by offering multiple solutions within the EHR, to enable clinicians to select an option based on their specific needs and preferences (Rosenbloom et al., 2011).

Data infrastructure supporting a rapid learning system for biomarker tests and molecularly targeted therapies requires linkages among the specific test ordered, the reported results, the treatments prescribed (whether based on the test result or not), and longitudinal clinical patient data. Diverse longitudinal data collected routinely during clinical practice would enable assessments of usage and multiple outcome measures to meet the evolving needs of clinical utility as defined by the broader health care stakeholder community (see Chapter 3). Therefore, to support the data infrastructure needs of a rapid learning system, the committee recommends that EHR and LIS vendors and relevant software developers should enable the capture and linkage of biomarker tests, molecularly targeted therapies, and longitudinal clinical patient data in the EHR (Recommendation 6a). Data structured in the EHR should include, at a minimum, biomarker test specimen requirements, specific test results and interpretation (particularly of actionable next-generation sequencing, or NGS, variants), treatments prescribed and other diagnostics ordered (including the reason for such orders, e.g., further need to refine treatment options), and longitudinal clinical patient data. Similarly, structured data within the LIS would include specific biomarker test descriptions,

including assay method, analytes assessed, test performance characteristics, quality metrics, and bioinformatics tools, when applicable. Taken together, these structured and linked data would provide the backbone of the infrastructure to facilitate clinical decision support development and ongoing research. As discussed previously, these fields should be populated in a manner that facilitates easy entry of data by clinicians in order to ensure consistency and use.

The committee further recommends that EHR vendors and relevant software developers should enable EHRs to facilitate point-of-care decision support for biomarker test ordering, reporting, and shared clinical decision making (Recommendation 6b). Decision support should be flexible, employing both highly focused prompts during clinic visits as well as more detailed educational support before or after the visit. EHRs should allow for the incorporation of practice guidelines or other treatment pathways into decision support systems, and include mechanisms for tracking compliance. Patient-facing EHR portals should provide biomarker test results in a patient-friendly format, and include linkage to relevant educational materials and the committee’s proposed pilot test labels (discussed in Chapter 3).

The committee recommends that health care institutions and physician practices should use EHRs that facilitate point-of-care decision support for biomarker test ordering, reporting, and clinical decision making. Because each practice will need to customize the EHR for their specific use, the customization of point-of-care decision support should align with available evidence-based clinical practice guidelines.

Finally, given the increasing educational role provided by CDSSs in a rapid learning system for biomarker tests and targeted therapies, the committee recommends licensing and specialty boards should recognize Continuing Medical Education (CME), Continuing Education Units (CEUs), and Maintenance of Certification (MOC) achieved through interaction with point-of-care decision support educational materials (Recommendation 6d). Professional schools, training programs, and specialty boards should ensure that clinicians are skilled in the use of these tools.

BIG DATA

In oncology, a relatively modest number of “driver” mutations are thought to underpin each patient’s cancer (Vogelstein et al., 2013). However, each patient’s cancer also harbors a long “tail” of mutations that, though infrequent, make each case unique (Garraway, 2015). This fact complicates efforts to define which biomarker test results may truly be clinically actionable for any given patient. Solving this challenge likely

will depend on the ability to leverage very large sample sizes coupled with detailed phenotype and clinical data in order to distinguish meaningful outcomes associated with the use of certain therapies (Abernethy, 2015). Recent research into genomic correlates of response to cancer immunotherapy likewise concluded that “detailed integrated molecular characterization of large patient cohorts may be needed to identify robust determinants of response and resistance to immune checkpoint inhibitors” (Van Allen et al., 2015, p. 207). The existing distributed and data-driven initiatives described in this chapter reflect these needs. Moreover, a small number of medical school curricula, with funding from the American Medical Association, are beginning to incorporate big data coursework to train physicians who are comfortable using such evaluations to explore and improve health care delivery (AMA, 2015). However, medical research in general has lagged behind other fields in the use of big data analytics, despite the potential usefulness of the vast amounts of data generated daily in clinical care (Krumholz, 2014).

Data Sharing

For large-scale data analysis to generate knowledge effectively, it will be important to aggregate evidence on promising biomarker tests and targeted therapies. Capture of clinical and other data from the off-label use of targeted therapies was discussed in the context of EHRs above, but data on biomarker tests or therapies that never make it to market due to failure to demonstrate benefit for patients in clinical trials also will need to be consistently documented. Studies have shown that despite obligations to do so, not all clinical trials are properly reported or updated on national registries such as ClinicalTrials.gov (Anderson et al., 2015), and the lack of widespread reporting of negative studies may impede research to develop effective molecularly targeted therapies. Furthermore, published medical research does not completely reflect all data generated during clinical trials (Riveros et al., 2013). Ensuring that clinical research data is made publicly available, when appropriate, is a first step toward building a culture of data sharing that would enable “big data” analysis.

The challenges and benefits of ensuring that clinical trial data are properly shared were described in the IOM report Sharing Clinical Trial Data. Acknowledging that the clinical trials landscape is changing, the report made broad recommendations, including the fostering of a data-sharing culture; adhering to pre-specified timeframes for sharing types of study data; and enhancing security and transparency through development of strategies, independent review panels, and data use agreements (IOM, 2015c). However, precision medicine may represent a blurring of the line between clinical care and clinical research; one example is treat-

ment for advanced cancer, which increasingly seeks to use the latest generation of molecularly targeted therapies when other treatments have been exhausted (IOM, 2015b). Comparative effectiveness research (CER), in which the relative benefits and harms of interventions are assessed, increasingly seeks to refine clinical practice through evaluating current clinical care options (see Box 4-5). Importantly, the clinical use of NGS platforms will require data sharing to ensure the validation of novel test results through assessment of concordance across clinical laboratories and patient care sites (IOM, 2012). The National Institutes of Health (NIH) considers data sharing integral to translating genomic discoveries to the benefit of human health, and has published a Genomic Data Sharing Policy⁴ that applies to all NIH-funded research that generates large-scale genomics data.

The health care and clinical research communities have been largely responsive to these data-intensive requirements of precision medicine; many public, private, and academic initiatives to create resources to assess the clinical significance of biomarkers are under way (see Table 4-2). These initiatives, however promising, represent silos of knowledge and potential obstacles to the building of larger datasets. A critical prerequisite for successful data sharing is overcoming the institutional, organizational, or other opposition to making data available. Sharing Clinical Trial Data featured a prominent discussion on the disincentives to sharing data, particularly the potential for researchers unrelated from the initial data gathering to misuse data or use the data to undermine the primary research (IOM, 2015c). Recent movement toward building data-sharing culture, including requirements for data sharing as a condition for manuscript submission, has helped demonstrate the potential of “symbiotic” secondary research on shared datasets (Dalerba et al., 2016; Longo and Drazen, 2016; Taichman et al., 2016). Emerging best practices for secondary researchers seeking to ensure trust should include a focus on novel research ideas, the careful selection of collaborators whose data may help test a hypothesis, and shared responsibility for conducting the research and subsequent report authorship (Longo and Drazen, 2016).

Despite the barriers to large-scale collaboration, promising projects are demonstrating potential to leverage shared data and expertise. The Global Alliance for Genomics and Health (GA4GH) seeks to position itself as an “honest broker” through convening stakeholders, fostering innovation and data-sharing culture, and harmonizing institutional processes (Lawler et al., 2015). GA4GH working groups (focusing on clinical practice, data, regulation, and security) have released work products, including data analysis tools as well as consent and data sharing policies

___________________

⁴ 79 CFR 51345.

(GA4GH, 2016). Initiatives under the umbrella of the GA4GH include an effort to globally pool BRCA research data to more rapidly establish therapeutic and preventive interventions for breast cancer patients, the development of a European “eCancer Hospital” through interoperable clinical laboratory processes and health information technology (IT) protocols, and the American Association for Cancer Research’s Genomics

TABLE 4-2 Existing Data Resources for Assessing Clinical Significance of Biomarkers

NOTE: AV = analytic validity; CU = clinical utility; CV = clinical validity; EGFR = epidermal growth factor receptor, a common therapeutic target in some cancers; NHGRI = National Human Genome Research Institute; NIH = National Institutes of Health; SNP = single-nucleotide polymorphism, a type of genetic variant.

SOURCES: Adapted from Bailey et al., 2014; Ramos et al., 2014.

Evidence Neoplasia Information Exchange (GENIE) (Lawler et al., 2015). The recently announced GENIE project seeks to “fulfill an unmet need in oncology by providing the statistical power necessary to improve clinical decision-making, particularly in the case of rare cancers and rare variants in common cancers,” and as such is a particularly relevant potential proof of concept for the committee’s proposed rapid learning system for biomarker tests for molecularly targeted therapies (AACR, 2016).

The Food and Drug Administration (FDA) has also signaled its willingness to innovate in the field of “regulatory science,” specifically stream-

lining evidence generation through the use of adaptive trial designs and by the incorporation of “real-world” patient data (Califf and Ostroff, 2015). FDA also recently announced that it intends to collaborate with the broader scientific community to develop future oversight of NGS and other data-intensive technologies through an open-access research infrastructure (Blumenthal et al., 2016; FDA, 2015b; Kass-Hout and Litwack, 2015).

The Common Rule⁵ governing human subjects research may be undergoing a similar policy evolution. The Department of Health and Human Services (HHS) has proposed modifications⁶ to the rule, including (1) a requirement for informed consent for all research on biospecimens, whether or not they are de-identified; (2) streamlining of the informed consent process; (3) the creation of uniform data security standards calibrated to the types of information being collected, thus removing the evaluation of privacy risks from Institutional Review Boards’ oversight, a task they may have not been well-suited to perform; and (4) the creation of a central repository for submission of adverse event data across all relevant federal agencies. Taken together, these modifications indicate a willingness to respect the privacy of research subjects in an increasingly data-driven research environment, while helping to facilitate data sharing by minimizing regulatory ambiguity for researchers (Hudson and Collins, 2015).

Challenges for Data Sharing

The Sharing Clinical Trial Data report concluded with observations on challenges to broad sharing of data, some of which are relevant for a rapid learning system to assess biomarker tests and molecularly targeted therapies. The primary technological challenge, beyond structured data discussed previously, is the need for infrastructure to store such a large volume of data while remaining nimble enough to permit continuous research (IOM, 2015c). The report acknowledged that the volume of data and the diversity of stakeholders involved in clinical research likely precluded the creation of a single database; rather, a federated query model could bridge existing resources by connecting them “in such a way that they can respond to queries as if all the data were in a single virtual database” (IOM, 2015c, p. 166). The federated query model also maintains institutional ownership of shared data, which was crucial in the context of clinical trial data and may be equally crucial for a repository of data for biomarker tests and molecularly targeted therapies (IOM, 2015c).

___________________

⁵ 45 CFR 46.

⁶ 80 FR 53931.

The ability for distinct electronic resources to transmit and receive structured data successfully (i.e., interoperability) is also critical to data sharing. For example, initiatives such as ASCO’s Cancer Learning Intelligence Network for Quality (CancerLinQ) program or the American Society of Radiation Oncology’s National Radiation Oncology Registry are attempting to demonstrate the feasibility of leveraging heterogeneous clinical data to support adherence to practice metrics (Efstathiou et al., 2013; Hudis, 2015). However, lack of interoperability in EHRs has been cited as a major roadblock to leveraging big data analytics to improve health care (ASCO, 2015). Moreover, ONC reported that data-blocking activities⁷ by health care providers or EHR vendors are potentially an additional obstacle to interoperability and widespread data sharing, though the report concluded that the scope of such activities is difficult to quantify (ONC, 2015b). The 21st Century Cures Act⁸ passed by the House of Representatives includes provisions designed to address some of these obstacles, including a prohibition of data blocking, a mandate for complete access and exchange of health information, and a broadly outlined plan for the development of interoperability standards. The development of such standards for interoperability is essential to the use of clinical patient data stored within EHRs for research purposes (Jensen et al., 2012). An ONC task force to evaluate the role of health IT for the Precision Medicine Initiative (PMI) likewise recommended closing the existing gaps in data and interoperability standards in order to facilitate the exchange of clinical data (ONC, 2015a). Pilot projects in EHR data interoperability have been promising, but limited in scope (Rea et al., 2012; Warner et al., 2015).

There will be a need to ensure the quality of data, particularly genomic sequence data, that are submitted to a shared database for biomarker tests for molecularly targeted therapies. In some, but not all, of the existing data resources in Table 4-2 there may be concerns around the quality of submissions (i.e., the strength of genotype-phenotype associations of sequences submitted). More specifically, quality needs to be addressed with respect to both analytic validity and clinical validity. For analytic validity, historically the only quality control for data emerging from research databases is if the report of the variant has been published in peer review. This is a very low bar. However, the sequence interpreta-

___________________

⁷ In the report, ONC described data blocking as activities beyond lack of coordination, divergent policy implementation, and inconsistency in standards that currently exist across health care systems and among states. Examples of health care providers or EHR vendors actively engaging in data blocking include: contractual terms that restrict access to health information, prohibitively expensive fees for information portability, and developing health IT in ways that will likely result in costly or complex data sharing or “locked in” users or data.

⁸ H.R. 6, 114th Congress (2015-2016).

tions relevant for a proposed database of variants used in clinical care will come from a clinical laboratory variant report, and the lab will have established analytic validity through the Clinical Laboratory Improvement Amendments certification process (or other related oversight processes, as discussed in Chapter 3).

The curation of clinical validity (i.e., whether a variant is likely pathogenic or pathogenic, or represents a legitimate target for molecularly targeted therapy) is indeed a source of confusion in existing public databases. However, the underlying rationale for these databases is to make variant interpretations transparent and publicly accessible, to include the underlying data that supports the interpretation, and to highlight the existence of discrepancies. Having a variant interpretation in a database does not mean it is correct; unfortunately, there is no reliable method for fully pre-curating the clinical validity of variants and knowing the “truth” about such variants prior to submitting them to a database. It is the availability of multiple interpretations, and the underlying evidence for these interpretations, that allows us to highlight discrepancies and allow the most accurate clinical validity assessments (Rehm et al., 2015). The power of such comparisons will be enhanced by existing collaborative efforts such as the Green Park Collaborative (GPC) and the MED-C collaboration to develop a core set of clinical data elements that all such repositories should collect and to identify data commonalities that promote cross-database compatibility and comparisons (CMTP, 2016).

Sustainability of data sharing is also paramount, particularly because the assessment of outcomes associated with therapy requires the use of longitudinal clinical patient data. For clinical trial data sharing, it was concluded that costs were borne by a fraction of stakeholders; the IOM therefore recommended transitioning to a model with more equitable distribution of economic responsibility (IOM, 2015c). A rapid learning system for biomarker tests and molecularly targeted therapies could be sustained through a variety of methods; for example, Chapter 3 discusses reimbursement models that could facilitate data collection for the ongoing assessment of clinical utility of biomarker tests. Given the expanding role of EHRs for both decision support and research as described above, and their potential to provide valuable clinical data to larger repositories, an additional mechanism to sustain the development and use of data sharing may take the form of payments to health care providers who submit data, similar to those provided by CMS to early adopters of EHRs.

Data security and privacy may pose challenges to broad data sharing for the purpose of rapid learning for biomarker tests and molecularly targeted therapies. The collection of longitudinal clinical and other outcomes data to support the clinical utility of biomarker tests and molecularly targeted therapies will necessarily preclude certain levels of de-

identification. Additionally, for genomic test results, true de-identification may be impossible as these data are inherently identifiable (Gymrek et al., 2013; Homer et al., 2008). The Common Rule currently allows de-identified specimens and data to be shared or used for research without a requirement for informed consent; however, variability in interpretation of regulations, particularly in delineating when research is additionally subject to the Health Insurance Portability and Accountability Act of 1996,⁹ has perpetuated an environment lacking in uniform standards for the use of data in research (IOM, 2009a). Given these facts, and the proposed changes to the Common Rule that will require informed consent for all research involving biospecimens and related data, the committee believes that patient consent measures are a reasonable requirement for data sharing to facilitate rapid learning for biomarker tests and molecularly targeted therapies. Such consent measures could take the form of the broad, open-ended consent documents suggested in the proposed Common Rule changes, and currently employed in some health care centers across the country.

Previous reports have addressed the competing needs of data sharing and privacy across varying health care research contexts, and in general have emphasized that privacy regulations should be prudently developed and deployed only when necessary and effective, in order to facilitate health care advances through research (IOM, 2009a, 2015c; NRC, 2011; Presidential Commission for the Study of Bioethical Issues, 2012). The NRC report Toward Precision Medicine stated that “there is little evidence that the public has the extreme sensitivity toward genetic data that many researchers anticipated 25 years ago” (NRC, 2011, p. 39). The Presidential Commission for the Study of Bioethical Issues likewise concluded that parsimonious regulation is justified because it will facilitate the sharing of data from autonomous research participants who desire to contribute to beneficial medical research (Presidential Commission for the Study of Bioethical Issues, 2012).

Nevertheless, data security and patient consent measures are insufficient to remove all concerns of potential breaches of privacy, and additional mechanisms could be used to minimize risk. In Sharing Clinical Trial Data, the IOM recommended data use agreements to govern the level of data sharing and potential uses for which data could be used, as well as transparency in policies and composition of any bodies that review research requests for shared data. Additionally, mandatory registration for researchers seeking to use shared data resources would facilitate penalties for willful re-identification of data or other misuse (IOM, 2015c). The PMI’s Proposed Privacy and Trust Principles likewise include data

___________________

⁹ Public Law 104-191.

use agreements and criminal penalties for the deliberate misuse of data. The PMI cohort paradigm seeks to create a new research model in which consented participants are equally as engaged as investigators, and the resulting focus is not on eliminating privacy risks but adequately communicating risk and minimizing such occurrences, with procedures in place for prompt notification and accountability in the event of a data breach (The White House, 2015).

Under the proposed changes to the Common Rule, standardized data security protections will be developed that are sensitive to varying clinical research scenarios, and the development of data-sharing repositories should reflect those standardized protections. Similarly, existing confusion regarding appropriate informed consent and requirements for sharing patient specimens and data will be clarified. However, existing research performed on stored and de-identified biospecimens may need to be re-evaluated in light of the proposed consent requirements. Moreover, whether or how specific data security, informed consent, and privacy modifications to the Common Rule will address potential discrimination on the basis of omics data is uncertain (see Box 4-6). Thus, the nature of the impact on translational research by the proposed updates to the Common Rule, while likely to be significant, remains unclear.

The Precision Medicine Initiative

The PMI, as described in the introduction to this report, reflects an appreciation for the scale of data and research infrastructure needed to explore associations between health and omics data. In a recently released PMI Working Group report on the PMI cohort study, the authors reflected on maximizing the opportunity to systematically study such a large research cohort, and recommended automated data collection whenever possible, rigorous data curation, centralized data resources, and coordination with CMS and other payers to facilitate integration of clinical data from EHRs (PMI Working Group, 2015). These recommendations are consistent with the role of supporting data infrastructure for biomarker tests for molecularly targeted therapies, as outlined in this chapter. The PMI is additionally relevant because of the funding set aside for the development of regulatory-grade databases to advance precision medicine (Blumenthal et al., 2016; OPS, 2015). FDA’s pilot collaborative data-sharing platform, precisionFDA, represents the first step toward such regulatory tools.¹⁰ NIH, through the National Human Genome Research Institute, is currently seeking multistakeholder input (including payers, patients, health care providers, researchers, professional organizations, policy makers, and

___________________

¹⁰ See http://precision.fda.gov (accessed January 5, 2016).

others) on the optimal design of a clinical sequencing program, including the ability to leverage existing and future data infrastructure to support the integration of genomics into clinical care (NHGRI, 2015). Similarly, the National Cancer Institute (NCI) intends to fund collaborative genomic and proteomic research networks across academic institutions in order to comprehensively characterize tumor types, investigate responses to drugs, and refine biomedical informatics approaches to working with these large omics datasets.¹¹

Thus, a well-identified need exists for broader collaboration and data

___________________

¹¹ Funding Opportunity Announcement Numbers include RFA-CA-15-018, RFA-CA-15-019, RFA-CA-15-020, RFA-CA-15-021, RFA-CA-15-022, RFA-CA-15-023.

sharing among all stakeholders in the health care system; such collaboration would multiply the usefulness of current research through the positive economics of data sharing (IOM, 2015a). To capitalize on the initial momentum described in this chapter (among clinical researchers, health care providers, health information technology vendors, regulatory agencies, and others), and to facilitate the continuous assessment of the evidence supporting the clinical use of biomarker tests and molecularly targeted therapies, the committee recommends the Secretary of HHS should charge FDA and NIH to convene a Task Force (comprising FDA, CMS, the Department of Veterans Affairs, NIH, the Department of Defense, the Patient-Centered Outcomes Research Institute, and other public and private partners) to develop a sustainable national repository of biomarker tests, molecularly targeted therapies, and longitudinal clinical patient data to facilitate rapid learning approaches. The repository would include data structured within EHRs (including biomarker test description, test results and interpretation, treatment decisions and outcomes, adverse reactions, and other relevant data generated during clinical practice), as well as clinical trial data, billing/reimbursement data, patient-reported outcomes, and other longitudinal clinical patient data. Given the fact that widely accepted EHR interoperability standards do not currently exist, the committee expects that the Task Force may need to define and develop a repository-specific interoperability standard, in order to ensure the incorporation of data from as broad a pool of clinical practice settings as possible.

In addition to data located in EHRs, this repository could also leverage data from other existing resources, such as FDA’s Sentinel Initiative to track adverse events (see Box 4-7), or those resources for assessing the clinical use of biomarker tests listed in Table 4-2. The repository may ultimately be composed of discrete databases for different indications (e.g., separate and dedicated resources for oncology, cardiology, cystic fibrosis, etc.). As mentioned above, the national repository should be built and made accessible with appropriate de-identification, data security, and patient consent measures, and sustainability should be provided, in part, through incentives put into place by HHS for data submission by all health care providers and health systems. The committee believes such a resource should provide de-identified datasets freely to researchers, health care providers, payers, and regulators, and standards and best practices for data sharing and analysis could draw upon existing cloud-based programs (Stein et al., 2015). The NCI’s Cancer Genomics Cloud Pilots are one such example of attempting to standardize access, analysis, and collaboration on large genomic datasets (NCI, 2015a).

SUMMARY AND RECOMMENDATIONS

The committee’s vision for a rapid learning system to assess biomarker tests and corresponding molecularly targeted therapies depends on robust data infrastructure; many of the recommendations presented throughout this report relate to the capabilities outlined here for EHRs and a national repository for shared data. The systematic capture of relevant clinical data will serve the complex and occasionally competing needs of regulators, payers, health care professionals, patients, and drug and diagnostic developers. Continuous research on these data through an openly accessible national data repository will be instrumental in assessing value and ultimately improving patient outcomes.

Goal 6: Ensure development and use of EHRs and related biomedical informatics tools and assessments that support the effective clinical use of biomarker tests for molecularly targeted therapies.

Recommendation 6a: Electronic health record (EHR) and laboratory information system (LIS) vendors and relevant software developers should enable the capture and linkage of biomarker tests, molecularly targeted therapies, and longitudinal clinical patient data in the EHR to facilitate data transfer into one or more national databases (as described in Recommendation 7).

The information to be structured in the EHR should include, at a minimum:

• Biomarker test specimen requirements (type, amount, handling).
• Specific biomarker test results and interpretation (including actionable panel or next-generation sequencing test results).
• Treatments prescribed and diagnostic tests ordered (whether based on the biomarker test result or not).
• Longitudinal clinical patient data.

The information to be structured in the LIS should include, at a minimum:

• Biomarker test descriptions (assay method, analytes assessed, test performance characteristics, quality metrics, and bioinformatics tools).

Recommendation 6b: Electronic health record (EHR) vendors and relevant software developers should enable EHRs to facilitate point-of-care decision support for biomarker test ordering, reporting, and shared clinical decision making.

• EHR decision support should be layered: highly focused for within the office visit and more detailed for before or after the visit.
• EHRs should allow for incorporation of practice guidelines and pathways as decision support, and also allow tracking compliance.
• Patient portals linked to EHRs should provide biomarker test result information in a patient-friendly manner.
• To enhance patient understanding, relevant educational materials should be accessible from within the portal.
• Portals should include linkage to test labels (see Recommendation 3).

Recommendation 6c: Health care institutions and physician practices should use electronic health records (EHRs) that facilitate point-of-care decision support for biomarker test ordering, reporting, and clinical decision making. This point-of-care decision support should align with available evidence-based clinical practice guidelines.

Recommendation 6d: Licensing and specialty boards should recognize Continuing Medical Education, Continuing Education Units, and Maintenance of Certification achieved through interaction with point-of-care decision support educational materials.

• Professional schools, post-graduate training programs, specialty boards, and continuing education programs should ensure that providers are skilled in the use of point-of-care decision support tools.

Goal 7: Develop and maintain a sustainable national database for biomarker tests for molecularly targeted therapies through biomedical informatics technology to promote rapid learning for the improvement of patient care.

Recommendation 7: The Secretary of the Department of Health and Human Services (HHS) should charge the Food and Drug Administration (FDA) and National Institutes of Health (NIH) to convene a task force (comprising FDA, the Centers for Medicare & Medicaid Services, the Department of Veterans Affairs, NIH, the Department of Defense, the Patient-Centered Outcomes Research Institute, and other public and private partners) to develop a sustainable national repository of biomarker tests, molecularly targeted therapies, and longitudinal clinical patient data to facilitate rapid learning approaches.

• This prospective, integrated, and structured database should include biomarker test description, test results and interpretation, treatment decisions and outcomes, other relevant electronic health record data generated during clinical practice, clinical trial data, billing/reimbursement data, patient-reported outcomes, and longitudinal clinical patient data.
• The national repository should be built and made accessible with appropriate de-identification, data security, and patient consent measures.
• HHS should provide incentives to encourage data submission by all health care providers/health systems.

REFERENCES

Berg, J. S., L. M. Amendola, C. Eng, E. Van Allen, S. W. Gray, N. Wagle, H. L. Rehm, E. T. DeChene, M. C. Dulik, F. M. Hisama, W. Burke, N. B. Spinner, L. Garraway, R. C. Green, S. Plon, J. P. Evans, G. P. Jarvik, and members of the CSER Actionability and Return of Results Working Group. 2013. Processes and preliminary outputs for identification of actionable genes as incidental findings in genomic sequence data in the clinical sequencing exploratory research consortium. Genetics in Medicine 15(11):860-867.

Birnbaum, J. K., F. O. Ademuyiwa, J. J. Carlson, L. Mallinger, M. W. Mason, and R. Etzioni. 2015. Comparative effectiveness of biomarkers to target cancer treatment: Modeling implications for survival and costs. Medical Decision Making. August 24, 2015 (Epub ahead of print).

Blanchette, P. S., A. Spreafico, F. A. Miller, K. Chan, J. Bytautas, S. Kang, P. L. Bedard, A. Eisen, L. Potanina, J. Holland, S. Kamel-Reid, J. D. McPherson, A. R. Razak, and L. L. Siu. 2014. Genomic testing in cancer: Patient knowledge, attitudes, and expectations. Cancer 120(19):3066-3073.

Blumenthal, G. M., E. Mansfield, and R. Pazdur. 2016. Next-generation sequencing in oncology in the era of precision medicine. Journal of the American Medical Association Oncology 2(1):13-14.

Bright, T. J., A. Wong, R. Dhurjati, E. Bristow, L. Bastian, R. R. Coeytaux, G. Samsa, V. Hasselblad, J. W. Williams, M. D. Musty, L. Wing, A. S. Kendrick, G. D. Sanders, and D. Lobach. 2012. Effect of clinical decision-support systems: A systematic review. Annals of Internal Medicine 157(1):29-43.

Califf, R. M., and S. Ostroff. 2015. FDA as a catalyst for translation. Science Translational Medicine 7(296):296ed299.

Chute, C. G., and I. S. Kohane. 2013. Genomic medicine, health information technology, and patient care. Journal of the American Medical Association 309(14):1467-1468.

CMS (Centers for Medicare & Medicaid Services). 2014. Eligible professional meaningful use core measures: Measure 10 of 17 (stage 2). https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/Stage2_EPCore_10_ClinicalLabTestResults.pdf (accessed October 19, 2015).

CMS. 2015. Electronic health records (EHR) incentive programs. https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/index.html?redirect=/ehrincentiveprograms (accessed October 19, 2015).

CMTP (Center for Medical Technology Policy). 2016. GPC and MED-C launch joint effort to harmonize standards for NGS registries and databases. http://www.prnewswire.com/news-releases/gpc-and-med-c-launch-joint-effort-to-harmonize-standards-for-ngsregistries-and-databases-300203152.html (accessed January 20, 2016).

Dalerba, P., D. Sahoo, S. Paik, X. Guo, G. Yothers, N. Song, N. Wilcox-Fogel, E. Forgo, P. S. Rajendran, S. P. Miranda, S. Hisamori, J. Hutchison, T. Kalisky, D. Qian, N. Wolmark, G. A. Fisher, M. van de Rijn, and M. F. Clarke. 2016. CDX2 as a prognostic biomarker in Stage II and Stage III colon cancer. New England Journal of Medicine 374(3):211-222.

Denny, J. C., L. Bastarache, M. D. Ritchie, R. J. Carroll, R. Zink, J. D. Mosley, J. R. Field, J. M. Pulley, A. H. Ramirez, E. Bowton, M. A. Basford, D. S. Carrell, P. L. Peissig, A. N. Kho, J. A. Pacheco, L. V. Rasmussen, D. R. Crosslin, P. K. Crane, J. Pathak, S. J. Bielinski, S. A. Pendergrass, H. Xu, L. A. Hindorff, R. Li, T. A. Manolio, C. G. Chute, R. L. Chisholm, E. B. Larson, G. P. Jarvik, M. H. Brilliant, C. A. McCarty, I. J. Kullo, J. L. Haines, D. C. Crawford, D. R. Masys, and D. M. Roden. 2013. Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data. Nature Biotechnology 31(12):1102-1110.

Deverka, P. A., and S. B. Haga. 2015. Comparative effectiveness research and demonstrating clinical utility for molecular diagnostic tests. Clinical Chemistry 61(1):142-144.

Dickson, D. J. 2015. The Molecular Evidence Development Consortium (MED-C): A shared method to advance personalized medicine. https://www.med-c.org/the-molecular-evidence-development-consortium-med-c-a-shared-method-to-advance-personalized-medicine (accessed July 12, 2015).

Efstathiou, J. A., D. S. Nassif, T. R. McNutt, C. B. Bogardus, W. Bosch, J. Carlin, R. C. Chen, H. Chou, D. Eggert, B. A. Fraass, J. Goldwein, K. E. Hoffman, K. Hotz, M. Hunt, M. Kessler, C. A. Lawton, C. Mayo, J. M. Michalski, S. Mutic, L. Potters, C. M. Rose, H. M. Sandler, G. Sharp, W. Tome, P. T. Tran, T. Wall, A. L. Zietman, P. E. Gabriel, and J. E. Bekelman. 2013. Practice-based evidence to evidence-based practice: Building the National Radiation Oncology Registry. Journal of Oncology Practice 9(3):e90-e95.

Etheredge, L. M. 2014. Rapid learning: A breakthrough agenda. Health Affairs 33(7):1155-1162.

FDA (Food and Drug Administration). 2015a. FDA’s Sentinel Initiative. http://www.fda.gov/Safety/FDAsSentinelInitiative/ucm2007250.htm (accessed September 15, 2015).

FDA. 2015b. Optimizing FDA’s regulatory oversight of next generation sequencing diagnostic tests—preliminary discussion paper. http://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM427869.pdf (accessed April 10, 2015).

French, D., and T. Kampfrath. 2014. Genetic data in health care: From data to information. Clinical Chemistry 61(1):307.

Fridsma, D. 2013. EHR interoperability: The structured data capture initiative. http://www.healthit.gov/buzz-blog/electronic-health-and-medical-records/ehr-interoperability-structured-data-capture-initiative (accessed October 20, 2015).

Friedberg, M. W., P. G. Chen, K. R. Van Busum, F. Aunon, C. Pham, J. Caloyeras, S. Mattke, E. Pitchforth, D. D. Quigley, R. H. Brook, F. J. Crosson, and M. Tutty. 2013. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Santa Monica, CA: RAND Corporation.

Friedman, C., J. Rubin, J. Brown, M. Buntin, M. Corn, L. Etheredge, C. Gunter, M. Musen, R. Platt, W. Stead, K. Sullivan, and D. Van Houweling. 2015. Toward a science of learning systems: A research agenda for the high-functioning learning health system. Journal of the American Medical Informatics Association 22(1):43-50.

Frueh, F. W., S. Amur, P. Mummaneni, R. S. Epstein, R. E. Aubert, T. M. DeLuca, R. R. Verbrugge, G. J. Burckart, and L. J. Lesko. 2008. Pharmacogenomic biomarker information in drug labels approved by the United States Food and Drug Administration: Prevalence of related drug use. Pharmacotherapy 28(8):992-998.

GA4GH (Global Alliance for Genomics and Health). 2016. Working groups. https://genomicsandhealth.org/working-groups (accessed January 20, 2016).

Garraway, L. A. 2015. From genomics to cancer precision medicine. Presentation to the Committee on Policy Issues in the Clinical Development and Use of Biomarkers for Molecularly Targeted Therapies, January 29, 2015, Washington, DC.

Ginsberg, G. 2014. Medical genomics: Gather and use genetic data in health care. Nature 508(7497):451-453.

Goddard, K. A., W. A. Knaus, E. Whitlock, G. H. Lyman, H. S. Feigelson, S. D. Schully, S. Ramsey, S. Tunis, A. N. Freedman, M. J. Khoury, and D. L. Veenstra. 2012. Building the evidence base for decision making in cancer genomic medicine using comparative effectiveness research. Genetics in Medicine 14(7):633-642.

Green, E. D., M. S. Guyer, and National Human Genome Research Institute. 2011. Charting a course for genomic medicine: from base pairs to bedside. Nature 470(7333):204-213.

Green, R. C., D. Lautenbach, and A. L. McGuire. 2015. GINA, genetic discrimination, and genomic medicine. New England Journal of Medicine 372(5):397-399.

Gymrek, M., A. L. McGuire, D. Golan, E. Halperin, and Y. Erlich. 2013. Identifying personal genomes by surname inference. Science 339(6117):321-324.

Health Affairs. 2015. Health policy briefs: The FDA’s Sentinel Initiative. http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief_id=139 (accessed June 15, 2015).

Healthwise. 2015. Patient resources: What you need to know about shared decision making. http://www.informedmedicaldecisions.org/patient-resources (accessed October 21, 2015).

Hellman, D. 2003. What makes genetic discrimination exceptional? American Journal of Law & Medicine 29(1):77-116.

Homer, N., S. Szelinger, M. Redman, D. Duggan, W. Tembe, J. Muehling, J. V. Pearson, D. A. Stephan, S. F. Nelson, and D. W. Craig. 2008. Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays. PLoS Genetics 4(8):e1000167.

Hripcsak, G., D. Vawdrey. 2013. Clinical Documentation Hearing Panel 1. Innovations in clinical documentation. https://www.healthit.gov/archive/archive_files/FACA%20Hearings/2013/2013-02-13%20Policy%3A%20Meaningful%20Use%20and%20Certification%20%26%20Adoption%20WGs,%20Clinical%20Documentation%20Hearing/muclinicaldocinnovation2013_0.pdf (accessed July 23, 2015).

Hudis, C. 2015. CancerLinQ: Role of biomarkers. Presentation to the Committee on Policy Issues in the Clinical Development and Use of Biomarkers for Molecularly Targeted Therapies, June 11, 2015, Washington, DC.

Hudson, K. L., and F. S. Collins. 2015. Bringing the Common Rule into the 21st century. New England Journal of Medicine 373(24):2293-2296.

IOM (Institute of Medicine). 2009a. Beyond the HIPAA privacy rule: Enhancing privacy, improving health through research. Washington, DC: The National Academies Press.

IOM. 2009b. Initial national priorities for comparative effectiveness research. Washington, DC: The National Academies Press.

IOM. 2012. Integrating large-scale genomic information into clinical practice: Workshop summary. Washington, DC: The National Academies Press.

IOM. 2013a. Best care at lower cost: The path to continuous learning health care in America. Washington, DC: The National Academies Press.

IOM. 2013b. Delivering high-quality cancer care: Charting a new course for a system in crisis. Washington, DC: The National Academies Press.

IOM. 2015a. Genomics-enabled learning health care systems: Gathering and using genomic information to improve patient care and research: Workshop summary. Washington, DC: The National Academies Press.

IOM. 2015b. Policy issues in the development and adoption of biomarkers for molecularly targeted cancer therapies: Workshop summary. Washington, DC: The National Academies Press.

IOM. 2015c. Sharing clinical trial data: Maximizing benefits, minimizing risk. Washington, DC: The National Academies Press.

Issa, A. M., W. Tufail, J. Hutchinson, J. Tenorio, and M. P. Baliga. 2009. Assessing patient readiness for the clinical adoption of personalized medicine. Public Health Genomics 12(3):163-169.

Iyengar, R., R. B. Altman, O. Troyanskya, and G. A. FitzGerald. 2015. Medicine. Personalization in practice. Science 350(6258):282-283.

Jarvik, G. P., L. M. Amendola, J. S. Berg, K. Brothers, E. W. Clayton, W. Chung, B. J. Evans, J. P. Evans, S. M. Fullerton, C. J. Gallego, N. A. Garrison, S. W. Gray, I. A. Holm, I. J. Kullo, L. S. Lehmann, C. McCarty, C. A. Prows, H. L. Rehm, R. R. Sharp, J. Salama, S. Sanderson, S. L. Van Driest, M. S. Williams, S. M. Wolf, W. A. Wolf, eMERGE Act-ROR Committee and CERC Committee, CSER Act-ROR Working Group, and W. Burke. 2014. Return of genomic results to research participants: The floor, the ceiling, and the choices in between. American Journal of Human Genetics 94(6):818-826.

Jaspers, M. W., M. Smeulers, H. Vermeulen, and L. W. Peute. 2011. Effects of clinical decision-support systems on practitioner performance and patient outcomes: A synthesis of high-quality systematic review findings. Journal of the American Medical Informatics Association 18(3):327-334.

Jensen, P. B., L. J. Jensen, and S. Brunak. 2012. Mining electronic health records: Towards better research applications and clinical care. Nature Reviews Genetics 13(6):395-405.

Johnston, J. J., W. S. Rubinstein, F. M. Facio, D. Ng, L. N. Singh, J. K. Teer, J. C. Mullikin, and L. G. Biesecker. 2012. Secondary variants in individuals undergoing exome sequencing: Screening of 572 individuals identifies high-penetrance mutations in cancer-susceptibility genes. American Journal of Human Genetics 91(1):97-108.

Kass-Hout, T. A., and D. Litwack. 2015. Advancing precision medicine by enabling a collaborative informatics community. http://blogs.fda.gov/fdavoice/index.php/2015/08/advancing-precision-medicine-by-enabling-a-collaborative-informatics-community (accessed September 20, 2015).

Kawamoto, K., C. A. Houlihan, E. A. Balas, and D. F. Lobach. 2005. Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success. BMJ 330(7494):765.

Kohane, I. S., D. R. Masys, and R. B. Altman. 2006. The incidentalome: A threat to genomic medicine. Journal of the American Medical Association 296(2):212-215.

Krumholz, H. M. 2014. Big data and new knowledge in medicine: The thinking, training, and tools needed for a learning health system. Health Affairs 33(7):1163-1170.

Kulikowski, C. A., E. H. Shortliffe, L. M. Currie, P. L. Elkin, L. E. Hunter, T. R. Johnson, I. J. Kalet, L. A. Lenert, M. A. Musen, J. G. Ozbolt, J. W. Smith, P. Z. Tarczy-Hornoch, and J. J. Williamson. 2012. AMIA board white paper: Definition of biomedical informatics and specification of core competencies for graduate education in the discipline. Journal of the American Medical Informatics Association 19(6):931-938.

Kullo, I. J., G. P. Jarvik, T. A. Manolio, M. S. Williams, and D. M. Roden. 2013. Leveraging the electronic health record to implement genomic medicine. Genetics in Medicine 15(4):270-271.

Lawler, M., L. L. Siu, H. L. Rehm, S. J. Chanock, G. Alterovitz, J. Burn, F. Calvo, D. Lacombe, B. T. Teh, K. N. North, C. L. Sawyers, Clinical Working Group of the Global Alliance for Genomics and Health. 2015. All the world’s a stage: Facilitating discovery science and improved cancer care through the Global Alliance for Genomics and Health. Cancer Discovery 5(11):1133-1136.

Leukemia & Lymphoma Society. 2015. Free information booklets. https://www.lls.org/resource-center/download-or-order-free-publications (accessed October 21, 2015).

Longo, D. L., and J. M. Drazen. 2016. Data sharing. New England Journal of Medicine 374(3): 276-277.

Lyman, G. H., and M. Levine. 2012. Comparative effectiveness research in oncology: An overview. Journal of Clinical Oncology 30(34):4181-4184.

Manolio, T. A., R. L. Chisholm, B. Ozenberger, D. M. Roden, M. S. Williams, R. Wilson, D. Bick, E. P. Bottinger, M. H. Brilliant, C. Eng, K. A. Frazer, B. Korf, D. H. Ledbetter, J. R. Lupski, C. Marsh, D. Mrazek, M. F. Murray, P. H. O’Donnell, D. J. Rader, M. V. Relling, A. R. Shuldiner, D. Valle, R. Weinshilboum, E. D. Green, and G. S. Ginsburg. 2013. Implementing genomic medicine in the clinic: The future is here. Genetics in Medicine 15(4):258-267.

Masys, D. R. 2002. Effects of current and future information technologies on the health care workforce. Health Affairs 21(5):33-41.

Masys, D. R., G. P. Jarvik, N. F. Abernethy, N. R. Anderson, G. J. Papanicolaou, D. N. Paltoo, M. A. Hoffman, I. S. Kohane, and H. P. Levy. 2012. Technical desiderata for the integration of genomic data into electronic health records. Journal of Biomedical Informatics 45(3):419-422.

Meric-Bernstam, F., A. Johnson, V. Holla, A. M. Bailey, L. Brusco, K. Chen, M. Routbort, K. P. Patel, J. Zeng, S. Kopetz, M. A. Davies, S. A. Piha-Paul, D. S. Hong, A. K. Eterovic, A. M. Tsimberidou, R. Broaddus, E. V. Bernstam, K. R. Shaw, J. Mendelsohn, and G. B. Mills. 2015. A decision support framework for genomically informed investigational cancer therapy. Journal of the National Cancer Institute 107(7):djv098.

Meystre, S. M., G. K. Savova, K. C. Kipper-Schuler, and J. F. Hurdle. 2008. Extracting information from textual documents in the electronic health record: A review of recent research. Yearbook of Medical Informatics 35:128-144.

Millenson, M., and R. Berenson. 2015. The road to making patient-centered care real: Policy vehicles and potholes. Washington, DC: The Urban Institute.

Miriovsky, B. J., L. N. Shulman, and A. P. Abernethy. 2012. Importance of health information technology, electronic health records, and continuously aggregating data to comparative effectiveness research and learning health care. Journal of Clinical Oncology 30(34):4243-4248.

NASEM (The National Academies of Sciences, Engineering, and Medicine). 2015. Improving diagnosis in health care. Washington, DC: The National Academies Press.

NCI (National Cancer Institute). 2015a. NCI cancer genomics cloud pilots. https://cbiit.nci.nih.gov/ncip/nci-cancer-genomics-cloud-pilots/docs/cancer-genomics-cloud-pilots-one-pager (accessed September 5, 2015).

NCI. 2015b. Patient education publications. http://www.cancer.gov/publications/patient-education (accessed October 21, 2015).

NHGRI (National Human Genome Research Institute). 2015. Request for information: Maximizing impact of a potential future program in clinical sequencing. http://grants.nih.gov/grants/guide/notice-files/NOT-HG-16-001.html (accessed November 10, 2015).

NHLBI (National Heart, Lung, and Blood Institute) Working Group, R. R. Fabsitz, A. McGuire, R. R. Sharp, M. Puggal, L. M. Beskow, L. G. Biesecker, E. Bookman, W. Burke, E. G. Burchard, G. Church, E. W. Clayton, J. H. Eckfeldt, C. V. Fernandez, R. Fisher, S. M. Fullerton, S. Gabriel, F. Gachupin, C. James, G. P. Jarvik, R. Kittles, J. R. Leib, C. O’Donnell, P. P. O’Rourke, L. L. Rodriguez, S. D. Schully, A. R. Shuldiner, R. K. Sze, J. V. Thakuria, S. M. Wolf, and G. L. Burke. 2010. Ethical and practical guidelines for reporting genetic research results to study participants: Updated guidelines from a National Heart, Lung, and Blood Institute Working Group. Circulation: Cardiovascular Genetics 3(6):574-580.

NIH (National Institutes of Health). 2013. Common data element (CDE) resource portal. https://www.nlm.nih.gov/cde/glossary.html#cdedefinition (accessed October 20, 2015).

NLM (National Library of Medicine). 2015. Genetics home reference: Your guide to understanding genetic conditions. http://ghr.nlm.nih.gov (accessed October 30, 2015).

NRC (National Research Council). 2011. Toward precision medicine: Building a knowledge network for biomedical research and a new taxonomy of disease. Washington, DC: The National Academies Press.

OHDSI (Observational Health Data Sciences and Informatics program). 2016. OMOP common data model. http://www.ohdsi.org/data-standardization/the-common-data-model (accessed January 21, 2016).

ONC (Office of the National Coordinator for Health Information Technology). 2014. Data analytics update. https://www.healthit.gov/FACAS/sites/faca/files/HITPC_DataAnalyticsUpdate_011414.pdf (accessed August 2, 2015).

ONC. 2015a. Precision Medicine Task Force: Preliminary recommendations to Health IT Standards Committee. www.healthit.gov/facas/sites/faca/files/HITSC_PMTF_2015-09-22_final.pptx (accessed November 5, 2015).

ONC. 2015b. Report to Congress on health information blocking. Washington, DC: Department of Health and Human Services.

OPS (Office of the Press Secretary). 2015. Fact sheet: President Obama’s Precision Medicine Initiative. https://www.whitehouse.gov/the-press-office/2015/01/30/fact-sheet-president-obama-s-precision-medicine-initiative (accessed June 3, 2015).

Pelak, K., K. V. Shianna, D. Ge, J. M. Maia, M. Zhu, J. P. Smith, E. T. Cirulli, J. Fellay, S. P. Dickson, C. E. Gumbs, E. L. Heinzen, A. C. Need, E. K. Ruzzo, A. Singh, C. R. Campbell, L. K. Hong, K. A. Lornsen, A. M. McKenzie, N. L. Sobreira, J. E. Hoover-Fong, J. D. Milner, R. Ottman, B. F. Haynes, J. J. Goedert, and D. B. Goldstein. 2010. The characterization of twenty sequenced human genomes. PLoS Genetics 6(9):e1001111.

PMI (Precision Medicine Initiative) Working Group. 2015. The Precision Medicine Initiative cohort program—building a research foundation for 21st century medicine: Report to the Advisory Committee to the Director, NIH. Bethesda, MD: NIH.

Presidential Commission for the Study of Bioethical Issues. 2012. Privacy and progress in whole genome sequencing. Washington, DC: Department of Health and Human Services.

Pulley, J. M., J. C. Denny, J. F. Peterson, G. R. Bernard, C. L. Vnencak-Jones, A. H. Ramirez, J. T. Delaney, E. Bowton, K. Brothers, K. Johnson, D. C. Crawford, J. Schildcrout, D. R. Masys, H. H. Dilks, R. A. Wilke, E. W. Clayton, E. Shultz, M. Laposata, J. McPherson, J. N. Jirjis, and D. M. Roden. 2012. Operational implementation of prospective genotyping for personalized medicine: The design of the Vanderbilt predict project. Clinical Pharmacology & Therapeutics 92(1):87-95.

Ramos, E. M., C. Din-Lovinescu, J. S. Berg, L. D. Brooks, A. Duncanson, M. Dunn, P. Good, T. J. Hubbard, G. P. Jarvik, C. O’Donnell, S. T. Sherry, N. Aronson, L. G. Biesecker, B. Blumberg, N. Calonge, H. M. Colhoun, R. S. Epstein, P. Flicek, E. S. Gordon, E. D. Green, R. C. Green, M. Hurles, K. Kawamoto, W. Knaus, D. H. Ledbetter, H. P. Levy, E. Lyon, D. Maglott, H. L. McLeod, N. Rahman, G. Randhawa, C. Wicklund, T. A. Manolio, R. L. Chisholm, and M. S. Williams. 2014. Characterizing genetic variants for clinical action. American Journal of Medical Genetics. Part C, Seminars in Medical Genetics 166C(1):93-104.

Rasmussen-Torvik, L. J., S. C. Stallings, A. S. Gordon, B. Almoguera, M. A. Basford, S. J. Bielinski, A. Brautbar, M. H. Brilliant, D. S. Carrell, J. J. Connolly, D. R. Crosslin, K. F. Doheny, C. J. Gallego, O. Gottesman, D. S. Kim, K. A. Leppig, R. Li, S. Lin, S. Manzi, A. R. Mejia, J. A. Pacheco, V. Pan, J. Pathak, C. L. Perry, J. F. Peterson, C. A. Prows, J. Ralston, L. V. Rasmussen, M. D. Ritchie, S. Sadhasivam, S. A. Scott, M. Smith, A. Vega, A. A. Vinks, S. Volpi, W. A. Wolf, E. Bottinger, R. L. Chisholm, C. G. Chute, J. L. Haines, J. B. Harley, B. Keating, I. A. Holm, I. J. Kullo, G. P. Jarvik, E. B. Larson, T. Manolio, C. A. McCarty, D. A. Nickerson, S. E. Scherer, M. S. Williams, D. M. Roden, and J. C. Denny. 2014. Design and anticipated outcomes of the eMERGE-PGx project: A multicenter pilot for preemptive pharmacogenomics in electronic health record systems. Clinical Pharmacology & Therapeutics 96(4):482-489.

Rea, S., J. Pathak, G. Savova, T. A. Oniki, L. Westberg, C. E. Beebe, C. Tao, C. G. Parker, P. J. Haug, S. M. Huff, and C. G. Chute. 2012. Building a robust, scalable and standards-driven infrastructure for secondary use of EHR data: The SHARPn project. Journal of Biomedical Informatics 45(4):763-771.

Rehm, H. L., J. S. Berg, L. D. Brooks, C. D. Bustamante, J. P. Evans, M. J. Landrum, D. H. Ledbetter, D. R. Maglott, C. L. Martin, R. L. Nussbaum, S. E. Plon, E. M. Ramos, S. T. Sherry, M. S. Watson, and ClinGen. 2015. ClinGen—the clinical genome resource. New England Journal of Medicine 372(23):2235-2242.

Reilly, P. 1980. When should an investigator share raw data with the subjects? IRB: Ethics & Human Research 2(9):4-5.

Ritchie, M. D., J. C. Denny, D. C. Crawford, A. H. Ramirez, J. B. Weiner, J. M. Pulley, M. A. Basford, K. Brown-Gentry, J. R. Balser, D. R. Masys, J. L. Haines, and D. M. Roden. 2010. Robust replication of genotype-phenotype associations across multiple diseases in an electronic medical record. American Journal of Human Genetics 86(4):560-572.

Riveros, C., A. Dechartres, E. Perrodeau, R. Haneef, I. Boutron, and P. Ravaud. 2013. Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Medicine 10(12):e1001566; discussion e1001566.

Robson, M. E., C. D. Storm, J. Weitzel, D. S. Wollins, K. Offit, and ASCO. 2010. American Society of Clinical Oncology policy statement update: Genetic and genomic testing for cancer susceptibility. Journal of Clinical Oncology 28(5):893-901.

Rosenbloom, S. T., J. C. Denny, H. Xu, N. Lorenzi, W. W. Stead, and K. B. Johnson. 2011. Data from clinical notes: A perspective on the tension between structure and flexible documentation. Journal of the American Medical Informatics Association 18(2):181-186.

Schilsky, R. L. 2014. Implementing personalized cancer care. Nature Reviews Clinical Oncology 11(7):432-438.

Schilsky, R. L. 2015. Moving from evaluation to value in cancer care. Clinical Cancer Research 21(5):947-949.

Sethi, P., and K. Theodos. 2009. Translational bioinformatics and healthcare informatics: Computational and ethical challenges. Perspectives In Health Information Management 6(Fall):1h.

Solit, D. B. 2014. Actionable Genome Consortium: Defining the actionable genome, setting technical standards. Presentation at the National Cancer Policy Forum, November 10, 2014, Washington, DC.

Stead, W. 2015. The future of diagnostics for clinical care—a challenge of integration. Presentation to the Committee on Policy Issues in the Clinical Development and Use of Biomarkers for Molecularly Targeted Therapies, April 1, 2015, Washington, DC.

Stead, W. W., J. R. Searle, H. E. Fessler, J. W. Smith, and E. H. Shortliffe. 2011. Biomedical informatics: Changing what physicians need to know and how they learn. Academic Medicine 86(4):429-434.

Stein, L. D., B. M. Knoppers, P. Campbell, G. Getz, and J. O. Korbel. 2015. Data analysis: Create a cloud commons. Nature 523(7559):149-151.

Taichman, D. B., J. Backus, C. Baethge, H. Bauchner, P. W. de Leeuw, J. M. Drazen, J. Fletcher, F. A. Frizelle, T. Groves, A. Haileamlak, A. James, C. Laine, L. Peiperl, A. Pinborg, P. Sahni, and S. Wu. 2016. Sharing clinical trial data—a proposal from the International Committee of Medical Journal Editors. New England Journal of Medicine 374(4):384-386.

Tanden, N., Z. Emanuel, T. Spiro, E. O. Lee, and T. Huelskoetter. 2014. Comparing the effectiveness of health care: Fulfilling the mission of the Patient-Centered Outcomes Research Institute. https://www.americanprogress.org/issues/healthcare/report/2014/01/24/82775/comparing-the-effectiveness-of-health-care (accessed October 10, 2015).

Van Allen, E. M., D. Miao, B. Schilling, S. A. Shukla, C. Blank, L. Zimmer, A. Sucker, U. Hillen, M. H. Foppen, S. M. Goldinger, J. Utikal, J. C. Hassel, B. Weide, K. C. Kaehler, C. Loquai, P. Mohr, R. Gutzmer, R. Dummer, S. Gabriel, C. J. Wu, D. Schadendorf, and L. A. Garraway. 2015. Genomic correlates of response to CTLA-4 blockade in metastatic melanoma. Science 350(6257):207-211.

Varmus, H. 2010. Ten years on—the human genome and medicine. New England Journal of Medicine 362(21):2028-2029.

Vogelstein, B., N. Papadopoulos, V. E. Velculescu, S. Zhou, L. A. Diaz, Jr., and K. W. Kinzler. 2013. Cancer genome landscapes. Science 339(6127):1546-1558.

Warner, J. L., S. E. Maddux, K. S. Hughes, J. C. Krauss, P. P. Yu, L. N. Shulman, D. K. Mayer, M. Hogarth, M. Shafarman, A. Stover Fiscalini, L. Esserman, L. Alschuler, G. A. Koromia, Z. Gonzaga, and E. P. Ambinder. 2015. Development, implementation, and initial evaluation of a foundational open interoperability standard for oncology treatment planning and summarization. Journal of the American Medical Informatics Association 22(3):577-586.

Welch, B. M., and K. Kawamoto. 2013. The need for clinical decision support integrated with the electronic health record for the clinical application of whole genome sequencing information. Journal of Personalized Medicine 3(4):306-325.

The White House. 2015. Precision Medicine Initiative: Proposed privacy and trust principles. https://www.whitehouse.gov/sites/default/files/docs/pmi_privacy_and_trust_principles_july_2015.pdf (accessed January 12, 2016).

Wilke, R. A., H. Xu, J. C. Denny, D. M. Roden, R. M. Krauss, C. A. McCarty, R. L. Davis, T. Skaar, J. Lamba, and G. Savova. 2011. The emerging role of electronic medical records in pharmacogenomics. Clinical Pharmacology & Therapeutics 89(3):379-386.

Williams, M. S. 2015. Clinical practice issues. Presentation to the Committee on Policy Issues in the Clinical Development and Use of Biomarkers for Molecularly Targeted Therapies, April 1, 2015, Washington, DC.

Wolf, S. M. 2012. The past, present, and future of the debate over return of research results and incidental findings. Genetics in Medicine 14(4):355-357.

Wolf, S. M., F. P. Lawrenz, C. A. Nelson, J. P. Kahn, M. K. Cho, E. W. Clayton, J. G. Fletcher, M. K. Georgieff, D. Hammerschmidt, K. Hudson, J. Illes, V. Kapur, M. A. Keane, B. A. Koenig, B. S. Leroy, E. G. McFarland, J. Paradise, L. S. Parker, S. F. Terry, B. Van Ness, and B. S. Wilfond. 2008. Managing incidental findings in human subjects research: Analysis and recommendations. The Journal of Law, Medicine & Ethics 36(2):219-248, 211.

Yu, P. P. 2015. Knowledge bases, clinical decision support systems, and rapid learning in oncology. Journal of Oncology Practice 11(2):e206-e211.