4

Generating Evidence During Implementation

The evidence base relied upon for deciding which genomic applications are implemented into clinical care varies in depth and lacks standards by which genetic associations are interpreted. However, it may be possible to collect new evidence as a genomics application is being implemented. Evidence collected may prove useful for assessing health outcomes, care provided, and the results of implementation. This chapter offers three case studies involving cancer, diabetes, and prenatal screening in order to explore how evidence can be collected concurrently with implementation.

PROGRAMS FOR ENROLLING PATIENTS AND TRACKING CANCER TREATMENT

Cancer treatment can be both compelling and frustrating for clinicians because of the high number of new drugs currently being approved, said Edward Kim, the Donald S. Kim Distinguished Chair for Cancer Research at the Levine Cancer Institute in the Carolinas HealthCare System. Specialists may be able to keep track of current drug approvals, but for generalists the volume of literature can be overwhelming, Kim said. Nonetheless, generalists provide much of the cancer treatment in the United States, he said. In the Carolinas HealthCare System more than 80 percent of patients are treated in community-based clinics, and most oncologists are generalists, treating patients with both hematological and solid-tumor malignancies. Generalists have a broad base of knowledge but may not necessarily be equipped to thoroughly research all of the options when multiple genomic markers and drugs are available. Oncology specialists typically have comprehensive knowledge of certain types of cancer but not all. “I spent 12 years studying lung cancer, so I can speak to pretty much any topic in non-small-cell and small-cell lung cancer,” Kim said. However, if a patient asked for a recommendation on how to treat breast cancer, for example, I would need to refer them to a breast oncologist, he said. In addition to the large body of knowledge that they must keep current with, clinicians are busy, and this means limited time for patients who want to see their physicians and nurses. Kim often receives requests from clinicians within the system for more staff, but certain constraints do not allow for the addition of extra staff.

Clinical researchers face several major challenges related to evidence gathering, one of which is the low patient accrual rates for clinical trials. Estimates suggest that less than 5 percent of adult cancer patients take

part in clinical trials in the United States (Sahoo et al., 2014). Although there are exceptions every now and then, enrollment levels across the board are dismal, Kim said. As cancer clinical trials become more complex, the number of eligibility criteria increases, and thus the study enrollment rates slow down (Kim et al., 2015). Attempts to streamline eligibility criteria have led to promising proposals, but many problems remain. The American Society of Clinical Oncology (ASCO) Cancer Research Committee recently recommended that stakeholders work on developing an algorithmic approach to streamlining eligibility criteria. In addition to patient accrual challenges, clinical researchers must also plan for sample collection, regulatory approval, and communication with providers, Kim noted.

To address the challenge of low enrollment rates and to foster awareness of clinical trials, the Carolinas HealthCare System has launched a proprietary system called EAPathways. EAPathways focuses on expanding access to clinical research and new therapies. Activated in May 2015, the program alerts practitioners to clinical trials that are actively enrolling patients. The program frees up time for study coordinators so that they can focus on specific trial- and patient-related issues such as insurance and follow-up. EAPathways tracks data on trial inquiries and pathway enrollments and conducts close-to-real-time modifications of trials and pathways. Every time a patient is enrolled into a pathway, the system captures the patient’s name, medical record number, and birth date. Different icons in the system represent open clinical trials, pending clinical trials, the need to collect specimens or conduct a genomics test, and clinician reminders (such as for a smoking cessation program). Generalists are not going to know about every open clinical trial, so EAPathways provides them with the information rapidly, Kim said.

EAPathways is designed to not disrupt a physician’s workflow and is very quick, Kim said. Once a patient is enrolled in a clinical trial pathway, the physician receives all the required documents, including the informed consent, study sheets, calendar, and order sheets. The system is compatible with iPads and mobile phones, and inquiries about specific trials are sent directly to researchers. Furthermore, the program can be operated by support staff and does not require information specialists. Even if some physicians are not enthusiastic about using the system, their staff members have embraced it, Kim said. The system eases the workload of physicians by presenting them with information rather than forcing them to search for that material, he said. One additional feature of

EAPathways is a link to a biospecimen repository, which generates an alert to collect samples from patients once they are enrolled in a clinical trial. Overall, Kim described EAPathways as having a patient-centered approach because it minimizes travel inconveniences and promotes consistency of diagnosis, treatment, and follow-up.

The implementation of the EAPathways system has worked better for physicians employed by Carolinas HealthCare System than for physicians who are just affiliated with the system, Kim said. The Carolinas HealthCare System tries to be very aware of the latest clinical trials and enrollment in clinical trials, he said, and in return, physicians are expected to attend multidisciplinary tumor boards and participate in a monthly disease-specific section.

The Levine Cancer Institute is participating in the Targeted Agent and Profiling Utilization Registry (TAPUR) study with ASCO, the Cancer Research Consortium of West Michigan, the Michigan Cancer Research Consortium, and the University of Michigan.¹ The primary objectives of the TAPUR study are to describe the anti-tumor activity and toxicity of commercially available, targeted anti-cancer drugs and to facilitate access to those drugs for patients with advanced solid tumors, B cell, non-Hodgkin lymphoma, or malignant melanomas with a known genomic variant. Many stakeholders could potentially benefit from the design of the TAPUR study. Patients will receive access to a therapeutic agent that is targeted to the genomic profile of their tumor, physicians will receive assistance interpreting complex genomic tests, and pharmaceutical companies, payers, and regulators will have access to data regarding off-label drug use and clinical outcomes (Schilsky, 2014). The TAPUR study will initially take place at 30 clinical sites in 4 states; however, the goal is to expand the program nationally. Studies such as TAPUR are designed to fit in community-based systems, provide patients with access to drugs, minimize travel for patients, reduce the need for generalists to speculate or hypothesize, and empower physicians with knowledge and easy access to clinical trials. When clinicians can access information on clinical trials more easily, they are able to reach more diverse populations, Kim said.

One way of determining whether a cancer drug is going to be implemented rapidly is by the number of people who pursue expanded access after a drug has been approved for one indication, Perlmutter said. She is leading a patient advocacy group associated with the TAPUR trial,

___________________

¹For more information on the TAPUR study, see http://www.tapur.org (accessed February 23, 2016).

which is designed to generate evidence on potential new cancer indications for drugs that have already been approved in one indication. Perlmutter expressed a concern that not enough work has been done to figure out how to use the evidence generated in the TAPUR trial. Currently no regulatory pathway exists to use the results of TAPUR in drug approval, she said. “If we just use the same old trials, we’re not going to get [drugs] to patients fast enough, and the patients will want to keep getting them through registries as opposed to through approval, and that is going to defeat the purpose.”

ENHANCING GENOMIC IMPLEMENTATION THROUGH A COLLABORATIVE RESEARCH NETWORK

Although this quotation comes from an analysis of innovation in service industries, it might as well have been written about genomics, said Stephen Kimmel, a professor of medicine and epidemiology at the University of Pennsylvania’s Perelman School of Medicine. Changing physician and health system workflows, the use of decision support for relaying new complex diagnostic information, the influence of reimbursement, and the development of an evidence base are challenges faced in many health care implementation scenarios. Other areas—such as the return of results to patients and family members and the interpretation of data—are more specific to the field of genomics, observed Kimmel.

Implementation strategies are designed to improve the uptake and sustainability of clinical interventions. Implementation efforts should manage the contingencies of various service systems or sectors, including the challenge of staff training and support, Kimmel said. Rigorously evaluating implementation strategies at early-, mid-, and late-stage endpoints is also important, he added. Failing to understand the barriers and facilitators to a specific intervention can lead to what Kimmel described as a “type III error”—attributing poor outcomes to a failed or ineffective intervention when they were actually a result of poor implementation.

The Implementing Genomics in Practice (IGNITE) Network,² is a program funded by the National Human Genome Research Institute (NHGRI), and Kimmel is the principal investigator of the program’s coordinating center. The goal of the IGNITE Network is “to enhance and accelerate the use of genomic medicine by incorporating genomic information into clinical care,” Kimmel said. The network includes six genomic medicine demonstration projects and a central coordinating center. The subjects of the demonstration projects range from detailed family histories to pharmacogenetics to monogenic diabetes. This is a very forward-thinking and creative approach by NHGRI to study implementation while expanding the knowledge base for genomic medicine, Kimmel said. The network uses the Consolidated Framework for Implementation Research³ to identify the core implementation components that are common across all of the demonstration projects and to determine which of those factors contribute to successful implementation (Damschroder et al., 2009).

The IGNITE Network research plan includes the identification of common outcomes that define success, the creation of a model for implementation, and the operationalization of constructs through a Likerttype questionnaire that can be adapted for each project. Through the questionnaire, clinicians can weigh in on workflow, knowledge, leadership, beliefs and attitudes, training and self-efficacy, value and utility, group efficacy, and strategies. Example inquiries from IGNITE questionnaires include

• Workflow: Staff have enough time to facilitate the integration of [genetic test] into clinical practice
• Knowledge: I can find/use reliable sources of the information I need to apply [genetic test] while caring for patients
• Leadership: Leaders have openly endorsed and supporting [genetic test] in visible ways
• Beliefs/Attitudes: The information generated by [genetic test] is important for patient care
• Training/Self-Efficacy: My training has prepared me to treat patients whose family history/genetics place them at high risk for medical conditions

___________________

²For more information on the IGNITE Network, see http://www.ignite-genomics.org (accessed February 23, 2016).

³For more information on the Consolidated Framework for Implementation Research, see http://www.cfirguide.org (accessed February 26, 2016).

• Value/Utility: [Genetic test] will improve my ability to care for patients
• Group Efficacy: The implementation leaders/team have the necessary qualities and skills to successfully incorporate [genetic test] into my clinical practice
• Strategies: A variety of strategies are being used to enable staff to use [genetic test] to assess patient risk

Information from the questionnaires gets collected across a broad range of practice patterns and communities and may help to serve as a sort of “biomarker of implementation,” Kimmel said. Such a tool could help indicate what the barriers are and where interventions might be necessary with a particular group of clinicians.

Most of the IGNITE projects do not yet have final results, Kimmel reported, but the network has provided a framework for asking the right questions. IGNITE also has shown that many, but not all, aspects of implementation are generic across projects. For example, one of the IGNITE Network projects is systematically collecting information on all of the barriers to implementation identified across all of the sites, with the barriers being rated by the sites according to how specific they are to genomics. About 25 percent of the barriers to implementation that have been identified are purely genomics-specific, approximately 31 percent are generic, and 43 percent have both genomic and generic components, Kimmel said. Among the genomics-specific challenges are the unknown effects of many novel genetic variants, the incorporation of genetic counseling with the return of results, the integration and formatting of genetic test results in electronic health records (EHRs), the timing and utility of EHR alerts, and reimbursement for testing. Referring to that last issue, Kimmel asked, “How do we do this if nobody is paying for it?” Among the generic challenges are issues with general integration with EHRs, institutional priorities, the underestimation of system-level challenges, provider engagement, and practice workflows. Even with these mixed barriers, the generic components can be rigorously studied to derive results useful for implementation in general. In summary, Kimmel said, genomics implementation has some unique characteristics, but many of its features are common to all implementation programs. The challenges “have to be measured carefully and formally assessed and analyzed so we know what works, what doesn’t work, and why.”

Both the successful and failed implementations can provide important lessons, Ginsburg said. Publishing such experiences “would be

highly valuable to the community,” he added. He also pointed out that the IGNITE Network is producing an implementation toolbox that could provide the community with a sense of what needs to be done. Furthermore, the toolbox is not limited to IGNITE investigators but rather will be open to other networks that are doing implementation of genomics. These other networks should be encouraged to join in and be part of that same toolbox so that information can be shared with all, Ginsburg said.

CELL-FREE DNA SCREENING FOR ANEUPLOIDY

Unbiased clinical effectiveness data is very important for implementing new clinical practices, and translation needs to be evidence-based instead of just rapid, said Mary Norton, a professor of obstetrics, gynecology, and reproductive sciences and the David E. Thorburn, M.D., and Kate McKee Thorburn Endowed Chair in Perinatal Medicine and Genetics at the University of California, San Francisco. The adoption of cell-free DNA screening for aneuploidy was both evidence-based and rapid and therefore makes an excellent case study.

The ability to detect cell-free fetal DNA (cffDNA) in the maternal bloodstream led to the development of non-invasive techniques for fetal aneuploidy screening (Allyse et al., 2013). cffDNA screens were developed from 2000 to 2010 and introduced into medical practice in October 2011 at the same time as the first clinical validation study was published. cffDNA screening is designed to detect fetal genetic abnormalities, such as trisomy 21 (Down syndrome), trisomy 18 (Edwards syndrome), and trisomy 13 (Patau syndrome). In carefully preselected populations, cffDNA screening has extremely high sensitivity, high specificity, and high positive and negative predictive values for Down syndrome, Norton said. Screening for trisomy and other chromosomal abnormalities traditionally has been done using two standard noninvasive tests, an ultrasound in the first trimester and a blood test in the second trimester. A high-risk woman or someone who tested positive to the initial screening undergoes one of two invasive diagnostic tests, chorionic villus sampling or amniocentesis. This conventional approach had a relatively low risk of complications, Norton said—roughly 1 in 1,000 in the most recent analysis, which is not different from the background rate of pregnancy loss. One advantage that cffDNA screening offers is that it has about a 99 percent detection rate for Down syndrome, while the most sensitive version of traditional screening has a detection rate of 92 to 93 percent, Norton said.

There are potential concerns about cffDNA screening, one of which is that while it is excellent at detecting trisomy 21, the test is far less effective at detecting other chromosomal abnormalities such as trisomy 18 or 13. Another concern with cffDNA screening, Norton said, is that it is more expensive than traditional methods. This raises the question of whether adequate counseling is being provided so that patients understand the limitations and benefits of the cffDNA screen. Norton and her colleagues published a comprehensive cost–utility analysis in which they recommended conventional screening methods until women were 40 years old and the risk of Down syndrome was elevated, at which point cffDNA screening became optimal and cost-effective (Kaimal et al., 2015).

The uptake of cffDNA screening has been among the most rapid of any new clinical test, Norton said. By 2014 more than 800,000 cffDNA tests were being performed worldwide (Bianchi, 2015). Evidence in support of cffDNA screening has been compelling, Norton said, although she added that the evidence has come largely from industry-sponsored trials that use carefully selected groups of patients. Another potential reason for the rapid uptake of cffDNA screening is that testing for Down syndrome has been widespread for decades and patients and clinicians are well informed about it. Professional societies have supported Down syndrome screening for many years, though many continue to recommend traditional screening rather than cell-free DNA screening, Norton said. The cffDNA screen was not developed to fill a quality gap; instead, it met a need that people were not asking for in a market that did not previously exist, Norton said. Fetal screening is a very competitive and lucrative industry, she added, which means that commercial laboratories have done considerable marketing.

Several major professional societies have published statements on cffDNA screening for fetal aneuploidy, and the general observation, Norton said, is that conventional screening methods are the most appropriate first line test. Any patient may choose cffDNA screening, according to these statements, but patients should be counseled appropriately regarding the limitations and benefits (ACOG, 2015). In addition, further diagnostic testing is required to confirm abnormal results before irreversible decisions are made.

Norton cited several challenges that have arisen during the rapid uptake of cffDNA screening. Many providers have inadequate knowledge of genetics and statistical factors such as the positive predictive value in low-risk patients, she said. In addition, standardized patient education is

lacking, and many patients think that the cffDNA screen is an alternative to invasive, risky testing, which is incorrect, Norton said. The results of the test also tend to be misunderstood. The lower risk the patient is, the less likely a positive result is to be a true positive, Norton said. For example, a study of 109 consecutive cases of women who had abnormal cffDNA screening results followed by diagnostic testing yielded a true positive or positive predictive value of 67 percent, with the performance higher for Down syndrome than for other less common chromosome abnormalities (Wang et al., 2015). Yet laboratory test results are essentially dichotomized to yes or no, which again can lead to misunderstanding. Poor understanding of the test results could lead some women to terminate normal pregnancies without confirmation.

Norton also called attention to the occurrence of incidental findings that can potentially come along with the cffDNA screening. The cffDNA test sequences both fetal and maternal DNA, so maternal health problems such as malignancies can be detected (Bianchi et al., 2015). Systematic data about incidental findings on mothers are now beginning to be collected, Norton said. The consent forms are usually very standard and indicate only that the patient is having a test for Down syndrome, she said. It is only now being recognized how problematic this can be for patients, she said. Professional societies are considering enacting a credentialing process for test providers along with standardized consent. The lesson that we can learn from cffDNA screening is that rapid implementation can lead to issues down the road, Norton said.

Several large integrated health systems and programs like the California Prenatal Screening Program⁴ are working on collecting high-quality evidence about cffDNA screening, Norton said. In addition, the U.S. Food and Drug Administration (FDA) recently called for improving the evidence base for laboratory-developed tests (Office of Public Health Strategy and Analysis, 2015). The FDA report cited cffDNA prenatal testing as one of 20 problematic case studies of laboratory-developed tests that may yield false positive and negative results and, thus, subsequent harm to patients.

cffDNA screening is a “perfect case study for implementation science,” said Alexandra Shields of Harvard Medical School and Massachusetts General Hospital. The challenge is to squeeze out the positive clinical benefits of this test and limit the deleterious potential downsides. Enacting a common reporting format for different laboratories would be

___________________

⁴For more information regarding the California Prenatal Screening Program, see http://www.cdph.ca.gov/programs/PNS/pages/default.aspx (accessed February 23, 2016).

an important advance, Shields said, as would creating universal patient educational information that is mandated by the FDA. Once women understand what additional information is going to be generated by that test, they could communicate what they would like back in the returned results, and this information could be flagged in the EHR. If a patient does not want certain types of information, the EHR could help minimize errors in disclosing information.

PROVIDING GUIDANCE FOR IMPLEMENTATION

Most innovations are not self-implementing, and those that are often are implemented inappropriately, Mittman said. For example, clinical practice guidelines would ideally be issued with accompanying implementation guidance. In an ideal world, the groups developing new practices would take on the responsibility of providing the supporting implementation guidance, the necessary tools, and patient and clinician education materials. This would require convening all of the stakeholders, including regulators, policy and practice leaders, fiscal intermediaries, payers, clinical leaders in the systems, and so on. Companies that produce genomic tests could get together with clinicians to develop standardized formats for delivering test results. “It’s another example of the principle of engagement, partnership, and collaboration,” Mittman said. Requirements from the FDA, the medical community, and professional societies may ensure that new innovations are accompanied by guidance and supporting tools, he said.

This page intentionally left blank.