Improving Genetics Education in Graduate and Continuing Health Professional Education: Workshop Summary (2015)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Educational Approaches Important Points Highlighted by the Individual Speakers • Innovations such as just-in-time and online education can get health care professionals the information they need when they are most likely to be able to use and learn it. • Technological advances can be a powerful adjunct to, but not a replacement for, effective instruction and the acquisition of clinical skills. • Interprofessional education can achieve better communication and collaboration among learners, practitioners, patients, cli- ents, families, and communities. • Avoiding conflicts of interest in educational materials requires a strong commitment by the leadership of the medical community. • Implementation research can promote the integration of re- search findings into routine use in an evidence-based manner. • The motivations of practitioners and the attributes of educa- tional resources are key determinants of whether genetics edu- cation will be embraced. New research findings and evidence-based approaches to education have transformed many learning environments and have improved learn- ing outcomes. Workshop speakers discussed several of these new ap- proaches, including just-in-time approaches to education, the use of social media and other technology platforms in health professional edu- cation, IPE, and the use of evidence and theory in designing educational 13 

14 IMPROVING GENETICS EDUCATION activities, along with a number of ongoing issues in medicine and genet- ics education. JUST-IN-TIME APPROACHES TO EDUCATION Disease-specific genetic knowledge is changing rapidly, with new variants being identified every day, said Benjamin Raby, associate pro- fessor of medicine at Harvard Medical School and the genetics section editor for UpToDate. “We need to get that information and figure out how to integrate it in a rapid way,” he said, so that patients are receiving the best care possible. However, in his role as the director of a pulmo- nology genetics center who often sees patients with extremely rare disor- ders, Raby has witnessed firsthand what he called the “dilemma” of providing genetics education. Late-stage fellows and junior faculty in his training program at Harvard Medical School, who should already have a solid training in genetics, are not able to answer such basic questions as what is a haplotype, a genotype, or an allele, or what genetic tests should be ordered in a given situation, or whether screening should be recom- mended for relatives. “We have to think about the fundamental basic ge- netic principles in our education,” Raby said, and also be concerned with how to convey variant-specific and disease-specific information. Just-in-time approaches to education offer a potential way to deliver an array of information at exactly the point in time at which it is most needed or useful. UpToDate is an online, subscription-based, just-in-time clinical decision support resource, which continuously reviews the medi- cal literature in order to provide clinicians with the information they need to make medical decisions. The UpToDate contributors review approxi- mately 460 medical journals covering more than 10,000 topics in 21 spe- cialties. Additions to the UpToDate resource undergo a three-tiered peer- review process—first by independent reviewers, then by a section editor, and finally by editorial staff. The resource is used by more than 850,000 clinicians from 29,000 institutions and practices in 164 countries, with 90 percent of U.S. academic medical centers having subscriptions. In 2013, it received more than a quarter billion queries, with an average of 484 topic reviews per minute. And its use has been associated with “demon- strable improvements in quality of patient care,” Raby said (Isaac et al., 2012). New knowledge is continually being incorporated into UpToDate, he added. The section editors examine the tables of contents from every 

EDUCATIONAL APPROACHES 15 journal on a monthly basis to identify articles that might be worthy of further review. They also respond to user suggestions for topics that could be included or revised, Raby said. Authors typically update topics once per year, but pressing topics are revised more frequently. In the area of genetics, UpToDate seeks to provide two types of content—disease-specific and foundational. Disease-specific content in- cludes information about specific monogenic and polygenic disorders as well as polygenic contributions to common diseases plus other disease- specific topics. Current content includes such specific topics such as breast and ovarian cancer, early onset Alzheimer’s disease, cardiomyopa- thy, and colorectal cancer, among many others. The types of variation, the genes considered to be relevant, what to tell a patient, and the next line of testing are all discussed within the disease context. Foundational content, by contrast, includes basic information such as a glossary of ge- netic terms, basic principles of genetic disease, an overview of genetic variation, and information on incidental findings. Entries also include hyperlinked references—for example, from specific disease context to related foundational information. The genetics section is unique within UpToDate, Raby said, in that other sections do not have similarly broad treatments of foundational issues. But the need for educational material in genetics is so pressing that the genetics section tries to provide that framework. At the same time, genetics information is included with content related to other spe- cialties because this is seen as an effective way of increasing understand- ing of genetics. The UpToDate resource tries to make information as accessible as possible by having the information directly linkable from electronic med- ical records, Raby said. UpToDate is also currently working on integrat- ing calculators of risk prediction and decision-management solutions into electronic medical records, which could be pre-populated with patient- specific information. Raby concluded by pointing to several remaining deficiencies in the UpToDate resource. One is the lag time in providing updated infor- mation because of the rapid evolution of genetic information. “New knowledge is always being developed around variants that have already been reported out,” Raby said. “What do we do when you get back a re- port that says, ‘This is likely benign,’ and 3 years later it turns out we made a mistake?” Furthermore, simply listing an identified gene variant on a report is of little value to clinicians. There is a need for supported reports that include the evidence for an interpretation and that tell clini- 

16 IMPROVING GENETICS EDUCATION cians specifically how to think about a given result. Such reports provide not simply data but knowledge, Raby said. Finally, the updating and sharing of genetic information remains a challenge. New systems are being developed to link genotypic and phe- notypic information across laboratories without violating privacy con- cerns. “If I see a new variant, and it turns out that it has never been published on before, but four other labs have seen the same rare variant and the same phenotype is linked, we can now make much better infer- ences,” he said. Such information can provide the basis for a continually updated clinical portal, with improved communications between labora- tories and physicians and continuous education. “These are the types of tools that we need to get into place for people to have the knowledge they need to move forward.” TECHNOLOGY IN MEDICAL EDUCATION Emerging technologies and media platforms offer other new and intri- guing ways to educate students and health care providers in genetics. But technology needs to serve patients as well as learners, said Alexander Djuricich, associate professor of clinical pediatrics and clinical medicine at the Indiana University School of Medicine. “I am interested in emerg- ing technology for myself, my learners, and patients for the purpose of improving education and ultimately patient care,” he said. The potential of technology as a learning tool has been widely rec- ognized. Technology can improve engagement with learners, provide educational opportunities outside of traditional lecture room settings, and can foster the wider dissemination of information, Djuricich said. These attributes have led a number of institutions and organizations to adopt technology for use in medical education. Djuricich said that the Universi- ty of California, Irvine, has been giving all of its incoming medical stu- dents iPads, for example. An asynchronous global emerging medicine journal club is now being hosted on a blog by the Annals of Emergency Medicine, with comments moderated on the blog and Twitter (Radecki et al., 2014), and audience response systems are becoming more common, the use of which has been shown by research to improve classroom en- gagement and learning (Forest, 2012). Twitter can be especially effective in conferences, Djuricich said, because it can act as a back channel for communication with the people at the meeting and elsewhere. Tweets can highlight important aspects of 

EDUCATIONAL APPROACHES 17 a meeting and can even be done as part of a formal presentation (Chaudhry et al., 2012; Desai et al., 2012; Logghe et al., 2013; Matta et al., 2014; Neill et al., 2014). For example, Djuricich’s institution has pi- loted tweeting grand rounds for pediatrics. In 17 presentations, 613 sepa- rate tweets were sent from 61 participants. “This provides a great example of the power of disseminating information, not just in the room for that 1 hour, but to disseminate it to others who can either read it then or at a later time.” Technology can be very effective in making information useful and meaningful for clinicians and patients, but its use must be balanced with still demonstrating humanity, Djuricich said. At the same time as the journal Pediatrics opted to move more toward a digital identity by de- veloping a full-text mobile app and becoming active in social media tools, such as Facebook and Twitter, in response to reader preferences for accessing journal content (First et al., 2014), an article was published in Academic Pediatrics warning against the overuse of technology and the effects of such overuse on interactions with patients and learners (Crain, 2014). Similarly, Colbert and Chokshi (2014) and Toll (2012) have cautioned about the potential drawbacks of technology. As the for- mer wrote, “Technology itself is no panacea. . . . Some fear that [learn- ers] may be losing sight of humanism in medicine” (Colbert and Chokshi, 2014). We need to be careful about how we use technology, and we need to provide instruction on how to use it effectively, Djuricich said. Emerging technologies are often treated as a tool by which to teach content. But technology itself can also be the content, as is the case in a workshop on how to use social media or a session on the best way to use audience response systems such as clickers. As an example of this ap- proach, Kind et al. (2014) have published a set of tips on using social media as a medical educator. They include • Identify and reflect on one’s digital identity. • Observe and establish comfort first—“Think, then contribute.” • Use social media to disseminate evidence-based health information. • Apply social media guidelines, such as those developed by aca- demic medical centers and professional groups. • Tap into the power of community, whether students, physicians, medical educators, or patients. 

18 IMPROVING GENETICS EDUCATION Gaglani and Topol (2014) have urged that training programs and medical schools integrate mobile health technologies into their curricula. Djuricich agreed that this is a promising approach and argued that this education should extend to continuing professional development as well, but he also emphasized that while technology can be effective as an ad- junct to instruction, it should not be seen as a replacement for effective instruction. Not all technologies are ready for use, and it can be difficult for clinicians to keep up with their development. Technologies tend to go through a hype cycle of inflated expectations, disillusionment, gradual enlightenment, and, finally, productive use.1 For example, new apps are being created so quickly that no one can keep up with all of them. But apps need to be evaluated, which has led to the development of rubrics and websites, such as imedicalapps.com and happtique.com, to assess the value of apps. Such apps are also subject to U.S. Food and Drug Administration (FDA) regulation if they can affect a user’s health, though so many prod- ucts are currently being developed that FDA is hard pressed to keep up. Technology can augment but not replace the acquisition of clinical skills, Djuricich concluded. In particular, it has the potential to increase engagement and to reach learners wherever they may be. INTERPROFESSIONAL EDUCATION IN GENETICS IPE is an approach to education in which individuals from two or more professions learn about, with, and from each other to improve col- laboration and the quality of health care. “About, with, and from” are the three key concepts, said Diane Seibert, professor and chair and director of the Family Nurse Practitioner Program at the Uniformed Services University of the Health Sciences. The concept of IPE is to learn about what another profession does, with other professionals in the same envi- ronment, and taught from different professions. This can be done, for instance, in a faculty-led or small-group environment where medical and nursing students are trained together, Seibert said. The goal of IPE is to improve collaborations between health care professionals in order to improve patient outcomes. Interprofessional learning may be spontaneous or strategically embedded in an educational program, but the ultimate goal is better communication and collaboration 1 See http://www.gartner.com/newsroom/id/2819918 (accessed January 16, 2015). 

EDUCATIONAL APPROACHES 19 among learners, practitioners, patients, clients, families, and communi- ties. “If people communicate better with one another, they will practice as a team, and you will get better patient outcomes,” Seibert said. “You have a diverse group of people bringing their perspectives to a common place and sharing and collaborating.” IPE is useful because health care systems are so complex that differ- ent perspectives are needed to get the best outcomes, Seibert explained. In addition, IPE fosters mutual respect among disciplines, countering the negative stereotypes about other disciplines that can begin early in pro- fessional education and shape how people view other professions for the rest of their careers. Health educators have been interested in IPE for a long time. The 1972 Institute of Medicine report Educating for the Health Team con- cluded that “a major deterrent to our efforts to fashion health care that is efficient, effective, comprehensive, and personalized is our lack of a de- sign for the synergistic interrelationship of all who can contribute to the patient’s well-being” (IOM, 1972, p. 4). More than 40 years later, IPE is still not where it needs to be, Seibert said. There are still not a lot of opportunities for students to learn in this manner. The Interprofessional Education Collaborative Expert Panel (2011) recently published Core Competencies for Interprofessional Col- laborative Practice, which has been “getting some forward traction,” according to Seibert. For example, in October 2012 the Liaison Commit- tee on Medical Education approved a new accreditation standard,2 which went into effect in July 2013, that requires that “the core curriculum of a medical education program must prepare medical students to function collaboratively on health care teams that include health professionals from other disciplines as they provide coordinated services to patients. These curricular experiences include practitioners and/or students from other health professions.” As a result of this new standard, students will have more opportunities to interact with other disciplines before they begin their professional careers, which in turn could have an effect on continuing medical education. While IPE is beginning to be more broadly embraced, it continues to face challenges, Seibert said. Not much evidence yet exists to show that IPE is more effective than standard approaches in terms of learning out- 2 See http://www.lcme.org/connections/connections_2014-2015/ED-19-A_2014-2015. htm (accessed November 11, 2014). 

20 IMPROVING GENETICS EDUCATION comes or health outcomes. Longitudinal research will be needed to demonstrate such effectiveness, Seibert said. In addition, developing a curriculum for very diverse audiences can be challenging. Faculty who have not had these experiences may not know how to develop such cur- ricula or how to facilitate small interprofessional groups. “Quite a bit of faculty development is needed in this area,” Seibert said. Scheduling poses other difficulties. For example, different profes- sions tend to hold their conferences at different times of the year, making it difficult for members of various disciplines to meet together, Seibert said. “We need to think about how do we break those silos down and get multiple disciplines together in the same room. [Currently] there is no cross talk.” Genetics and genomics are cross-cutting topics that could help break down those barriers and attract members of various health care profes- sions, Seibert said. Already, patients with rare single-gene disorders tend to be treated through team-based approaches. However, many members of the health care community have not had much experience with genet- ics and genomics because of limited educational experiences and the rap- idly advancing state of the field. As a result, while the audience for genetics education is diverse, its members tend to have similar learning needs, which could address some of the challenges of developing an in- terprofessional curriculum. People will be most interested in IPE events that meet their needs, Seibert said. “The topic will attract the audience.” For example, nurse practitioners are very interested in family health histories and risk as- sessment. Many individuals are also interested in genetic testing, espe- cially testing for common complex diseases that affect large numbers of patients. A rare disorder that health care professionals are unlikely to see in their practice may not be particularly attractive. Members of many different professions are also interested in ethical, legal, and social issues or counseling because many professionals have relatively weak skill sets in those areas, Seibert said. Diversity at every level is key in such events, Seibert said, and this includes having diverse audiences, diverse faculty members, diverse panels, and diverse small groups. Advertising such an event requires care, because genetics can scare people away. Not mentioning disciplines will help keep people from excluding themselves. And more complex case-based approaches can attract audiences, Seibert said, while at the same time providing an opportunity to teach participants about less common disorders or topics. 

EDUCATIONAL APPROACHES 21 Techniques to foster engagement, including problem solving, peer instruction, advanced simulations, and built-in interactions, can enhance the success of such events. Another critical element of an interprofes- sional learning opportunity is allowing everyone to have a voice—for example, by including more than one representative of a profession with- in each group so that people are more encouraged to speak and represent multiple viewpoints within a single profession. An emphasis on building professional networks across communities can also increase engagement. In this way, participants get more out of an educational experience than just information, Raby said. “They get to meet people who they will be encountering often, maybe over their practice lifetime.” Raby suggested that individuals could be further engaged by shadowing other profession- als in their clinics. “Hands on is [a] really important means of learning from colleagues,” he said. This could be incentivized within the hospital system, he added. Maren Scheuner, chief of medical genetics in the Vet- erans Affairs Greater Los Angeles Healthcare System, added that focus- ing on a specific condition and finding the venue where “everyone gets together” is a valuable way of facilitating IPE. Attending and regularly participating in tumor boards has been helpful in implementing Lynch syndrome screening in her region, she said. Finally, collecting data during and after IPE events can advance the science and help achieve the goals of IPE, Seibert said. Such research can improve teamwork and facilitate the ultimate goal of improving pa- tient outcomes. IDENTIFYING AND AVOIDING CONFLICTS OF INTEREST IN EDUCATIONAL MATERIALS3 Many physicians have a misplaced confidence that marketing bias is easy to recognize and straightforward to avoid, said Jean Silver-Isenstadt, executive director of the National Physicians Alliance. That belief makes it easier for marketers to embed themselves in medical education, re- search, and practice at many levels. But these marketing efforts to influ- 3 For further information on managing conflicts of interest and facilitating innovation, see: http://www.nap.edu/catalog/18723/conflict-of-interest-and-medical-innovation-ensuring- integrity-while-facilitating (accessed January 16, 2015) and http://www.nap.edu/catalog/ 12598/conflict-of-interest-in-medical-research-education-and-practice (accessed January 16, 2015). 

22 IMPROVING GENETICS EDUCATION ence physicians are “a destructive force,” Silver-Isenstadt said. They can lead to the unnecessary use of tests, drugs, procedures, and devices, driv- ing up costs that could be avoided. They also can direct curiosity and research away from preventive measures and other promising approaches that do not yield profits. The National Physicians Alliance was founded in 2005 to strengthen physicians’ civic engagement on behalf of patients and to build commu- nity within the profession. As part of its efforts to help restore trust and integrity in medicine, it accepts no funding from pharmaceutical or med- ical device companies. “The founders [of the alliance] felt that medicine was losing its way and that what had been a profession was becoming more and more grounded in business interests,” Silver-Isenstadt said. “There was real risk and loss in that.” The founding campaign of the National Physicians Alliance, which continues today, is called the Unbranded Doctor. Silver-Isenstadt de- scribed it as a consciousness-raising effort that includes such elements as support for the Physician Payments Sunshine Act, which has created a publicly searchable database of gifts and payments from industry to pro- viders,4 opposition to the sale of individual prescriber data to drug com- panies for marketing purposes, and encouragement for clinicians to avoid free lunches, samples, visits from drug company representatives, and so on. Another core effort of the alliance is the Promoting Good Steward- ship in Clinical Practice project, which is funded by the American Board of Internal Medicine Foundation. It was designed to champion evidence- based care, and it sparked the Choosing Wisely campaign,5 which pro- motes the ability of patients to choose care that not only is supported by evidence but is not duplicative with other tests that have been performed previously, that is free from harm, and that is necessary for their care. One aspect of the Promoting Good Stewardship project was the creation of “top five” lists of common practices within internal medicine, pediat- rics, and family practice that are not supported by clinical evidence and guidelines, such as prescribing antibiotics for a viral infection, rushing to order a computed tomography (CT) scan at the first sign of uncomplicat- ed lower-back pain, or prescribing a brand-name statin rather than a ge- neric—“things that people know better [than to do] but happen routinely,” Silver-Isenstadt said. The Choosing Wisely campaign has 4 See https://openpaymentsdata.cms.gov (accessed January 16, 2015). 5 See http://www.choosingwisely.org (accessed November 11, 2014). 

EDUCATIONAL APPROACHES 23 helped expand the development of these lists—at the time of the work- shop, more than 60 specialty societies had generated lists—which have reached millions of people. “It is an inspiring story because it forces all of us to consider the question of why, across specialties, people are doing so many things that are counter to the evidence,” she said. Perry Pugno, the senior advisor for education for the American Academy of Family Physicians (AAFP), observed that today’s gold standard therapy may, in the future, subsequently be found to be detri- mental to the health of patients. “This context should make us humble in our biases and how strongly we feel about them,” he said. Disclosure is a common way of dealing with conflicts of interest, but, Silver-Isenstadt said, while disclosure is necessary for addressing conflicts of interest, it is not sufficient. Disclosure can suggest that someone has special expertise, it can heighten trust, and it can increase bias in advice (Cain et al., 2005). Some people point to disclosure as an obstacle to meaningful reform because it is seen as the solution to the problem. As a result, time is spent trying to tweak disclosure forms, caus- ing frustrations for those who have to fill out the forms, Silver-Isenstadt said, rather than spending time on “the real problem, which is what is be- hind what is being disclosed.” When making consumer decisions, people routinely consult inde- pendent evaluations of products’ safety, cost-effectiveness, and price. Yet physicians routinely allow those with a personal stake in drugs, tests, and devices to fund and conduct continuing medical education, to ghost- write journal articles, to make sales visits, to set up paid consulting agreements, and to distribute free products, Silver-Isenstadt said. Marketing-free education requires a strong cultural commitment by the leadership of the medical community. “Vigilance about conflict of interest needs to be built into the workflow,” Silver-Isenstadt said. What policies and safeguards are in place? How can they be stronger? Are they built into compliance training so that people are familiar with them? Are they well enforced? Are there consequences? Are they celebrated by the leadership? Are they points of pride? Resources are available to address the issue. The American Medical Student Association has rated medical schools with letter grades across domains on the strength of their conflict-of-interest policies, and the asso- ciation’s new scorecard will rate teaching hospitals.6 The Pew Charitable 6 The rating is available at www.amsascorecard.org (accessed November 11, 2014). 

24 IMPROVING GENETICS EDUCATION Trust has issued best-practice recommendations for academic medical centers, including such topics as pharmaceutical samples, ghostwriting, and meals sponsored by industry.7 Community Catalyst has put together toolkits for creating a new conflict-of-interest policy or strengthening an existing policy at a medical school or academic medical center.8 The Na- tional Physicians Alliance has an online archive of a webcast known as the Grand Rounds series, which addresses conflict-of-interest issues.9 Internal conflicts of interest are pervasive as well, a participant not- ed. These can be manifested in fee-for-service clinical medicine or in the incentives around career advancement. All people have personal perspectives and biases, Silver-Isenstadt said, and “to think that we could ever shed those is delusional.” But peo- ple need to recognize that they are susceptible to influences and be vigi- lant for the sake of patients. “For us to allow the conflation of marketing and education is to abdicate a responsibility to patients,” Silver-Isenstadt concluded. “And to fix it is going to require a lot of courage.” PRINCIPLES OF EVIDENCE IN DESIGNING EDUCATIONAL PROGRAMS Educational programs fall under the rubric of implementation re- search, which has been defined by the Fogarty International Center as “the study of methods to promote the integration of research findings and evidence into health care policy and practice,”10 as Maren Scheuner of the Department of Veterans Affairs explained. Similarly, Rubenstein and Pugh (2006) have defined implementation research as “scientific investi- gations that support movement of evidence-based, effective health care approaches (e.g., as embodied in guidelines) from the clinical knowledge base into routine use.” 7 The recommendations are available at http://www.pewtrusts.org/en/research-and-analysis/ reports/0001/01/01/conflictsofinterest-policies-for-academic-medical-centers (accessed Nov- vember 11, 2014). 8 Information is available at http://www.communitycatalyst.org (accessed November 11, 2014). 9 The series is available at http://www.npalliance.org/conflict-free (accessed November 11, 2014). 10 See http://www.fic.nih.gov/ResearchTopics/Pages/ImplementationScience.aspx (accessed November 14, 2014). 

EDUCATIONAL APPROACHES 25 The success of implementation research can be measured using two methods, Scheuner said. In formative evaluations, a rigorous assessment process is designed to identify potential and actual influences on the pro- gress and effectiveness of implementation efforts (Stetler et al., 2006). Such evaluations can use both qualitative and quantitative methods, in- cluding semi-structured interviews, direct observation, surveys, and tools that assess the fidelity of adoption of the implementation strategy. Form- ative evaluations can also provide much better understanding of the vari- ation seen in the uptake of guidelines. In summative evaluations, by contrast, a systematic process is undertaken of collecting and analyzing data on impacts, outputs, products, outcomes, and costs in an implemen- tation study. In conducting implementation research, the use of theory can help in planning the implementation strategy, aid in the conduct of evaluations, identify unanticipated elements critical to successful implementation, and help in understanding findings, including relationships between domains or constructs. Additional insights can also be gained about the theory itself, which can benefit future studies. There are three basic types of theories: explanatory theories, which include hypotheses and assump- tions about how implementation activities will facilitate a desired change as well as about the facilitators and barriers for success; process theories, which describe how implementation should be planned, organized, and scheduled; and mixed theories, which include elements of both explana- tory and process theories. Often, multiple theories will be used in imple- mentation research, Scheuner said. Scheuner has used these methodologies—formative and summative evaluation as well as theory—to examine genetic testing applications that have evidence of clinical utility, particularly with regard to family histo- ry. The project, known as Family History Education to Improve Risk Assessment for Hereditary Cancer, sought to improve the recognition and referral of patients at risk for hereditary cancer syndromes by primary care clinicians at women’s health clinics. The program reflected research showing that sequenced, continuous, and multifaceted activities can lead to a change in practice (Mazmanian and Davis, 2002) and that multifaceted interventions targeting different barriers to change are more likely to be effective than single interventions (Grimshaw et al., 2001). It also incorpo- rated research indicating that provider behavior is influenced by a number of factors, including provider characteristics, organizational structures and processes, external environmental factors, patient characteristics, and the encounter between the patient and the clinician (Rubenstein et al., 2000). 

26 IMPROVING GENETICS EDUCATION During the pre-implementation phase, the formative evaluation in- cluded recognizing determinants of current practice, identifying barriers and facilitators, assessing the feasibility of the proposed intervention, and integrating findings into the intervention design and refining the design prior to implementation. This evaluation revealed that the electronic health records lacked standards for family history documentation. Inter- views with primary care clinicians at the facility revealed that the pro- viders wanted a standardized template for the family history in the electronic health record, better organization of the family history in the record, and data gathered from patients before their office visits through a kiosk, personal health records, or some other method. The providers gave a high rating to clinical reminders that could help with documenta- tion, would enable easier recognition of conditions which might be inher- ited, and that prompt referrals for consultation or testing. When asked the reason for giving high ratings to such reminders, the providers pointed to, among other things, a lack of knowledge, familiarity, and confidence in genetic risk assessment, diagnosis, and testing. “Most primary care clinicians do not want any reminders,” Scheuner said, “but they were open to it [in this case] because they knew they needed help to better document family history.” In response, Scheuner and her colleagues created a draft reminder tool designed to identify family history red flags for hereditary cancer, eventually expanding the reminder template to gather even more com- prehensive family history information. Ultimately, a multifaceted strate- gy that included clinical interventions, information interventions, and behavior interventions was developed as part of a family history toolkit. For example, the strategy included a lecture series, information sheets for providers and patients, a website with additional information, and quar- terly practice feedback reports. With the formative evaluation, project developers sought to assess discrepancies between the implementation plan and its execution, to un- derstand and document the nature and implications of local adaptation, to monitor impacts and indicators of progress toward project goals, to use data to inform the need for modifying the original strategy, and to pro- vide positive reinforcement to high performers and negative reinforce- ment to low performers. The project led to a doubling of cancer family history being docu- mented in progress notes, Scheuner said. Missed opportunities to make genetics referrals also decreased over time. Individualized reinforcement, both positive and negative, was given to providers. 

EDUCATIONAL APPROACHES 27 Post-implementation interviews with clinicians sought to assess the usefulness and value of the intervention from the stakeholders’ perspec- tives, to elicit stakeholder recommendations for further intervention re- finements, to assess satisfaction with the intervention and implementation process, and to identify additional barriers and facilitators. Almost all of the clinicians interviewed said that they were doing a much better job of documenting the cancer family history and that they were much better informed about when to make a referral, Scheuner said. Effective programs tend to be multifaceted and ongoing, Scheuner concluded, and the use of theoretical models can help inform the devel- opment and evaluation phases for such programs. Djuricich added that much could also be learned from examining the partnerships that clini- cians in specialties that do not have enough practitioners to meet health care needs, such as adolescent medicine, child psychiatry, and geriatrics, enter into with primary care organizations in order to disseminate educa- tion. Investigating how well these partnerships work might be a way to understand how to advance genetics education. Scheuner said that there is a need to better understand what works in different health settings and systems in order to identify what works locally and what could work na- tionally. “I would encourage you to think about developing implementa- tion strategies for your genetics education programs and generate evidence about the effectiveness,” she said. MEASURING THE EFFECTIVENESS OF EDUCATIONAL INTERVENTIONS Much can be learned throughout the course of an intervention, rather than waiting until the intervention has been completed, Scheuner said. In projects at the Department of Veterans Affairs, ongoing interviews are conducted with stakeholders, including front office staff. In particular, she emphasized the importance of quality improvement and the use of best practices, which requires quality indicators. “Everybody needs to agree on that,” she said. “Then you have to go and look. If you can cap- ture them in administrative data, that might be great. Oftentimes it is not that simple.” Djuricich pointed out that educational researchers have studied the impact of educational interventions—for example, by using the seven levels developed by Moore et al. (2009): 

28 IMPROVING GENETICS EDUCATION 1. Participation 2. Satisfaction 3. Learning 4. Competence 5. Performance 6. Patient health 7. Community health Murray Kopelow, the president and chief executive officer of the Accreditation Council for Continuing Medical Education, noted that nu- merous systematic reviews have been performed that demonstrate that continuing education does exactly what it is intended to do (Al-Azri and Ratnapalan, 2014; Bloom, 2005; Davis and Galbraith, 2009; Forsetlund et al., 2009; Lowe et al., 2009; Mansouri and Lockyer, 2007; Marinopoulos et al., 2007; Mazmanian et al., 2009). The issue, he said, is “that profes- sionals do not do what they are supposed to do. This is a barrier.” Putting resources into the further evaluation of continuing medical education initiatives would not be well spent. Instead, he suggested doing research on the problems that exist, such as the professional practice gap, and how those problems can best be addressed. Davis emphasized the importance of having a gold standard to pro- vide a measure against which performance can be gauged. “We need a guideline that says, in this disease, for this condition, with these parame- ters of age or gender, we need to do this test. . . . Then we can do meas- urements in population health standards.” He also said that national campaigns like Choosing Wisely, combined with local agreements to adopt a particular guideline, can be means to facilitate changes in practice. REACHING COMMUNITY PRACTITIONERS A number of factors can influence the motivation that community practitioners have for embracing and using a new practice approach, Perry Pugno said, and these factors include • It directly benefits patients the practitioners see frequently. • It is simple and straightforward to apply. • It reduces practice costs or saves time. • It reduces professional liability risks or costs. • It helps secure privileging. 

EDUCATIONAL APPROACHES 29 • It is part of a maintenance-of-certification (MOC) requirement. • It leads to financial remuneration. Pugno emphasized a point made in the previous chapter: Giving cli- nicians information does not necessarily mean that they will integrate a new approach into their practice “People do not learn by just telling them things,” he said. “We have to make them do something. They have to actually use the information for them to integrate it and make it part of what they do on a daily basis.” Adoption and use are further complicated when dealing with genetics information, Pugno said, because of the gap between the promise of ge- netics and the current reality. Genomics was touted as making it possible to deliver targeted care with treatments that would have a higher rate of success and also to make it possible to use genetic information to predict the risk of or prevent the occurrence of disease. However, the majority of applications for genetics have been focused on rare disorders, they are infrequently relevant to practice, and can be an expense to the patient or practice with little or no benefit currently. AAFP has taken several steps to equip its 100,000 practicing physi- cians with information and skills that they will embrace and use. For ex- ample, it has developed a MOC module known as Measuring, Evaluating, and Translating Research Into Care (METRIC), which ena- bles the capture of performance information in order to improve patient care and outcomes. A recent study on the efficacy of METRIC found that in nearly 12,000 patient encounters with family physicians who had un- dergone the intervention, 75 percent of the quality markers in diabetes care showed improvement (Bird et al., 2013). AAFP also has developed such tools as self-evaluation modules with online patient simulators, fast- track board review with audience response and engagement systems, a translation-to-practice credit system, geospatial mapping of needs as- sessments, and the use of social media to facilitate interaction. “There are a lot of efforts that are going on right now to try to provide education that matters that clinicians will accept and use in the care of their patients,” Pugno said. If educational resources are to help capture the interest of practicing clinicians and turn that interest into effort, the resources should have cer- tain features, Pugno said. First, relevance to daily practice is crucial, he said, echoing comments made by previous speakers. There should also be easy access to the resources, such as a website that contains most of the information that will be helpful to clinicians and patients. Integration 

30 IMPROVING GENETICS EDUCATION with electronic health record decision support can also be important, as can frequent updates aimed at addressing timely questions from patients. It is also effective to offer learning in small chunks at the convenience or time of need of the clinician. The Family Doctor website (FamilyDoctor.org) de- veloped by AAFP has a number of these attributes. “Challenges persist,” Pugno concluded, and “there are no magic an- swers.” But examples of things that work exist and can point the way toward future innovations and improvements. MOTIVATING CLINICIANS One of the biggest barriers to making genetics an integral part of continuing education for health professionals is motivating the clinicians to participate, which in turn depends upon them recognizing that there is a significant gap in knowledge that could impact their care of patients. As Raby described it, the clinicians need to “understand that there is de- mand for [this knowledge], that there is need for it.” Scheuner pointed to the importance of convincing the leaders of an institution of the need for such education. For example, she talked with the leader of the Veterans Affairs Greater Los Angeles Healthcare System clinical service about BRCA1 and BRCA2 genetic testing, to which the leader was “very re- ceptive.” Clinicians also were interested in educational activities because they helped provide the context for the new tool they were using to ob- tain family histories. Raby said that clinicians could also be motivated by the coming “tsunami” of readily available genetic information. The public is going to expect physicians to have an understanding of genetics, whether the ge- netic information they want explained comes from testing at a clinic or from personal genetic testing. Testing for research and clinical purposes has been increasing dramatically, and this testing is producing infor- mation that is directly relevant to patients, even if it is not yet actionable. With “most of the patients I see,” Raby said, “the thing they are most grateful about is that I can actually tell them what is wrong with them, and I can explain to them why this has happened to them. The more in- formation that is going to be out there, the more we are going to have to tell them.” Pugno said that if clinicians are to be interested in continuing medi- cal education, they have to be able to answer “Yes” to three questions: Does this benefit my patients in a measurable way? Does this save me 

EDUCATIONAL APPROACHES 31 and my patient time? Does this save me and my patient’s money? “If it addresses those three things, you will be beating them away from the door.” David Davis summed up the issue of motivation in this way: “Knowing that every clinician gets up in the morning trying to do the best job they can, that is the clue to engagement for all of us.” EDUCATION AND QUALITY IMPROVEMENT Education initiatives and quality improvement initiatives are typical- ly conducted by two different groups of people within health care institu- tions. The critical question, Djuricich said, is how to integrate these two sets of activities at the point of care. Electronic health records may pro- vide the answer, he said. These records cannot contain 45-minute educa- tional modules, but they can contain what he called “educational nuggets.” Packets of information that are quite short, less than a minute or two long, could help a clinician at the point of care, when that infor- mation is needed most to help a particular patient. The accreditation of quality improvement projects for physicians and for multidisciplinary allied health staff teams has worked well for some institutions, according to one workshop participant. “For example,” the participant said, “I work with pathologists, and they are constantly doing quality improvement with regard to turnaround time or improving la- boratory reports.” Physicians are extremely busy, but if they can receive continuing medical education in something that they are already doing, they can use that information to improve the quality of care for their patients. Davis said that education will fail if the improvement component is not included. “If there are Venn diagrams, they overlap extensively.” It is necessary to begin thinking about including a focus on improving the quality of patient care as a component of continuing education, he said. Seibert pointed to the more general need to connect practice envi- ronments with educational communities in more formal ways. For exam- ple, many medical schools have adopted more integrated curricula. There are places where physicians, nurse practitioners, and other providers can work together on problems within the health care system, she said. “That may bring those educational pieces together.”