Important Points Highlighted by the Individual Speakers
- Medical organizations that accredit residency and fellowship training or that certify physicians for independent practice can shape the content of graduate and post-graduate education programs.
- Standards or guidance for the clinical use of genomics findings is needed to take advantage of accreditation as a lever for change.
- Pharmacy organizations that accredit residency and fellowship training are updating their competency requirements in genetics and genomics for pharmacists to include training in applied pharmacogenomics and broader exposure to genomic medicine.
- Focusing on the critical points to master relating to a specific disease process or condition is effective for learner retention.
- The following core competencies are emphasized in postgraduate residency or fellowship training: patient care skills, clinical knowledge, practice-based learning and improvement, communication skills, professionalism, and systems-based practice.
The Accreditation Council for Graduate Medical Education (ACGME), the body that accredits physician residency and fellowship programs, could serve as a lever for expanding genomics education at the graduate and post-graduate levels, said Kevin Weiss, ACGME’s senior vice-president for patient safety and institutional review. The organization not only responds to changes in medical education but is one of the forces that shapes those changes.
The development of medical professionals is not a linear pipeline but a series of interrupted, challenging activities, Weiss said. This collection of activities includes pre-medical studies, medical school, specialty (residency) education, subspecialty (fellowship) training, and continuing education. Underlying these activities are the accrediting organizations and other bodies that certify individuals. “There is a series of tickets that have to be punched, both at the educational organization level, as well as the individual,” Weiss said. “You can’t just push one and expect the entire [collection of activities] to respond. You almost have to work as a group.”
This is the context within which ACGME works. Its mission, Weiss said, is to “improve health care by assessing and advancing the quality of resident physicians’ education through accreditation.” In the United States, roughly 110,000 to 120,000 people are taking part in residency programs accredited by ACGME. These training programs must address six core competencies: patient care and technical skills; medical knowledge; practice-based learning and improvement; interpersonal and communication skills; professionalism; and systems-based practice. The chief focus of a residency program is on molding individual trainees into highly developed clinicians. However, Weiss said, “graduate medical education is not just about preparing clinicians for independent practice. It is also about preparing them to become proficient teachers and managers of resources, roles that take a lot of time and effort to develop.”
ACGME is in the midst of a major effort to overhaul its accrediting process over the next 10 years, creating what is known as the Next Accreditation System (Nasca et al., 2012). The organization is shifting to focus continuously on improved patient outcomes. The current goal is to encourage program sponsors to maintain a humanistic educational environment that assures the safety and quality of the care that residents provide to their patients both today and in their future practices. Weiss also said that the organization may one day incorporate patient outcomes di-
rectly as a measure in the accreditation process. “We are at least a 5- to 7-year cycle before I think any of us can see that happening easily,” he said, except perhaps with the clinical data registries that some health systems have been establishing.
To reach ACGME’s current goal, the Next Accreditation System was built as a continuous accreditation model with annual updating based on data that residency programs must submit each year. This system includes a 10-year “self-study visit,” Weiss said, and also a revision of standards every 10 years at a minimum, with reviews occurring much more frequently than that. To stay accredited, medical centers and institutions that sponsor residency or fellowship programs must also undergo a Clinical Learning Environment Review visit.
The annual review process for accreditation examines a series of outcomes, including the particularly important feature of milestone data, which is one of several elements used to assess residents’ progress. The residents must demonstrate mastery of the requisite skills, knowledge, and behaviors in each of the six core competencies. “You can begin to see how competency domains may translate from genomics and genetics into a milestone,” Weiss said. The milestones also create a framework of observable behaviors and attributes associated with residents’ development as physicians.
Aggregate performance on the milestones will be used as 1 of 10 indicators for measuring a program’s educational effectiveness as part of ACGME’s continuous accreditation monitoring, Weiss said. The milestones “are probably even more important for residency programs because they can help guide curriculum development,” he said. “They can help target individual residents for specific needs and improvement for the underperformers.” Medical centers and institutions that sponsor residency or fellowship programs must undergo a learning environment review every 18 to 24 months, examining six focus areas: patient safety, supervision, professionalism, health care quality and health care disparities, duty hours and fatigue management, and transitions of care.
There is a great opportunity for ACGME to assimilate genomics education because “we can broadly shape the thinking in the residents and fellows through our requirements,” Weiss concluded. As with the introduction of CT scans decades ago, the medical world is now facing “a rapid and not well-rationalized diffusion” of technology, he said—in this case, genomic technology. “We need to have a set of standards or some sort of guidance that is a clear signal [versus the] noise that really can take advantage of a lever that is as strong as accreditation.”
The pharmacy profession has made substantial progress in the clinical implementation of pharmacogenomics and in national education efforts, said Grace Kuo, professor of clinical pharmacy and associate dean for academic clinical affairs at the University of California, San Diego. Pharmacogenomics promises to optimize drug therapy, minimize the trial-and-error approach to prescribing, and prevent avoidable adverse drug reactions. More than 100 medications approved by FDA include pharmacogenomics information on their labels at present,1 and the number has been rising. Finally, applied pharmacogenomics offers a clear opportunity for spreading genomic medicine, as many institutions have adopted pharmacogenomic testing as an initial foray into genomic medicine practice.
More than half of the roughly 287,000 pharmacists practicing in the United States in 2013 did so in the community pharmacy setting, with another 22 percent practicing in hospital settings and 2 percent in non-patient care settings (e.g., administration, teaching, or research). 2 While postgraduate pharmacy residency and fellowship training is strongly supported by several national pharmacy organizations, relatively few Pharm.D. graduates pursue such training—largely because training slots are limited and post-graduate education is not a mandatory requirement for many pharmacy positions. In 2012–2013, only 42 percent of the 13,551 Pharm.D. recipients applied for residency, with half of those being accepted into accredited programs.3 There is a critical need to continue developing the capacity for post-graduate clinical pharmacy training, Kuo said.
There is a growing profession-wide recognition that clinically applied pharmacogenomics offers an additional tool for optimizing the safe, effective, and affordable use of drugs, Kuo said. In 2009 the American Pharmacists Association convened a workshop to discuss the role of the pharmacist in using clinical pharmacogenetic data to direct patient care by effectively using electronic health records to exchange relevant health data among
1See http://www.fda.gov/drugs/scienceresearch/researchareas/pharmacogenetics/ucm083378.htm (accessed November 12, 2014).
3See https://www.natmatch.com/ashprmp/stats/2013applstats.html (accessed January 16, 2015).
the entire health care team (Reiss, 2011). Subsequently, large clinical implementation efforts were initiated to test the effectiveness of different care delivery models for pharmacogenomics (Hoffman et al., 2014; O’Donnell et al., 2014; Shuldiner et al., 2014), with various research studies, surveys, and focus groups underscoring the need for focused education in genetics, genomics, and pharmacogenomics (McCullough et al., 2011; Sansgiry and Kulkarni, 2003).
The PharmGenEd project was also developed as a continuing education curriculum focusing on pharmacogenomics primer concepts and clinical applications in various therapeutic areas (Kuo et al., 2011). Disseminated to health care professionals and students through a train-the-trainer program, the PharmGenEd project makes shared curriculum modules, video seminars, and relevant journal articles available online through its website (https://pharmacogenomics.ucsd.edu). “The program has now reached over 100 countries with 23,000 unique users and over 3,000 registrants,” Kuo said, and it has been used by pharmacists, pharmacy technicians, pharmacy students, physicians, nurse practitioners, genetic counselors, researchers, and others.
PharmGenEd also evaluates outcomes and processes through two online modules, Kuo said. Continuing education credit can be earned through the American Society of Health Systems Pharmacists (ASHP) by passing the self-assessment exam on the society’s website. For both modules, 97 percent of all users taking the exams passed.
While the pharmacy profession continues to play a key role in clinical pharmacogenomics, there is increasing momentum toward team-based care and interprofessional collaboration among front-line clinicians—both in genomics and in other clinical paradigms, Kuo said. To foster such collaboration the Genetics/Genomics Competency Center for Education (G2C2) project of the National Institutes of Health was instituted. The project provides high-quality educational resources in genetics and genomics for health care educators and practitioners. 4 G2C2 has so far collected educational competencies for nurses, physicians, physician assistants, and genetic counselors, and it is currently in the process of adding a new educational competency on genetics and genomics for pharmacists (Feero et al., 2012), which builds on a previous one developed in 2002 by the American Association of Colleges of Pharmacy (Johnson et al., 2002).
In order to adequately prepare and train the future pharmacy workforce to practice effectively, pharmacy faculty, national professional organizations, and the Accreditation Council for Pharmacy Education recently announced that the forthcoming 2016 accreditation standards for all U.S. pharmacy schools will, for the first time, require that pharmacogenomics be taught to all students. An important next step, Kuo said, may be to develop equivalent standards for post-graduate residency and fellowship training programs, although these are accredited by a separate organization (ASHP).
Certifying boards define a discipline through certification, they track trends in medical knowledge application, and they provide assessment, which can also drive learning, said Rebecca Lipner, the senior vice president for evaluation, research, and development at the American Board of Internal Medicine (ABIM), Founded in 1936, the not-for-profit ABIM is an independent umbrella organization representing 24 certification boards in medical specialties, including internal medicine, allergy and immunology, medical genetics, preventive medicine, and urology. The organization’s mission is “to enhance the quality of health care by certifying internists and subspecialists who demonstrate the knowledge, skills, and attitudes essential for excellent patient care.” Roughly one in four practicing physicians in the United States is certified in internal medicine, which itself includes a multiplicity of subspecialties ranging from cardiovascular disease to transplant hepatology. Generally, the domains of medical knowledge or content that these disciplines cover are large, Lipner said, and genetics can fit into many of these areas.
To become board certified in internal medicine, a physician must complete a 3-year internal medicine residency with 2 to 3 years of further fellowship training required for subspecialty certification. Doctors must achieve satisfactory faculty ratings in the six core competencies specified by ACGME and pass a summative, high-stakes cognitive examination, Lipner said.
ABIM often uses board certification exams to help define the breadth of a medical discipline, Lipner explained. The advantage of a written exam is that “you can put a lot in there in a short period of time
and cover a lot of area,” but the trade-off is that “you usually do it in an artificial way, like with multiple choice questions or short answers.” The idea is to broadly sample a physician’s knowledge, judgment, attitudes, and ability to diagnose and treat patients, typically using questions involving clinical vignettes about patient cases. The test may require recognizing both common and rare conditions. Among the key features of these exams, Lipner said, are that they assess the core knowledge that a physician “should know and carry around in their head without looking it up” and that they emphasize content that reflects current best clinical practices, which, Lipner noted, “might be changing out from under us.”
The starting point for ABIM’s pro-active process to fit applicable content into the certification exam is a “practice analysis” or job analysis. This is a systematic procedure for collecting practice-related information—usually through a survey using a questionnaire developed out of research and input from focus groups and subject-matter experts—and defining the knowledge base in the discipline. The purpose of the practice analysis is to link the knowledge, skills, and attributes that are tested on the exam with the responsibilities of the profession. “Somehow, we have to create an exam, but we don’t want to do it without understanding where the profession is coming from, what are the tasks that have to be done on that job,” Lipner said. “Tasks are important, and person-oriented skills are also important.”
Based on its analysis, Lipner explained, ABIM creates a blueprint to frame critically important exam characteristics (including the weight or percentage of the test devoted to different content areas), provide guidance to expert test question authors, and document historical performance. The blueprint for the medical oncology certification exam, for instance, shows that questions about genetics and tumor biology make up 3 percent of the exam. (That percentage does not, however, include many additional questions about genetic testing that are embedded within other content categories such as breast cancer and colon cancer.)
The evaluation of exam items takes into consideration two factors: difficulty, or how hard or easy a question is; and discrimination, or how well an individual question captures the overall ability of the examinees. These factors indicate whether certain content areas are obsolete and whether emerging medical evidence may be outpacing question content, Lipner said.
The exam blueprint is evaluated through an annual review process by subject-matter experts who assess changes in clinical practice guidelines
and emerging content or practices to determine which topics should be included on the exam. The annual review is augmented by surveys or post-exam comments that offer feedback from doctors on, for instance, whether the blueprint is missing or overemphasizing certain content or skills.
Interestingly, cognitive science research (Larsen et al., 2008; Roediger and Butler, 2011) has found that “the act of taking an exam improved performance above and beyond simply studying for it,” Lipner said. “Frequently taking tests is actually a good thing.” Currently the board certification exams are required every 10 years. Because this “is probably not as good for retention [of information],” she said, ABIM is sponsoring an initiative called Assessment 2020 to explore the best practices for assessing physicians’ skills that would be in addition to, or in place of the comprehensive 10-year exam.5 Thus far the discussion has centered on such ideas as testing more frequently, targeting areas of weakness for individual physicians, and holding an open-book exam, Lipner said. Alexander Djuricich of Indiana University noted that it is possible to send personalized follow-up quizzes to individual examinees’ smart phones. He also pointed out that the New England Journal of Medicine offers a learning program called Knowledge Plus™, which delivers test review questions for the internal medicine board exam via smartphone and tablet. The program uses adaptive learning technology to assess a learner’s progress and identify content to reinforce with additional questions or content. ABIM is also planning to provide individual physicians more detailed feedback on areas for improvement relative to responses for specific content, Lipner said. The aggregate performance results will be shared with medical specialty societies in order to identify the areas of strength and weakness in the group of physicians completing residency training in order to assist quality improvement within education programs.
Offering insights from the learner’s perspective, Samantha Bazan, a public health nurse in her second year of nurse practitioner training at the Uniformed Services University of the Health Sciences, shared her thoughts
on effective teaching and learning in genetics. Bazan agreed with David Davis’s earlier observation that simply providing more information in graduate and continuing education is not necessarily the best approach. “You can only retain so much information,” she said. An alternate approach could include a sharp focus on the critical points to master relating to a specific disease process or condition—including genetic implications or associations. “Those things are going to be what sticks with you in clinical practice that triggers your brain to say, ‘Oh, okay, I remember this. This is something we need to look for.’”
It is also valuable for trainees to understand why a genetic test is needed, what it is going to tell the patient, and what the implications of the results are, Bazan said. “If we learn that in our graduate programs, then we can better explain to the patient why we think they either need to go see a specialist or why we think, hey, you don’t need this.”
As far as preferred learning tools and strategies, Bazan said she finds UpToDate useful. During moments of downtime—for instance, while waiting to present a patient case to a preceptor—she will use the resource on her smartphone as a knowledge check. Clinical practice guidelines are also helpful for new practitioners, Bazan said, especially guidelines with algorithms that direct decision making. “Case studies are [another] important way to learn,” she said. 6 “But it is more productive to work through case studies together in class, where you can benefit from hearing others’ thought processes in reaching a conclusion, rather than being asked to do it on your own.”
Bazan confessed to a “love–hate relationship” with the patient simulation center. “I absolutely hate it,” she said. “I hate being videoed and having to go back and look at it.” But the simulation sessions offer more realistic training than role-playing between students and faculty. “When you are actually doing it with simulation center patients who are there role-playing professionally, it works out really well. I have learned a great deal from that.”
Graduate students can occasionally leap ahead of the faculty with their assimilation of knowledge of genetics and genomics, one workshop participant said. Three years ago the participant’s institute started offering medical students the option of concurrently enrolling in a master’s degree program in genomic medicine. The institution anticipates that medical school graduates with this additional degree will end up being an educational resource for the practices that they are entering.
Weiss said that “there is a real excitement to teaching the learners and getting them as an inoculum there to make change.” However, as an isolated strategy, it might carry higher risk. When residents have received quality improvement training, but then move to practices that are unsupportive of this type of approach, the enthusiasm and engagement of that trainee can “just die on the vine,” Weiss said. “That is a very painful thing to watch.” Faculty need to plan for that potential circumstance, Weiss said, to keep young learners’ engagement in quality improvement or genomics going. He also suggested that one strategy for spreading genomics knowledge into residency programs might be to encourage collaboration between faculty and residents on experiential projects.
Bruce Blumberg said that at Kaiser Permanente a similar issue arose related to efforts to provide medical residents with education about quality improvement. “We are learning pretty rapidly that we have faculty that don’t know very much about quality improvement,” he said. “Whenever a new body of knowledge comes into a field, you have that problem—where the learners and the people recently coming out of training have a skill or a body of knowledge that doesn’t necessarily exist throughout the faculty.”
Given such obstacles, implementing a genomics education component for currently practicing physicians is a grand challenge. One participant, for example, described how medical students at his institution could take an elective course in personalized medicine and genomes that offered them (and interested faculty) the chance to analyze either their own pharmacogenomics data or data from an anonymous person; then throughout their second year the students are taught how that genetic information could be useful in clinical practice. However, the faculty in charge are not entirely comfortable with the genetics material themselves, the participant said, and thus when students enter their third, fourth, and residency years, the follow-through on this earlier learning is lost.
Assessment is a chief driver of learning, and at times a new body of information must enter a field of practice. The critical challenge, Lipner said, is determining which persuasive sources to trust in terms of which new information is critical to assimilate.
This page intentionally left blank.