- A lackluster in-person presentation is just as uninteresting online. The quality of teaching and education matters, regardless of the medium used to transfer the information. (Desai)
- Some skills are best learned in person and not through the online instruction format. (Desai)
- Given the huge opportunity massive open online courses (MOOCs) offer, it would seem to be an excellent avenue for promoting interprofessional education (IPE). (Gawron, Jeffries)
- As the health system moves more toward technology as the platform for providing care, are we creating even more disparities? (Meleis)
Forum member Jack Kues from the Alliance for Continuing Education in the Health Professions moderated a “flipped classroom” session on technology and innovations in assessment. In the flipped-classroom format, speakers post an online presentation before the workshop, and Forum members and workshop participants are given first exposure to the session material in this format. At the workshop itself, the speakers briefly discuss their presentation and then respond to questions from Forum members and workshop participants. The flipped-classroom structure allows participants to engage more deeply with the speakers to develop a richer understanding of technology and innovation use in learning and assessment.
Kues first provided a brief background to the topic. Many changes have occurred in health care that have prompted changes in how and where health professionals are trained. These changes have been going on for some time. Traditionally, the clinical experience took place in a hospital, which is where most care was given. This is no longer the case. Training has been spread out into the community into all kinds of environments, both clinical and nonclinical.
The educational model, however, is still trying to catch up with the shift in health care. And the educational training of students is still largely episodic, meaning that students in a clinical environment may see a patient once or twice in the course of rotation, which is not consistent with the chronic care model that is emerging.
Another change, said Kues, is that faculty are increasingly pressured to tie more of their work to the bottom line in reimbursement and financial models not only for themselves, but also for the institutions for which they work or teach. That means that busy practitioners, while they may be willing to continue to teach, do not have the same time availability they used to have.
A fortunate change is in the area of technology innovation, which has helped educators and practitioners overcome some the challenges inherent in both educational and assessment models. In this session at the workshop, speakers described how technology is being used for assessment in three different settings: (1) patient engagement, (2) clinical competency, and (3) independent online learning.
AMERICAN NURSE PRACTITIONER FOUNDATION AND THE LEADING REACH PATIENT ENGAGEMENT MOBILE PLATFORM
Margaret Crump, American Nurse Practitioner Foundation
In her presentation, Margaret Crump described a patient engagement mobile platform that is being rolled out by the American Nurse Practitioner Foundation (ANPF). ANPF supports nurse practitioner education, enables innovative research, and provides the tools and resources to develop practice-based, data-driven solutions to public health problems. One such tool was embraced by ANPF in early 2013 and focuses on patient engagement. Known as Leading Reach, this mobile application (app) provides a communication platform designed for touch screen and portable devices like tablets and smartphones (ANPF, n.d.). Through this mobile app, practitioners can send patients accurate health information and administrative issues related to their upcoming clinic visit; however, the greatest value, as it relates to assessment, is the ability for patients to respond to their practitioner using this application, possibly providing patient feedback about practitioner or team performance.
In addition to providing a two-way communication platform, this mobile application also tracks and scores how well practitioners and patients are connecting. It does this by collecting interaction data between patients and practitioners and assigning a score in four specific categories of engagement—new patient information, patient education, patient satisfaction and social media, and referral information. Each category contains specific elements easily customized by the provider, clinic, or health care system to direct health professionals’ behaviors around such issues as revenue, cost, time saved, and healthier patients.
Leading Reach has been used in more than 75 countries worldwide and was provided to some of ANPF’s nurse practitioners so they could study how well the mobile app connects the nurse practitioners to their patients, whether quality of care is improved, and whether ANPF should make it more widely available to their nurses.
Crump was asked why ANPF decided to use this particular patient engagement app over the other communication device options, why it chose this method for rolling it out, and what is the business case for its rollout. She began her response by citing statistics on what is currently known about the state of health care in the United States:
- Fifty-five percent of doctors do not communicate with their patients between visits (Televox, 2011).
- Seventy-two percent of hospital patients do not schedule a follow-up appointment (Scott, 2012).
- Eighty-three percent of patients do not follow treatment plans (Televox, 2011).
- Seventy-eight percent of U.S. consumers are interested in mobile health solutions (Float Mobile Learning, n.d.).
- Seventy-four percent of U.S. households have Internet access (U.S. Census Bureau, 2012).
Then she explained that within the first 175 days of existence, more than 100 clinics and 800 referring doctors used Leading Reach for processing and generating thousands of referrals and emails that produced more than 75,000 content downloads by their patients.
With that as background, Crump went on to explain that ANPF selected this particular app to increase their scholarship and research grant area by partnering with a group focused on technology. In her opinion, having technology partners at the table and as part of the conversation is key to development. ANPF studied many different technology-based innovations, but it selected Leading Reach because of the ability to link providers to patients on an educational and informational platform and because of
positive experiences expressed by other health professionals already using the system.
In 2014, ANPF will make Leading Reach more widely available to nurse practitioners for free so they can start building a capacity and an understanding of the technology. Crump said ANPF recognizes it does not have all the answers. The pilot study is part of its business model so it can research and better understand how the app is improving patient outcomes and behavior change by studying the communication process.
IMPROVING NURSING SKILLS THROUGH SIMULATION: TOOLS FOR ASSESSING IMPACTS ON PATIENT SAFETY
Barbara Gawron, University of Illinois College of Nursing
Barbara Gawron presented how she and her colleagues at the University of Illinois College of Nursing use simulation to formatively assess the clinical competency of nursing students in an effort to improve patient safety. To do this, data are collected by clinical instructors at the time of the simulation using the Creighton Competency Evaluation Instrument (C-CEI). This is a tool developed at the Creighton University School of Nursing for conducting observational analysis of students in simulated clinical environments (Creighton University, 2013). Structured around the American Association of Colleges of Nursing (AACN) core competencies, it includes 22 behaviors organized into four areas that include assessment, communication, critical thinking, and technical skills. Each clinical instructor completes a form rating how well the student performed during the simulation exercise.1
The purpose of collecting these data is to see how students are doing at the time of their simulation. As an example, Gawron shared a video demonstrating a respiratory distress simulation for prelicensure nursing students. In the video, faculty members used their iPads to collect data then discuss their results with the student during the debriefing as a formative assessment. Students who did not meet their learning objectives were brought back into the simulation lab. Because of the assessment tool and the extra time built into the program, faculty were able to immediately correct the student learning to meet the objectives for understanding how to care for a patient in respiratory distress. The data collected in each student’s simulation performance were aggregated then analyzed to identify patterns, weaknesses, or gaps in the class’s understanding, and enabled faculty to revise the content.
1An example of a form can be found at http://www.cod.edu/academics/conted/business/nursing_symposium/pdf/csei.pdf (accessed April 18, 2014). The C-CEI is also included in Appendix B.
This simulation is undertaken by students early in the nursing curriculum. As the learners progress, they have to master 24 different patient safety scenarios that increase in complexity. By graduation, all the nursing students understand their roles and responsibilities as individuals and members of a team. It is during the simulation exercises that students learn to delegate, to collaborate, and to work effectively as a team.
Gawron was asked whether she thought her simulation exercises improved patient safety. She responded that the purpose of collecting these data is to see how students are doing at the time of their simulation, but she does have anecdotal data showing an improvement. In the example she described in her presentation, a student had 5 minutes to introduce herself to the patient and recognize safety concerns that above all included a patient experiencing respiratory distress. The student did not respond to the low oxygen level. This was discussed in her debriefing with an instructor before the student repeated the exercise until she could correctly identify and correct the patient’s low oxygen levels. This student passed the simulation exercise and returned from her clinical experience telling Gawron about a patient with a “pulse ox of 88” and how she knew exactly what to do.
In another example, Gawron was asked if she could assist with decreasing particularly high rates of readmission for coronary heart failure patients at a local hospital. In response, Gawron created a translational care program for her students using simulation. She then sent her students out into the community to track the coronary heart failure patients in an effort to identify and correct causes for the high readmission rates. These are two examples where Gawron believes her simulation exercises are having a positive impact on patient care and community health.
ASSESING VIRTUAL LEARNING AND TEACHING THROUGH THE KHAN ACADEMY PLATFORM
Rishi Desai, Khan Academy
Rishi Desai described his work in medical education at Khan Academy, which is a free online platform for education. Khan Academy’s website attracts roughly 10 million unique users per month through its four categories of content that include videos, questions and assessments, text, and games. The website content is geared to a variety of audiences, such as patients, students, and health professionals based on the depth of content the user selects. Unlike other courses that begin in March and end in May, Khan Academy provides information that can be accessed whenever and for whatever length of time the user has available. In this way, Khan Academy provides a lifelong resource for lifelong learning for all. It is online, it is
free, and it can be taken offline so information can be extended to those in remote areas that do not have Internet connectivity.
Currently, Khan Academy is partnering with the Association of American Medical Colleges (AAMC) and the Robert Wood Johnson Foundation to develop content for the Medical College Admissions Test so anyone can study for the exam free of charge. Additionally, Khan Academy is partnering with AACN and the Jonas Center to develop content for the National Council Licensure Examination. These are some of the activities Khan Academy has under way at this time, but because education is not static, neither is Khan Academy. For example, if thinking changes around a particular topic, Khan Academy takes information off its website and posts new information. It is a live system that is particularly beneficial to health providers for continuing and supplementing their education.
One of the questions posed to Rishi Desai was how Khan Academy ensures the accuracy of its information. To frame his response, Desai commented that Khan Academy staff increasingly think of their users not just as content consumers, but as content producers. Many of the games Khan Academy uses for educational purposes come from the users of its website. For example, a game was made for Khan Academy explaining what will happen if the myelin sheath—needed for proper functioning of the nervous system—is gone. The game explored complex issues using a fun and engaging approach. It is incredibly instructive having young viewers making games around medicine and health because it teaches them and also teaches the community about important information. To encourage development, Khan Academy sponsors national competitions for video and game development that are open to anyone interested in competing.
To better ensure accuracy of content, Khan Academy is also introducing a peer-review system that Desai believes is a tremendous step forward in quality. This, he says, separates Khan Academy from some of the other massive open online courses (MOOCs) that do not have a quality control mechanism.
EXPANDING QUALITY EDUCATION THROUGH TECHNOLOGY
Following the presentations, Forum members posed a series of questions to the speakers that addressed how technology could expand the quantity and quality of education for all learners, including students, practitioners, laypersons, and patients. The discussions and speakers’ responses are detailed in the sections below and address a wide range of issues including the following:
- How might virtual collaborations among health professionals, other professionals, and educators function?
- How might technology empower patients and communities through improved education and communication?
- How might the advancement in technology worsen disparities in health?
Shifting the Focus from the Individual to the Interprofessional
The moderator of the session, Jack Kues, asked the first question: How might the models and methodology presented move from focusing primarily on individuals or individual professions to ones that center on teams and multiple disciplines?
Crump responded first by saying the mobile app she described is being tested as a tool for five different interprofessional teams in central Texas. These teams work in transitional care units for patients with chronic conditions, and they are led by a nurse practitioner. Each team includes a dietitian, dentist, social worker, and in some cases, a physician or physician assistant. The goal of this test is to assess the content and delivery of information from the team to the patient in order to determine how a long-term engagement of patients through a virtual connection affects patients’ behaviors.
Gawron then commented that her school is not tied to a medical center. And like other universities in this situation, providing robust interprofessional educational opportunities is a challenge. While she attempts to get more resources to her university, Gawron is using the work of others who have made their interprofessional education (IPE) curricula and assessment sources freely available. Because she does not have access to a medical center, her hope is to develop an IPE curriculum in the community rather than focus on inpatient care for training.
In Desai’s response to the question, he noted that some skills are best learned in person and not through the online instruction format set up by MOOCs like Khan Academy. Communication, leadership, and management are necessary elements to work as a team and are probably best taught in person. A common mistake by educators is to fill classroom time with didactic information, he said. Khan Academy is moving the didactic piece online so classroom time can be used for more experiential learning. But it is important that the online experience be engaging because a lackluster in-person presentation is just as uninteresting online. The quality of teaching and education matters, regardless of the medium used to transfer the information.
Forum member Pamela Jeffries from Johns Hopkins University School of Nursing commented that their school of nursing recently posted a MOOC for dementia care. Following some high-publicity advertisement, there are now 17,000 students enrolled in this MOOC that has not yet started. Had this same course been taught in the classroom, there may have been anywhere from 50 to 100 enrollees—nothing close to the 17,000 persons who signed up for the MOOC. Given the huge opportunity MOOCs offer, it would seem to be an excellent avenue for promoting IPE, but in her opinion, what is lacking is a more unified agreement over the required content for competency in IPE.
Gawron agreed that educating about IPE through MOOCs would definitely address the needs of smaller academic institutions that have limited capabilities for doing IPE. She speculated that observers could have a defined role in the online simulation activity. For example, other schools might have observers watching the Johns Hopkins IPE simulation activity who would communicate and debrief through online video conferencing. In this way, schools would share resources and innovative practices, and students as well as faculty could become more familiar with technologies used for education and improving communication.
She added that technology keeps young students interested and engaged. In fact, the younger generation is pushing the use of technology in new ways, such as showing patients relevant health care videos on their smartphones. These students are transforming the health care system and breaking down barriers to technology. Transformation and innovation are valued by Gawron and her colleagues, so they are now requiring all their students to have some proficiency of smart technology coming into the classroom. This poses no barrier for most of her younger students, said Gawron, but it does create challenges for some of the older learners coming back for a second career. However, given that education and care continue to move deeper into technology-based innovations, she feels these are critical skills all her students need to be successful at the institutional level now and in the future.
Kues commented that increasingly, interprofessional care does not mean that all the professions are physically in the same room. There are a lot of team skills being learned online by people that do not know each other. There are games being played by teams of people that have been working together for years in different parts of the world or different parts of the
country. They become a very good, tight, close team of integrated friends, even though they have never seen each other and probably never will.
Looking at this from an educational perspective, it is often thought that to have team-based education, one needs to figure out how to bring all the different health professional students into the same room at the same time. Those who work at academic health centers know that this does not happen easily. Students of different professions are in different places and have different schedules. One of the biggest challenges is achieving physical presence of all the team members. Using technology, Kues questioned whether it would be possible to develop team-based skills without learners ever seeing each other in person or if physical presence is an absolute requirement for interprofessional education.
At Khan Academy, said Desai, the staff use a tool called HipChat that is fairly well known in Silicon Valley. Essentially, it is a tool for creating virtual teams that can be accessed on a desktop. On his desktop screen, Desai has a tab linking him to a team working on analytics, another for a team working on website content, and a third for the team looking at the overall success of the entire project. Within seconds, Desai can stay connected with all three conversations taking place in the virtual space. Taking this example to a health care setting, Desai could imagine including a patient or including students as part of the virtual teams using a virtual communication device like HipChat. It could custom develop teams corresponding to the different components of the patient’s care. This could be especially useful for complicated patients that have several members on their health care team.
In fact, Khan Academy has tested a similar idea using teenage students acting as patients for learning purposes. In one example, Desai’s student “received” the drug isonicotinylhydrazine (INH) for treatment of latent tuberculosis. After watching online videos about the disease and the medications to treat it, the student commented that his liver function tests went up, but based on the video, his levels do not meet the threshold for stopping INH. He then guessed his liver function tests would need to be rechecked. According to Desai, this teenager understood the mechanics of his simulated disease and treatment; however, the challenge in a wider audience will be determining how to bridge the gap between the up-to-date scientific information available on PubMed and websites that provide generic information to consumers. Desai believes a site can be created where both health professionals and laypersons can go to obtain quality information.
One participant questioned the paternalistic mentality of many health care providers who still believe that patients do not need information about
their own health and health care. And how might one overcome institutional barriers to embracing new technologies?
In response, Crump cited a study by the Pew Research Center that found one in three adults in the United States have used the Internet for diagnosing health conditions over the past year (Fox and Duggan, 2013). Taking control of one’s care is certainly laudable, but many providers are frustrated by all the misinformation patients are downloading from the Internet. With the new app that Crump presented, providers control the content and format of the information patients receive, which can be written text or videos. The important piece is that the system is bidirectional, so patients and providers can ask questions. However, this raises several other issues for the provider, like whether all providers want to have that kind of direct connection with their patients. Another potential issue is how the providers may be reimbursed for their time corresponding with the patient, if the length of virtual communication extends the length of the patient visit. Although there are complexities with such a tool, Crump believes it is necessary to at least start the conversation so some of the challenges can be addressed and, it is hoped, improved.
After hearing Crump’s response, one participant asked whether the technology was just a communication device or whether it could be used for chronic care management like tracking blood sugar or monitoring blood pressure. If it is just about patient engagement, it still maintains the uneven relationship that was brought up in the previous question. Crump responded that it depends on how the provider or the team decides to use the tool. It is bidirectional, so providers could use it to monitor a patient’s condition. And although it could be interfaced with other systems like the electronic patient health record, it is ideally set up to start and maintain conversations around the data (like blood glucose readings) that could be supplied by the patient and shared with the provider or taken at the time of the visit and shared with the patient. What is unique about this system is it is mobile and it records how the team influenced patient care through their dialogue with the patient. Although it was not set up as an educational tool, Crump could see the usefulness of bringing students into the team to learn from the communication skills of professionals working on teams.
Desai was not aware of a tool that put the patient in control of his or her team, but he could envision such an instrument. The example he used was a patient with anxiety or depression. This patient requires a fairly intensive level of support that would not be possible for a busy provider. Instead, the patient could work with a coach to assemble her own virtual team that might include her mother, her husband, her care provider, and her best friend. It would be the patient who determines the team members who would help her follow her care plan, which might consist of meditating, going running every morning, attending yoga class twice a week, and
eating more salads. This, said Desai, could be a step in the right direction toward putting individuals in charge of their own health.
With all the discussion about technology, Forum Co-Chair Afaf Meleis from the University of Pennsylvania School of Nursing wondered whether these advances would create or exacerbate the present disparities in health. A large percentage of the population is illiterate, not computer literate, or does not have access to a computer, she said. As the health system moves more toward technology as the platform for providing care, are we creating even more disparities?
Although this is a valid concern, Crump also pointed out that almost 80 percent of U.S. households have Internet access, and this number continues to grow (Miniwatts Marketing Group, 2013). According to the Miniwatts Marketing Group, a similar trend can be found in developing countries where far less than half the population currently uses the Internet, but the percentage of users has grown exponentially since 2000 (see Table 4-1).
Despite these trends toward greater connectivity, Crump admits that
TABLE 4-1 World Internet Usage and Population Statistics: June 30, 2012
|World Regions||Population (2012 Estimate)||Internet Users Dec. 31, 2000||Internet Users June 30, 2012||Growth 2000–2012|
|Middle East||223,608,203||3,284,800||90,000,455||2,639.9 percent|
|North America||348,280,154||108,096,800||273,785,413||153.3 percent|
|Latin America/Caribbean||593,688,638||18,068,919||254,915,745||1,310.8 percent|
|World Total||7,017,846,922||360,985,492||2,405,518,376||566.4 percent|
SOURCE: Miniwatts Marketing Group, 2013.
moving into the future will require a variety of platforms to reach all the different populations living in different situations. Forum member Harrison Spencer from the Association of Schools and Programs of Public Health suggested testing the new technologies to see what works in changing educational and health care environments; but, he added, there needs to be greater tolerance for ambiguity as such technologies, like the ones presented, are tested in new environments. Crump agreed, saying that some level of risk has to be accepted; however, there is still a responsibility by the researchers to test those theories that are based on sound knowledge and information.
According to Desai, the mentality at Khan Academy is to be relevant and to get products tested even if they are not perfect. The idea is to change the tool in response to consumer testing. In this way, Khan Academy’s work and their products are ever changing and remain relevant to the changing needs of its consumers.
Desai also said Khan Academy is attempting to address some disparities by figuring out ways to get its hardware available in clinics so waiting room time can be used to educate patients. This is often a time when patients or caretakers are motivated to learn about health issues. Because language can be a major barrier, Khan Academy is translating its content into multiple different languages including Arabic, Farsi, and Spanish. Relevant content can be shown to patients during their sick or well-patient visit but also before and after the appointment. Accessing the video after the visit can be especially helpful in maintaining the accuracy of the information that might need to be shared with multiple family members or caretakers who were unable to be at the appointment. Desai admitted that a criticism of using Khan Academy videos for patient education is that their library is not complete. So, for example, a provider can direct her patient to a video on diabetes or asthma but there is not a similar video for arthritis. Staff at the Khan Academy are working on increasing their content but this is an impediment to pushing its use throughout all health systems although pilot studies are underway in a variety of health care settings to better understand the gaps this sort of tool could fill.
ANPF (American Nurse Practitioner Foundation). n.d. Patient engagement app. http://anpfoundation.org/tools-resources/patient-engagement-mobile-app (accessed February 16, 2014).
Creighton University. 2013. Successful nursing outcomes central to Creighton-developed evaluation tool. http://www.creighton.edu/publicrelations/newscenter/news/2013/october2013/october102013/nursingsimnr101013/index.php (accessed January 6, 2014).
Float Mobile Learning. n.d. Is mHealth poised to explode? http://floatlearning.com/wpcontent/uploads/2012/03/mhealth-infographic-float.jpg (accessed February 16, 2014).
Fox, S., and M. Duggan. 2013. Health online 2013. http://www.pewinternet.org/2013/01/15/health-online-2013 (April 2, 2014).
Miniwatts Marketing Group. 2013. Internet world stats: World Internet usage and population statistics: June 30, 2012. http://www.internetworldstats.com/stats.htm (accessed January 6, 2014).
Scott, R. 2012. 12 surprising facts about patient engagement. http://www.dorlandhealth.com/dorland-health-articles/12-Surprising-Facts-About-Patient-Engagement (accessed January 6, 2014).
TeleVox. 2011. A fragile nation in poor health: Realities about why so many Americans fail to follow their doctor’s orders; strategies for improving patient cooperation. http://www.televox.com/downloads/fragile-nation-in-poor-health-flip/#?page=0 (accessed February 16, 2014).
U.S. Census Bureau. 2012. Computer and internet trends in America. https://www.census.gov/hhes/computer/files/2012/Computer_Use_Infographic_FINAL.pdf (accessed June 16, 2014).