Multiuse Infrastructure for Continuous Learning
KEY SPEAKER POINTS
- Sustainability requires that learning become part of the fabric of care and that health care organizations embrace infrastructure support as essential to their survival, Patrick Conway said. Furthermore, he noted, that infrastructure needs to become part of the financial model in a way that enables investments in the research infrastructure that will drive continuous learning to improve population health and the efficiency of care delivery.
- Research works best, Brent James said, when it is integrated into routine care delivery at the level of data systems and organizational structure.
- An organization can foster a culture in which its leadership team and medical professionals work together consistently to create a virtuous cycle between research and operations to create value at speed, according to Scott Armstrong.
- Being a part of PCORnet is allowing institutions to pick up the pace of learning and improvement by catalyzing efforts to extract information and create synergy among the many data systems that exist at most large institutions, John Warner said. He added that PCORnet represents an opportunity to build the infrastructure needed to maximize return on the huge investment that health systems have made in EHRs.
- PCORnet should consider collecting cost data, Warner said. “You simply can’t measure clinical effectiveness without it,” he said. “If we are going to make the types of investment that are going to be required from our health systems into these types of research, we have to measure cost in a way that we can transparently provide the information to our patients and their families.”
- Armstrong said that cost data will be critical to determining which investments are providing the expected rate of return and which should be discontinued.
This session of the workshop included brief, prepared comments by panel members followed by moderated roundtable discussion among the panel and the workshop participants on the challenges and opportunities to establish and maintain the infrastructure for continuous learning, including but not exclusive to PCORnet. Members of the panel included Patrick Conway, the chief medical officer and deputy administrator for innovation and quality at CMS; Brent James, the chief quality officer and executive director of Intermountain Healthcare’s Institute for Health Care Delivery Research; Scott Armstrong, the president and CEO of Group Health; and John Warner, the vice president and chief executive officer of the University of Texas Southwestern Medical Center. Sarah Greene, a senior program officer with PCORI’s CER Methods and Infrastructure Program, moderated the discussion that followed the panel presentations.
Patrick Conway, Centers for Medicare & Medicaid Services
In a recent paper, Conway and two colleagues described four categories of payment (Rajkumar et al., 2014):
- Category 1: fee-for-service payments with no link to quality or cost
- Category 2: fee-for-service linked to quality and value
- Category 3: alternative payment such as ACOs, bundled payments arrangements, and advanced primary care models
- Category 4: population-based payments
CMS is trying to drive Category 1 payments to as close to zero and as fast as possible. Category 2 payments are the majority of the Medicare spend, Category 3 payments now account for approximately 15 percent and growing of CMS’s outlays, and population-based Category 4 is also
growing, said Conway. He noted that shifting more of the payments to higher-level categories in as short a time as possible will require the efforts of PCORnet and others. CMS is currently working on the goals for these four payment categories for the next 2 years, Conway said, and rapidly implementing change based on evidence will be critical to meeting these goals.
Doing population-based management of health care is challenging, Conway acknowledged, and although some systems do it better than others, even the good ones will admit that they are still learning how to implement change faster and more efficiently. CMS is currently working on developing performance metrics that states and communities can use to assess their progress in moving toward population-based health management. Noting the importance of linking a patient’s perspective with clinical data, Conway said he was excited to see that this is happening in the systems represented at the workshops.
The greatest challenge, Conway said, is creating a sustainable system starting with the infrastructure that PCORI is seeding with its funds. Sustainability, he said, requires that learning becomes part of the fabric of care and that health care organizations embrace infrastructure support as essential to their survival. For a pioneer ACO, for example, the infrastructure needs to become part of the financial model in a way that enables investments in research infrastructure that drives continuous learning to improve population health and the efficiency of care delivery.
Conway concluded his remarks by saying that he was fairly confident that the nation’s health care system is heading in the right direction, both in terms of quality and cost. The pace of change, however, worries him. “If we don’t get the pace right, I worry that we are going to end up with those blunt cuts that actually harm Americans,” he said. “The question is, can we accelerate change through PCORnet and other innovations? Can we accelerate that pace of improvement so that we really get those population health outcomes at lower cost at scale across the nation?”
Brent James, Intermountain Healthcare
A couple of years ago, James said, he and Intermountain Healthcare’s vice president for strategic planning conducted an internal analysis that showed that some 70 percent of the care the company provides is “quasi-capitated,” and as a result, Intermountain Healthcare launched a series of major initiatives that are now integrated with inpatient and outpatient care. The fruit of these initiatives is a reduction in hospital volumes that exceeded projections and a few years of the best financial performance the system has ever experienced. “The models do work, and they work well,” James said. He noted that trying to predict when the transition from fee-for-service to fee-for-performance will occur is like trying to time
the stock market, so the best approach is to assume the risk and start managing care today.
Intermountain Healthcare started this process in 1996, James said, and the health system currently has some 60 longitudinal, disease-specific registries running in its enterprise data warehouse, with data pulled from many sources, including the EHR. Those 60 registries account for approximately 80 percent of the care that Intermountain Healthcare provides, thereby enabling the health system to truly manage care on a large scale. He explained that determining what data to collect in the EHR is a critical factor in developing a system that can lead to learning and improvement. Once the data systems are running, it is important to develop an organizational structure that promotes champions for specific projects who are paid to own projects and oversee them as their full-time job.
At Intermountain Healthcare, Level 1 research is aimed at answering questions that will have a relatively immediate impact on the company’s care delivery performance. This covers what James characterized as a tremendously broad and interesting set of research questions, often around competing treatments for the same condition, and answering these questions has proven to be a fruitful area of research that generates more peer-reviewed publications than most academic medical centers generate. “The reason it works is that it is integrated into routine care delivery at the level of data systems and organizational structure,” James said. He added that Intermountain Healthcare has a master data agreement that took 15 months to develop and cost approximately $500,000 but which, once in place, enabled studies to start within days of the study being conceived. He concluded his comments by saying that the process to select topics for research should depend on organizational priorities.
Scott Armstrong, Group Health
Group Health, Armstrong said, is a 1,200-physician group practice with employed physicians and a series of medical centers across Washington State and northern Idaho that covers about 600,000 insured patients. Unlike Intermountain Healthcare, Group Health is 100 percent capitated and about 95 percent of the system’s revenues come from premiums, with the rest coming from co-pays. Because of this structure, Armstrong said, Group Health, with input from the Group Health Research Institute, is a laboratory for aligning the entire organization, from health plan to care delivery, around population health and total outcomes for all of the system’s patients. Armstrong also noted that Group Health has a cohort of patients who have been members since 1947, offering what may be an unprecedented longitudinal health record.
Group Health operates as a 501(c)(3) not-for-profit HMO with a board elected from among the organization’s members. This organizational structure results in a high level of accountability to patients, but it also creates an expectation among members that they themselves play a critical role in achieving desired outcomes. Armstrong credits this two-way accountability for Group Health’s early adoption of EHRs and almost immediate access to those records by patients through any computer. He also said that the organization’s culture is such that its leadership team and medical group consistently work together on advancing the virtuous cycle between research and operations to create value at speed. Group Health’s culture of continuous learning has created value in the marketplace through lower costs, confidence in the care delivery experience, and quality as measured in a number of ways. As an example, he cited the way in which patients, clinicians, and accountants worked together to build a primary care medical home model in one of the organization’s 50 medical centers. “To me, the value of this was that we could get it up and running,” Armstrong said. “We could check how it was working and make refinements. At some point, I had confidence that I would not have had otherwise to very quickly expand that model to our other 50 medical centers at a pace that we would never have moved, if it weren’t for this kind of relationship.”
John Warner, University of Texas Southwestern Medical Center
The University of Texas Southwestern Medical Center (UT Southwestern) is a large, complex university health system that not only operates the university hospitals and clinics, Warner said, but also provides faculty who account for more than 90 percent of the physicians at the Parkland Health and Hospital System. UT Southwestern Medical Center also provides faculty physicians who staff both Children’s Medical Center in Dallas and the Dallas Veterans Administration Hospital. Furthermore, UT Southwestern delivers some component of the medical training of more than 50 percent of the health care providers in Dallas as well as offering training to the lay community. As a result, Warner said, his organization has the opportunity and obligation to provide a learning environment not just for its campus, but for a city and a region.
UT Southwestern, Warner told the workshop, has had a very positive experience thus far working with PCORnet as a participant in the Greater Plains Collaborative CDRN. According to an informal poll that he conducted just prior to the workshop, staff who have worked with the network reported that their experience has been that being a part of PCORnet allows the organization to pick up the pace of learning and improvement. One reason for this, he said, is that being part of PCORnet has catalyzed an effort to create synergy among more than 70 internal and external data registries,
including its EHR, used at UT Southwestern. “Being part of PCORnet has allowed us to more effectively submit and extract data in both internal and external registries,” Warner said. He noted that no one medical center or hospital has enough information technology staff to do this alone. “Having partners in this 10-hospital system collaborative network,” he said, “has helped us move the needle more quickly, allowing us to build the infrastructure needed for us to extract data; to take our existing data, compare it with other research databases; and to answer new questions so we can begin to look at value and care improvements in a very different way.”
Concerning the opportunities that PCORnet creates, Warner said that he would like to see cost added to the data being collected. “You simply can’t measure clinical effectiveness without it,” he said. “If we are going to make the kind of investment into the types of research that will be required from our health systems, we have to measure cost in a way that allows us to transparently provide the information to our patients and their families.”
To start the discussion, Greene asked the panel members how each of them can put research at the top of their priority list, given all of the other demands for time and resources that they face in serving the good of the entire organization. James replied that the entire purpose of the research institute that he runs at Intermountain Healthcare is to answer questions for the system’s administration that will allow the administration to make good decisions, and it is why the institute is 100 percent internally funded. As an example, he said that the administration was trying to decide whether it should put psychologists in its primary care clinics based on the argument that doing so would reduce utilization enough to pay for extra nurses. A prospective cluster randomized trial showed that argument to be true, enabling administration to make an evidence-based decision. James then seconded Warner’s call to track cost as part of the learning health system, something that Intermountain Healthcare has been doing for years.
Warner, James, and Armstrong all commented that they appreciate the value that research creates because of what they hear from the clinical service lines, from the nurses and physicians who are providing care, and from patients. What is critical, James said, is that research lead to deployment. “If you have got the structure in place with the data and the organizational structure, a natural consequence of doing the investigation is you have got a deployed system,” he said. Armstrong added that the question that he wants to answer now, with all of the data systems in place, is which investments are not providing the expected return. “Quite frankly,” he said, “I think we are carrying enormous costs that aren’t giving us the return that we used to get.”
Conway added the perspective of a policy maker who has compiled data on the percentage of decisions that he makes with the help of data. He said that when he first started at CMS, that figure was about 2 percent, and while it has now risen to somewhere between 40 and 50 percent, it has been stuck there for 2 years. He said that he has come to realize that the problem is not that he and others are not looking hard for the evidence but that the data and evidence just do not exist.
Steven Lipstein of BJC Healthcare asked if the panelists were concerned that the desire to generate value with speed would lead to unintended consequences that would actually harm certain segments of the provider and patient communities. Armstrong replied that there is a balance that needs to be struck but that, overall, the health care industry is grossly conservative and far too slow to change. The key is to protect patients in terms of safety and quality, Armstrong said, and the industry should be proud that it does that well, but it needs to accelerate the pace of change. Conway agreed that there is a balance between risk and benefits, and while he said that he does not know the right answer, he said the pace of change today is not too quick, and he worries more that the opposite is true. He added that focusing on things that have no unintended consequences will not enable the system to transform itself at the pace that the nation needs it to occur.
Robert Kaplan from AHRQ asked the panelists if they had questions that they would not try to answer with PCORnet and for which they would demand data from an RCT before enacting change. James said that there is a set of principles that can determine which trial method is appropriate and that he and his colleagues use a full range of designs, the choice of which is determined by the question that needs answering. He did note that there is technical body of work using a formal empirical evaluation of internal validity within a study, known as the confidence profile methods, with which James said he can design a good quasi-experimental study that will produce higher internal validity than most RCTs. He estimated that Intermountain Healthcare does use RCTs for about 1 of every 20 studies it runs. “We tend to use as rigorous a design as we can do in a reasonable length of time that matches the need and the circumstances of the data,” he said.
Selby agreed with James that observational data in PCORnet can complement RCTs, which are costly and thus limited in the number that any health system can afford to run. But aside from that, PCORnet can facilitate more efficient RCTs when such trials are indicated. Selby pointed to the REDUCE MRSA trial as a good example of how the two types of trials are complementary, noting that there was a randomized trial followed by observational studies to watch the impact of the interventions. He said that the PCORI methodology committee is spending time on this very topic.
Jonathan Tobin from the Rockefeller University Center for Clinical and Translational Science asked if it would be possible to systematically
design a series of RCTs that had parallel registry studies conducted, either in the same settings or in similar settings, in such a way that it would be possible to generate answers and effect sizes from the randomized trials and effect sizes from the observational studies. The results could provide some insights into when it is necessary to invest in an RCT and when questions can be answered with registries. Warner said that in cardiology, his area of practice, this type of comparison has been used often to look at practice variation, and it has proven to be a good approach.
Sean Tunis from the Center for Medical Technology Policy said that the sustainability challenge for PCORnet is to enable and demonstrate the value of observational studies that are conducted with greater rigor and higher-quality data than is the case with the typical quality-improvement study. He wondered if there is a business case to be made for investing in a system that does raise the quality and reliability of observational studies. James said that there is a business case to be made, but that it is necessary to match the method to the problem. Warner acknowledged Tunis’s point and said that it is still an open question whether UT Southwestern has realized the full potential of its decade-long investments in its EHR. He said that health care systems need to expect more from their EHRs and that PCORnet represents an opportunity to build the infrastructure to extract more meaningful data from EHRs. Greene added that creating a sustainable public good is also going to require broad patient engagement.
Joel Allison of the Baylor Health Care System said that there is a general lack of individual accountability with respect to health decisions and asked how the health care system can change patients’ behavior to become more engaged in personal accountability. He also expressed concern about the role of health literacy in changing behaviors and asked if there is ongoing research that addresses issues involving health literacy. Warner agreed that it is important to understand the impact of health literacy on the decisions that patients make and said he believes that the requirement for PCORnet members to pay attention to public engagement is a positive step toward gaining that understanding. Armstrong agreed that health care systems need to do a better job of using their patient-centered relationship to explore how patients understand the information they are given and the consequences of the choices they make outside of the exam room. He added that Group Health is involving patients in system change in a way that asks them if processes are designed in a way that increases the likelihood that they will pay attention.
Armstrong also said that Group Health is taking advantage of the fact that some 15,000 employees and their families are also covered by Group Health’s insurance plan. These employees are given the opportunity to receive up to $750 in premium discounts based on the beneficiary’s improvement in body mass index, blood pressure, and other health screening tests.
Group Health has partnered with Kaiser Permanente Colorado to serve as a control to enable a true assessment of the impact of financial incentives to advance this kind of engagement on the part of the beneficiaries. “I don’t know how that translates into an agenda for PCORnet, but I think it is an area that we need to be investing much more time and attention in,” Armstrong said. Conway added that CMS’s Innovation Center is considering broadening its research portfolio to look at ways of engaging consumers and patients in a real way with the decision-making process.
Rachel Hess from the University of Pittsburgh commented that she was impressed with how operational motivations have been embedded within a research infrastructure. James said that this has been an intentional development to achieve both performance improvement and learning at the same time. He noted that any time he and his colleagues launch an investigation—even a rapid-cycle quality-improvement project—they always add a little bit of data to the system, which lowers the overhead for the system and eventually speeds the pace of change.
Kenneth Mandl from Boston Children’s Hospital asked for the panel’s input on the part of the virtuous cycle that is involved with returning evidence back to the point of care, particularly with respect to criticisms of the rigidity of EHRs. James said that there are a number of applications designed to work on top of the EHR and that are organized around the continuum of care. He noted that when systems are developed to capture data in an EHR that are concerned with creating an effective care management system, those data turn out to be the data that are needed to run trials effectively. What disappoints him, he said, is that very few people think about this in advance when building their systems. Conway said that what is important is to think concurrently about both the questions that need answering and the deployment of the results, and Armstrong added that systems have to be bidirectional in that they have to be pushing information out while they are receiving information.
This page intentionally left blank.
