National Academies Press: OpenBook

Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary (2007)

Chapter: 3 Barriers to Quality Improvement and Quality Improvement Research

« Previous: 2 Approaches to Quality Improvement Research
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

3
Barriers to Quality Improvement and Quality Improvement Research

In the early 1990s, a number of hospitals created rapid-response teams or medical-emergency teams to identify and intervene early in the care of clinically critical patients. The promise of these teams was that they could help provide better care to many of a hospital’s most at-risk patients, and in some instances the rates of cardiac arrest, post-surgical complications, and overall mortality were shown to have improved, at least informally. Over time, rapid-response teams became increasingly popular. In 2003 there were about 100 such teams in U.S. hospitals, and by 2005 there were a couple of thousand teams, said Frank Davidoff of the Institute for Healthcare Improvement.

This widespread adoption, however, was not accompanied by a strong evidence base. Between 1992 and 2004, Davidoff said, only 17 reasonably creditable accounts of rapid-response teams were published. In fact, the strength of evidence regarding the effectiveness of rapid-response teams remains only moderate. The question that must be raised, Davidoff commented, is this: Why has the evidence taken so long to develop?

There is very little data available to guide the development of quality improvement research, of health sciences research, and of medicine in general, stated Harold Pincus of Columbia University and New York-Presbyterian Hospital. The lack of data, coupled with the insufficient development of the basic science of quality improvement research, Jeremy Grimshaw described, has led to a situation

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

where quality improvement has not been guided by evidence-based learning. Quality improvement has been based largely on experiential learning, but this knowledge has yet to be adequately captured in the literature. The spread of ideas from experiential learning has thus been poor.

This session of the workshop was devoted to discussing the barriers to quality improvement and, in particular, to developing an evidence base for use in quality improvement and in quality improvement research.

BARRIER OF FOCUS

Many workshop speakers emphasized the need to concentrate on the particular purpose of quality improvement projects and research. Davidoff noted that quality improvement efforts can have many divergent purposes. Some believe the purpose is improving performance, a process that occurs mainly through experiential learning and which differs significantly from scientific research, whose purpose, Davidoff noted, is to discover generalizable truths through hypothesis testing.

The emphasis on experiential learning that has evolved may lead to the conclusion that many of those doing quality improvements are uninterested in studying and writing about their experiences, Davidoff said. For them, discovering the generalizable truths about the efficacy and effectiveness of quality improvement interventions may be largely a secondary consideration.

THE ROLE OF CONTEXT

Understanding specific contexts and what is generalizable across settings is extremely valuable in the implementation of interventions, Grimshaw said. He also stated that work in the field attempts to generate evidence while considering context and its effect on processes.

If local contexts are not considered, then the lessons learned from interventions will not be generalizable and will fail to improve the health care system, Batalden cautioned. For example, the practice of health care policy is local, while the policy of health care is not, Batalden added. The local uptake of health care policies, thus, must be considered when working to improve care. Active research and knowledge development are needed, both locally and across local settings.

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

WHERE DOES IT BELONG?

Contributing to the infrastructure problem is lack of an agreed academic area for where quality improvement research should be taught, researched, and developed, noted Pincus. While the field could be part of schools or hospitals, it is unclear what school—i.e., public health, medicine, or nursing—or department would be most appropriate to hold it. One answer might be that health care quality improvement research should be considered an interdisciplinary research field, serving as a bridge between multiple disciplines. As an interdisciplinary field, Pincus suggested, it could potentially become a discipline of its own, following the precedent set by other fields, such as neuroscience.

But while many agree in theory with the concept of interdisciplinary research, it is extremely difficult in practice, Pincus said. Disincentives to interdisciplinary research outnumber incentives. In particular, Pincus identified three disincentives to interdisciplinary research: conceptual, procedural, and structural and financial. The conceptual barriers include the lack of common understanding and language across different disciplines. “When people with completely different scientific backgrounds get together to solve a common problem, they have to learn a different way of speaking, a different language,” Pincus quoted Nobel Prize Winner Alan MacDiarmid. Procedural barriers include disincentives in career development, for example the time it takes to learn about all these various areas of study. The inherent departmental nature of academic medicine raises questions about whether a department will sponsor faculty who receive grants technically falling into other departments. This, along with issues concerning how indirect costs will be shared, are examples of structural and financial barriers. All of these, Pincus said, must be considered in identifying strategies for enhancing and expanding quality improvement research.

RESOURCE BARRIERS

Limited data exist about the resources allocated to health care quality improvement. According to the Coalition for Health Services Research, an estimated $1.5 billion of federal funding was spent on health services research in fiscal year 2006 (Coalition for Health Services Research, 2006). In another study, about 1.5 percent of 2002 biomedical research funding was in health services and policy research, equating to less than 0.1 percent of total U.S. health care expenditures (Moses et al., 2005), cited Pincus. Because these statistics refer to funding for all of health services research, not just

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

quality improvement research, even less is actually spent on understanding what works to improve performance. In countries outside the United States, Grimshaw said, the situation is no better, with funding policies that are often insufficient and inconsistent.

A related issue, Pincus said, is stakeholders’ lack of motivation to build the infrastructure that is needed to attract health services researchers. The main sources of funding for health services research include several federal agencies such as the Agency for Healthcare Research and Quality (AHRQ), the U.S Department of Veterans Affairs (VA), and the Centers for Medicare and Medicaid Services (CMS), foundations, and to a lesser degree, the National Institutes of Health (NIH). Very little funding comes from industry (i.e., hospitals, insurance, and pharmaceuticals) and voluntary health organizations, Pincus said. Thus it is important to develop strategies for diversifying funding sources and enhancing contributions from both industry and the NIH.

During the discussion, Richard Kahn of the American Diabetes Association (ADA) noted that as a voluntary health organization, the ADA does not receive many applications for quality improvement research grants. The applications that it does receive tend not to be well put together and are not of highly sophisticated research designs. Despite the general feeling that the organization awards few grants for quality improvement research, Kahn said that more funding would be provided if better applications were received.

BARRIERS TO RECRUITMENT AND TRAINING

Davidoff characterized the following “mismatch” between training and practice: Most people doing medical quality improvement projects have little or no research training, while most people with research training are not doing quality improvement projects. Furthermore, he observed, few people know how to study quality effectively. Quality improvement research is unfamiliar to most practitioners, mainly because quality improvement is, at its core, more a social process of behavior and organization change than a biological or physical process.1 This mismatch is a large barrier to improving the state of quality improvement research, Davidoff said.

1

The roles of behavior and organization change are extremely relevant to the understanding of quality improvement and implementation science, but because of the scope of the workshop, discussion of these issues was limited.

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

Recruitment

The science behind recruiting and training researchers to study quality improvement is not well understood, Pincus said. There is some evidence that such strategies as involving people in research during medical school or before, undertaking full-time research fellowships, protecting faculty time, and training people in research-intensive departments can help produce successful researchers, Pincus said. In particular, exposure to research experiences is critical to the recruitment of future researchers.

Another issue is how potential quality improvement and patient safety researchers should be recruited into an interdisciplinary field. Pincus likened it to marketing and the strategy of market segmentation. Different strategies are needed for involving and recruiting different audiences—undergraduates as opposed to residents and post-doctoral students or health professionals versus those in fields outside of the health professions.

The problem of ownership presents yet another barrier to recruitment. It is often difficult to attribute ideas and quality improvement interventions to one specific person. Additionally, rewarding a single person for an idea may not be appropriate because quality improvement has to become part of the culture, and therefore belongs to everyone, noted Jay Berkelhamer of Children’s Healthcare of Atlanta. This is a problem, and reward systems are yet to be built and may indeed reward multiple people, Davidoff agreed. Pincus also noted the trend of moving toward a “team science” approach.

One further difficulty is building a critical mass of interested people, Davidoff said. Although it is not clear whether a critical mass has yet been reached in quality improvement research, there are at least some examples of movement toward that goal. For example, Davidoff said, the Institute for Healthcare Improvement’s annual meetings gathers around 6,000 people, and both Batalden and Davidoff commented on the number of residents they have seen who are interested in this work. Quality improvement is now also on the agendas of many medical specialty certifying boards, said David Stevens of the Association of American Medical Colleges. Andrea Kabcenell of the Institute for Healthcare Improvement commented that getting involved in quality improvement needs to be made more democratic and accessible. However, recruitment is confounded by the problems of publication and lack of career opportunities.

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

Training

The types of training needed to be successful in quality improvement have not been specified, but before evidence can be accumulated on the issue, Pincus said, it will be necessary to develop an infrastructure to train and develop people to go into the field of quality improvement research. There are a few models for this type of training, such as the VA Quality Scholars Program and the Robert Wood Johnson Foundation’s Clinical Scholars Program. Curricula should be developed, Pincus suggested, by focusing on those skills currently believed to be important for quality improvement researchers. Values, mentorship, research opportunities, and flexibility must be part of the environment provided. People should also be taught practical skills, such as tips on conducting successful research and receiving grants. And, Pincus said, if quality improvement is to be treated as an interdisciplinary field, special attention should be given to the criteria for how promotion and tenure should be executed.

Quality improvement research involves not only those researchers who will become principal investigators but also many other professionals, such as clinician educators and administrators, whose roles and development must also be considered. When developing training strategies, professionals from other disciplines should be included. Early recruitment and proper training are well-supported strategies, but difficult to implement, Pincus said.

LEVERS FOR STRENGTHENING QUALITY IMPROVEMENT RESEARCH

Pincus identified three levers to improve quality improvement research. The first is humanistic—that is, the research will ultimately result in better care for patients. The second lever is in the policy arena. Policy levers, such as accreditation or payors providing matching funds for quality improvement research, must be utilized. Quality improvement research can also be leveraged by strategies focused on individual researchers, for example, providing salary support for protected research time or altering tenure and promotion policies to respond to the special barriers of interdisciplinary research.

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

BARRIERS TO PERFORMING QUALITY IMPROVEMENT AND QUALITY IMPROVEMENT RESEARCH

Ethics

Ethical oversight in quality improvement remains largely ambiguous. For example, Davidoff, citing the work of continuing education expert Philip Nowlen, said that what distinguishes professionals from other people is “the obligation of professionals to ‘move unceasingly toward new levels of performance.’” From this perspective, quality improvement can be seen as an intrinsic element of clinical care. Others, however, believe quality improvement to be a form of clinical research, which raises the question of whether quality improvement research is human-subjects research. This is an important question because human-subjects research requires ethics review and institutional review board (IRB) approval.

The purpose of IRB approval is not to decide whether clinical care is ethical, Davidoff said, but the prospect of undergoing IRB approval, which can be extremely slow and inconsistent, has deterred some people from studying quality improvement. At the heart of this issue is determining whether a project falls under the rubric of quality improvement, which would not be subject to an ethics review, or whether it is research that would require ethics review. Currently, the distinction between these types of projects is not well delineated, Davidoff said. Constructs need to be developed that can help sharpen the distinction.

One member of the audience brought up the issue of confidentiality, asking how the Health Insurance Portability and Accountability Act (HIPAA) impedes researchers’ abilities to collect data. Davidoff responded that HIPAA does not prevent quality improvement research from being conducted, although there are many rules that need to be followed, referencing the more complete discussion of this issue in a report from the Hastings Center (The Hastings Center, 2006).

These concerns argue, Davidoff said, that it would be best if the health care system itself developed ethical guidelines for quality improvement instead of allowing the task to be subsumed by the administrative structure responsible for clinical research ethics.

Methodology

Methodological differences between the biological sciences and the social sciences offer another barrier, Davidoff said. Quality improvement research faces the same challenges—such as biases,

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

confounders, and difficulties with measurement—that clinical research does, but there are also methodological problems specific to quality improvement research. For example, clinical interventions are often studies of efficacy that are conducted in highly controlled environments with rigorous population sampling and close monitoring of response rates. By contrast, quality improvement studies are not performed under such tightly controlled conditions. This is by design, Davidoff noted, but it makes it difficult to know if “proven interventions” in quality improvement research are generalizable and actually yield improved outcomes.

Another common challenge in methodology, which is compounded in quality improvement studies, is containing the intervention. As Davidoff explained, if individual participants or small sections of single large institutions are the units of study, they need to be prevented from sharing ideas with others in the organization to the greatest extent possible, so as not to “contaminate” the trial. This is extremely difficult to accomplish; an alternative, Davidoff said, is to let the unit of study be entire organizations (cluster trials), but that gives rise to another problem: that of heterogeneity. In this approach, all types of care settings—large, medium, or small; teaching or nonteaching; rural or urban—would be included in the trial. This may increase generalizability, but it would also decrease the internal validity of such a trial.

Such methodological problems have caused many studies of quality improvement to be methodologically flawed, Davidoff concluded.

Publication

Publication is seldom seen as an essential element of quality improvement, Davidoff said, because quality improvement studies are often dependent on local context and do not identify and share generalizable truths. Furthermore, Grimshaw said, much of what is published is poorly reported. This stems from a lack of writing experience by those doing quality improvement work, Davidoff suggested. When writing about complex systems and social processes, the need for writing experience becomes even more pronounced. Unfortunately, there is limited guidance as to how published articles documenting quality improvement efforts should be structured. One exception is an article written by Davidoff and Batalden that proposes guidelines for how write-ups of quality improvement studies should be structured in an effort to improve them in the eyes of reviewers, editors, and readers (Davidoff and Batalden, 2005).

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

Journals should begin to be more active in and receptive to quality improvement research, which would in turn help stimulate interest in the field by future researchers. Journals need to begin to rethink some of what is published, Davidoff said, offering the section in Annals of Internal Medicine on improving patient care as an example.

Collectively, these issues of role and structure result in a relatively small amount of learning being published, Davidoff said, and much of what is published is not generalizable and so fails to provide a basis upon which to base future efforts.

Communication

The lack of a common vocabulary for quality improvement and implementation research terms is a barrier to further progress. This is compounded by the fact that frameworks for how this research should be approached are not widely known. The result, Grimshaw concluded, is that those doing research in these areas have difficulty communicating with each other, which contributes to the problem of studies not building on previous findings, as discussed previously. The difficulties are augmented when the research is performed in an interdisciplinary setting, Pincus added.

BARRIER OF SUSTAINABILITY

Scot Webster spoke of the important role that culture change plays in improving quality. In particular, Webster noted Medtronic’s culture of grass roots, bottom-up sustainability. Due to this corporate culture, individual employees and units are able to initiate improvement projects on their own. If the employees did not believe in quality improvement as part of their culture, sustained change would not occur, Webster said.

Marita Titler observed that overcoming problems with employee engagement requires addressing the false notion that interventions and improving the quality of care do not affect employees. This means that employees must understand why interventions are important. Otherwise, interventions are at risk of being seen merely as additional short-term projects adding to workloads, and not as priorities. People have to believe in the improvements, not just see them as short-term solutions, Webster agreed, adding that culture change and change management must be included as areas of focus.

Titler also emphasized the barriers caused by problems at the system level. System-level issues that can potentially detract from

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×

advances in quality improvement include accounting for change, managing competing demands, and understanding that change is complex. Purely mechanistic approaches for change in complex systems are often inadequate; instead, more complex, adaptive approaches may be necessary. These barriers must be addressed in order to induce change, Titler concluded.

Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 27
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 28
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 29
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 30
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 31
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 32
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 33
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 34
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 35
Suggested Citation:"3 Barriers to Quality Improvement and Quality Improvement Research." Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/11884.
×
Page 36
Next: 4 Opportunities »
Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Institute of Medicine's Forum on the Science of Health Care Quality Improvement and Implementation held a workshop on January 16, 2007, in Washington, DC. The workshop had its roots in an earlier forum meeting when forum members discussed what is meant by the terms "quality improvement" and "implementation science" and became convinced that they mean different things to different people. At the time, the members also discussed the need to identify barriers to quality improvement research and to implementation science. Thus the purpose of this workshop was to bring people together from various arenas to discuss what quality improvement is, and what barriers exist in the health care industry to quality improvement and also to research about quality improvement.

The summary that ensues is thus limited to the presentations and discussions during the workshop itself. We realize that there is a broader scope of issues pertaining to this subject area but are unable to address them in this summary document.

The workshop's first session was devoted to experiences that various institutions have had with quality improvement. Recognizing the wealth of experiences available outside of health care services, the workshop included presenters from outside the health care service industry as well as from inside. This includes discussions from a variety of perspectives: non-health care services, health plans, hospitals, and nursing. Advancing Quality Improvement Research: Challenges and opportunities - Workshop Summary describes the events of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!