7
Strategic Opportunities

This panel was designed to provide reactions to the previous speakers from the perspectives of barriers to quality improvement research.

ETHICS

Jeffrey Cohen of HRP Associates, Inc., discussed the questions surrounding quality improvement research and the ethics of human subject protections. In most institutions, quality improvement research will face ethics review by IRBs, which make decisions about research by bringing together ethical principles and regulations. The basic principles governing human subject research and IRBs are respect for persons, beneficence, and justice. These principles, however, do not necessarily govern quality improvement research because it is not evident that much work has been done to determine what the ethical principles underlying quality improvement research are, Cohen said.

IRBs make decisions about what must be reviewed by considering the definition of regulations as inclusive of two parts: research and human subjects. Research was defined as “a systematic investigation designed to develop or contribute to generalizable knowledge,” and a human subject as a “living individual about whom an investigator conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary 7 Strategic Opportunities This panel was designed to provide reactions to the previous speakers from the perspectives of barriers to quality improvement research. ETHICS Jeffrey Cohen of HRP Associates, Inc., discussed the questions surrounding quality improvement research and the ethics of human subject protections. In most institutions, quality improvement research will face ethics review by IRBs, which make decisions about research by bringing together ethical principles and regulations. The basic principles governing human subject research and IRBs are respect for persons, beneficence, and justice. These principles, however, do not necessarily govern quality improvement research because it is not evident that much work has been done to determine what the ethical principles underlying quality improvement research are, Cohen said. IRBs make decisions about what must be reviewed by considering the definition of regulations as inclusive of two parts: research and human subjects. Research was defined as “a systematic investigation designed to develop or contribute to generalizable knowledge,” and a human subject as a “living individual about whom an investigator conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary information.” Assessing whether quality improvement research is actually “research” may be a problem because the definition states that investigations must contribute to generalizable knowledge, Cohen said. Communication is necessary between IRBs and quality improvement researchers to discuss this issue. The second question is determining whether quality improvement research is human subject research. The definition uses the term “about whom,” which begs the question of who subjects are in quality improvement research. In many instances it could be argued that the subjects are institutions and systems, while others target individual patients. Quality improvement researchers should clearly identify subjects in discussions with IRBs to determine whether IRB approval is necessary. A better framework explaining what quality improvement research is and what the research model is should also be clearly communicated to IRBs. Cohen agreed with Tierney’s point that quality improvement researchers should become members of IRBs to help facilitate this dialogue. IRBs have the flexibility to efficiently and effectively review nonbiomedical research. Some believe the regulations do not match social science research, but Cohen does not believe regulations to be problematic for social science or quality improvement research. Having separate IRBs focus on just quality improvement is a highly contested issue as well as a problem in social and behavioral research. Although there is some merit to separating ethics review for biomedical research and other types of research, the amount of overlap in protocols makes it difficult to separate the research. Another barrier to separating into different IRBs is expertise because there is often not enough expertise to fill multiple IRBs. IRBs must have sufficient knowledge and expertise in order to rule on quality improvement research. Although this creates a burden for IRBs, there is also a burden on quality improvement researchers to develop clear research frameworks and ethical standards as well as to educate IRBs, Cohen said. RESEARCH TRAINING Evidence-based medicine and evidence-based management need to work together to sustain improvement in quality of care, said Steve Shortell of the University of California, Berkeley. Evidence-based medicine can be defined as using the best available evidence in making treatment decisions, interventions, and technologies to improve care for patients. Evidence-based management draws on the social and behavioral sciences, human factors engineering, and

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary health services research for managers to make the best decisions with a given allocation of resources to be used to, for example, make organizational changes and implement different strategies. The gross underuse of evidence by both clinical practitioners and managerial leaders can be partially addressed in how future leaders are educated and trained. Because the health care system cannot afford to wait until the next generation to change, Shortell said, health care workers should take advantage of the many opportunities to work with clinical and managerial leaders. Some opportunities include designing short courses to educate leaders, encouraging team-based learning, and developing online distance learning. To make information gleaned from meta-analyses and synthetic review more accessible and to train people how to use that evidence appropriately, Shortell suggested the development of a National Center for Evidence-Based Health Care Management, following the examples of similar centers in Canada and the United Kingdom. This center would identify the best evidence about effective strategies found in the organizational behavior and social science literature and would be shared with clinical and managerial leaders in a meaningful manner. These leaders would, in the long run, be able to integrate evidence with efforts taking place in evidence-based medicine. Researchers and leaders must be trained in process improvement skills, Shortell said. This includes becoming more interdisciplinary, perhaps asking engineers and psychologists to discuss human factors engineering approaches, change management, conflict management, and culture management. Education of future leaders begins with targeting current students of medicine, nursing, pharmacy, public health, and health services management. These students should be trained in the above-mentioned disciplines, but also should be exposed to evaluation research and study design. The role of accrediting bodies such as the Association of American Medical Colleges could be leveraged. Training of researchers can be divided into two parts: clinical and social science. Likening quality improvement research to health services research, Shortell identified the need for students to study epidemiology, biostatistics, study design, ethics, and cost–benefit and cost-effectiveness analyses. Approximately 20 to 25 years ago, the Robert Wood Johnson Foundation Clinical Scholars Program trained physicians in health services research. At the time, the field had not been widely developed, but as a consequence of those scholars’ continued support and interest in health services research, that field is much better defined today. Students should be trained to

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary triangulate different methods, theories, and concepts. For those students in interdisciplinary doctorate programs, the emphasis is on social science backgrounds with some clinical background. Shortell believes these students should have not just a broad understanding of many subjects, but also in-depth training in one discipline, which is extremely useful when working with multidisciplinary teams. This additional training does not have to span a number of years and can be taught in intense focused sessions, and the skills would be reinforced on the job. The notion of learning organizations needs to be built on to fundamentally change practices. Shortell offered some first steps about resource needs. For example, development of awards for investigators and postdoctoral students as well as career development in quality improvement research would encourage both current and future researchers. Additionally, fellowships in quality improvement research would also improve engagement. Highlighting a recommendation from a joint National Academy of Engineering and IOM report, Shortell suggested the development of approximately 10 centers for engineering and improving health care delivery. These centers would house interdisciplinary groups of engineers, health services researchers, social scientists, managers, and clinical and managerial leaders to work together in certain settings to work on these largely interdisciplinary issues. Development of these centers would be a great opportunity to train researchers, Shortell said. The United States produces very competent providers of health care who are equipped with 21st-century knowledge of biomedicine and technologies. Those providers are then turned over to a delivery system largely still in the 19th century. Quality improvement is one of the most apt tools to bridge that gap. PUBLICATION General Major medical journals can have great impact with their broad circulation, media coverage, and website hits, said Cathy DeAngelis of the Journal of the American Medical Association (JAMA). JAMA is the most widely read medical journal in the world, with 365,000 print circulation and a million hits online every week. Because of this high visibility, JAMA is extremely selective in the articles it publishes. Journals are rated by a measure called impact factor, which divides the number of citations from the journal over a 2-year period (the numerator) by the number of articles of original research and

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary others (the denominator) calculated by Thompson ISI, DeAngelis explained. The denominator is not a clearly defined calculation, but it does not include letters, editorials, commentaries, or perspectives. To improve its impact factor, JAMA, like some other journals, could make adjustments to categories counted in the denominator or publish only basic science and RCT articles to increase citations. JAMA does not publish articles only to increase its impact factor, as exemplified by its issues dedicated to topics that result in few citations, such as its annual issues covering medical education. Studies with high likelihood for publication have a number of characteristics. The most sought-after articles are those showing causality with significant impact on patient care, primarily RCTs. Large cohort studies, which do not show causality but can provide association, also can generally be published. Research on quality improvement that can be studied in randomized or prospective cohort studies focused on specific interventions or ensuring that individuals receive certain diagnostic tests. Rigorous trials on systems are far more difficult to conduct, DeAngelis said. To be of most use, studies must establish rules that improve care, not guidelines. The difficulty in establishing rules is that they must account for context, including the type of intervention required, the type of patient involved, and the methodology. General guidelines, while useful, are not often followed. Methods sections are extremely important for diffusing knowledge and ability to be replicated. The average JAMA article is about 3,500 words, while articles with full explanations of methodology tend to be much larger. This space limitation deters from the journal’s publication of qualitative research, and thus these articles are not often accepted for publication. In addition to its length, qualitative research is hypothesis generating, not hypothesis testing. High-citation papers tend to be those that are hypothesis testing. One alternative offered is to publish a more complete methodology in a different kind of journal and publish the results section in a journal like JAMA, referring readers to the methodology paper. Meta-analyses and systematic reviews, if well done, can be published, DeAngelis said. Although these are hypothesis generating, they often help in the practice of medicine or health care. These types of studies are difficult to conduct, but worthwhile. The difficulty with meta-analyses and systematic reviews in quality improvement is that they tend to have only a few articles to analyze, which are often of poor quality and thus do not provide a strong base.

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary Focus Journals In some respects, a goal for quality improvement research should be to increase both the quantity and quality of research, said Mittman of the VA and the journal Implementation Science. Increasing the quantity requires more than just funding; it requires the research community to more wisely use its funding. Improvements in the health care delivery system’s commitment to train researchers and facilitate collaboration with other researchers are also needed. Improving the quality of research is a multidimensional issue in which journals have a role, Mittman said. Journals should view themselves as partners in supporting the progress of the field. Journals should not just passively be places for researchers to publish, but rather, journals should be more proactive in stimulating interest and in identifying key challenges facing research, Mittman said. Journals and their editorial boards share the burden of identifying the types of research questions and articles being pursued with individual researchers and funding agencies. In a more proactive way, these stakeholders can collectively help guide the field to address the right questions. Mittman recognized the need to lay out a vision for documentation of future research. This includes ensuring full details of studies be published before studies begin, including the motivation, the literature review, design and methods, hypotheses, and conceptual frameworks. Of particular importance to quality improvement research is publication of accurate baseline data, which are good indicators of gaps in quality to be addressed. These components should be made accessible to all. Journals that are highly specialized, such as Implementation Science, have the responsibility to publish supporting analyses and details of studies. With the increasing popularity of online publication without page limits, documentation of all details is possible. Better communication strategies are needed to prevent information overload. Different messages will be required for different audiences; thus, different types of information should be placed in different journals. Cumulative knowledge is a fundamental goal of science. Research is often published in a stand-alone manner. An adequate amount of effort has not been taken to explain and understand the interrelations among past findings, current implications, and future needs. Journals should emphasize this goal and should help ensure that studies are documented in a way that facilitates knowledge accumulation. Mittman also suggested the need for journals to increase their

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary roles in stimulating and supporting interaction and debate about articles. Some efforts have begun in this direction, such as rapid response letters, but more could be done. FUNDING Agency for Healthcare Research and Quality Funding for quality improvement has not been well funded federally, said Dougherty of AHRQ. Although funding has increased over time due to investments in patient safety and health information technology, AHRQ’s budget of $318 million is used toward achieving its mission to improve quality and the safety, efficiency, and effectiveness of health care for all Americans. AHRQ has also modified its approaches to funding and reviewing quality improvement projects. Not satisfied with the results of awarded grants, AHRQ used cooperative agreements directly with implementers of quality improvement to accelerate change in 2002. Rather, these grants and contracts were designated for large national or regional organizations with the potential to be agents for change in professional behavior; academic institutions could not apply. However, by not requiring the kinds of rigorous evaluations that academic researchers would have used, summarizing evaluations of these cooperative agreements was made difficult. In 2004, the administration and Congress focused AHRQ’s improvement work on health information technologies, including electronic health records and regional health information exchanges. Given that electronic health records and regional health information exchanges were implemented in large settings with little opportunity for randomization, making meaningful evaluations a challenge. Health information technology was a focus again in 2005, with a slant toward patient safety. This year (federal fiscal year 2007), the focus will be on improving ambulatory safety and quality, with a total of $22 million in grants, including cooperative agreements. AHRQ also issued special emphasis notices to encourage research on policy, systems, and organizational changes, particularly in low-resource settings. Small grants for implementation of emerging or existing research findings and related tools are also encouraged. Other funded areas relevant to improvement in the current fiscal year include the Accelerating Change and Transformation in Organizations and Networks program, the PBRNs, and health services research demonstration and dissemination grants.

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary In 2008, the President’s budget asked Congress to provide AHRQ with about $6 million in new research for ambulatory safety and quality and $1.7 million for new research in patient safety. Patient safety organizations will be funded to help form baseline data on patient safety. In addition, contracts related to value-driven health care will be funded for $3.7 million, Dougherty said. Also in 2007, AHRQ will launch the Healthcare Innovations Exchange, a web-based tool to disseminate information in a meaningful way. In addition to providing grants and contracts, AHRQ conducts evidence syntheses through its Evidence-based Practice Centers. In particular, one EPC conducted condition-specific reviews about what was known about quality improvement in high-priority areas as identified by the Institute of Medicine (IOM), as discussed in Chapter 3 by Heidenreich. From her own perspective and not that of AHRQ, Dougherty commented on the future importance of interdisciplinary team building to help make breakthroughs in quality improvement. She noted that good ideas lead to funding. Better funding for interdisciplinary team building has been an area of focus for some funders and should continue to be strengthened. The California Endowment The mission of The California Endowment (TCE) is to improve the health of Californians, said Ignatius Bau of TCE. In particular, TCE emphasizes the underserved in terms of racial and ethnic minorities and other uninsured and poor populations, many of whom bear the brunt of quality gaps. TCE also believes in a broad public health approach, stemming from the belief that health is influenced largely by social and environmental factors, not just visits to the doctor or hospital. Quality improvement is thus a small piece of TCE’s strategies. TCE’s view is very broad and expansive in its focus on access to care. With respect to quality, TCE would focus on the two aims of the IOM’s six aims of quality receiving the least attention: patient-centeredness and equity. From the perspective of a foundation like TCE, Bau said, ideas with the potential to make lasting change are the most intriguing. Bau encouraged potential grantees to consider developing ideas within particular levels of the health care system—the individual provider, the health care team, the organization, and the system. Developing ideas in this context of a specific level will help grantees understand the challenges facing both research and implementation

OCR for page 44
The State of Quality Improvement and Implementation Research: Expert Views, Workshop Summary in sustaining change. Individual champions often seek funding for their change ideas in order to provide validation to their organizations. On the other hand, team leaders tend to ask for funding when a new structure is taking place, such as during times of reorganization or new leadership. At the organizational level, adverse events and bad publicity are often the motivators to improve quality. At the system level, change can often be leveraged through legislation, regulation, accreditation, or financing. Using these examples, Bau observed that sustainable changes in quality improvement are generally much more reactive to opportunity or crisis than proactive. Stakeholders often neglected in planning change are consumers and purchasers. In talks with unions and businesses, Bau said their focuses are often on access to care and cost, without regard for quality. Creating a demand for quality through purchasers, including smaller businesses, could be a promising way to advance quality improvement. Patients should be more involved in this, especially in moving beyond measures of patient satisfaction (e.g., being treated with respect and having long wait times) to really being able to judge whether care was of high quality. Patients should be empowered with enough information to create expectations for their care without having to know the technical details of medicine. If health care delivery and research were more patient centered, the system could begin to break down the barriers to providing coordinated care. Quality improvement should not be based on condition, disease, or procedure, but on people interacting with the health care system.