Learning What Works Infrastructure Required for Comparative Effectiveness Research: Workshop Summary (2011) / Chapter Skim
Currently Skimming:
2 The Work Required
2 The Work Required
Pages 87-152
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 87... ...
Aimed at both individual patient health and overall health system improvement, CER assesses the risks and benefits of competing interventions for a specific disease or condition as well as the systemlevel opportunities to improve health outcomes. To meet the ultimate goal of providing information that is useful to guide the healthcare decisions of patients, providers, and policy makers, the work required includes conducting primary research (e.g., clinical trials, epidemiologic studies, simulation modeling)
|
From page 88... ...
Erin Holve and Patricia Pittman from AcademyHealth estimate that approximately 600 comparative effectiveness studies were ongoing in 2008, including head-to-head trials, pragmatic trials, observational studies, evidence syntheses, and modeling. Costs for these studies range broadly, but cluster according to study design.
|
From page 89... ...
Particular attention is needed as to which functions might be best supported by centralized versus local, decentralized approaches THE COST AND VOLUME OF COMPARATIVE EFFECTIVENESS RESEARCH Erin Holve, Ph.D., M.P.H., Director; Patricia Pittman, Ph.D., Executive Vice President, AcademyHealth Overview In the ongoing discussion about CER, there has been limited understanding of the current capacity for conducting CER in the United States. This report intends to help fill this gap by providing an environmental scan of the volume and the range of costs of recent CER.
|
From page 90... ...
. 3 During the interviews we attempted to identify research by more specific types, asking questions about pragmatic trials, registry and modeling studies, and systematic reviews.
|
From page 91... ...
; 2. observational studies (including registry studies, prospective cohort studies, and database studies)
|
From page 92... ...
The interviews with funders identified 617 comparative effectiveness studies, of which approximately half were observational studies (prospective cohort studies, registry studies, and database studies)
|
From page 93... ...
. Within the the Agency for Healthcare Research and Quality, the prioritization and research coordination efforts for comparative effectiveness studies are undertaken as part of the Effective Health Care Program.
|
From page 94... ...
Both the DCRI and CREST programs have a strong emphasis on clinical research using both randomized experimental study designs and observational designs. Other funding for training includes the National Institutes of Health (NIH)
|
From page 95... ...
Some respondents noted that differences in training manifest themselves in disagreements about the benefits of various observational study designs. Nevertheless, most individuals interviewed as part of this study felt that RCTs, observational studies (including registry studies, prospective cohort studies, and quasi-experiments)
|
From page 96... ...
Appendixes C and D, respectively, are the recommended priority CER studies proposed in 2009 by the IOM Committee on Comparative Effectiveness Research Prioritization and the Federal Coordinating Council for Comparative Effectiveness Research. Introduction CER has been defined as "rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients," (CBO, 2007)
|
From page 97... ...
But there is often an insufficient body of evidence demonstrating its relative effectiveness compared to existing or alternative treatment options, especially in realworld settings. This limited scope of information increases the likelihood that clinical decisions are not based on evidence but rather on local practice style, institutional tradition, or physician preference.
|
From page 98... ...
Participants at this meeting identified the need for the development of what they initially termed a "national problem list" to illustrate key evidence gaps and to prompt discussions leading to national studies. National Priority Assessment Inventory After initial work was done by IOM staff, the proposed project was renamed the National Priority Assessment Inventory.
|
From page 99... ...
A prioritized list of 100 Medicare research priorities generated by the Medicare Evidence Development and Coverage Advisory Committee was also reviewed, as was a list of 14 priority conditions created for AHRQ's Effective Health Care Program. A special effort was made to identify topics related to the care of children because so much effort is usually concentrated on adults and the elderly.
|
From page 100... ...
All of the summaries are organized in the same manner: • description of the condition or problem, • available treatments or interventions, • current evidence about the treatments or interventions, • issues needing research and conclusions, and • brief list of references. Lessons Learned from This Project The original, perhaps naïve, intent of this project was to produce a list of the 20 or so "best" or "most important" comparative effectiveness studies that need to be done immediately.
|
From page 101... ...
0 THE WORK REQUIRED TABLE 2-2 The Comparative Effectiveness Studies Inventory Project Identified 16 Candidate Topics for Comparative Effectiveness Research Study Topic Study Type Age Group Condition Treatment of attention Comparative Children Mental diseases deficit hyperactivity effectiveness disorder in children: treatment studies drugs, behavioral across modalities interventions, no prescription Treatment of acute Comparative Adults Heart and thrombotic/embolic effectiveness vascular diseases stroke: clot removal, treatment studies reperfusion drugs across modalities Treatment of chronic Comparative Adults Heart and atrial fibrillation: effectiveness vascular diseases drugs, catheter treatment studies ablation, surgery across modalities Treatment of chronic Comparative Adults Neurological low back pain effectiveness diseases treatment studies across modalities Gamma knife surgery Comparative Adults Neurological for intracranial lesions effectiveness diseases vs. surgery and/or treatment studies whole brain radiation across modalities Treatment of localized Comparative Adults Cancer prostate cancer: effectiveness watchful waiting, treatment studies surgery, radiation, across modalities cryotherapy Diagnosis and Diagnostic studies Adults Cancer prognosis of breast cancer using genetic tests: human epidermal growth factor receptor 2 and others Over-the-counter drug Drug–drug Children Respiratory treatment of upper and drug–placebo disorders respiratory tract treatment studies infections in children Drug treatment of Drug–drug and drug– Adults Mental disorders depression in primary placebo treatment care studies continued
|
From page 102... ...
Once general topics had been selected, literature reviews often found either too much or
|
From page 103... ...
In some cases, the extant literature is sufficient to assess comparative effectiveness, and systematic reviews with or without formal meta-analysis can be created from data taken from previous studies. This type of study is usually the fastest to complete.
|
From page 104... ...
More often than not, clinical trials and the resulting meta-analyses report on work done in academic centers with highly selected patients. Except for some very recent trials, most do not include minority or gender diversity.
|
From page 105... ...
Currently Available Healthcare Databases: Claims and Electronic Health Records Most existing healthcare databases were created for purposes other than research. Currently, payers (including managed care organizations, insurance companies, employers, and governments)
|
From page 106... ...
The absence of these data may be critical when making comparisons between specific therapeutic options, as they may strongly influence the study results or the conclusions drawn, but they cannot be managed in the design or analysis of these databases. The problem of confounding of comparisons between therapeutic options is why randomization is such a powerful aspect of clinical trials.
|
From page 107... ...
Electronic medical record systems independent of a single healthcare delivery system: General Practice Research Database (united Kingdom) , The Health Improvement Network (based in the united Kingdom)
|
From page 108... ...
. In the context of comparative effectiveness studies, differences in effectiveness (usually considered to be benefits)
|
From page 109... ...
By engaging the clinician directly at the point of data entry, the record system itself can help standardize the capture of data, can solicit additional detail, or can push summary information and links to additional resources. Existing databases are generally considered to be reasonably complete with respect to determining "exposure" to particular drugs (and, to a lesser CLINICAL CONSENSUS Diagnostic Criteria CAMDEX ICD-10 ICD-9 DSM-IV DSM-IIIR DSM-III 0 5 10 15 20 25 30 Prevalence (%)
|
From page 110... ...
Looking Ahead Given the limitations of working with available databases, it seems likely that more robust data collection at the clinical interface would improve insights from, and the quality of, observational databases. Accumulating higher-quality data could help provide insight into current practice and help improve care.
|
From page 111... ...
This model is being revived across the United Kingdom in the Research Capability Programme in the Connecting for Health initiative, which includes educating the public on the importance of participating in clinical trials as a way of contributing to medical knowledge and advancement. Making such trials practical in the context of primary care may well require either structural changes in the process of care, in order to facilitate patient and physician participation, or adjustments to how these trials are conducted, so that the interruptions to the usual process of care are minimized.
|
From page 112... ...
regulations. Making links across hospitals, claims, electronic medical records, and other data sources will be best accomplished by using unique personal identifiers (e.g., social security numbers)
|
From page 113... ...
For example, a Duke/ FDA/Industry Cardiac Safety Research Consortium exists, the purpose of which is to create an electrocardiogram (ECG) library from clinical trials that could be used to identify early predictors of cardiac risk (Cardiac Risk ECG Library)
|
From page 114... ...
would be smaller steps that could generate huge returns in the quality of patient care. The ultimate goal is to begin both building targeted studies and enhanced data capture capabilities into the framework of existing medical care databases as well as making information flow both from and to healthcare providers in a way that is immediately beneficial and effective in informing their care of the patient.
|
From page 115... ...
Physicians seeking information based upon grades of clinical evidence face a similar dilemma. For example, microvolt T-wave alternans is an office-based test that predicts the risk of life-threatening cardiac arrhythmia.
|
From page 116... ...
However, comparative effectiveness studies are not required as part of this review, so users lack information on which medication works best for a particular disease scenario. Another division of the FDA, the Center for Devices and Radiological Health (CDRH)
|
From page 117... ...
Deficiencies of the Current State of Comparative Effectiveness Review As noted above, some systematic reviews of clinical evidence and some clinical practice guidelines lack scientific rigor, relying instead on a con
|
From page 118... ...
) √ √ √ √ √ √ √ colonoscopy Disease management Ambulatory blood pressure √ √ √ √ √ √ √ monitoring Intermittent intravenous insulin √ √ √ √ √ therapy Diagnosis CT angiography for suspected √ √ √ √ √ √ √ coronary artery disease Microvolt T-wave alternans √ √ √ √ √ √ √ Wireless capsule endoscopy √ √ √ √ √ √ √
|
From page 119... ...
. The Kaiser Permanente entries represent all Kaiser regions.
|
From page 120... ...
Recommendations by the Institute of Medicine Committee on Reviewing Evidence to Identify Highly Effective Clinical Services To address the deficiencies noted above and in fulfillment of the charge to the IOM by the Robert Wood Johnson Foundation, a Committee on Reviewing Evidence to Identify Highly Effective Clinical Services (HECS) was convened.
|
From page 121... ...
Research study designs would include RCTs, cohort studies, case-control studies (identifying factors that may contribute to a medical condition by comparing a group of patients
|
From page 122... ...
. Infrastructure requirements include funding, staffing, and stakeholder involvement; common agreement on the hierarchy of clinical evidence; a body of evidence sufficient to allow for systematic review; capability of satisfying the needs of different stakeholders, including consumers, physicians, purchasers of health care, and payers, among others; a repository for evidence reports; a process for systematic updating and revision; and an infrastructure sufficient to compare different treatments (IOM, 2008)
|
From page 123... ...
Further, it is impossible, unethical, and prohibitively expensive to randomize "everything." To fill this gap, balancing-score methods coupled with rigorous study design can approximate randomized trials. They are less controlled but
|
From page 124... ...
Unfortunately, a longitudinal birth-to-death patient-centric health record, populated largely with discrete values for variables that would be useful for both streamlined randomized and balancing-score-based clinical trials, has not been brought to fruition. Instead, clinical information remains locked in narrative, mostly within segregated institutional silos.
|
From page 125... ...
Personalized • Patient-specific strategic decision support Reengineering Randomized Controlled Trials Following intense preliminary work, several cardiac surgical centers began designing a randomized trial to answer a simple question: Is surgical ablation of nonparoxysmal atrial fibrillation accompanying mitral valve disease effective at preventing the return of the arrhythmia? It took a short time -- weeks -- to design this study, but then it had to be vetted through committees, review boards, and the FDA, leading to multiple revisions, additions, and mounting complexity.
|
From page 126... ...
Third, adding administrative and bureaucratic complexity to many RCTs is needed for investigational device exemptions and new drug exemptions from the FDA. This introduces considerable delay by an organization that should itself promote efficient study designs focused on safety and efficacy.
|
From page 127... ...
Available computer-based clinical data could and should be used for patient screening, recruiting, and data gathering. With electronic patient records composed of "values for variables" (discrete data elements)
|
From page 128... ...
. Are all the clinical trials that are being performed actually necessary?
|
From page 129... ...
Shouldn't insurers be interested in sponsoring clinically relevant RCTs, including making data available from the trials to the scientific community or at least bearing the patient costs of RCTs? Approximate Randomized Clinical Trials What effect does chronic exposure to urban pollution have on the risk of developing pulmonary disease or cancer?
|
From page 130... ...
. What distinguishes these patients from those in an RCT is that at one end of the spectrum of propensity scores, only a few who actually received treatment A match those who actually received treatment B, and at the other end of the spectrum, only a few patients who actually received treatment B match those who received treatment A
|
From page 131... ...
NOTE: COPD = chronic obstructive pulmonary disease; LVEF = left ventricular ejection fraction; NYHA = New York Heart Association. 2-3 fixed image is known explicitly and equally distributes both known and unknown factors, propensity score methods can at best account for only those selection factors that have been measured and recorded, not for those that are unknown.
|
From page 132... ...
Notice at low propensity scores, the numbers of group 1 patients dominate over those of group 2, and at the other extreme, the numbers of group 2 patients dominate over those of group 1. Within each quintile, patient charac Figure 2-4, fixed image teristics are well matched between groups, but these characteristics progressively change across quintiles (for example, low-risk profile in quintile 1 and high-risk profile in quintile 5)
|
From page 133... ...
Third, the most appropriate method of comparing outcomes after propensity matching remains controversial and probably requires developing new statistical tests. Semantically Interpreting, Querying, and Exploring Disparate Clinical Data Computerized Patient Records In 1991 the Institute of Medicine described what it called the computerbased patient record (CPR)
|
From page 134... ...
They provide the observational data for developing propensity scores, balancing scores, and conducting studies of comparative clinical effectiveness. If a true longitudinal record is created, then discrete data may identify adverse events and the substrate by which unsuspected correlated events may be identified, quite possibly with the use of artificial intelligence and computer learning techniques.
|
From page 135... ...
Blackstone's move to Cleveland Clinic in late 1997 provided the opportunity to pursue development of the CPR, but in the test bed of a highly productive cardiovascular clinical research environment. Clinical researchers know, of course, that discrete data are required for statistical analysis, and for the preceding 25 years, human abstractors at the clinic had laboriously extracted data elements from narratives for every patient undergoing a diagnostic or interventional cardiac procedure, resulting in the Cardiovascular Information Registry.
|
From page 136... ...
, underlie the data model and enable semantic searches. An investigator can search for patients and their data without knowing anything about underlying data structure.
|
From page 137... ...
Fourth, a semantic-ontology approach also permits truly intelligent patient search of medical concepts. This is becoming increasingly important as patients seek out information about their medical conditions.
|
From page 138... ...
Computer Learning Methods As much as one can dream of a longitudinal database that might permit innovative research for information-based medical care, it is important to ask, "If we actually had these data, would we know what to do with them? " One useful way to look at the issue is to use a "trees and woods" analogy in which individual patients, their data, and their genes are like the individual trees, and groups of patients or populations are the woods (Blackstone, 2007)
|
From page 139... ...
Joel Verter once said that RCTs are "sledgehammers, not fine dissecting instruments." Medicine needs to head toward fine dissecting instruments, toward personalized strategic decision support. With n = 1, a new paradigm of RCT needs to be developed for genomic-based personalized medicine (Balch, 2006)
|
From page 141... ...
Locked in the medical literature even today are static risk factor equations that could be used in dynamic mode for strategic decision support for a patient such as this (Levy et al., 2006)
|
From page 142... ...
Heterogeneity in practice facilitates approximate randomized trials via propensity score methods that are inexpensive and widely accessible but which require patient-level clinical data stored as discrete values for variables. Emerging semantic technology can be exploited to integrate currently disparate, siloed medical data -- responding to investigators' complex queries and patients' imprecise ones -- and in the near future holds the promise to automate discovery of unsuspected relationships and unintended adverse or surprisingly beneficial outcomes.
|
From page 143... ...
The cost would double if extensive integration of data was necessary, possibly over healthcare networks. For $100 million, it would be possible to conduct more than 1,000 such approximate randomized trials.
|
From page 144... ...
While it builds on an appreciation for the role of technology assessment, comparative study designs, and the increased role of health information technology to gather evidence and distribute it to the point of care, the capacity and infrastructure for this research has received less targeted attention. Understanding the landscape of organizations and health systems undertaking CER is challenging but essential.
|
From page 145... ...
The United States is not accustomed, nor organized effectively, to conduct this type of research. In part, this is because the system tends not to grow researchers who have the capacity to move beyond what might be described as a parochial mind set regarding the types of research study designs, a mind set which has limited the capacity to readily generate hypotheses and study designs appropriate for CER.
|
From page 146... ...
There is general sentiment that CER can be a positive thing if it is done fairly, is well designed, and is transparent. This is important because of the potential impact of CER on many different sectors -- not just patients, but also industry and health plans.
|
From page 147... ...
Otherwise, the rigorous study designs needed to move this field forward will never be developed. Successes, however, make the fundamental concept of CER worthwhile.
|
From page 148... ...
2006. Randomized clinical trials in surgery: Why do we need them?
|
From page 149... ...
2007. Comparative clinical effectiveness and cost effectiveness research: Background, history, and overview.
|
From page 150... ...
2009. Comparative effectiveness studies inventory project.
|
From page 151... ...
2007. The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials.
|
From page 152... ...
2007. Bridging the inferential gap: The electronic health record and clinical evidence.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.