Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
2 What Is Comparative Effectiveness Research? Abstract: Comparative effectiveness research (CER) provides an oppor- tunity to improve the quality and outcomes of health care by providing more and better information to support decisions by the public, patients, caregivers, clinicians, purchasers, and policy makers. Several organiza- tions have developed definitions of CER; for purposes of this study, the Institute of Medicine committee has defined CER as âthe generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.â CERâs distinguishing characteristics include informing a specific clinical or policy decision, comparing at least two approaches or interventions, describing results at the subgroup level, measuring benefits in real-world populations, and applying appropriate methods and data sources. Several federal agencies and many other organizations are involved in important CER activities, which are summarized in this chapter. However, the exist- ing incentives for developing CER evidence are uneven, the infrastructure for supporting the development of such evidence has gaps, and better coordination of research and translation of evidence into clinical practice and health policy is needed. 29
30 INITIAL NATIONAL PRIORITIES FOR CER The need for more and better evidence of what works in health care While the U.S. health care system continues to make progress in im- proving health, there is wide agreement that large gaps remain in the qual- ity and outcomes of health services delivered to many Americans (IOM, 2001; Orszag, 2007). Moreover, the need for better value in the health care system becomes apparent when overall health care spending and outcomes are considered. Health care expenditures were $2.4 trillion in 2008 and are projected to grow by an average of 6.2 percent per year for the next 10 years, more than triple the projected rate of overall gross domestic product (GDP) growth (Sisko et al., 2009). The Congressional Budget Office (CBO) projects that under current law, health care will consume more than 30 percent of GDP by 2035 (CBO, 2008). Regional variations in treatment patterns and cost growth provide deeper insight into the need for more informed medical decision making. Researchers at the Dartmouth Institute for Health Policy and Clinical Prac- tice have shown that patients in the highest-spending regions of the country receive 60 percent more health services than those in the lowest-spending regions, yet this additional care is not associated with improved outcomes (Fisher et al., 2003). The Dartmouth research suggests that physicians in higher-spending areas are more likely than physicians in other regions to recommend costly interventions that have not been definitively shown to be effective (Fisher et al., 2009). Nationwide, the Institute of Medicine (IOM) has estimated that less than half of all treatments delivered today are sup- ported by evidence (IOM, 2007). Even the most thoughtfully conceived and sophisticated practice guidelines have inadequacies in their evidence base, whereas many guidelines that are evidence-based and well-supported are often not translated into clinical practice. A recent review of practice guide- lines developed by the American College of Cardiology and the American Heart Association found that relatively few recommendations were based on high-quality evidenceârandomized controlled trials, for instanceâand many were based solely on expert opinion, individual case studies, or standard of care (Tricoci et al., 2009). A similar study revealed that more than two-thirds of recommendations contained in 51 guidelines for treating lung cancer were not evidence-based (Harpole et al., 2003). Researchers as- sessing recommendations for preventing and treating breast and colorectal cancers concluded that the overall quality of available guidelines for these cancers was âmodestâ (Vigna-Taglianti et al., 2006). Thus, providers need better information to provide the appropriate care for their patients. Providers, hospitals, and clinics are not the only groups with a stake in the evidence base for health care decisions. A growing movement of educa- tion and empowerment for consumers, as well as the public and patients
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 31 taking a more active role in making decisions about their own health care, has emerged together with health insurance plans shifting a greater share of expenses to patients. New models of shared decision making promise to bring greater emphasis to informed patient choice for âpreference-sensitiveâ care (Wennberg et al., 2007). Indeed, new and better research could im- prove the quality, safety, and effectiveness of health care by providing both patients and their health care providers with the evidence to assist in informed decision making. To accomplish this, however, patient-focused research is needed to identify not only the population-level effects of health interventions, but also the patient-level predictors of both positive and negative outcomes and the role of patient preferences in making choices (CBO, 2008). Beyond consumers and health care providers, those that bear substan- tial financial responsibility for the health care needs of populations need better evidence; they include employers, federal programs such as Medicare and Medicaid, and private insurers. These organizations must allocate re- sources across a panoply of medical services, procedures, and technologies in an attempt to maximize health benefits while keeping coverage afford- able. Moreover, these organizations must function within a growing array of models for the delivery and financing of health care, many of which are not yet supported with robust evidence of effectiveness. Optimizing Evidence Study Populations Representative of Clinical Practice Many studies of the effects of medical interventions on health address efficacy rather than effectiveness. Efficacy reflects the degree to which an intervention produces the expected result under carefully controlled condi- tions chosen to maximize the likelihood of observing an effect if it exists. Many randomized controlled trialsâgenerally considered to be the gold standardâare efficacy studies, particularly those conducted to win regula- tory approval. The study population and setting of efficacy studies may differ in important ways from those settings in which the interventions are likely to be used. By contrast, effectiveness research intends to measure the benefits and harms of an intervention in ordinary settings and broader populations, and therefore can often be more relevant to policy evaluation and the health care decisions of providers and patients. Care needs to be taken, however, in effectiveness research such as observational, database, registry, and other studies to recognize that without randomization, uniden- tified bias and confounders may weaken the level of evidence, and that ef- ficacy studies may be strengthened by broadening eligibility of populations or settings for trials and other attempts to increase generalizability. Issues
32 INITIAL NATIONAL PRIORITIES FOR CER regarding efficacy and effectiveness, and other aspects of CER studies are further examined below. Focus on the Individual Rather Than the Average Patient With the growing knowledge of disease mechanisms, systems biology, genomics, and other sciences that create the potential for more targeted therapies, patients and providers are increasingly seeking evidence not only from representative populations, but also from relevant subgroups. Increasing emphasis on patient-level attributes that may modify the balance of benefits or harms can lead to more personalized medicine, reducing the pressure to try alternatives found to be ineffective in similar subgroups. Study Two or More Interventions in Direct Comparison Randomized, placebo-controlled studies demonstrating safety and ef- ficacy for the purpose of gaining approval from the Food and Drug Admin- istration (FDA) frequently serve as the basis for informing clinical decisions. Although the FDA does require âactive comparatorsâ in some clinical cir- cumstances, there is a paucity of head-to-head studies, and the public needs more studies that compare evidence-based alternatives (including usual care), particularly in subgroups that preapproval studies often omit. Beyond specific medical interventions and technologies, there is a need for evidence evaluating the clinical and resource effects of innova- tions in health care delivery models, including new benefit designs, cost- sharing techniques, integrated organizational models, public health and population-level strategies, and interventions to improve the quality of care. Approaches to the organization, delivery, and payment for health care services are seldom evaluated for their impact on patient outcomes and overall value, yet they need to be studied directly using the same prin- ciples described previously. Because these interventions are often imple- mented at provider or regional levels, the methods required to evaluate themâsuch as cluster randomized trialsâmay differ from those used to evaluate patient-level interventions. Finally, merely generating better evidence is not enough to meet the decision-making needs of consumers, patients, health care providers, and purchasers. To maximize its impact on the quality and value of health care, these parties must use evidence when making clinical and policy decisions. Disseminating evidence into clinical practice must be accompanied by on- going evaluation and feedback to decision makers, the key characteristic of a true learning health care system. However, this is not happening con- sistently. In a review of adherence to 439 indicators of health care quality for 30 acute and chronic conditions as well as preventive care, McGlynn
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 33 and colleagues concluded that American adults received only 55 percent of recommended care (McGlynn et al., 2003). Similarly, another study found that children and youth received only 46.5 percent of recommended care (Mangione-Smith et al., 2007). Even proven screening tests and other pre- ventive servicesâsuch as influenza vaccines and mammograms in adultsâ are utilized by only 75 percent of Medicare beneficiaries (GAO, 2002). Furthermore, only 77 percent of U.S. children aged 19-35 months received all of the recommended doses of six childhood vaccines in 2006 (CDC, 2007). This gap between evidence and execution underscores the need to identify more effective tools to help patients, providers, and policy makers to use the available evidence. Indeed, as the number of elective treatments and procedures has grown, so has the need for patient-centered research that compares their effective- ness. The promise of comparative effectiveness research (CER) is that it provides evidence that is better focused on the decisions of daily medical practice than existing evidence and therefore helps patients, caregivers, providers, payers, and policy makers make informed decisions about health care. defining comparative effectiveness research CER can be very broad in scope depending on what is âcompared,â how one defines âeffectiveness,â and what constitutes âresearch.â Virtually any applied biomedical inquiry is fundamentally about improving health, avoiding unnecessary costs, or both. In the current public policy environ- ment, defining and describing CER is important: the 111th Congress made a $1.1 billion investment in CER and has created a Federal Coordinating Council to coordinate CER within the Department of Health and Human Services (HHS). According to the American Recovery and Reinvestment Act (ARRA) of 2009 (P.L. 111-5), this Council must provide Congress and the Secretary of HHS with an annual report describing its activities and specific recommendations for infrastructure and coordination of CER activities within relevant federal departments and agencies. Debate continues among stakeholders seeking to shape what will be studied, how it will be studied, and how the results will be appliedâa debate reflected in the extensive response to the present studyâs efforts to encourage stakeholder input to its assigned task of setting research priori- ties (as described in Chapter 3). New and expanded CER efforts will build on a solid base that is gaining wider recognition for its importance and applicability to clinical decision â American Recovery and Reinvestment Act of 2009, P.L. 111-5, 111th Congress, 1st ses- sion (February 17, 2009).
34 INITIAL NATIONAL PRIORITIES FOR CER making. As expectations for CER rise, more and more gaps in both knowl- edge and research infrastructure appear. These gaps must be filled if CER is to fulfill its promise of more informed decisions and better outcomes. In this chapter, the committee takes inventory of different authoritative defini- tions of CER, derives from these the disciplineâs defining characteristics, and briefly describes the range of ongoing activities that offer a foundation for new investments in this rapidly evolving field. Existing Definitions Although research comparing the effectiveness of health care strategies and interventions has been conducted for more than a century, the term âcomparative effectiveness researchâ has taken on new meaning in recent years. Table 2-1 displays several definitions, all of which were developed by leading public and private-sector authorities in the United States. These definitions all emphasize CERâs role in helping to inform health care decisions. All denote the evaluation of at least two alternatives, each with the potential to represent best practice. Most CER definitions suggest that the objective is to learn what works best in actual practice for a defined population. CER includes both systematic reviews of existing data and the collection and analysis of primary data. A contested issue in defining CER is whether costs or cost-effectiveness are appropriate outcomes of interest. The main justification for including economic considerations is that the overall value of a strategy can be under- stood best by considering costs and benefits together. Another view is that CER in general, and the use of cost-effectiveness analysis in particular, will inevitably discourage the use of expensive forms of care and lead to denial of needed care. While CER may identify ways to obtain better outcomes for the same or lower spending, cost-effectiveness analysis and CER may also lead to the conclusion that the more expensive approach offers better value than lower-cost approaches. For example, in the setting of breast cancer, magnetic resonance imaging (MRI) screening was found to be cost-effective relative to mammography in carriers of the BRCA1/2 mutation (Plevritis et al., 2006). Similarly, adjuvant trastuzumab was found to be cost-effective compared to conventional chemotherapy without trastuzumab in early stage HER2/neu-positive breast cancer (Kurian et al., 2007). When CER examines differences in costs as well as outcomes, its aim is to identify the approach that offers the better value; it does not necessarily promote or favor low-cost care. â Ernest Amory Codman, M.D. (1869â1940), was a Boston surgeon and pioneer in the study of quality and safety outcomes. According to one biographer, he never stopped in his effort to link care, errors, and end results and to measure, report, and improve.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 35 TABLE 2-1 Definitions of CER Organization Definition Congressional Budget A rigorous evaluation of the impact of different options that are Office available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy. The analysis may focus only on the relative medical benefits and risks of each option, or it may also weigh both the costs and the benefits of those options. In some cases, a given treatment may prove to be more effective clinically or more cost-effective for a broad range of patients, but frequently a key issue is determining which specific types of patients would benefit most from it. Related terms include cost-benefit analysis, technology assessment, and evidence-based medicine, although the latter concepts do not ordinarily take costs into account. IOM Roundtable The comparison of one diagnostic or treatment option to one or on Evidence-Based more others. In this respect, primary comparative effectiveness Medicine research involves the direct generation of clinical information on the relative merits or outcomes of one intervention in comparison to one or more others, and secondary comparative effectiveness research involves the synthesis of primary studies to allow conclusions to be drawn. Secondary comparisons of the relative merits of different diagnostic or treatment interventions can be done through collective analysis of the results of multiple head- to-head studies, or indirectly, in which the treatment options have not been directly compared to each other in a clinical evaluation, and inferences must be drawn based on the relative effect of each intervention to a specific comparison, often a placebo. American College of Comparative effectiveness analysis evaluates the relative (clinical) Physicians effectiveness, safety, and cost of two or more medical services, drugs, devices, therapies, or procedures used to treat the same condition. Although the use of the term comparative effectiveness broadly refers to the evaluation of both the relative clinical and cost differences among different medical interventions, it is notable that most comparative effectiveness research engaged in and used by stakeholders in this country focuses solely on evaluating relative clinical differences to the exclusion of cost factors. IOM Committee on Comparison of . . . the impacts of different options for caring for Reviewing Evidence a medical condition for a defined set of patients. The comparison to Identify Highly may be between similar treatments, such as competing Effective Clinical prescription medications, or for very different treatment Services approaches, such as surgery or radiation therapy. Or, the comparison may be between using a specific intervention and its nonuse (sometimes called watchful waiting). This report uses the terms âeffectiveness,â âclinical effectiveness,â and âcomparative effectivenessâ interchangeably. continued
36 INITIAL NATIONAL PRIORITIES FOR CER TABLE 2-1 Continued Organization Definition Medicare Payment Comparative-effectiveness analysis evaluates the relative value of Advisory Commission drugs, devices, diagnostic and surgical procedures, diagnostic tests, and medical services. By value, we mean the clinical effectiveness of a service compared with its alternatives. Comparative-effectiveness information has the potential to promote care of higher value and quality in the public and private sectors. Agency for Healthcare A type of health care research that compares the results of Research and Quality one approach for managing a disease to the results of other approaches. Comparative effectiveness usually compares two or more types of treatment, such as different drugs, for the same disease. Comparative effectiveness also can compare types of surgery or other kinds of medical procedures and tests. The results often are summarized in a systematic review. The direct comparison of existing health care interventions to determine which work best for which patients and which pose the greatest benefits and harms . . . the core question of comparative effectiveness research (is) which treatment works best, for whom, and under what circumstances. SOURCES: AHRQ (2009a); American College of Physicians (2008); IOM (2007, 2008); Medi- care Payment Advisory Commission (2008); Orszag (2007); Slutsky and Clancy (2009). Ultimately, CER aims to provide data that can influence clinical deci- sions for the better. If CER results are integrated into health care delivery and facilitate improved decision making, health care outcomes can be expected to improve. The concept has been formalized as the value of information, or the difference between the value of the outcome given the decision one would make in the absence of additional information and the value of the outcome of the decision that would be made as additional in- formation became available as a result of the research (Garber and Meltzer, 2009). In such a circumstance, value may be judged from the perspective of the patient, provider, or payer. The Spine Patient Outcomes Research Trial (SPORT) provides several examples of the potential utility of CER. The investigators demonstrate the value of spinal stenosis surgery in subjects without degenerative spon- dylolisthesis compared to nonsurgical interventions, as well as to subjects with degenerative spondylolisthesis (Tosteson et al., 2008; Weinstein et al., 2008). The Veterans Administration instituted a variety of interventions based upon their initial observations following performance scores obtained from an external peer review program in the mid-1990s, demonstrating re- markable improvement in 12 of 13 quality indicators over a 5-year period (Jha et al., 2003).
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 37 characteristics of CER Six defining characteristics of CER may be drawn from these definitions and the committeeâs deliberations over national priorities: 1. CER has the objective of directly informing a specific clinical deci- sion from the patient perspective or a health policy decision from the population perspective. The range of potential objectives for CER studies gives the field a broad scope. Clinical questions re- fer to the health care of individual patients, including preventive, screening, diagnostic, therapeutic, monitoring, or rehabilitative in- terventions. Policy questions refer to the health and health care of populations through knowledge synthesis and transfer strategies, public health programs, or initiatives involving the organization, delivery, or payment for health services. This characteristic has a major implication; because CER contributes to such important de- cisions, all relevant stakeholders (including patients and the public) and decision makers would reasonably be included throughout the CER process, including priority setting, study design, and imple- mentation of results (Tunis et al., 2003). Consumer involvement in trials began in earnest in the AIDS era (Epstein, 1996). Early efforts in CER embraced involvement of patients, caregivers, clini- cians, and other decision makers in setting research boundaries and in defining populations, settings, comparisons, and outcomes that should be addressed in the research (Helfand, 2005; Santaguida et al., 2005; Whitlock et al., 2009). 2. CER compares at least two alternative interventions, each with the potential to be âbest practice.â Compared clinical strategies may be very similar (drug vs. drug) or very different (drug vs. surgery). For many clinical decisions, âoptimal usual careâ reflecting current standards is an appropriate potential comparator. CER studies may also include the alternative of âwatchful waitingâ when it is considered to be a reasonable strategy in the clinical context. CER highlights research that compares a test intervention with viable alternatives. Interventions are implemented in accord with usual practice, which includes co-interventions and practice preferences. For policy decisions, a comparator may be the status quo. 3. CER describes results at the population and subgroup levels. The primary outcome of a clinical trial is a measure of the âaverage effectâ of an intervention, usually as estimated in the population assigned to the intervention in the trial. Even when selecting from
38 INITIAL NATIONAL PRIORITIES FOR CER âprovenâ strategies, clinicians must judge whether a particular pa- tient is sufficiently similar to the trial population or if technological or scientific advancement has outdated the empirical results. By its focus on subgroup results and clinical prediction rules to identify patients likely to benefit from an intervention, CER assists providers and patients in individualizing decisionsâgoing beyond the average effects to the effect in subjects with common clinical characteristics. This focus of CER reflects the growing potential for individualized and predictive medicineâbased on advances in genomics, sys- tems biology, and other biomedical sciencesâthrough the analysis of subgroups with demographic, ethnic, physiologic, and genetic characteristics that could be useful factors in clinical decisions. 4. CER measures outcomesâboth benefits and harmsâthat are im- portant to patients. The committee is using the term âeffective- nessâ in reference to the extent to which a specific intervention, procedure, regimen, or service does what it is intended to do when used under real-world circumstances. This can be contrasted with âefficacy,â which is the extent to which an intervention produces a beneficial result under controlled conditions (Cochrane, 1971; Higgins and Green, 2008). This implies an important distinction between much clinical research and CER, in that CER places high value on external validity, or the ability to generalize results to real-world decision making. Harms or risks of unintended consequences are also outcomes of interest, because they influ- ence the net benefits of an intervention. Including and giving weight to patient-reported outcomes is particularly important for CER studies in which patient ratings of effectiveness or adverse events may differ from clinical measures. Finally, resource utiliza- tion may be highly relevant to net benefits when comparing the full clinical course of interventions over time. Cost-effectiveness analysis is a useful tool of CER, allowing evaluation of the full range of treatment outcomes in relationship to the difference in costs. Robust evidence of comparative clinical effectiveness is a building block necessary for resource allocation decisions. More- over, just as clinical effects may vary in different settings, costs vary as well, so a given set of cost-effectiveness results is often not generalizable. 5. CER employs methods and data sources appropriate for the deci- sion of interest. CER includes at least three broad categories of â The net benefit of a particular intervention is the balance of the harms and benefits.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 39 research methods. Where evidence is lacking, CER may generate it either in nonexperimental studies (observational settings) or in experiments (randomized and cluster randomized, as well as non- randomized, controlled trials). For decisions that have been the topic of substantial previous research, synthesis of existing Âstudies (systematic reviews and meta-analysis, technology assessments and decision analysis) may be appropriate. Box 2-1 describes these methods in greater detail. Data sources for CER may thus include published studies, existing data from the delivery of care (admin- istrative claims, medical charts, electronic health records), clinical registries, and information collected by clinical investigators, either retrospectively or prospectively. To ensure the wide availability and use of these data sources will require new CER infrastruc- ture to support a highly robust national CER program, including methods, workforce, and data networks. For example, multisite data networks, analyzed simultaneously via a distributed protocol (distributed network analysis), are needed. Considerations in the selection of methods and data sources for CER include the quality and relevance of previously published evidence, the availability and potential for confounding in observational data sources, and the time and resources available for primary data collection. 6. CER is conducted in settings that are similar to those in which the intervention will be used in practice. Consistent with the definition of effectiveness, the settings of CER studies are a defin- ing characteristic. Studying interventions in a realistic practice setting has implications for both CER trials and observational studies. For experimental studies, investigators should deliver the intervention in settings that are as close to actual practice as pos- sible, which is a strength of observational studies of actual clinical practice. The call for CER should not be interpreted to mean that all research must have these characteristics. Early studies of an intervention are likely to compare it to a placebo, standard care, or no intervention. In fact, during early development of a new intervention, it is critical to determine safety and efficacy under a defined set of circumstances. For example, the I Â schemic Optic Neuropathy Decompression Trial was a landmark examina- tion of an innovative surgical technique used by ophthalmologists (Ischemic Optic Neuropathy Decompression Trial Research Group, 1995). It found no Â beneficial effect and possible harm compared to no therapy. Once an intervention has been shown to be effective against a placebo, head-to-head trials address the critical question âWhat works best for whom?â
40 INITIAL NATIONAL PRIORITIES FOR CER BOX 2-1 Methods Commonly Used in CER Experimental studyâA category of studies in which the investigators actively intervene to test a hypothesis. Controlled trials are experimental studies in which an experimental group receives the intervention of interest while one or more com- parison groups receive an active comparator, a placebo, no intervention, or the standard of care, and the outcomes are compared. In a randomized controlled trial (RCT), the participants are randomly allocated to the experimental group or the comparison group. Because patients are enrolled prospectively, RCTs can require many years to enroll a sufficient number of subjects and observe the outcomes of interest. Cluster randomized trials are RCTs in which the subjects are assigned to intervention or control in groups (clusters) defined by a common feature, such as the same physician or health plan. Head-to-head trials are a method of CER, because they compare more than one active intervention. Observational or nonexperimental studyâA category of studies in which the investigators do not seek to intervene but simply observe the course of events. Observational studies may be prospective or retrospective. Prospective obser- vational studies, such as registries, can require years to accumulate the needed numbers of patients and outcomes. In cohort studies, groups with certain expo- sures or characteristics are monitored over time to observe an outcome of inter- est. In retrospective case-control studies, groups with and without an event or condition are examined to see whether a past exposure or event is more prevalent in one group than in the other. Cross-sectional studies determine the prevalence of a condition or an exposure at a specific time or time period. Case series de- scribe a group of patients with a characteristic in common, for example, individuals undergoing a new type of surgery or the users of a new device. Retrospective CER is limited by the intrinsic methodologies it utilizes. In some cases, bias or confounders may limit the reliability of data obtained through ret- rospective means. Alternatively, prospective, randomized clinical trials may take so long or be so expensive as to render the study unfeasible or unethi- cal. As a result, it is imperative that the specific methodology be matched to the question being asked and to the population and/or data base that is available. In order for CER to impact health care delivery or outcomes, the results must be integrated into the health care delivery system and be utilized by patients and providers. Failure to utilize the findings minimizes their potential impact. How the data are utilized has also been a matter of significant debate. Some have expressed concern that CER could be inap- propriately used to limit access to care (Gottlieb, 2009), however, coverage decisions can only benefit from better information on what works (Avorn, 2009; Garber and Tunis, 2009).
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 41 observational studies can be designed and analyzed in existing databases within a matter of months, but provide more limited data. Research SynthesisâA category of studies in which investigators seek to summarize the information from multiple studies addressing similar research questions. A comparative effectiveness systematic review summarizes available scien- tific evidence in which investigators collect, evaluate, and synthesize studies in accordance with an organized, structured, explicit, and transparent methodology. They seek to provide decision makers with accurate, independent, scientifically rigorous information for comparing the effectiveness and safety of alternative clinical options, and have become a foundation for decision making in clinical practice and health policy, including informing coverage decisions for therapeutics in health care. Meta-analysis is the process of using statistical methods to combine the results of similar studies quantitatively in an attempt to allow inferences to be made from the sample of studies and applied to the population of interest. Technology assessments assess the effectiveness of medical technologies us- ing either single studies or systematic reviews. SOURCES: Adapted from IOM (2008). See also Chou et al. (2007); Cochrane Collaboration (2005); Haynes et al. (2006); Last (2001); West et al. (2002). IOM National Priorities Committee Definition Proceeding from the definitions in Table 2-1 and the preceding consid- erations, the committee developed the following working definition of CER to guide its deliberations: Comparative effectiveness research (CER) is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will im- prove health care at both the individual and population levels. This definition is consistent both with the rationale for CER expressed by other groups and with federal legislation providing support for such
42 INITIAL NATIONAL PRIORITIES FOR CER research. In addition, it encompasses the breadth of interventions that influ- ence the individual and public health of Americans, whether through pre- vention, diagnosis, or management of disease. The fundamental principle underlying this definition is that CER must be relevant to decision makers, particularly patients and providers. examples of CER studies: coronary artery disease A better understanding of the scope of and potential methodologies employed by CER is best illustrated by several examples. The selected methodology is based on the sources of information available to answer the question and dictates the rapidity and cost involved in study performance. Treatment of coronary artery disease has been the subject of numerous CER studies. Table 2-2 provides four distinct examples of CER studiesâemploying very different methodologiesârelated to commonly performed interven- tions for coronary revascularization in patients with coronary artery dis- ease. These procedures are among the most widely performedâand most costlyâin patients with coronary heart disease, the leading cause of death in the United States. Numerous trials have addressed the efficacy of three alternative treatment modalities: drug therapies; percutaneous coronary intervention (PCI) with balloon angioplasty, stents, or both; and coronary artery bypass graft (CABG) surgery. Uncertainty remains about which modality is best for specific patient subgroups. Moreover, the question of how to manage a patient with myocardial ischemia is often urgent, requir- ing the cardiologist to quickly explain the risks and benefits of alternative interventions to a patient and then agree on a timely course of action. In this context, an individualized recommendation based on evidence from rigorous CER has the potential to improve outcomes. Randomized Controlled Clinical Trial The 5-year randomized controlled trial (RCT) comparing all three modalities at a single center in Brazil is an example of a comparative ef- fectiveness trial with some significant limitations (Hueb et al., 2007). Its strengths include internally valid results (the most powerful advantage of a study that randomly assigns patients to one of the target interventions) and long-term follow-up (namely, no differences in mortality over 5 years), yet it had too few subjects for robust analyses of secondary endpoints and subgroups. Moreover, it took many years from design to publication of results, during which new modalitiesâincluding glycoprotein IIb/IIIa in- hibitors, clopidogrel, and drug-eluting stents, none of which were included in the studyâachieved routine use.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 43 Systematic Review and Meta-Analysis of Existing Data A systematic review of PCI and CABG reported in 2007 included a meta-analysis of 23 RCTs and a comparison of findings of five large ob- servational studies (Bravata et al., 2007). While corroborating the earlier finding of no difference in survival after PCI or CABG, the study filled important gaps by offering, on a limited scale, analyses of subgroups and secondary endpoints. By combining many studies, the statistical power to detect or rule out relatively small but still clinically useful differences in subgroups can be determined. These analyses demonstrated that sur- vival outcomes were equivalent in diabetic and non-diabetic patients, and that CABG tended to lead to angina relief, but to more procedure-related strokes, whereas repeat revascularization procedures were more common after PCI. Pooled Analysis of Patient-Level Data from Many Clinical Trials Even more recently, data from 10 RCTs were pooled to create one large dataset that included the patients from the 10 trials. The authors sought to identify patient-level predictors of outcomes (Hlatky et al., 2009). The study confirmed equivalent overall survival outcomes with PCI and CABG. However, it showed that CABG might be a better option for elderly and diabetic patients, because mortality was lower in these subgroups. By pool- ing individual-level data (in contrast to study-level data as in the systematic review by Bravata, described earlier), the authors were able to use powerful, statistically valid methods (such as Cox regression analysis with interaction terms) to identify subgroup effects. Health Services Research Finally, in an example of a policy-level CER study, researchers analyzed a Hospital Quality Incentive Demonstration Project, piloted by the Centers for Medicare & Medicaid Services (CMS), in the care of patients with acute myocardial infarction (AMI) (Glickman et al., 2007). The intervention of- fered pay-for-performance economic incentives to hospitals with the highest quality scores in five clinical conditions, including AMI. Using a database of administrative and clinical data from both incentivized hospitals and a con- trol group of hospitals, the researchers found that the pay-for-performance economic incentives produced neither significant benefits nor unintended adverse consequences. This hospital-level observational study gave useful information about the effects of incentives on hospitals. Because the study used existing electronically stored data, it was relatively inexpensive and quick to perform. However, like all observational studies, the authors could
44 INITIAL NATIONAL PRIORITIES FOR CER TABLE 2-2 Selected CER Studies of Management of Acute Coronary Syndrome Type of cer Randomized Clinical Trial Systematic Review Study (Authors) (Hueb et al., 2007) (Bravata et al., 2007) Objective To compare relative effectiveness To compare the effectiveness of CABG with that of PCI of PCI and CABG in or MT in patients with patients for whom coronary symptomatic multivessel revascularization is clinically coronary artery disease requiring indicated. revascularization. Comparators CABG, PCI, and MT (stepped PCI (including balloon care using nitrates, aspirin, beta- angioplasty or coronary blockers, angiotensin-converting stents) and CABG (including enzyme inhibitors, lipid-lowering standard or minimally invasive agents, or combination of these techniques). unless contraindicated). Population and Patients with angiographically Twenty-three unique RCTs, Subgroups documented proximal multivessel which enrolled a total of Studied coronary stenosis of >70% and 9,963 patients, met inclusion documented ischemia (but no criteria; subgroups were prior revascularization or AMI), defined by diabetes, age, sex, randomly assigned to undergo smoking, and number of CABG (n = 203), PCI (n = 205), diseased vessels; five major or MT (n = 203). clinical registries met inclusion criteria. Outcomes of Primary: overall mortality, Q- Survival, myocardial Interest wave MI, or refractory angina infarction, stroke, angina, requiring revascularization and use of additional Secondary: angina status and revascularization procedures. stroke or cerebro-vascular accident. Methods and RCTâMultivariate Cox Systematic review of aggregate Data Source(s) proportional hazards 5-year data from published trials survival model to assess the and large observational relationship between pairwise studies; meta-analysis of treatment comparisons and RCTs computed summary risk primary endpoints. differences and odds ratios between PCI and CABG and the 95% CI for each outcome of interest at annual intervals; observational studies were summarized qualitatively. NOTE: AMI = acute myocardial infarction; CABG = coronary artery bypass graft; CI = con- fidence interval; CMS = Centers for Medicare & Medicaid Services; MT = medical therapy; PCI = percutaneous coronary intervention.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 45 Pooled Quantitative Analysis of Trials Observational Study (Policy) (Hlatky et al., 2009) (Glickman et al., 2007) To assess whether the effects of CABG and To determine if pay-for-performance was PCI on mortality are modified by patient associated with either improved processes characteristics. of care and outcomes or unintended consequences for AMI at hospitals participating in the CMS pilot project. CABG and PCI (balloon angioplasty and Quality improvement intervention, bare-metal stents). including bonus payments to hospitals with performance scores in the top two deciles and potential future financial penalty for those with the poorest performance, compared to usual care. Subgroups defined based on Patients with acute non-ST-segment demographics, cardiac risk factors, clinical elevation. MI enrolled in a national quality- manifestations, and angiographic factors. improvement initiative at 54 program hospitals (n = 10,325) and 446 control hospitals (n = 95,058). Primary: All-cause mortality Six process measures used by CMS to Secondary: Death or myocardial evaluate the care of patients with non-ST- infarction; death or repeat segment elevation AMI, and eight process revascularization; death, myocardial measures that are designated âuseful and infarction, or repeat revascularization. effectiveâ by the American College of Cardiology and American Heart Association, but are not currently tracked by CMS. Pooled analysis of individual patient data Observational, patient-level analysis of data from 10 clinical trials that randomly abstracted from hospital administrative assigned patients with multivessel and clinical records; temporal trends in coronary disease to either CABG or PCI individual and composite processes of care and that reported at least 3 years of for both CMS and non-CMS measures at follow-up. pay-for-performance and control hospitals were identified using nonlinear mixed-effects models.
46 INITIAL NATIONAL PRIORITIES FOR CER not be sure that they had adjusted statistically for unmeasured differences between rewarded hospitals and control hospitals; these unmeasured dif- ferences could affect the quality of care independently of the incentives and conceal true differences in this apparently negative study. These examples illustrate both the breadth and diversity of CER meth- ods applied to a single clinical context. Clearly, definitive answers relevant to specific clinical situations are difficult to obtain, and these studies remind us that CER may provide answers to very specific questions to assist in clinical decision making, but that clinical uncertainty may persist to the extent that the study fails to encompass all individual patient characteris- tics. The challenge is to increase the probability of an optimal outcome in a particular patient if it is a clinical decision or a population if it is a policy decision. Existing CER ACTIVITY in the united states Many research activities in the United States have one or more of the six characteristics of CER described by the committee earlier in this chapter. Some activities are propelled by regulatory and reimbursement requirements, while others respond to public and private research funding opportunities. This section reviews some of those initiatives and derives lessons that can guide new investments in CER. Federal Government-Sponsored CER Activities To date, the federal government has sponsored a series of systematic comparative effectiveness reviews as well as a considerable amount of CER, primarily through the Agency for Healthcare Research and Quality (AHRQ) and the National Institutes of Health (NIH). CMS, the Veterans Administration (VA), and the FDA also support specific forms of CER. Ad- ditional agencies and departments, such as the Centers for Disease Control and Prevention and the Department of Defense, engage in research, some of which may meet the definition of CER. Agency for Healthcare Research and Quality AHRQâs Effective Health Care Program has been the federal govern- mentâs leading effort to conduct systematic comparative effectiveness re- views and database studies that compare health care interventions. Created by Congress in 2003 under the Medicare Prescription Drug, Improvement, and Modernization Act, the Effective Health Care Program had a budget of $30 million in 2008. Its research priorities are set by the Secretary of HHS
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 47 and informed by an explicit priority-setting process that includes nomina- tions from all stakeholders. The Effective Health Care Program uses existing studies to make head- to-head comparisons of treatment alternatives, including drugs, devices, surgical procedures, and other types of interventions. Its activities are conducted by 14 Evidence-based Practice Centers (EPCs), 13 Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Centers, 14 Centers for Education and Research on Therapeutics (CERTs), and the John M. Eisenberg Clinical Decisions and Communications Science Center. This multicenter infrastructure is capable of a wide range of evidence develop- ment and synthesis activities. The EPCs review relevant scientific literature on clinical, behavioral, and organization and financing topics to produce evidence reports and tech- nology assessments. Reviews may be systematic reviews of a given interven- tion, or comparative effectiveness reviews of a disease entity. These reports are used for informing and developing coverage decisions, quality measures, educational materials and tools, guidelines, and research agendas. The EPCs also conduct research on the methodology of systematic reviews. The DEcIDE network links research centers to generate new CER evidence. It conducts accelerated practical studies about the outcomes, comparative clinical effectiveness, safety, and appropriateness of health care products and services. The network comprises health system-based research organizations with access to electronic health information databases and the capacity to conduct distributed network analysis. The mission of the CERTs is to conduct research and provide education that will advance the optimal use of drugs, devices, and biological products; increase awareness of the benefits and risks of therapeutics; and improve quality while cutting the costs of care. The CERTs program consists of 14 research centers and a coordinating center (AHRQ, 2009b). The John M. Eisenberg Clinical Decisions and Communications Sci- ence Center translates complex scientific research produced in the Effective Health Care Program into publications that are intended for use by the public, clinicians, and policy makers. The Eisenberg Center creates and dis- seminates a variety of products for each audience, ranging from summary guides to decision aids and other materials. To date, the AHRQ Effective Health Care Program has released ap- proximately 30 products, including 14 comparative effectiveness reviews from the EPCs; 18 summary guides for clinicians, patients, and policy mak- ers; and 10 reports from the programâs DEcIDE research network. â Many more projects are under way, and all of the programâs products are publicly available at AHRQâs website: http://www.effectivehealthcare.ahrq.gov.
48 INITIAL NATIONAL PRIORITIES FOR CER National Institutes of Health NIH is the principal federal agency that performs and funds biomedi- cal research, including clinical trials, systematic reviews, and observational studies. An internal survey to determine the extent of NIH funding of studies meeting the definition of CER is under way, but to date it has pro- vided limited information. Numerous obvious examples of CER exist; for example, the National Heart, Lung, and Blood Institute has funded or co- funded numerous comparative trials, including the CASS (Coronary Artery Surgery Study), ALLHAT (Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial), and ACCORD (Action to Control Cardio- vascular Risk in Diabetes) studies. The Cardiovascular Research Network is taking advantage of electronic data, covering more than 10 million patients, to do observational research to answer questions about hypertension detec- tion and management in communities, warfarin use in atrial fibrillation and venous thromboembolism, and the outcomes and costs of implantable defi- brillators for primary prevention of sudden cardiac death (Lauer, 2009). The Cooperative Group Program housed within the National Cancer Institute was established in 1955 to promote and support clinical trials of new cancer treatments, explore methods of prevention and detection, and assess quality of life during and following treatment (NCI, 2006). Currently, 1,700 institutions contribute more than 22,000 new patients into sponsored clinical trials annually. Many of these studies meet strict criteria for CER and generate important data for subsequent patient-level therapeutic decision making. For example, a large study of paclitaxel adju- vant treatment in over 4,900 women with breast cancer demonstrated an improved disease-free survival and overall survival in women negative for human epidermal growth factor receptor type 2 (HER2), as compared to those who were HER2 positive, regardless of their hormone receptor status (Sparano et al., 2008). The National Institute for Neurological Diseases and Stroke has funded several trials that could be described as CER because they compared in- terventions that were already employed by practicing physicians, a key characteristic of CER. Ongoing trials include the CREST trial (Carotid Revascularization Endarterectomy Versus Stenting Trial), which compares carotid endarterectomy to stenting; ARUBA trial (A Randomized Trial of Unruptured Brain Arteriovenous Malformations), which compares medical care to more invasive procedures in patients with arteriovenous malforma- tions; a trial performed in the Clinical Research Collaboration comparing propranolol and topiramate for migraine prophylaxis; and the CombiRx trial, which compares the combination of glatiramer acetate and beta in- terferon to each therapy alone in patients with multiple sclerosis (Johnston and Hauser, 2009).
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 49 Another example of NIH-funded CER is a trial supported by the Na- tional Institute for Arthritis, Musculoskeletal and Skin Diseases, called the Spine Patient Outcomes Research Trial (SPORT), which is the largest study comparing operative to non-operative treatment for the three most common conditions for which spine surgery is performed in the United States (disc herniation, spinal stenosis, and degenerative spondylolithesis with stenosis). This study was carried out in actual clinical practices in 11 states and 13 centers (Birkmeyer et al., 2002). The NIH Pharmacogenetics Research Network was established in 2000 to enable multi-disciplinary research groups to address questions in phar- macogenetics and pharmacogenomics. The goal is to populate a knowledge base for the purpose of identifying safe and effective drug therapies for individual patients (NIH PGRN, 2009). Not all NIH trials that compare interventions completely meet the defi- nition of CER. Trials of this type may, for example, restrict entry to a small subset of the population actually treated in practice, or monitor subjects in ways inconsistent with routine care, changing the focus of the research to efficacy rather than effectiveness. Pragmatic trials and other new methods are being developed to better evaluate treatments in routine use while at- tempting to control for the risk of introducing confounding factors that can occur in real-world settings (Lohr, 2007). Food and Drug Administration The FDA has had growing involvement in developing the standards and methods for comparative studies, as well as assessing them to support the regulation of medical products. In some clinical contexts, the FDA has required active comparators for product approval when existing treatments have demonstrated significant benefit for a âmajor disease.â In addition, the FDA Amendments Act (FDAAA) of 2007 (P.L. 110-85) enhanced the FDAâs ability to enforce requirements for post-approval studies. While many such studies are focused on safety concerns, some provide additional evidence about comparative effectiveness (for example, longer-term or more defini- tive comparisons of effectiveness, and analyses involving additional types or subtypes of patients). In addition to exercising regulatory authority to ensure the safety and effectiveness of medical products, the FDA is developing standards and methods for CER studies, as well as their interpretation to support its regulatory mission. The FDAâs Critical Path Initiative seeks to identify pre- dictors of response to therapy, a key characteristic of CER. For example, collaborations between FDA scientists and academic groups validated genetic predictors of enhanced sensitivity to warfarin and other drugs me- tabolized by liver enzymes. One of the statutory goals of the Reagan-Udall
50 INITIAL NATIONAL PRIORITIES FOR CER Foundation, authorized under the FDAAA but not yet funded, is to sup- port public-private collaborations on applied research questions relevant to CER, such as the Sentinel Program, which endeavors to link very large numbers of medical records and claims data to facilitate post-marketing surveillance for unusual complications of new therapies. Centers for Medicare & Medicaid Services The mission of CMS is to âensure effective, up-to-date health care coverage and to promote quality care for its beneficiariesâ (CMS, 2009a). CMS is a key stakeholder in the health care system in generalâits ben- eficiaries generate about one-third of national health expenditures (CMS, 2009b)âand in CER in particular. Although Medicare does not explicitly base coverage decisions on CER evidence, it uses such evidence in its cov- erage reviews. Medicareâs predominance in the U.S. health care market provides product developers with a strong incentive to tailor the evidence they produce to the needs of CMS reviewers. The Medicare Evidence De- velopment and Coverage Advisory Committee, a large group of clinical experts, researchers, industry representatives, and consumer advocates, provides CMS with guidance on specificâusually controversialânational coverage decisions. When CMS determines that the available evidence is insufficient to support a definitive coverage decision, it can employ a relatively new policy option, âCoverage with Evidence Development,â which authorizes CMS to cover items or services only if the patient participates in a registry or a clinical trial. This mechanism serves to expand coverage to unapproved interventions, giving them an opportunity to prove themselves while simul- taneously enlarging the evidence base (Tunis and Pearson, 2006). Department of Veterans Affairs The Veterans Healthcare System at the VA is suited to both produce and apply evidence on treatment effectiveness because it cares for a very large patient population, has a sophisticated electronic health records system that supports clinical research, and provides an array of medical services. Through its Cooperative Studies Program (CSP), the VA provides support to its own investigators conducting multi-center, collaborative clinical trials. The programâs research infrastructure includes five Coordinating Centers that provide data management and analysis support, a Clinical Research Pharmacy, four epidemiological Research and Information Centers, and a â Food and Drug Administration Act of 2007, P.L. 110-85, 110th Congress, 2nd session (September 27, 2007) 121.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 51 Health Economics Resource Center (Holve et al., 2008; Veteranâs Health Administration, 2009). CSP trials study a range of conditions, from schizo- phrenia to stroke (Veteranâs Health Administration, 2009), and have pro- duced notable resultsâfor instance, a large RCT with 38,546 participants demonstrated the effectiveness of a Herpes zoster vaccine (Oxman et al., 2005), and a very recent RCT of 1,791 diabetic military veterans showed that intensive control of blood glucose was not superior to usual care in preventing cardiovascular complications (Duckworth et al., 2009). The VA conducts comparative effectiveness reviews and guides for policy makers through its Evidence-based Synthesis Program, which works directly with top clinical managers and the Office of Quality and Per- formance to select priorities (VA, 2009). The VA is well-suited for CER activities because it has broad clinical and research capacity and feeds patient data recorded in the VA electronic health record system into a data warehouse that contains health records on the entire VA population. These data capabilities are a major national resource and a model for other data research networks. As a result of the application of the above techniques, the VA system has modified practices to markedly improve both the process of health care delivery and its outcomes (Jha et al., 2003). Nongovernmental CER Activities The private sector contains the life sciences industry, a number of orga- nizations with the capacity to conduct CER reviews, and some organizations focused on evidence development. However, not all of the work of these organizations meets the IOM definition of CER, and some of the work is pro- prietary, and available by contract or purchase, if at all. Representative non- governmental organizations involved in CER are described briefly below. Life Sciences Industry Manufacturers of drugs, devices, and other medical products have made large investments in research on their products, motivated in part by regulatory requirements set by the FDA. While most studies submitted to the FDA compare the new drug to a placebo, manufacturers also submit to the FDA studies comparing the new treatment to previously approved products with increasing frequency. Head-to-head studies may be necessary when comparison to a placebo is unethical, or to understand how the new treatment compares to an existing standard of care. The latter may be used to influence the content of the package insert and subsequent marketing of the product. Consequently, the FDA has been developing the standards and methods for comparative studies, and using the results to support the regulation of medical products.
52 INITIAL NATIONAL PRIORITIES FOR CER The second major motivation for manufacturers to conduct CER is to increase the likelihood that health systems will list their products in their formularies, and that payers will reimburse them. Determination of safety and efficacy does allow entry into the market, but it does not ensure that a third-party payer will cover a product, or that safety and efficacy will be the sole basis for determining payment levels. Health care payers asked to pay a higher price for a new product increasingly require evidence that the higher price buys additional clinical benefit. For example, drugs with similar risk- benefit profiles may end up on different formulary tiers, depending on their price. Consequently, reimbursement pressures may encourage manufactur- ers to support CER to demonstrate that their products provide benefit at the margin relative to existing products. Although these incentives may be important motivators to do CER related to particular products, they are less effective motivators to study medical practices and processes, products that are not âon patent,â novel uses of approved products, or âclass effectsâ or combinations of products. Blue Cross and Blue Shield Association Technology Evaluation Center The Blue Cross and Blue Shield Association Technology Evaluation Center (TEC) was established in 1985. Its mission is to provide health care decision makers with scientifically rigorous assessments that synthesize the available evidence on the prevention, diagnosis, treatment, and manage- ment of disease. Its assessments review the evidence that specific medical procedures, devices, and drugs improve health outcomes such as length of life, quality of life, and functional ability (Blue Cross and Blue Shield As- sociation, 2009). TEC produces 20 to 25 assessments per year for clients including CMS, Kaiser Permanente, and other private health plans. Its re- ports are publicly available. TEC is an EPC of the AHRQ Effective Health Care Program. Cochrane Collaboration Established in 1993, the Cochrane Collaboration is an independent, multinational nonprofit organization that creates and distributes systematic reviews of health care interventions. These reviews are prepared by 52 Co- chrane Review Groups. Quality standards, which are published regularly in a handbook, are maintained by editorial teams that oversee the prepara- tion and maintenance of the reviews. Cochrane review abstracts and plain- â When a payer places a drug on a high tier in the payerâs formulary, the patient is charged a higher out-of-pocket co-payment when purchasing the drug and the drug is less likely to be prescribed.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 53 language summaries are free and publicly available online, and complete reviews are available with a subscription (Cochrane Collaboration, 2001). As of April 2009, The Cochrane Database of Systematic Reviews contained a total of 5,785 systematic reviews of medical interventions, methodological studies, and diagnostic test accuracy (Cochrane Collaboration, 2009). A number of smaller nonprofit and for-profit enterprises are actively involved in a variety of CER activities. A sampling of organizations includes The ECRI Institute and Hayes, Inc., which provide exclusive proprietary in- formation and do not make their reports publicly available (ECRI Institute, 2009; Hayes Inc., 2009). The Drug Effectiveness Review Project (DERP) is a collaboration of public entities: the Center for Evidence-based Policy and the EPC at Oregon Health and Science University. These organizations produce systematic reviews of the comparative effectiveness and safety of drugs (Oregon Health and Science University, 2009), which served as a significant source for âConsumer Reports Best Buy Drugsâ (Findlay, 2006). The ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ Institute for Clinical and Economic Review produces publicly available assessments of new medical interventions to support value-based insurance benefit designs, coverage and reimbursement policy, and patient-clinician decision support tools (Institute for Clinical and Economic Review, 2009). The Center for Medical Technology Policy (CMTP) provides a forum for ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ï¿½ the collaborative design and implementation of CER, including pragmatic trials, adaptive designs, and clinical registries (CMTP, 2009). Lessons from Ongoing CER Activities This account of CER initiatives supports several conclusions. First, considerable CER is under way, often as a result of regulatory and reim- bursement incentives, with the support of a variety of programs in the public and private sectors. Although these programs vary in scope, goals, and activities, all seek to provide timely and useful evidence to health care decision makers on questions of patient care and policy significance. In its report, the committee acknowledges that this is not an exhaustive descrip- tion of the activities of U.S. organizations involved in CER. The Federal Coordinating Councilâs June 30, 2009, report to the Secretary of HHS will also describe the CER activities of federal agencies. Even limiting itself to its description of federal CER activities and the study of systematic reviews in Knowing What Works in Health Care, the committee found consider- able duplicated effort, which is one reason to propose a mechanism to coordinate CER activities throughout the nation. New public investments in CER should complement these ongoing initiatives. Federal coordination would allow for systematic identification and rapid dissemination of best practices, improved prioritization of research topics for future funding,
54 INITIAL NATIONAL PRIORITIES FOR CER reduction of duplication, encouragement of collaboration, improvement and standardization of methodology, and generally leverage of private-sec- tor initiatives with the goal of more rapidly and efficiently generating results on priority topics. Second, leaders of CER must evaluate the present CER workforce in light of the requirements of an expanded, sustained program equal to the task of undertaking the priority research described in Chapter 5. Expansion of CER, as envisioned by ARRA, requires assessment of current capacity and planning to support a national CER program. For example, the size of the qualified CER workforce is not known and the workforce needed to perform CER must be defined, assessed, and trained. NIH, FDA, AHRQâs DEcIDE Research Network, CERTs, CMS, VA, and CMTP are among the organizations focused on the development of new evidence from well- d Â esigned comparative clinical trials and observational studies. Priorities for new CER must address the research infrastructure required to generate new data to answer questions of interest to patients and policy makers and must recommend investment in new capacity where needed. Third, the United States lacks a large-scale national infrastructure for learning from the delivery of health care through observational research using existing clinical and administrative data sources. Moreover, a meth- odological framework is needed to guide the translation of clinical and policy relevant questions into answerable CER questions and to match CER questions to appropriate CER methods. Finally, the value of high-quality CER depends on successfully dissemi- nating and incorporating the results into routine practice. The means to the latter end include evidence-based guidelines, clinician and patient decision support tools, models of shared, informed decision making, reimbursement policy, and benefit design. Current law provides direct support for the syn- thesis and dissemination of CER (through the AHRQ-sponsored EPCs) to inform clinical and policy decision makingâproviding the building blocks for evidence-based policy. Conclusion Greater investment in CER has the potential to help improve the qual- ity, outcomes, and value of health care in America. But what exactly is CER? The committee has derived from several existing definitions six characteristics of CER studies, as well as a new working definition to guide priority setting. CER is defined by the pragmatic aim of informing a specific health care or health policy decision, and the explicit comparison of clini- cally credible, alternative interventions in a representative study population. CER studies seek to inform population-level and subgroup-level decisions alike, using outcomes, methods, and data sources appropriate to answer
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 55 the specific question within the limitations of the study design. CER en- compasses the collection of new experimental and observational data, the analysis of existing observational data, studies that synthesize completed research, and the translation and dissemination of research findings into clinical practice. Past clinical effectiveness studies have sought to answer questions about the effectiveness of medical interventions and strategies, but not all clini- cal effectiveness studies are comparative effectiveness studies. CER studies compare the intervention under study against its best or most commonly used alternatives in practice or in development, rather than against a pla- cebo. Furthermore, the studies address effectiveness (i.e., how the interven- tion performs with real-world patients) rather than assessing efficacy (i.e., how it performs in highly selected patient groups who receive study-related care in parallel with usual care). CER techniques are varied. Although RCTs, prospective cohort studies, and patient registries are among its most important tools, CER also uses other forms of information, such as systematic reviews, electronic health records, patient registries, and other observational datasets. Prospective studies based on new trials or primary analyses of patient- level data are a very important aspect of CER. However, CER also includes secondary analysesâsuch as meta-analyses, or formal pooled analyses of multiple studies. Pooled analyses are often critically important because they can be used to draw conclusions that could not be inferred from individual studies. Besides helping health care providers and patients make better clinical decisions, CER information can improve care in other ways. For example, hospitals and health systems might organize their facilities and personnel to better support care that is revealed by CER to be superior. Professional societies are likely to be both sources of CER and users of the informa- tion, incorporating CER into the development of clinical guidelines (IOM, 2008). A common misapprehension is that CER will lead to uniform, âone- size-fits-allâ care that ignores the ways that patients differ. In fact, CER done well should give providers the means to tailor the choice of treatment to the individual patientâs characteristics and preferences. Better compara- tive effectiveness studies will make it possible to measure the implications of individual differences in disease severity and the presence of comor- bidities, to identify predictors of response to treatment, and to incorporate other aspects of a personâs health and preferences. For example, CER might assess the added value of using genomic information in addition to traditional clinical predictors to determine the best treatment for a cancer in a particular patient. It might suggest formal assessment of patient pref- erences in those situations in which patient and caregiver desires might
56 INITIAL NATIONAL PRIORITIES FOR CER alter the decision to proceed with one or another treatment strategy. These techniques might allow physicians to tailor therapy to reflect the goals and desires of each patient. Better information can only help physicians do a better job of matching their care to an individual patientâs needs. Indeed, improvements in CER methods to support the use of targeted therapies are urgently needed. CER has been under way in a number of venues in the United States and has received notable support from the private and public sectors. The committeeâs review of these initiatives indicates that there is considerable capacity for evidence synthesis. However, the incentives for doing primary CER are uneven, the infrastructure for supporting the development of new evidence is in an early stage of development, and a wide gap exists between CER results and their translation into consistent clinical practice and health policy. New federal investments in CER must address these infrastructure and translational priorities in addition to the information needs on specific clinical topics. References AHRQ (Agency for Healthcare Research and Quality). 2009a. AHRQ Effective Health Care glossary http://effectivehealthcare.ahrq.gov/tools.cfm?tooltype=glossary&TermID=118 (accessed April 4, 2009). âââ. 2009b. Centers for Education & Research on Therapeutics (CERTs) http://www.certs. hhs.gov/ (accessed April 9, 2009). American College of Physicians. 2008. Improved availability of comparative effectiveness information: An essential feature for a high-quality and efficient United States health care system. Philadelphia, PA. Avorn, J. 2009. Debate about funding comparative-effectiveness research. New England Jour- nal of Medicine 360(19):1927-1929. Birkmeyer, N. J. O., J. N. Weinstein, A. N. A. Tosteson, T. D. Tosteson, J. S. Skinner, J. D. Lurie, R. Deyo, and J. E. Wennberg. 2002. Design of the spine patient outcomes research trial (SPORT). Spine 27(12):1361-1372. Blue Cross and Blue Shield Association. 2009. Technology Evaluation Center criteria http:// www.bcbs.com/blueresources/tec/tec-criteria.html (accessed May 1, 2009). Bravata, D. M., A. L. Gienger, K. M. McDonald, V. Sundaram, M. V. Perez, R. Varghese, J. R. Kapoor, R. Ardehali, D. K. Owens, and M. A. Hlatky. 2007. Systematic review: The comparative effectiveness of percutaneous coronary interventions and coronary artery bypass graft surgery. Annals of Internal Medicine 147(10):703-716. CBO (Congressional Budget Office). 2008. Technological change and the growth of health care spending. In CBO paper. Place Published: United States Congressional Budget Of- fice. http://www.cbo.gov/ftpdocs/89xx/doc8947/01-31-TechHealth.pdf (accessed April 30, 2008). CDC (Centers for Disease Control and Prevention). 2007. Morbidity and mortality weekly report. 56:880-888. Chou, R., R. Fu, S. Carson, S. Saha, and M. Helfand. 2007. Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. Journal of Clinical epidemiology 60(1):18-28.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 57 CMS (Centers for Medicare & Medicaid Services). 2009a. Mission, vision and goals overview http://www.cms.hhs.gov/MissionVisionGoals/ (accessed May 22, 2009). âââ. 2009b. National health expenditures aggregate, per capita amounts, percent distribu- tion, and average annual percent growth, by source of funds: Selected calendar years 1960-2007 http://www.cms.hhs.gov/NationalHealthExpendData/downloads/tables.pdf (accessed May 1, 2009). CMTP (Center for Medical Technology Policy). 2009. Welcome to the Center for Medical Technology Policy http://www.cmtpnet.org/ (accessed April 9, 2009). Cochrane, A. L. 1971. Effectiveness and efficiency: Random reflections on health services. London: Nuffield Provincial Hospitals Trust. Cochrane Collaboration. 2001. Cochrane Collaboration consumer network http://www. cochraneconsumer.com/ (accessed April 28, 2009). âââ. 2005. Glossary of terms in the Cochrane Collaboration. Version 4.2.5. www.cochrane. org (accessed November 27, 2006). âââ. 2009. Product descriptions-record counts http://www.interscience.wiley.com/egibin/ mrwhome/106568753/ProductDescriptions.html (accessed April 24, 2009). Duckworth, W., C. Abraira, T. Moritz, D. Reda, N. Emanuele, P. D. Reaven, F. J. Zieve, J. Marks, S. N. Davis, R. Hayward, S. R. Warren, S. Goldman, M. McCarren, M. E. Vitek, W. G. Henderson, and G. D. Huang. 2009. Glucose control and vascular complications in veterans with type 2 diabetes. New England Journal of Medicine 360(2):129-139. ECRI Institute. 2009. Pioneering applied scientific research in healthcare https://www.ecri. org/about/pages/default.aspx (accessed April 24, 2009). Epstein, S. 1996. Impure science: AIDS, activism, and the politics of knowledge. In Medicine and society. Place Published: University of California Press. http://www.loc.gov/catdir/ bios/ucal051/96016805.html (accessed April 21, 2009). Findlay, S. D. 2006. Bringing the DERP to consumers: âConsumer Reports Best Buy drugsâ. Health Affairs 25(4):w283-w286. Fisher, E. S., D. E. Wennberg, T. A. Stukel, D. J. Gottlieb, F. L. Lucas, and E. L. Pinder. 2003. The implications of regional variations in Medicare spending. Part 2: Health outcomes and satisfaction with care. Annals of Internal Medicine 138(4):288-298. Fisher, E. S., J. P. Bynum, and J. S. Skinner. 2009. Slowing the growth of health care costsâ Lessons from regional variation. New England Journal of Medicine 360(9):849-852. GAO (Government Accountability Office). 2002. Medicare: Beneficiary use of clinical preven- tive services. Report to the Chairman, Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, House of Representatives. Washington, DC. Garber, A. M., and D. O. Meltzer. 2009. Setting priorities for comparative effectiveness re- search. Paper presented at Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact, Engleberg Center for Health Care Reform at Brookings: Wash- ington, DC. Garber, A. M., and S. R. Tunis. 2009. Does comparative-effectiveness research threaten per- sonalized medicine? New England Journal of Medicine 360(19):1925-1927. Glickman, S. W., F. S. Ou, E. R. DeLong, M. T. Roe, B. L. Lytle, J. Mulgund, J. S. Rumsfeld, W. B. Gibler, E. M. Ohman, K. A. Schulman, and E. D. Peterson. 2007. Pay for performance, quality of care, and outcomes in acute myocardial infarction. JAMA 297(21):2373-2380. Gottlieb, S. 2009. Congress wants to restrict drug access: A bill in the House could tie your doctorâs hands. Wall Street Journal, January 20, 2009. Harpole, L. H., M. J. Kelley, G. Schreiber, E. M. Toloza, J. Kolimaga, and D. C. McCrory. 2003. Assessment of the scope and quality of clinical practice guidelines in lung cancer. Chest 123(1 SUPPL.):7S-20S.
58 INITIAL NATIONAL PRIORITIES FOR CER Hayes Inc. 2009. Hayes: Transforming healthcare with evidence http://www.hayesinc.com/ hayes/ (accessed May 8, 2009). Haynes, R. B., D. L. Sackett, G. H. Guyatt, and P. Tugwell. 2006. Clinical epidemiology: How to do clinical practice research. 3rd ed. Philadelphia, PA: Lipincott Williams & Wilkins. Helfand, M. 2005. Using evidence reports: Progress and challenges in evidence-based decision making. Health Affairs 24(1):123-127. Higgins, J., and S. Green. 2008. Cochrane handbook for systematic reviews of interventions: Version 5.0.1 Place Published: The Cochrane Collaboration, 2008. http://www.cochrane. org/resources/handbook/index.htm (accessed April 2, 2009). Hlatky, M. A., D. B. Boothroyd, D. M. Bravata, E. Boersma, J. Booth, M. M. Brooks, D. CarriÃ©, T. C. Clayton, N. Danchin, M. Flather, C. W. Hamm, W. A. Hueb, J. KÃ¤hler, S. F. Kelsey, S. B. King, A. S. Kosinski, N. Lopes, K. M. McDonald, A. Rodriguez, P. Serruys, U. Sigwart, R. H. Stables, D. K. Owens, and S. J. Pocock. 2009. Coronary artery bypass surgery compared with percutaneous coronary interventions for multivessel disease: A collaborative analysis of individual patient data from ten randomised trials. The Lancet 373(9670):1190-1197. Holve, E., S. Mercer, J. Kupersmith, and S. Phurrough. 2008. Exploring comparative effec- tiveness: Activities of CDC, VA and CMS to advance evidenced-based health. National Health Policy Forum The George Washington University, http://www.nhpf.org/library/ details.cfm/2646 (accessed October 3, 2008). Hueb, W., N. H. Lopes, B. J. Gersh, P. Soares, L. A. C. Machado, F. B. Jatene, S. A. Oliveira, and J. A. F. Ramires. 2007. Five-year follow-up of the medicine, angioplasty, or surgery study (MASS II): A randomized controlled clinical trial of 3 therapeutic strategies for multivessel coronary artery disease. Circulation 115(9):1082-1089. Institute for Clinical and Economic Review. 2009. Mission. http://www.icer-review.org/index. php/about.html (accessed April 9, 2009). IOM (Institute of Medicine). 2001. Crossing the quality chasm: A new health system for the 21st century, Edited by J. M. Corrigan. Washington, DC: National Academy Press. http://www.nap.edu/catalog/10027.html (accessed April 20, 2009). âââ. 2007. Learning what works best: The nationâs need for evidence on comparative effectiveness in health care. http://www.iom.edu/ebm-effectiveness (accessed April 15, 2009). âââ. 2008. Knowing what works in health care: A roadmap for the nation. Edited by J. Eden, B. Wheatley, B. J. McNeil and H. Sox. Washington, DC: The National Academies Press. Ischemic Optic Neuropathy Decompression Trial Research Group. 1995. Optic nerve decom- pression surgery for nonarteritic anterior ischemic optic neuropathy (NAION) is not effective and may be harmful. JAMA 273(8):625-632. Jha, A. K., J. B. Perlin, K. W. Kizer, and R. A. Dudley. 2003. Effect of the transformation of the Veterans Affairs health care system on the quality of care. New England Journal of Medicine 348(22):2218-2227. Johnston, S. C., and S. L. Hauser. 2009. Comparative effectiveness research in the neurosci- ences. Annals of Neurology 65(2):A6-A8. Kurian, A. W., R. N. Thompson, A. F. Gaw, S. Arai, R. Ortiz, and A. M. Garber. 2007. A cost-effectiveness analysis of adjuvant trastuzumab regimens in early HER2/neu-positive breast cancer. Journal of Clinical Oncology 25(6):634-641. Last, J. M. 2001. A dictionary of epidemiology. New York: Oxford University Press. Lauer, M. S. 2009. Comparative effectiveness research: The view from the NHLBI. Journal of the American College of Cardiology 53(12):1084-1086.
WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH? 59 Lohr, K. N. 2007. Emerging methods in comparative effectiveness and safety: Symposium overview and summary. Medical Care 45(10 SUPPL. 2):S5-S8. Mangione-Smith, R., A. H. DeCristofaro, C. M. Setodji, J. Keesey, D. J. Klein, J. L. Adams, M. A. Schuster, and E. A. McGlynn. 2007. The quality of ambulatory care delivered to children in the United States. New England Journal of Medicine 357(15):1515-1523. McGlynn, E. A., S. M. Asch, J. Adams, J. Keesey, J. Hicks, A. DeCristofaro, and E. A. Kerr. 2003. The quality of health care delivered to adults in the United States. New England Journal of Medicine 348(26):2635-2645. Medicare Payment Advisory Commission. 2008. Report to the Congress: Reforming the de- livery system. Washington, DC. NCI (National Cancer Institute). 2006. NCIâs clinical trials cooperative group program http://cancertrials.nci.nih.gov/cancertopics/factsheet/nci/clinical-trials-cooperative-group (accessed June 15, 2009). NIH PGRN (National Institutes of Health Pharmacogenetics Research Network). 2009. PGRN mission statement http://www.pharmgkb.org/network/pgrn_mission_statement. jsp (accessed June 12, 2009). Oregon Health and Science University. 2009. Drug Effectiveness Review Project http://www. ohsu.edu/drugeffectiveness/ (accessed April 8, 2009). Orszag, P. R. 2007. Research on the comparative effectiveness of medical treatments: Op- tions for an expanded federal role: CBO testimony before the Subcommittee on Health, Committee on Ways and Means, U.S. House of Representatives. U.S. Congressional Budget Office. http://www.cbo.gov/ftpdocs/82xx/doc8209/Comparative%5FTestimony. pdf (accessed April 28, 2009). Oxman, M. N., M. J. Levin, G. R. Johnson, K. E. Schmader, S. E. Straus, L. D. Gelb, R. D. Arbeit, M. S. Simberkoff, A. A. Gershon, L. E. Davis, A. Weinberg, K. D. Boardman, H. M. Williams, J. H. Zhang, P. N. Peduzzi, C. E. Beisel, V. A. Morrison, J. C. Guatelli, P. A. Brooks, C. A. Kauffman, C. T. Pachucki, K. M. Neuzil, R. F. Betts, P. F. Wright, M. R. Griffin, P. Brunell, N. E. Soto, A. R. Marques, S. K. Keay, R. P. Goodman, D. J. Cotton, J. W. Gnann Jr, J. Loutit, M. Holodniy, W. A. Keitel, G. E. Crawford, S. S. Yeh, Z. Lobo, J. F. Toney, R. N. Greenberg, P. M. Keller, R. Harbecke, A. R. Hayward, M. R. Irwin, T. C. Kyriakides, C. Y. Chan, I. S. F. Chan, W. W. B. Wang, P. W. Annunziato, and J. L. Silber. 2005. A vaccine to prevent herpes zoster and postherpetic neuralgia in older adults. New England Journal of Medicine 352(22):2271-2284+2365. Plevritis, S. K., A. W. Kurian, B. M. Sigal, B. L. Daniel, D. M. Ikeda, F. E. Stockdale, and A. M. Garber. 2006. Cost-effectiveness of screening BRCA1/2 mutation carriers with breast magnetic resonance imaging. JAMA 295(20):2374-2384. Santaguida, P. L., M. Helfand, and P. Raina. 2005. Challenges in systematic reviews that evaluate drug efficacy or effectiveness. Annals of Internal Medicine 142(12_Part_2):1066-1072. Sisko, A., C. Truffer, S. Smith, S. Keehan, J. Cylus, J. A. Poisal, M. K. Clemens, and J. Lizonitz. 2009. Health spending projections through 2018: Recession effects add uncertainty to the outlook. Health Affairs 28(2):w346-357. Slutsky, J. R., and C. M. Clancy. 2009. AHRQâs Effective Health Care Program: Why com- parative effectiveness matters. American Journal of Medical Quality 24(1):67-70. Sparano, J. A., M. Wang, S. Martino, V. Jones, E. A. Perez, T. Saphner, A. C. Wolff, G. W. Sledge, Jr., W. C. Wood, and N. E. Davidson. 2008. Weekly paclitaxel in the adjuvant treatment of breast cancer. New England Journal of Medicine 358(16):1663-1671. Tosteson, A. N. A., J. D. Lurie, T. D. Tosteson, J. S. Skinner, H. Herkowitz, T. Albert, S. D. Boden, K. Bridwell, M. Longley, G. B. Andersson, E. A. Blood, M. R. Grove, and J. N. Weinstein for the SPORT Investigators. 2008. Surgical treatment of spinal stenosis with and without degenerative spondylolisthesis: Cost-effectiveness after 2 years. Annals of Internal Medicine 149(12):845-853.
60 INITIAL NATIONAL PRIORITIES FOR CER Tricoci, P., J. M. Allen, J. M. Kramer, R. M. Califf, and S. C. Smith, Jr. 2009. Scientific evi- dence underlying the ACC/AHA clinical practice guidelines. JAMA 301(8):831-841. Tunis, S. R., and S. D. Pearson. 2006. Coverage options for promising technologies: Medicareâs âcoverage with evidence development.â Health Affairs 25(5):1218-1230. Tunis, S. R., D. B. Stryer, and C. M. Clancy. 2003. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA 290(12):1624-1632. VA (Veterans Affairs). 2009. Evidenced-based synthesis program http://www.hsrd.research. va.gov/publications/esp/ (accessed May 22, 2009). Veteranâs Health Administration. 2009. Cooperative studies program (CSP) http://www. research.va.gov/programs/csrd/csp.cfm (accessed May 1, 2009). Vigna-Taglianti, F., P. Vineis, A. Liberati, and F. Faggiano. 2006. Quality of systematic reviews used in guidelines for oncology practice. Annals of Oncology 17(4):691-701. Weinstein, J. N., T. D. Tosteson, J. D. Lurie, A. N. A. Tosteson, E. Blood, B. Hanscom, H. Herkowitz, F. Cammisa, T. Albert, S. D. Boden, A. Hilibrand, H. Goldberg, S. Berven, H. An, and the SPORT Investigators. 2008. Surgical versus nonsurgical therapy for lumbar spinal stenosis. New England Journal of Medicine 358(8):794-810. Wennberg, J. E., A. M. OâConnor, E. D. Collins, and J. N. Weinstein. 2007. Extending The P4P agenda, part 1: How Medicare can improve patient decision making and reduce unnecessary care. Health Affairs 26(6):1564-1574. West, S., V. King, T. Carey, K. Lohr, N. McCoy, S. Sutton, and L. Lux. 2002. Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment No. 47. (Pre- pared by the Research Triangle Institute-University of North Carolina Evidence-based Practice Center under Contract No. 290-97-0011). AHRQ Publication No. 02-E016. Rockville, MD: Agency for Healthcare Research and Quality. Whitlock, E. P., S. A. Lopez, S. Chang, M. Helfand, M. Eder, and N. Floyd. 2009. Identi- fying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health care program. http://effectivehealthcare.ahrq.gov/repFiles/ 20090427IdenttifyingTopics.pdf (accessed June 5, 2009).