Patient Outcomes Research Teams (PORTS): Managing Conflict of Interest (1991)

To support studies on the outcomes of health care services and procedures, the Omnibus Budget Reconciliation Act of 1989 (P.L. 101–239) established the Agency for Health Care Policy and Research (AHCPR), replacing the National Center for Health Services Research and Health Care Technology Assessment (NCHSR/HCTA) within the Department of Health and Human Services. Within AHCPR, the Center for Medical Effectiveness Research has primary responsibility for administering grant and contract research under the Medical Treatment Effectiveness Program (MEDTEP).

MEDTEP is charged to improve "the effectiveness and appropriateness of medical practice by developing and disseminating scientific information regarding the effects of presently used health care services and procedures on patients' survival, health status, functional capacity, and quality of life" (AHCPR, 1990:1). To fulfill this charge MEDTEP supports multidisciplinary research groups called PORTs—patient outcomes research teams.¹ In fiscal year 1990, AHCPR received $100 million, $38 million of which was allocated to MEDTEP, including the PORTs.

Table 2.1 lists the 11 PORTs that had been funded as of October 1990. With one exception, they result from investigator-initiated grant applications.² These applications are peer reviewed for scientific merit.

Afterward they are periodically reviewed by the agency study section that first awarded the grant.

Each PORT focuses on a specific acute or chronic condition "to identify and analyze the outcomes and costs of current alternative practice patterns in order to determine the best treatment strategy and to develop and test methods for reducing inappropriate variations" (AHCPR, 1990:2). In selecting conditions to study, MEDTEP has given priority to conditions occurring frequently, especially among Medicare beneficiaries, conditions and interventions for which risks and costs are particularly high, treatments are particularly variable, and outcomes uncertain. PORTs also evaluate the effects of their findings and recommendations on practice patterns.

PORTs are large multidisciplinary projects that employ larger and more diverse groups of researchers than is typical in health services research. MEDTEP requires that each PORT include academic and practicing community-based health care providers, and that each team have expertise in the following areas: pertinent clinical specialties, research design, literature synthesis (including meta-analysis), epidemiology, biostatistics, economics, decision analysis, survey research, data management, and research dissemination. PORTs are multi-institutional and, in some cases, international in composition. Box 2.1 shows the disciplines, institutions, and professional affiliations of members of the currently funded PORTs.

To enhance inter-PORT coordination MEDTEP has established six work groups to focus on specific methodological issues: meta-analysis, decision analysis, outcomes assessment, use of claims data, cost of care, and dissemination. Each work group comprises designated PORT representatives who will meet to discuss common problems and possibly common solutions for use by PORTs and other researchers.

According to AHCPR's Program Note on medical treatment effectiveness research (AHCPR, 1990), PORTs must include at least the following four components in their research:

1. Comprehensive literature review and synthesis on the condition or treatment being assessed. The literature review is the basis for developing research hypotheses to analyze associations between practice variations and outcomes and to identify gaps in knowledge.

Meta-analysis is a quantitative approach used to synthesize the clinical literature. It includes an explicit method for locating studies, the use of specified criteria for admission of studies, a system for classifying and coding study characteristics using a common scale, and methods for aggregating and interpreting study findings (Thacker, 1990). It includes studies ranging from large scale, double-blind randomized controlled trials to anecdotal reports as well as information from both published and unpublished sources. For instance, meta-analysis conducted in the Johns Hopkins University PORT study of cataract treatment will attempt to estimate probabilities of specified outcomes for each patient management strategy. Steinberg et al. (1990) estimated that more than 3,000 potentially relevant articles on cataract surgery were published during the past 10 years. Because most of this literature consists of observational studies rather than randomized controlled trials, the PORT will need to adapt traditional meta-analytic techniques (Steinberg et al., 1990). Methods of meta-analysis are still evolving; in aggregating findings for meta-analysis, at least one approach explicitly identifies biases in studies, estimates the magnitude of the bias, adjusts, and weights the results of the studies accordingly (Eddy, 1990; Eddy et al., 1990).

2. Collection and analysis of data including variations in medical practice and associated patient outcomes. PORTs are charged to identify ''findings important for patient decisionmaking'' and critical gaps in knowledge that should be addressed by subsequent studies (AHCPR, 1990:5). Sources of information may include Medicare and Medicaid data, hospital discharge abstracts, state health department records, chronic disease data banks, and insurance records. The availability of large data bases from insurance claims, such as Medicare Parts A and B and the Canadian provincial health data bases, makes it possible to estimate the probabilities of relevant outcomes. Data on mortality, repeat surgery, and other major complications are identifiable in the claims data bases.

To obtain primary data about outcomes and preferences that are not available from insurance data bases, PORTs may also conduct patient and practitioner interviews or surveys and abstract patient records. For instance, the Indiana PORT, which is evaluating total knee replacement, is gathering information directly from patients by conducting cross-sectional telephone surveys every six months in Indiana, western Pennsylvania, and Ontario to assess functional status and perceived pain (Freund et al., 1990).

PORTs may use decision analysis. This is a systematic approach to decision making in the face of uncertainty. It includes an explicit formulation of the problem, important outcomes, and alternative choices available to the decision maker. Associated with each outcome is the probability of its occurrence and an estimate of patient values (utilities) for that outcome (Mulley, 1990).

The Harvard and Dartmouth assessment teams, for example, use the methods described above. The Harvard PORT is studying treatment of acute myocardial infarction (AMI), more specifically the use of diagnostic and therapeutic interventions during and shortly after hospitalization for AMI and assessing their value in improving patient survival, health status, functional capacity, and quality of life in the chronic post-MI period. The first project in this effort involves a study of the value of aggressive management of patients during the first three months after an AMI. "Aggressive" in this context is determined by the decision to perform coronary angiography and then revascularization procedures: coronary artery bypass grafting and percutaneous transluminal coronary angioplasty. The PORT is using claims analysis (both Medicare and other data bases) to examine regional variations in use rates of coronary angiography and revascularization and their subsequent effect on patient outcomes and cost of care. The claims data will be used to define longitudinal cohorts of AMI patients starting in 1987 and in all available subsequent years to measure utilization, including hospital admissions for AMI-related procedures, and outcomes.

The PORT's decision-analysis working group will develop probability estimates for decisions to perform elective coronary angiography during the period immediately following an AMI; the group will then compare the results of this analysis to actual use of the diagnostic procedure as measured in claims data bases. Data on key probabilities such as life expectancy and quality-adjusted life expectancy for various patient subgroups will be derived from meta-analysis of the clinical literature. The meta-analysis group continually updates its literature search and performs statistical analyses to pool the results of randomized controlled trials.

The Harvard PORT will also gather primary data from patient surveys and medical records. The objectives of such primary data collection include (1) examining the reasons for regional variations in utilization of angiography, noninvasive tests, and revascularization procedures, (2) examining regional differences in a broad range of outcomes, including functional capacity and quality of life that are not obtainable from the Medicare files, and (3) providing data on case mix, process of care, and patient preferences (Pashos and McNeil, 1990; Chris L. Pashos, Project Director, personal communication, 1991).

The PORT will use the results of these analyses to develop and disseminate recommendations that will be used to help form clinical practice guidelines for targeted physicians. Simultaneously it will conduct a controlled trial of the effects of specific dissemination techniques on physician practice patterns.

The PORT has a senior national advisory board and a regional advisory board. It is seeking the cooperation and support of the American College of Cardiology, American College of Physicians, and the American Heart Association.

The Dartmouth assessment team study of benign prostatic hypertrophy (BPH) and localized cancer of the prostate actually antedates PORT funding. Its early studies of treatment for BPH focused on the reasons for and outcomes of three treatments: transurethral prostatectomy, open prostatectomy, and watchful waiting. The present study incorporates prospective assessments of emerging technologies such as the use of microwave diathermy, balloon dilation, and several new drugs. The Dartmouth assessment team has developed new methods to incorporate prospective evaluation and systematic follow-up of patient cohorts in each treatment arm because so few data on the new treatments are available in claims data bases (Wennberg, 1990a). Rather than employ classic randomized clinical trials, Wennberg and his colleagues rely on "preference trials," in which patients exercise their preferences by determining their treatment, thus providing direct comparison and follow-up of new and existing alternative technologies. The researchers have used large claims data bases to develop probability estimates for such outcomes as operative mortality and reoperations. In addition, they conducted interviews with patients before and after surgery to ascertain any changes in symptoms and quality of life in response to therapy. After demonstrating that surgery based on a preventive theory (that early surgery for BPH prevented later disability) was unfounded, the Dartmouth assessment team became convinced that patients must make their own decisions about surgery based on their symptoms and attitudes about the risks of surgery.

Practicing urologists from many regions of the country are members of the PORT team. Early findings about small area variations in rates of prostatectomy were shared with urologists in northern New England. A concurrent study to examine an apparent elevation in mortality rates for transurethral prostatectomies compared to the open prostatectomy is being conducted in collaboration with the American Urological Association.

3. Dissemination of findings about effectiveness. Based on their analysis and with the involvement of practicing physicians, PORTs are to develop recommendations related to the "prevention, diagnosis, treatment, and/or clinical management of health conditions" (AHCPR, 1990:5). These

recommendations and findings are to be disseminated according to an explicit plan.

To aid patients in making choices based on their symptoms, attitudes, and values, the Dartmouth assessment team has used interactive video technology to present information to patients in a consistent manner, to gather preference and outcome information for all relevant treatment strategies, and to estimate the impact of information on patient treatment choices. Such outcome information includes the likelihood, not now known, that men not undergoing surgery will develop acute urinary retention.

4. Evaluation of the effects of dissemination. PORTs are to evaluate the influence of their findings on provider behavior and assess how information can be most effectively presented to and assimilated by practicing health care providers. Their methods should encourage voluntary change, and their evaluations should assess the extent of reduced variation in practice patterns, more appropriate use of health care resources, and improvement in patient outcomes.

The research being conducted by PORTs is in many ways similar to traditional clinical, health services, and epidemiological research. By combining these types of studies, however, PORTs take on several features that justify giving them special attention in relation to conflicts of interest.

First, PORT research will be used in the development of practice guidelines. As such, PORT findings can be expected to influence medical practice more directly and broadly than the results of a clinical trial of a particular modality.

Second, research results that identify effective and ineffective practices are likely to influence third-party reimbursement decisions under public and private programs. In particular, as a high-visibility program backed by the desire of Congress to reduce health care expenditures, PORT research is likely to affect Medicare expenditures for conditions that are costly in terms of prevalence, technological intensity, or both. The findings of PORTs thus can have significant financial implications for both manufacturers of medical devices and drugs and for practitioners who rely heavily on modalities that may be found to be relatively ineffective.

Third, some PORTs may have multiple sources of funding in addition to federal funding. Finally, PORTs are multi-institutional and multidisciplinary. These last two aspects complicate the usual process of research oversight.

This summary of PORT characteristics suggests some ways PORTs differ from other forms of clinical research, especially randomized controlled trials. The difference between efficacy, as measured in clinical trials, and effectiveness, as measured in the community setting, is the primary distinguishing characteristic of outcomes research conducted by PORTs. Randomized controlled trial protocols are strictly determined at the outset, and outcomes are narrowly defined—generally, as survival or physiologic or anatomic end points. Clinical research faces many uncontrollable or unknown confounding variables—most notably patient variability, variability in health care delivery, variability in the quality of care provided by other health care professionals involved in patient care, the need to rely on data bases that were not designed for research purposes, and changing technology and practices. Clinical trials attempt to control for as much of this variability as possible by defining patient populations narrowly, using selected sites and practitioners, and restricting changes in technology. This makes it virtually impossible for clinical trials to be generalized to practice outside of highly skilled and carefully controlled settings. PORT research, however, seeks to measure the effectiveness of care in community settings. This results in large sources of variability and change which necessitate ongoing investigator judgment about study questions, design, and analysis.

Clinical trials usually compare a limited number of technologies (for instance, two surgical alternatives, or a pharmaceutical treatment compared with a placebo), rather than comparing all alternative treatments including new applications or "off-label" uses of older technologies (Wennberg, 1990a). PORTs, on the other hand, are expected to use retrospective data to compare all existing alternatives and to use prospectively gathered information to augment those data with a variety of outcome measures that are seldom part of randomized, controlled clinical trials.

Another research area that might be compared to the work of PORTs is so-called "pharmaco-epidemiologic" of three kinds: traditional postmarketing surveillance, cost-effectiveness research, and effectiveness research. Postmarketing surveillance involves the traditional recording of drug side effects, including both passive and active reporting systems. One example of an active reporting system is the prescription-event monitoring program in the United Kingdom (Inman, 1990). Cost-effectiveness research, like clinical trials, does not begin from a condition-specific perspective to compare all alternative treatments, but rather compares pharmaceuticals (e.g., the cost-effectiveness of antibiotic A versus that of antibiotic B in a hospital setting).

In searching for ways to promote the value of a new pharmaceutical in comparison to a competitor's, pharmaceutical companies have also

expressed greater interest in measures of clinical effectiveness and have begun to employ a variety of quality-of-life and health status measures (e.g., see Spilker, 1990). This form of effectiveness research does not focus on a condition to compare all alternative treatments either. Of the three kinds of pharmaco-epidemiologic research, work on clinical effectiveness is most similar to PORT research, especially to the extent that it uses measures in the public domain and reports the instruments and results of the research in the scientific literature so that they are accessible to other health services researchers (Bergner, 1990).

PORTs are notable for their combination of many of these methods, which opens up many points for researchers' judgment and discretion. For example, in reviewing the literature, PORT investigators must decide what studies to include and, once included, how much weight to give each of them. Thus, in the publication and peer review of PORT findings, careful attention must be given to the potential biases introduced during any stage of the research—analysis and synthesis of the literature, study design, incorporation of new treatment modalities, data analysis, and allocation of patients to alternative treatment groups. These issues are discussed in greater detail in Chapter 4.

This chapter describes the structure and methods of PORTs. These methods include: review of the literature, including meta-analysis; data collection, including claims analysis, prospective data collection from records and patients, and decision analysis; dissemination of findings; and evaluation of the effects of dissemination. The chapter identifies special aspects of PORT research that distinguish it from clinical research and that justify giving PORTs special attention in relation to conflicts of interest. The next chapter describes the varieties of ties among individuals and institutions that may give rise to conflicts of interest and the ways they have been addressed.