APPENDIX D1. PORT RESEARCH COMPARED WITH CLINICAL RESEARCH: CONFLICTS OF INTEREST IN PATIENT OUTCOMES RESEARCH
David Asch, M.D.1
Several forces in the last two decades have worked to increase the collaboration of industry, government, and academic biomedical research centers.1 First is the increasing scope and expense of medical research programs. Second is the threat of a simultaneous contraction of federal research support for these projects. Third is the competitive pressure to facilitate efficient translation of the products of academic research into marketable goods and services. The resulting growth of different collaborative agreements among universities, industries, and government-supported researchers has highlighted concerns about the kinds of conflicts of interest that can result from these complex arrangements.
An enlarging literature addresses these concerns. It is a given that even the most conscientious investigators cannot eliminate the subtlest biases that affect their work. But when researchers receive support from the companies that produce the products or services they are investigating, when they enter into consulting arrangements with them, or when they share in the ownership of these products, their commitment to upholding professional norms may be or appear compromised. Such personal economic interests increase the likelihood that researchers will lose their
objectivity—consciously or unconsciously—and so widen the opportunities for misleading research results. And even when objectivity is preserved, the public recognition of such potential conflicts of interest may erode confidence in biomedical research.2
The substance of these concerns is illustrated in the recent controversy surrounding several investigators who conducted research supporting the wide indications for an ophthalmic ointment while owning large amounts of the manufacturer's stock.3,4,5 Later research by different investigators challenged the original findings.6,7 Attempts to address these problems have recognized the need to balance the potential for misguided research with the legitimate social goals underlying economic incentives. The many advantages to these economic incentives have been extensively discussed elsewhere. Links between investigators and for-profit concerns create efficiencies not only in the conduct of research, but also in the commercialization and distribution of the products of research. In part because of this continuum between efficiency and conflict, conflict of interest in this setting has defied simple definition.8 There is, nevertheless, widespread consensus that conflicts of interest by any definition need attention from within the biomedical research community. The past few years have seen statements by federal funding agencies,9 universities,10 scientific journals,11 professional organizations,8 ,12 ,13,14 and individual research teams.15
None of these statements, however, distinguishes among various types of biomedical research or among the differences in research goals or methods that may encourage or deter conflicts of interest for investigators. One relatively new form of medical research uses somewhat nontraditional research methods to compare and evaluate the effectiveness of different medical practices in achieving desired patient outcomes. These projects are supported by the Agency for Health Care Policy and Research (formerly the National Center for Health Services Research) under their Medical Treatment Effectiveness Program. Research is conducted by multidisciplinary Patient Outcomes Research Teams (PORTs) using novel methodologies that span a wide area of expertise. The methodologies employed by PORTs are novel enough, and the intensity of their projects are deep enough, to raise concerns that the existing guidelines on conflicts of interest need to be expanded.
This paper introduces principles of outcomes research and examines ways in which members of PORTs may be at risk for conflicts of interest different from those faced by more traditional clinical researchers.
Patient Outcomes Research Teams
The goal of PORTs is to foster effective medical approaches to specific clinical problems. They seek to achieve this goal by evaluating and comparing the outcomes of existing variations in medical practice, and disseminating that information in the form of practice guidelines. The principle underlying outcomes research is that medical practices—for example the use of particular surgical techniques or diagnostic tests—ought to be subject to the same standards of safety and efficacy as are pharmaceutical agents. The measured outcomes themselves ought to be robust enough to include most of the parameters patients typically value: survival, health status, functional capacity, and quality of life. Medical practices will be more effectively and appropriately utilized if their existing variation is perceived as an observational forum for natural selection. Through the Medical Treatment Effectiveness Program those practices that achieve expressed goals will be identified and will continue, while those that fail to achieve these goals will fall into disuse. The hope is that medical practices will evolve in the right direction with this kind of cherry-picking. These natural evolutionary forces may substitute for the expensive, time consuming, and cumbersome randomized clinical trials that have been the traditional way to evaluate alternative treatment strategies.19
Each PORT focuses on a specific clinical problem, for example benign prostatic hyperplasia or gallstones, with the aim of evaluating the medical alternatives to this problem and identifying those that best achieve desired patient goals. While each PORT may be narrowly focused on only one clinical problem, it approaches that problem in an extremely comprehensive way. This depth requires that PORT research be divided into a sequence of interrelated subtasks. The model for PORT research is to: (1) review and synthesize existing published evidence regarding treatment alternatives in order to identify current practices and controversies, to obtain first estimates for decision outcomes, and to identify gaps in the scientific knowledge base; (2) analyze insurance claims databases to estimate the probabilities of relevant outcomes, for example mortality, reoperation, and other complications; (3) interview patients and practitioners to obtain primary data regarding preferences, decision making, and outcome measures not available from insurance databases; (4) develop decision analytic models to provide a rational framework for understanding the tradeoffs in probability and outcome involved in the examined treatment alternatives, and to help explain observed variations in medical practices; (5) formulate practice guidelines based on the findings in the earlier steps; (6) disseminate these guidelines, and examine their effects on service utilization and appropriateness, practice variation, and patient outcome.
The diversity of these subtasks requires that PORTs be complex multidisciplinary teams with expertise in epidemiology and biostatistics, clinical decision and utility analysis, claims data analysis, psychometrics and survey research, medical education, and the relevant clinical disciplines. While these teams may focus narrowly on a single clinical problem, they approach those problems with great depth.
Although PORTs are designed to evaluate existing alternatives in the management of common clinical syndromes, they serve a related goal of linking medical management decisions with specific patient outcomes. Patient preferences for these outcomes are diverse. The better able physicians are to predict the various outcomes of treatment alternatives, the better equipped they will be to tailor their medical management to the individual goals and risk preferences of their patients. Because the personal goals of many patients may be best achieved by medical strategies of ''watchful waiting,''20 resource utilization and the intensity of medical service delivery may actually decrease as physicians learn to better tailor their decisions to reflect patient preferences. On a case by case basis, outcomes research helps not only to define the best treatment alternatives, but also to define how much is too much.
The emphasis that PORTs have placed on individual patient preferences has important societal consequences. Evidence from the studies of benign prostatic hyperplasia suggest that the individual interests of many patients may be best served by less costly medical strategies. 20 It is certainly no coincidence that the opportunity to articulate convincingly that less may be better arrives in the setting of deafening societal cries for reduced health care expenditures. At the same time, the emphasis PORTs place on individual patient preferences, on the understanding of practice variation, and on the development and dissemination of practice guidelines, dovetails neatly with rising patient consumerism, the perception of health care as a commodity, and the recognition of the health care system within an industrial model. Patients and payers have become more influential stakeholders in the medical enterprise, and PORTs evaluate medical treatment alternatives in a format that is particularly well adapted to the interests of those stakeholders. Understanding the effects of power shifts among stakeholders may help us predict the different kinds of conflicts of interest to which PORTs may become susceptible.
Conflicts of Interest in Patient Outcomes Research Teams
Like the outcome research supported by PORTs, randomized clinical trials also are designed to compare and evaluate treatment alternatives
according to predetermined outcome measures. While the outcomes evaluated in clinical trials may be more narrowly defined—survival alone, for example—there is nothing intrinsically different between the goals of the Medical Treatment Effectiveness Program, and the goals of more traditional randomized clinical trials. PORTs rely on insurance claims data systems for their analyses, and so must focus their efforts on existing medical products and services. Only these treatment alternatives have the track record necessary for the kinds of analyses that PORTs do. Prospective clinical trials are not limited in the same way. Aside from this difference in potential subjects for study, however, PORTs and more traditional investigators involved in clinical trials differ only in their methods. By and large they share common goals. PORTs are likely to find themselves susceptible to qualitatively the same kinds of conflicts of interest as more traditional investigators.
Opportunity and Motive
Vulnerability to conflicts of interest is a story of opportunity and motive. Although investigators' personal interests may conflict with their professional responsibilities, they will not really be at risk for betraying those trusts if they have no opportunity for infecting their research with bias. Investigators engaged in clinical research can be influenced, consciously or unconsciously, in their research design, in their interpretation of findings, and in the timing and forum they choose to report their results.11 Peer review does much to reduce these opportunities for bias, but does not eliminate them. Moreover, the peer review system requires that investigators act responsibly in revealing all aspects of their study design so that referees can knowledgeably review the findings. There are abundant opportunities for conflicts of interest in traditional clinical research.
PORTs also are vulnerable to infecting their methods, consciously or not, with interests that conflict with professional research goals. Some might argue that compared with investigators involved in randomized clinical trials, PORTs are less vulnerable because they rely so heavily on secondary data sources that are, by and large, out of their control. Insurance claims data, for example, exist already on computer tapes waiting to be analyzed.21 But within the complex research design and multiple subtasks necessary for outcome research are many areas of vulnerability. The techniques of meta-analysis of the existing literature are highly subjective: some published studies (and, in general, most unpublished studies) will be excluded because they fail to meet criteria for rigor established by the investigators. While insurance claims data tapes already
exist, the data must be "cleaned" and transformed to make it suitable for the kinds of analyses PORTs do. It may be difficult for publication referees to review these transformations. When primary data must be collected about functional status, for example to assess outcomes with greater resolution than mortality or reoperation rates, biases and subjectivity can as easily be introduced as in any other primary research. The development of practice guidelines, although based on the initial subtasks of outcome research, is highly subject to personal interpretation.
These are not necessarily weaknesses of the PORT methodology, but they represent PORTs' vulnerabilities. While PORTs rely heavily on secondary data sources, they do not remain at arm's length from those data. Numerous opportunities exist for conflicts of interest to bias the methodologies of these teams. These opportunities are multiplied by the large numbers of diverse team members required to complete these complex projects. At the same time, pluralistic teams may introduce checks and balances that limit opportunities for bias. PORTs are at least as vulnerable to conflicts of interest as are investigators involved in more traditional clinical research.
These vulnerabilities represent the opportunity for personal interests to alter the direction of research so that it satisfies different goals. What might those interests be? Given the opportunity to redirect research findings, what might be the motives?
Traditional Conflicts of Interest: Funding, Consulting, and Equity
Investigators in PORTs might be subject to the same kinds of conflicting interests as investigators in clinical research. It is not uncommon for investigators involved in clinical research, particularly pharmaceutical research, to have all or part of their research supported by the industries whose products they are evaluating. While outcome research in general is less likely to receive direct funding from industries, industries may understandably look upon PORTs as centers of excellence in their clinical areas and so choose to support related research by members of the team. This is especially likely to occur in PORTs because they are designed to approach narrow clinical problems with depth. Moreover, because of the expense of these large scale research programs, and the incentives to limit unnecessary duplication of research efforts, PORTs may have near research monopolies on their clinical problems. This concentration focuses industry interests on a smaller group. For the same reason, members of PORTs are especially likely to be offered and enter into consulting arrangements with industries that produce related products and services. If PORTs really have their clinical playing fields all
to themselves, the absence of competing and confirming research teams will make the potential end results of conflicts less easily or less quickly detected.
Nevertheless, there are obvious social advantages to such academic and industry collaboration, because it embodies the kind of information and technology transfer that makes for the efficient commercialization of academic findings. Industries often are able to support such research efforts with products, services, and information that would otherwise be unavailable. Both industry and agency funds, for example, may support the staff that is in effect the machinery of the PORTs. While this overlap represents an efficiency, it highlights the difficulty researchers may have in keeping their interests separate. Conflicts may arise because the findings of PORTs in their outcomes research may have profound effects on the industries that support their other research. PORTs are particularly vulnerable to this kind of conflict because they focus on existing services and, among these, the big ticket items.
Besides receiving industry support for related research, and the possibility of having conflictual consulting arrangements, members of PORTs are at risk for conflicts if they have equity interests in related companies. The conflict created when an investigator has a direct financial stake in the results of his research is not fundamentally different between investigators in outcome research and in other kinds of research, but members of PORTs may be particularly susceptible. Like investigators involved in pharmaceutical research, PORTs focus on existing products and services. The end results of their research may have clearer market implications than the results of investigations undertaken at a more basic level. At the same time, industries providing established products and services are more likely to be publicly traded and so offer better defined avenues for exploiting the inside information available to PORTs.
Spinoff Ventures and Intellectual Property Rights
Although PORTs focus on existing medical practices, there are nevertheless opportunities for entrepreneurialism that may conflict with the goals of academic research or may represent the diversion of federal funds for private gain. One not so surprising finding of the PORT investigating benign prostatic hyperplasia, for example, is that the subjective importance of various symptoms and outcomes varies greatly among patients with prostatism.22 If patients knew more about these outcomes and the probabilities of achieving them, they would be better equipped to make informed choices to reflect their personal preferences and goals. For this reason the PORT is developing an interactive
computer video disk to help inform patients of various options and outcomes. The video disk can be perceived as a diagnostic test to determine true patient preferences.16
Such video disks have obvious commercial potential, not only for prostatism, but also for innumerable other common conditions for which patients and practitioners want educational tools. Although the commercial development of such products will serve many social goals, members of PORTs will be uniquely positioned to benefit from these commercial applications because they have investigated precisely those issues necessary for such educational tools. Moreover, their primary data collection on patient preferences for outcomes can be seen as an extensive predevelopment marketing survey that will convey additional advantages in bringing such a product to its commercial potential. Small publication delays will enhance these competitive advantages.
PORTs with different kinds of expertise might be able to commercialize other talents as well. Insurance companies are potentially hungry audiences for information about the appropriateness of various medical practices in different clinical situations, and members of PORTs may be particularly qualified to provide this information. Sideline consulting firms with competitive advantages sustained by PORT research represent, in part, the harnessing of the federally funded PORT machinery for private gain. Even though PORTs are not specifically involved in new product development, they are nevertheless vulnerable to conflicts of intellectual property rights.
Investigators in any kind of research are vulnerable to conflicts that arise because of funding or consulting arrangements with industry, owning equity in evaluated products, or participating in spinoff ventures. PORTs and their members may be vulnerable, in addition, to pressures from stakeholders typically unrepresented in more traditional research. Because PORTs may be perceived as influential evaluators of existing medical practices, for example, they represent a potential challenge to the providers of those services.
Such perceptions may play out in different ways. The fundamental justification for the PORT methodologies is to hasten and improve the evaluation of medical practices. Practitioners heavily invested in existing technologies may feel vulnerable when those practices come under evaluation.23 PORTs rely on such practitioners in their primary data collection about preferences for risks and outcomes. In 1986, 32 percent of urologists' Medicare revenue came from transurethral prostate
resections.24 Can a practicing urologist, who might receive most of his income from transurethral prostate resections, objectively contribute to a study that may suggest that this procedure does not yield the most desired outcomes?25 By the nature of research priorities, PORTs focus on the "cash cows" of clinical services, and physicians may feel that these practices are on trial.
It is not only individual physicians who may introduce these conflicts. While investigators in more traditional clinical trials may develop conflicts through the participation of industry collaborators, PORTs may develop conflicts because of the participation of professional societies. These societies may offer consulting arrangements to PORTs or support them in related research to confirm or refute findings or test new technologies that might increase the portfolio of skills that their members share.26 PORTs may be especially vulnerable to such arrangements because of their relative exposure on their clinical playing fields and the influence they therefore may carry. It is unusual to be concerned about the direct effects of conflicts introduced by clinician stakeholders or their professional societies.
Health insurers are another group of stakeholders likely to be drawn to outcome research. Practice guidelines represent one promising arm in a general strategy to reduce health care costs. The ability to compare the health outcomes of high intensity and low intensity geographic areas introduces the appealing possibility that cost and quality may not always be in conflict. While there are already societal pressures on PORTs to support findings of this sort, insurance companies may intensify these pressures, as well as bring them within arm's length. Like other industries, insurance companies may introduce conflicts through consulting arrangements, or by funding related research. Even without these arrangements, however, PORTs must collaborate with insurers for access to claims data. Even the nonprofit insurance systems cannot be perceived as disinterested in the findings. These potential conflicts are not fundamentally new, but they are pressures from stakeholders not generally represented in more traditional clinical research.
Conclusions: What's Different about Outcomes Research?
If there is really something that distinguishes outcome research from more traditional types of clinical research it is that it was born in a period when power in the medical enterprise was shifting among the various interest groups. PORTs have available to them powerful tools for evaluating the medical profession from the outside. For this reason, they are well positioned to meet the awakening interests of patients and payers,
and they also pose a substantial potential threat to the autonomy of existing practitioners and the integrity of the medical profession. 27 It is difficult to speculate how influential PORTs may become, but their emphasis on evaluation and the development of practice guidelines is consistent with broad trends in health care. Through their efforts, PORTs may facilitate access to information about medical treatment decisionmaking and so increase the chance that market forces of supply and demand— rather than medical professionalism—will determine the efficient delivery of health care. These are distant goals, but the direction is clear and the first steps are taken.
Because the PORT methodology is so well adapted to these social agendas, members of PORTs are vulnerable to the conflicts of interest introduced by those interested in cost containment. At the same time, members of PORTs are likely to feel reactive pressures from clinicians and professional societies struggling to defend their professional and monetary interests from previously silent stakeholders. These are pressures investigators involved in more traditional clinical research are not likely to feel.
The products of outcome research are of concern to more interest groups than are the products of more traditional clinical research. But is outcome research sufficiently different that separate guidelines need to be constructed to prevent the socially undesirable consequences of industry links? The stakes may be higher, and there may be a few extra players, but the object of the game is fundamentally the same. Separate guidelines constructed for different types of medical research establish multiple standards. If certain collaborative arrangements are tolerated in clinical research but not in outcome research, which standard is right? The tendency to gravitate toward the strictest standards can de facto impede future social progress.
Complex interrelationships among governmental, academic, and industrial institutions create opportunities for social progress while creating opportunities for potentially destructive conflicts of interest. Restrictive guidelines that eliminate the possibility of conflict will also eliminate opportunities for the efficient translation of research findings into marketable goods and services. The best institutional guidelines will not eliminate industry collaboration, but will help investigators navigate toward its socially desirable products and away from its socially undesirable products. Those who write guidelines must fight the urge to increase restrictions and instead work for specificity in their applications. What is needed is clarity, not increased strigency.
1. Hanna KE. Collaborative research in biomedicine: resolving conflicts. In: Institute of Medicine. Government and Industry Collaboration in Biomedical Research and Education. Washington: National Academy Press. 1989.
2. Relman AS. Economic incentives in clinical investigation. New Eng J Med 1989;320:933–4.
3. Booth W. Conflict of interest eyed at Harvard. Science 1988;242:1497–9.
4. Lichter PR. Biomedical research, conflict of interest, and the public trust. Ophthalmology 1989;96:575–8.
5. Tseng SCG, Maumenee AE, Stark WJ, Maumenee IH, Jensen AD, Green WR , Kenyon KR. Topical retinoid treatment for various dry-eye disorders. Ophthalmology 1985;92:717–27.
6. Soong HK, Martin NF, Wagoner MD, et al. Topical retinoid therapy for squamous metaplasia of various ocular surface disorders: a multicenter, placebo-controlled double-masked study. Ophthalmology 1988;95:1442–6.
7. Gilbard JP, Huang AJW, Belldegrun R, Lee JS, Rossi SR, Gray KL. Open-label crossover study of vitamin A ointment as a treatment for keratoconjunctivitis sicca. Ophthalmology 1989;96:244–6.
8. Council on Scientific Affairs and Council on Ethical and Judicial Affairs. Conflicts of interest in medical center/industry research relationships. JAMA 1990;263:2790–3.
9. U.S. Department of Health and Human Services. Request for comment on proposed guidelines for policies on conflict of interest. NIH Guide for Grants and Contracts 1989;18(32):1–5.
10. Harvard University Faculty of Medicine. Policy on Conflicts of Interest and Commitment. March 22, 1990.
11. Relman AS. Dealing with conflicts of interest. N Engl J Med 1984;310:1182–3.
12. American Thoracic Society. The potential for conflict of interest of members of the American Thoracic Society. Am Rev Respir Dis 1988;137:489–90.
13. Association of Academic Health Centers Task Force on Science Policy. Conflicts of Interest in Academic Health Centers. 1990.
14. Association of American Medical Colleges. Guidelines For Dealing With Faculty Conflicts of Commitment and Conflicts of Research. February 22, 1990.
15. Healy B, Campeau L, Gray R, et al. Conflict of interest guidelines for a multicenter clinical trial of treatment after coronary-artery bypass-graft surgery. N Engl J Med 1989;320:949–51.
16. Wennberg JE, Mulley AG, Hanley D, et al. An assessment of prostatectomy for benign urinary tract obstruction. JAMA 1988;259:3027–30.
17. Stewart AL, Hays RD, Ware JE. The MOS short-form general health survey. Reliability and validity in a patient population. Med Care 1988;26:724–35.
18. Stewart AL, Greenfield S, Hays RD, Well K, Rogers WH, Berry SD, McGlynn EA, Ware JE. Functional status and well-being of patients with chronic conditions: results from the medical outcomes study. JAMA 1989;262:907–13.
19. Greenfield S. The state of outcome research: are we on target? N Engl J Med 1989;320:1142–3.
20. Barry MJ, Mulley AG, Fowler FJ, Wennberg JW. Watchful waiting vs immediate transurethral resection for symptomatic prostatism: the importance of patients' preferences. JAMA 1988;259:3010–17.
21. Wennberg JE, Roos N, Sola L, Schori A, Jaffe R. Use of claims data systems to evaluate health care outcomes: mortality and reoperation following prostatectomy. JAMA 1987;257:933–6.
22. Fowler FJ, Wennberg JE, Timothy RP, Barry MJ, Mulley AG, Hanley D. Symptom status and quality of life following prostatectomy. JAMA 1988;259:3018–22.
23. Council on Long Range Planning and Development. Health care in transition: consequences for young physicians. JAMA 1986;256:3384–90.
24. Health Care Financing Administration. Part B Medicare Annual Data Base System. 1986.
25. Loos NP, Wennberg JE, Malenka DJ, et al. Mortality and reoperation after open and transurethral resection of the prostate for benign prostatic hyperplasia. N Engl J Med 1989;320:1120–4.
26. Brailer DJ, Nash DB. Uncertainty and the future of young physicians. JAMA 1986;256:3391–2.
27. Ellwood PM. Outcomes management: a technology of patient experience. N Engl J Med 1988;318:1549–56.
APPENDIX D2. PORTS: THEIR IMPACT ON HEALTH SERVICES RESEARCH, TECHNOLOGY INNOVATION, AND PAYMENT POLICY
Judith Lave, Ph.D.1
I was asked by the staff at the Institute of Medicine to write a paper on the impact of PORTs on health services research, technology innovation and on payment policy. They asked that the paper be a provocative and forward looking document rather than a scholarly one. It is difficult to resist such a charge. But before turning to my charge, it is important to discuss the PORTS themselves briefly.
PORTS are the major component of the medical effectiveness initiative of the Agency for Health Care Policy and Research. PORTS and the POARP studies that preceded them are large scale, multi-faceted multi-disciplinary projects that must meet special requirements. The goals of the PORT project are to "identify and analyze the outcomes and costs of current alternative practice patterns in order to determine the best treatment strategy and to develop and test methods for reducing inappropriate variations." The basic model to be followed is one developed by John Wennberg and his colleagues at Dartmouth University. Each PORT will be funded at a level of at least $5 million dollars. PORTS thus represent a huge commitment of resources.
PORTS are the most visible symbol of a new era in health services research. In recent years there has been an increased effort to expand the boundaries of traditional research endeavors. In my field, economics, researchers have moved beyond examining the linkage between payment policy and utilization change to examining the effect of utilization changes
on health outcomes. In the clinical arena, researchers have moved from studying the effect of an intervention on specific clinical indicators to examining its effects on a broader set of health status measures. RAND has brought us both the National Health Insurance Experiment and the Medical Outcomes studies. Dartmouth has brought us the variations study and almost everything you want to know about prostatism. The University of Pittsburgh has not been isolated from these trends. In the past year I have been involved in the development of proposals to study the cost effectiveness of alternative methods for treating depression, to examine alternative treatments for epilepsy, to evaluate a geriatric assessment project, and to investigate enteral feeding and COPD. These projects, which are still in the developmental or review stages, are characterized by their interdisciplinary nature.
With this introduction, I will now turn to my charge and consider briefly PORTS and health services research, technology innovation and payment policy. This is obviously a mammoth and impossible charge—and so I will only touch on the more salient issues.
Health Services Research
The effect of the PORT initiative on the field of health services research should be significant. It should lead to a nonmarginal expansion in the relevant human capital because it will further legitimize health services research within academic medicine as well as attract economists, sociologists and others to the field. Thus, young and ambitious researchers will be attracted to health services research because it will be viewed as an area of inquiry that will be blessed with federal funding. This expansion of capacity will increase the likelihood that the effectiveness initiative will be a successful one.
PORT projects should lead to significant methodological advances in a number of different areas. I expect that we will make significant advances in the analyses of claims data. We will develop better methods of linking files to produce information on episodes of illness as well as in assessing the effect of treatment by determining surrogate health status measures from the procedure and diagnostic information contained in such files. We will make significant improvements in the development of health outcomes measures—and in creating measures that are useful for policy makers and clinicians. I also expect that there will be some advances made in methods for disseminating information. I am hopeful that the improvements in methods which result from this initiative will exceed those resulting from the RAND national health insurance experiment another
large research project which brought together researchers from the many disciplinary backgrounds that make up the field of health services research.
Let me discuss outcomes measurement in more detail. I expect that PORTs will evolve in such a way there will be some outcome measures that will be used by each of the teams. This will allow researchers to compare the effectiveness of treatments both within and across conditions. I also expect that there will be major advances in determining methods of finding out what is important to patients in treating specific conditions. This will create a variety of outcome measures that will complement the traditional set of outcome measures used by clinicians.
It is useful to look at some specific characteristics of PORTS. Here I consider briefly the structure of the design of the PORTS, the cost of implementing the PORT initiative, and some of the implications of developing visible centers of expertise on certain conditions.
The structure of the funded POARPs and PORTs is modeled after the study on prostatism that was pioneered by Wennberg. Briefly, large interdisciplinary teams will conduct meta-analyses of the condition of interest, examine claims data files, undertake some prospective clinical studies, develop models of care, inform practicing physicians about their findings, implement educational programs to change physician practice patterns, and evaluate the success of these educational interventions. This is a set of heroic tasks.
This project design has many strengths. One of the major strengths is that the condition of interest will be examined by a multidisciplinary team. This means that the perspective that each of the disciplines brings to bear on the problem should be integrated in the various analyses undertaken by the team. I have often been impressed at how little members of one discipline search out the research results of scientists in another discipline, even when they are both examining aspects of the same question. For example, do gynecologists know as much about post-hysterectomy depression as do sociologists, psychologists and psychiatrists?
As an aside, it should be pointed out that the multidisciplinary nature of the team may mitigate against some of the conflict of interest concerns. Each team will have to work out arrangements for managing conflict and for determining rules for accessing data collected by it. It is likely that the working arrangements that need to be developed to keep the multi-site, multidisciplinary group together will lessen the probability that bias in the point of view of a particular investigator will be reflected in the judgments taken by the team or in the final analyses.
Second, since the design of each PORT project is similar, each team will face a common set of problems, and working groups have been (or will be) established to address some of these problems. As noted above, this should lead to significant advances in analytical methods and measurement. Additionally there should be enough similarity in the analytical approach taken by each PORT that it will be relatively easy to compare their findings.
Third, I expect this research initiative to lead to improvements in how we measure and convey differences in the effectiveness and the cost effectiveness of treatments for different conditions. Although we will learn that some practices are simply inappropriate, and that some are in fact harmful, we will also learn that different practices differ in their effectiveness or that they are more or less effective for different clinical populations and in different settings. Health services research will have to do research on better methods of ''risk assessment'' and "risk communication."
Possible Threats to PORT Success
There are, however, some problems with this project design that may threaten the success of the PORTs. Given the visibility of PORTs, it is important to be aware of such threats. Some of these are discussed below.
Flexibility. The approach taken is fairly inflexible. Although the POARP and PORT studies are not contracts, there may be a tendency for the project officers to "force" the projects into a standard approach. Although, as noted above, standardization is a positive feature of the PORTs initiative, if it is too rigidly enforced it could restrict, rather than encourage, creativity.
Use of Claims Data. One has to be careful not to put more weight on the findings that the nature of the analytical data can support. We have come a long way in making use of claims data, and no doubt we will move much further along as a result of the PORT initiative. However, claims data do not contain clinical information nor do they contain much information on specific treatment technologies. This is particularly true in cases where the procedures under investigation involve the insertion of devices or prostheses which are continually being modified. Claims analyses will often lead to hypotheses that must be tested through the collection of additional primary data rather than to concrete conclusions.
Prospective Data Collection. All the PORTs are planning to collect data prospectively. In general, these studies will be conducted in a limited
number of sites. These studies are designed to study how medical care is actually practiced, and by using statistical controls, to determine whether there is any difference in the outcomes associated with different practices that are in effect today. This prospective component is a major strength of the project. (The alternative strategy is to conduct large multi-site clinical trials.) However, critics may argue that the sites are too limited and that therefore the results are not generalizable, or that the types of questions being addressed by the teams should be asked only in the context of a clinical trial. These criticisms are more likely to be raised if the findings of the teams are used in setting payment policy.
Study Population Size. The clinical condition of the patient can vary significantly. The PORT findings about clinical effectiveness could be subject to the criticism that they were based on inadequate data. Are the proposed clinical studies large enough so that it will be possible to determine the effect of treatment on many different groups of patients?
Legal Implications of PORT Recommendations. Each PORT is multidisciplinary. However, there is one discipline that is lacking—the legal one. If the PORT teams end up making recommendations about the practice of medicine, are there any legal implications to these recommendations about which they should be aware?
Scope of Task. Finally, PORTs are responsible for undertaking many tasks. I am concerned that the research programs may be too ambitious and that, as a consequence, they may fail at some of their assigned tasks. In particular, PORTS are supposed both to implement an educational program to inform physicians about their findings and then to determine whether the educational programs have influenced physician practice patterns. While there is no doubt that this is a critical component of the PORT initiative, is it reasonable to expect that successful campaigns can be mounted; and that, even if the programs are successful, that their success rates will be discernable over this time period? Although a number of studies have shown that physician practice patterns do respond to economic incentives, the effects of educational strategies are not encouraging—particularly in the short run. This point is made in an editorial in the May 24 1990 New England Journal of Medicine which was commenting on a study in which researchers tried to change physician test ordering behaviors:
I often hear physicians lament the difficulty of changing their patients' behavior regarding issues such as diet and smoking, even when the medical data are incontrovertible. Despite our common
frustration with individual patients, however aggregate changes in life style have probably contributed substantially to the impressive decline in mortality from coronary heart disease in the United States. Perhaps we as a profession should complain less about the difficulty of altering patients' behavior and learn more about how to change our own lest the pot be accused of calling the kettle black.
Cost of the PORTS
The PORTS are very expensive, and the expense of the PORTS may have some implications for the field of health services research. Three implications are discussed below.
Allocation of Health Services Research Funds. A significant proportion of the funds of the new agency are to be allocated to effectiveness research. It is possible that this area of research will flourish while others will be seriously underfunded. Some of these areas, such as research on alternative methods or strategies for paying for new and evolving technologies and research on alternative delivery systems, are complementary to the effectiveness initiative.
Allocation of Effectiveness Research Dollars. There are many approaches that can be used to study the effectiveness of medical care. In the short run, PORTs may absorb most of the effectiveness dollars. This, of course, will mean that most of the funds will go to a small number of researchers located in a few centers across the country. This strategy could stifle the research efforts of other individuals who are interested in effectiveness work but who are not interested in undertaking projects on the scale of the PORTs. However, Gordon De Friese at University of North Carolina does not believe that this will be a significant problem. He argues that we are simply going to illustrate that it is the well-designed study that can answer the types of fundamental questions raised by the PORTs, not just the size of the grant that supports the research.
Allocation of Resources to Study a Particular Condition. How will the existence of PORTs influence decisions to fund other proposals to study a condition which is being examined by a PORT? Can anybody but Wennberg get funded to do health services research for prostatism? If somebody submits a proposal to examine knee surgery; will the "knee jerk" response be that "Indiana is taking care of that?" Along the same lines, how will ROls by members of the PORT teams to study additional questions related to the PORT be received? Given a limited research
budget, it may be appropriate that additional funding to study a given condition be limited, although such decisions need to be carefully assessed.
Concentration of Expertise
The existence of PORTs will mean that some centers and individuals will develop a particular type of expertise on treatment approaches for a particular condition. Does this concentration of expertise have any important implications for the field of health services research?
Consulting. Expertise in the field will mean that the members of the teams will be called upon to use that expertise in the role of a consultant. Drug companies, firms that manufacture prostheses, and home health associations beckon. Is there anything different about PORTs? Do the members of PORTs need to follow a different set of rules or guidelines as they consider the options available to them? The answer to this question depends, in part, in whether PORTs are viewed as being significantly different from other groups of researchers who study and develop expertise in specific conditions. However, I would imagine that the same conflict of interest and consulting arrangements that guide faculty members should be applicable here.
Contracts. Members of the PORT teams will have a comparative advantage in responding to any Requests for Proposal or Program Announcements related to the condition of interest. They will have assembled data bases, organized networks of providers, and obtained an "institutional" knowledge base that should enhance their ability to compete for funds. However, they are no different from other expert teams who because of their expertise have an advantage in the competitive market place.
Reviewers. Because of their expertise, members of PORTs will be called upon to serve as reviewers of other proposals related to their field or of articles that have been submitted for publication. However, it is unlikely that this role is likely to give them any measure of "control" in a particular field. Although they are experts, there will be many other experts in the field who also will be called upon to serve these functions. Although PORT teams have received significant levels of funding from the new agency, the size of this funding fades into insignificance given the amount of funding other researchers have received from NIH.
It is difficult to determine the impact of PORTS on technology innovation without a clarification of the role of PORTs. PORTs could be considered to be simply groups of investigators who are conducting large scale research projects on particular conditions, or they could be considered quasi-regulatory groups whose mandate it is to evaluate new and emerging technologies that are related to the condition of interest. Initially, PORTS followed the earlier model. They focused on general treatment strategies such as the decision to operate, or the decision to admit patients to an acute care facility rather than treating them on an ambulatory basis. The practice patterns examined by PORTS were established practices, and it is difficult to see how studies of this sort would have much of an impact on technology innovation. However, it would be naive to assume that PORTs should or will continue to have such a benign role. The members of the teams are in an ideal position to identify new treatments and new procedures as they are being introduced into medical practice. They also may be called upon to conduct some technology assessments by manufacturers who may want a "PORT Seal of Approval."
One force leading to the effectiveness initiative was that procedures are often introduced into medical practice without formal evaluation. 1 Thus PORT researchers will be called upon to do and will initiate a number of technological assessments. From the perspective of health services research, the teams should improve methods for technology assessment. In addition, the teams should make advances in determining:
(a) methods for deciding which technologies should be evaluated, (b) methods for deciding at what particular part of the product life cycle a technology should be evaluated and (c) methods for determining when a product should be reevaluated.
Technology assessment has become an important activity. It is difficult to see how PORTs will be any more than one of the many participants in this activity. However, I think that we can be sure that the ultimate effect of all of this evaluation will be that technology innovation and diffusion will be different than it would have been in the absence of such activity.
The research conducted by PORTS will generate much needed information about the effectiveness of certain treatments. PORTs must also determine methods for disseminating their findings both to practicing physicians and to patients. We can expect this information to be used in many ways.
Kinds of Uses for Information
Physician Decision Making. PORTs should provide information to physicians that will influence their decision making. In fact, as noted above, each PORT is required both to develop strategies for disseminating their findings to the physician community and to evaluate their impact on physician practice patterns. Thus physicians will have much better information about what works and what does not, and on how effective certain treatments are. They will also have better information about the implications of alternative treatment strategies. This should both influence their practice directly as well as influencing the type of information that is given to patients as treatment options are considered.
Consumer Choice. Information about PORT findings should facilitate consumer choice. Different treatments for the same condition will have different outcomes, and patients will place different values on these outcomes. For example, surgery for prostatism may decrease symptoms of prostatism; at the same time it carries its own risks. The Dartmouth team has developed videos that convey to patients the implications of "watchful waiting" versus "surgery"—information that is designed to enable patients to make more informed choices about their therapies.
Managed Care. The nature of the data generated by PORTs should be useful to organizations such as PPOs and HMOs in their analysis of the practice patterns of physicians who are either affiliated with these organizations or who are being considered for possible affiliation. These organizations will have much better information against which to assess the practice patterns of physicians who manage the kinds of conditions that are the focus of specific PORTs.
Payment Guidelines. We can expect that PORT findings will be used in the development payment guidelines by the Medicare program, the Medicaid programs, and by other third parties that reimburse providers on a fee-for-service basis. At minimum we would expect payors to develop policies to deny payment for treatments or procedures that were found by PORTS to be harmful or inappropriate. We would also expect the information to be used to form the basis for guidelines on care that was found to be marginally ineffective.
Magnitude of Impact
While we can expect that the findings of PORTs will certainly influence payment policy, it is difficult to determine the extent to which they will influence payment. The nature of medical treatment is complex. (The term treatments is used generically here—it can be used to refer to a procedure, a device, a decision to admit, etc.). Some treatments may be clearly useless; some treatments may be clearly harmful with no offsetting benefits; and some treatments may be more costly but equally as effective (along all relevant dimensions) as others. It should be easy to agree that such treatments should not be paid for. However, most cases are not so clear cut. Many treatments can be useless in some cases, but useful in others in others; or harmful in some cases and helpful in others. It is sometimes necessary to have a considerable amount of clinical information in order to determine whether a particular treatment is likely to be "beneficial," and it will be difficult for payors to clearly define the conditions under which some treatments will be paid for without developing an administrative structure that absorbs much of the savings. This is particularly true for many of the diagnostic tests.
Even more important, treatments can be very beneficial, somewhat beneficial or marginally beneficial. Again, some treatments may have positive impacts on some health status indicators but have no impact on other measures. PORTs will provide a considerable amount of information about the effectiveness of treatment. However, in order to use some
of this information the payors will have to make decisions about what it is they want to pay for.
The PORTs initiative is of great importance. They are the most visible symbol of a new way of looking at health care in the United States. PORTs are being funded because it is hoped that they will provide information that will lead to an improvement in the quality of medical care delivered in the United States, and that they will promote the delivery of cost-effective care.
I believe that the research conducted by PORTs will, indeed, lead to improvements in the quality of health care provided and will promote the delivery of more cost-effective care. However, I am concerned that the effectiveness of this initiative will be judged by the Congress by whether it has had a measurable impact on the cost of health care in the United States, and I do not think that it will have a measurable impact on costs.
Costs have not been increasing because the amount of ineffective care provided has increased, but rather because of the spread of new diagnostic testing methods and treatments. Henry Aaron has said it this way, "There is no indication that the technological creativity that has been largely responsible for the very rapid growth in medical outlays is abating. Scientific imagination that has given us various kinds of transplants and the new methods of treatment and diagnosis that have been driving up expenditures, is likely to push even harder in the future." Others would argue that the increase in the number of physicians, per se and the increase in the number of specialists has an independent effect on costs. If these are major sources of growth of medical expenses, it is highly unlikely that the findings of PORTs will be powerful enough to have an impact on that trend.
The myth at the moment is if only we could get rid of "inappropriate" care, we would have enough resources to pay for all effective care. Consider a familiar example—the per capita cost of medical care is higher in Boston than it is in New Haven, and there are no data to suggest that this difference has resulted in different health outcomes. However, it is not clear that if one carefully examined the practice patterns of individual physicians in Boston that they would be found to be any more "inappropriate" than the practices of individual physicians in New Haven, although they would be found to be different. We do not know how to use payment policy to make providers in Boston practice medicine in the same way as they do in New Haven.
The PORT program is an important initiative for the field of health services research. Not only should it lead to an expansion of the capacity in the field, it ought to result in research that will result in significant methodological strides. In turn, the research should generate findings about the effectiveness of treatments and the relative effectiveness of different treatments for the same conditions so that one can assess their cost effectiveness. Since the researchers are charged with developing methods to disseminate their findings, the initiative should lead to improvements in the quality of health care and in the efficiency with which care is provided. However, many of these changes will be quite subtle, and it may be difficult to measure them.
There is some likelihood that PORTS will be evaluated on the basis of their impact on the overall level of health care costs in the United States. If this is the case, then PORTs will be doomed to failure. There are many factors contributing the rising costs; given the current level of funding, it is highly unlikely that PORTs will have a significant impact on those factors.
Goldman, L. (Editorial) Changing Physician Behavior: The Pot and the Kettle. New England Journal of Medicine 322:1524–1525, 1990.
Heilbrunn, S.M. (Letter to the editor) Indiscriminate Use of Laser-assisted Angioplasty. New England Journal of Medicine 322:1531nnn, 1990.
Strandness, D.E., Jr., Barnes, R.W., Katzen, B.T., et al. (Letter to the editor) New England Journal of Medicine 322:1531–1532, 1990.
APPENDIX D3. THE EVOLUTION AND CURRENT STATUS OF CONFLICT OF INTEREST REGULATION IN BIOMEDICAL SCIENCE
Barbara McNeil, M.D., Ph.D.
Michael W. Roberts, J.D., Ph.D.1
This effort to formulate guidelines regulating conflicts of interest arising in connection with the Patient Outcomes Research Teams (PORTs) coincides with a temporary lull in a rising clamor of public and regulatory concern in this area. The problem, as succinctly stated recently by the Executive Council of the Association of American Medical Colleges, is the fear that "financial or other personal considerations may compromise, or have the appearance of compromising, an investigator's professional judgment in conducting or reporting research."1 A well-documented increase in the number and importance of academic-industrial relationships over the last decade2 has added to concerns that the excellence of American academic science might become a casualty of these new financial influences, or as Rep. Ted Weiss bluntly frames the question, "Is Science for Sale?"3 This concern is particularly intense in relation to clinical medicine, in which the implications of scientific innovation for public health, welfare and safety are more obvious and immediate.
A review of recent developments in the regulation of conflicts of interest in biomedical research suggests that the workshop may wish to surpass the minimal standards currently mandated by the federal agencies and most institutions as well as those proposed by clinical and educational consortia. These standards indicate that at a minimum, the workshop should recommend a policy of full confidential disclosure by all PORT participants of financial interests in businesses in a position to profit from products or procedures under examination ("interested businesses"). Recent developments in regard to self-regulation by investigators in clinical trials also suggest the wisdom of a self-denying rule whereby all key scientific participants in PORTs would forego financial ties to interested
businesses during the course of such studies and for a period of time afterwards.
Over a twenty-five year period, the recurrent theme in regard to conflict of interest has been self-regulation in anticipation of standards imposed by Washington. At the height of federal support for academic research in the mid-1960's some agencies, notably the Atomic Energy Commission, sought to deter conflicts arising from undisclosed financial interests in connection with research paid for by the government, e.g., the purchase of equipment from, or the orientation of research to benefit, a private firm in which the faculty member had an interest. At that time, stringent government-wide regulation was forestalled when research institutions through their consortia (the American Association of University Professors and the American Council on Education) resolved to adopt individual institutional standards providing advice and guidance to faculty in planning outside relationships.4
Within a short time, most major research institutions adopted policies which rejected ''comprehensive and detailed codification" while providing very general guidelines by which faculty were to determine the existence of a conflict. Many delineated activities which were "clearly permissible," such as receipt of nominal honoraria and publication royalties, from those which, while requiring some degree of review, were generally allowable with disclosure, such as most faculty consulting. A third category consisted of activities thought ''likely to involve conflicts," including, for example, the assumption of executive responsibilities with a for-profit company. It was a characteristic of most of these early policies that they vested in faculty discretion to decide when a conflict existed, in which case there was to be disclosure to the departmental chair or supervisor, who would consult with the dean in appropriate cases.
This structure survived with minor alterations until the early 1980's, when the rise of biotechnology led to a new era of academic-industrial cooperation as business, particularly in the life sciences, turned to academe for scientific expertise.5 These relationships brought enormous benefits for both sides and for the public, which the Congress recognized in enacting federal legislation intended to foster such collaborations by permitting academic institutions and scientists to benefit financially if their federally-sponsored research led to commercial products.6 At the same time, the spiraling growth in the costs of biomedical research began to outpace federal funding for such activities, requiring universities and hospitals to seek new sources of support.
These developments were greeted with concern in some quarters, and in 1981 the Oversight Subcommittee of the House Committee on Science and Technology called on the American Association of Universities (AAU) to develop ethical guidelines for university-industry collaboration, pointing with alarm to "the metamorphosis of our scientific research force from educators to entrepreneurs . . . "7 After considering the Subcommittee's report, the AAU Committee on University-Industry Relations concluded that uniform guidelines were unnecessary but agreed to facilitate the sharing of information on research collaboration among universities, industry, government and the pubic, and established a Clearinghouse on University-Industry Relations for this purpose in 1983.8
Many institutions moved at this time to revise existing policies. Some, such as Yale, commenced institution-initiated disclosure in the form of an annual report of the level of commitment and the organizations involved in all non-university professional work, including consulting, equity, board memberships and managerial positions.9 Others, such as Harvard, retained faculty-initiated mechanisms, but sought to refine and improve policies regulating university-industry sponsored research agreements and concomitant bilateral agreements between such companies and faculty.10 Such policies were intended to insure:
the priority of the institution's commitment to education and training and to the development of basic knowledge in preference to commercial applications;
the intellectual integrity of the appointments process;
the openness of agreements and relationships and a disfavor of secrecy; and
the unfettered freedom of faculty to choose the direction of their research and whether and when to publish.
Throughout this period, institutions continued to stress voluntary disclosure and avoided per se prohibitions.
The regulation of academic conflict of interest was also affected in the 1980's by the rise of state laws governing this area. In a number of states local laws applicable to public university or hospital employees will consequently dictate conflict of interest rules for public institutions. Hence university employees are sometimes forbidden to participate in an official capacity in, or receive extra compensation in connection with, a transaction between the university and a business entity in which the employee has a substantial interest. Public university scientists are often also precluded from accepting employment which might impair their independence of judgment in the performance of their university duties and from having investments in a business entity which would create a
substantial conflict between their private interests and their university responsibilities.11 Such statutes are diverse, and state court decisions interpreting them are generally few in number and more likely to deal, for example, with issues such as participation by professors in state agency proceedings.12 Certainly a state university or hospital employee must carefully consider any applicable state statute in planning outside relationships.
It should also be noted that since the mid-1980's some federal agencies have prohibited "organizational" (as opposed to "personal") conflicts of interest, in which the institution's self-interest could be thought to interfere with its obligations as a government contractor.13 By this means, such agencies seek to avoid practices such as ''wired bidding," whereby a contractor designs a request for proposals on which it subsequently bids. While these rules appear to be rather narrowly focused, their definitions are sometimes broad enough to reach some personal conflicts, e.g., a "relationship . . . whereby . . . a contractor (including chief executives, directors, proposed consultants or subcontractors) has interests which (1) may diminish its capacity to give impartial, technically sound, objective assistance or advice. . . . ''14 Although there is currently no Public Health Service (PHS) rule regarding institutional conflicts, the workshop may wish to consider at least a mechanism to prevent wired bidding on subcontracts. One possibility would be to include on the proposal form for subcontracts a question, "Did you participate in the preparation of this RFP?" An affirmative answer would require the AHCPR to consider whether it wished to waive the conflict in appropriate cases.
Recent Concerns Regarding Clinical Studies
Concern about conflict of interest has intensified in the last two years as a result of a few egregious cases receiving great attention. One such case involved a medical researcher, Dr. Sheffer Tseng, who undertook direction of a clinical study at the Harvard-affiliated Massachusetts Eye and Ear Infirmary (MEEI) on a therapeutic agent, the rights to which he had assigned to a company, Spectra Pharmaceutical Services, Inc., in which he subsequently became a major shareholder. 15 The FDA-approved protocol, which was designed to assess the effectiveness of topical treatment with a Vitamin A analogue (tretinoin) for the "dry eye" syndrome, was funded by Spectra and supervised by Dr. Kenneth Kenyon of MEEI, who also acquired stock in Spectra after commencement of the study. In the course of the study, Dr. Tseng deviated significantly from the protocol without approvals and failed to report discouraging results of his research for a number of months. When these results were publicly
announced by the company in 1987, the stock price fell significantly. It has been reported that Tseng had sold his holdings by this time and realized a large profit. An institutional review at Harvard Medical School concluded, inter alia, that "a significant conflict of interest developed after the clinical study began, (that] proper safeguards were not in place to protect the study from potential bias; and that Dr. Tseng was improperly supervised."16
The Tseng case sharpened the focus of discussion with respect to conflict of interest upon the particular problems associated with clinical medicine. During 1989, articles appeared in the Wall Street Journal, New York Times, Boston Globe and other publications, pointing to the increasing frequency of financial interests, particularly equity stakes, held by researchers in firms whose products they evaluated. 17 Among the situations reported:
activity by a prominent scientist in publicizing and promoting a major new drug while simultaneously holding options to acquire a significant stock interest in the manufacturer of that drug;
receipt by one researcher of stock options from the sponsor of his work on an antiviral preparation with a portion of the options contingent upon FDA approval of the drug;
a NIH study on the efficacy of antibiotics on ear infections in children conducted while the principal investigator received private research grants, speaking honoraria and travel expenses from companies that manufactured the tested drugs;
concern that consulting and stockholder relationships between a small pharmaceutical company and the Dean and four of five department heads of a College of Pharmacy could have influenced the evaluation by the institution's researchers of claims of toxicity with respect to company products developed by one of the stockholding scientists; and
questions concerning the possibility that equity or option-holding scientists could have delayed or minimized data about potential complications of their company's drug.
The subject was seized upon in Congress by Representatives Ted Weiss (Chairman, House Government Operations Committee's Subcommittee on Human Resources and Intergovernmental Relations (the oversight committee of the Department of Health and Human Services]) and John Dingell (Chairman, Energy and Commerce Committee and Chair of its Subcommittee on Oversight and Investigations). Rep. Weiss held hearings in September 1988 at which witnesses testified that some investigators in Phase II clinical trials of anti-blood clot medications (the TIMI or "Thrombolysis in Myocardia Infractions" trials) held stock or options in the
company manufacturing one of the drugs (Genentech and TPA).18 Weiss has continued to be vocal, holding a second round of hearings in 1989 and publishing several articles, of which the following excerpt is representative:
One recurrent finding has been that universities can be surprisingly blind to potential conflicts of interest; they are adamant that consulting fees and other forms of remuneration will not affect the impartiality of their faculty members . . . When the federal government is paying for the research, that research should not be tainted by any possibility of bias due to financial conflict of interest."19
NIH Activity in the Area
As the principal funding support for the biomedical scientific community, the National Institutes of Health (NIH) inevitably stepped into this debate, beginning in 1987 with an amendment of the PHS Grants Administration Manual. This new rule on "Standards of Conduct for Employees" required grant recipients to "establish safeguards to prevent" the use by investigators of their positions for purposes "that are, or give the appearance of being,'' motivated by a desire for private financial gain.20 Grantee institutions were also required to implement "written policy guidelines on conflict of interest": (1) indicating proper and improper relationships; (2) providing for notice of such conflicts by investigators to a grantee official; and (3) specifying possible sanctions. This development was greeted quietly since the agency had not attempted to prescribe substantive standards and was only requiring what most research institutions had already had in place for some time.
In the wake of the Tseng case, a further step followed in January 1989, when an amendment to the NIH Guide to Grants and Contracts stated the agency's "expectation" that participating investigators "will not have financial interests in organizations that produce drugs, devices, or other interventions studied in a controlled clinical trial."21 In June, NIH jointly sponsored with the Alcohol, Drug Abuse and Mental Health Administration (ADAMHA) a two-day meeting on conflict of interest including representatives of academe, industry, independent research institutions and government.22 This was followed in September by the publication for comment in the NIH Guide of a far-reaching but highly ambiguous document entitled "Proposed Guidelines for Policies on Conflict of Interest." ("Proposed NIH Guidelines").23
These guidelines would have required, in essence, the avoidance of any prospective financial conflict of interest in connection with an NIH award. Specifically prohibited were:
receipt by investigators of equity or stock options in any company that would be "affected" by the outcome of NIH or ADAMHA research or whose product or equipment was being evaluated in such research;
receipt by investigators of honoraria or fees from, or engagement in a management position by, a company if the recipient were involved in any NIH or ADAMHA project evaluating or testing that company's product;
disclosure of agency-funded research results to any company "with which a conflict exists" until such results are "publicly available."
Waivers to these rules would have been permitted only where the financial interest involved was "insignificant" and there existed no "potential of influencing research results." The proposal would also have required the awardee institution to obtain full financial disclosure of outside relationships at the time of the proposal and annually thereafter and to notify the agency if a waiver were proposed to be granted or if a conflict were detected after the award.
Vehement protests during the ninety-day comment period from some seven hundred representatives of the research and industrial communities led Department of Health and Human Services (DHHS) Secretary Louis Sullivan to withdraw the draft guidelines on December 29, 1989.24 Secretary Sullivan has promised a new proposal but indicates that it will be published in the Federal Register for notice and comment, rather than in the more informal and legally dubious format of the NIH Guide. Representative Weiss has publicly warned DHHS not to be deterred by criticism from the research community25 and has threatened to add conflict of interest standards to the NIH authorization bill this year.26
Recent Self-Regulation in Clinical Studies
Public discussion of this topic has led to two notable instances of self-regulation in connection with clinical trials and to calls from within the biomedical community for more of the same. One case involves the Post Coronary Artery Bypass Graft Surgery (Post-CABG) Clinical Trial, a seven-year multi-center randomized trial study of the impact of cholesterol reduction and anti-thrombotic treatment upon the development of atherosclerosis in patients who have undergone coronary artery bypass graft surgery. After noting that the trial will evaluate the efficacy of phar-
maceutical products of three U.S. corporations and that the study's results could affect the profitability of these products, the principal investigators announced in April of last year that they would not buy, sell or hold stock or stock options in companies providing medications for the study, nor would they serve as paid consultants to such companies. 27
This ban will last from the commencement of patient recruitment through termination of the investigator's (or institution's) involvement in the study and will extend to the investigator's spouse and dependents. All "key investigators" are covered, excluding persons providing "primarily technical support or who are purely advisory," i.e., without direct access to the trial participants or data, unless they are "in a position to influence the study's results or have privileged information on the outcome." Each person subject to the policy will be required to file with the trial's coordinating center a conflict of interest statement and to update it annually. In a similar move, the Phase III team for the TIMI clinical trial referred to above has also adopted guidelines which will provide comparable restrictions and has taken the further step of prohibiting financial ties with involved companies for a year following the conclusion of the study. 28
A number of commentators within the biomedical community have expressed approval for the self-denying approach taken by the Post-CABG and TIMI trial teams while urging even more extensive steps to curtail the acquisition by researchers of financial interests in the new drugs and clinical devices they are studying. New England Journal Editor Arnold Relman called in April 1989 for disclosure of all financial ties between researchers and the products and procedures they are investigating both to sponsors and to publishing journals.29 He also recommended policy reviews by all institutions sponsoring clinical research or employing clinical investigators, leading to "a broader and more institutionalized approach." Dr. Arthur Caplan, director of the Center for Biomedical Ethics at the University of Minnesota, has also called for a flat ban on financial stakes of any kind in clinical trials: "These people should divest their ownings or disqualify themselves if they have a stake in the outcome. . . . The standards for human research have to be higher than those of other studies." 30
Recent Moves by Consortia and Individual Institutions
Under the impetus of these suggestions and with a determination to insure continued public confidence in the integrity of the biomedical research enterprise, the major professional consortia have reviewed the subject and promulgated new recommendations to their memberships, and
some institutions have revised their own policies. Such reviews have grappled with two questions: (1) should present provisions for disclosure be intensified, and (2) are any situations of potential conflict so dangerous that they should not be allowable? Recent reports by the Association of Academic Health Centers (AAHC) and the Executive Council of the Association of American Medical Colleges (AAMC) have answered the first question affirmatively while deferring to institutional discretion as to the second.
The AAMC paper calls on its members to develop procedures for "full disclosure to the institution, and to the interested public, of financial and professional interests that may influence, or may be perceived to influence, research activity or other scholarly responsibilities. . . ." This disclosure is to occur at least annually and would include all relevant personal interests, e.g., equity stakes, outside professional positions and salary, gifts, honoraria and loans, of the faculty member as well as immediate family members. Cases should be reviewed by supervisors, and if there is "any reasonable question of conflict or legitimacy regarding the situation, then all relevant information should be passed on to a designated institutional committee." The policy expresses the hope that an institutional common law may develop as this committee gains experience.31
The AAHC similarly endorses regular disclosure by faculty, researchers, staff and students for themselves and their immediate family members of "significant financial, personal or professional relationships that raise a potential conflict of interest between their academic role and outside interests."32 With respect to sponsored research, the report also calls for disclosure of "[s]ignificant financial, persona, or professional relationships that raise a potential conflict of interest. . . . in all speeches, writings, advertising, public communications, or collegial discussions" relating to the research. 33
At the institutional level, Johns Hopkins last November joined a number of schools in intensifying disclosure requirements and instituted a flat ban on the ownership by investigators of shares in companies sponsoring their work.34 According to Science, however, Stanford University School of Medicine has so far resisted per se prohibitions but has begun "spot auditing" of its annual disclosure forms and has added questions on an investigator's financial ties to its human subject review protocol.35
Existing rules have also been extensively revamped at Harvard Medical School, where the Faculty Council and the thirteen-institution consortium of affiliated teaching hospitals recently approved a new policy. This document includes a broader, more precise definition of conflict of interest (any outside financial interest touching in any way upon faculty respon-
sibilities, e.g., teaching, research, patient care or administration), mandatory written financial disclosure not less frequently than annually and a much elaborated set of guidelines (with detailed definitions) identifying the most common sensitive situations.36
What is new in the Harvard policy is a category of outside activities to be subject to maximum scrutiny, focusing upon faculty who conduct clinical research and those who own stock in companies sponsoring their work. The policy mandates intensive review by a standing faculty committee on conflict of interest for every case in which a faculty member proposes to participate in any way in clinical research (including FDA or other committees reviewing such research) on technologies or drugs in circumstances of possible financial benefit. This rule covers all equity interests and consulting income but excludes salary and royalty income paid through sponsored research relationships. Also subject to mandatory committee review will be every case in which a faculty member proposes to have research support from a company in which he owns stock. Research support is broadly defined to cover equipment, biological materials and drugs as well as cash. This rule also covers basic as well as clinical research.
Other high-scrutiny situations under the Harvard policy include:
the assumption of executive positions with outside for-profit businesses;
clinical referrals to a business in which the faculty member has a financial interest (but excluding school-or hospital-affiliated institutions and individual or group private practice plans); and
circumstances in which a faculty member expects to receive financial returns from businesses competing with the school or the employing hospital.
Finally, Harvard faculty will also be expected to disclose in conjunction with any published or formally presented research results any financial interest relating to those results.
Conclusions and Applications to the PORTs
Emerging standards in conflict of interest regulation suggest a number of common themes to be considered in formulating guidelines for PORTs:
an increasing reliance on regular, detailed, mandatory financial disclosure that includes relevant information with respect to spouse and dependents;
maximum scrutiny of financial relationships with organizations producing drugs, devices or other interventions which the financial stakeholder is studying, especially a the research approaches the clinical trial stage;
greater concern with respect to equity relationships, with other ties, such as consulting and management positions, increasingly suspect as the research comes closer to evaluation or testing of actual or prospective products;
the possibility that disclosure, supervisory controls and monitoring may be sufficient to allow some conflict relationships to continue, provided:
monitoring mechanisms are stringent and provide reasonable and publicly acceptable assurances that research will be free from bias;
there is no early, financially advantageous disclosure of research results; and
any reference to the relevant research in speeches, writings, advertising or collegial discussions is accompanied by disclosure of the relationship.
notwithstanding (4), a trend in clinical trials towards a self-denying rule whereby key scientific participants agree to forgo financial ties to interested businesses during such studies.
Conflict of interest guidelines for PORTs investigators should also take into account several specific features of these studies. These considerations relate to the composition or study teams, to subsequent dissemination of the results, to expertise developed in the course of the study, and to funding sources. Depending upon the financial arrangements involved, these features may pose particular risks of conflict of interest for study investigators.
The breadth of expertise in the individual PORTs teams will lead to increased credibility for the research results, likely far greater than that achieved by any single publication or study. Seldom in clinical research do groups consist not only of subject-matter experts and biostatisticians but also of decision analysts, epidemiologists, meta-analysts and economists. In many cases, National Advisory Bodies further enrich the PORT. Thus, there will be many requests for information and opinions from the PORTs. These will take the form of informal conferences or lectures on the results as well as on general methodologic issues. In addition, as part of the experimental design itself, there will likely be formal dissemination of research results to specific groups of physicians. The ability to influence large numbers of students, physicians, patients and policy makers will be great.
In the course of executing its research strategy, each of the teams is likely to become expert, perhaps uniquely so, in a number of areas of interest to investigators in other aspects of health services research or health policy. Expertise might include development of any of the following: specialized software and/or hardware for analysis of claims data; instruments or survey tools for assessing outcomes; approaches for assessing utilities; educational programs describing alternative treatments; and miscellaneous software programs. In the absence of a regulatory framework, this expertise could lead to the unmonitored formation of businesses (either for-profit or not-for-profit) in which members of the PORT have a financial interest.
Funding for PORTs is another vehicle by which conflicts of interest may arise. Currently, although the funding level is approximately $1 million (direct plus indirect) yearly, the breadth of problem areas within the purview of any PORT and the depth of investigation possible within a given area are larger than can be adequately covered by that amount of money. The ready-made group of investigators and study sites makes PORTs an attractive vehicle for funding by industry; in addition, the possibility of supplemental funding by industry is obviously desirable for the teams themselves. Currently, it would appear that industrial funding would be used primarily for support of clinical research activities.
A further consideration is the desirability of reasonable uniformity. It is important to note that whatever conflict of interest guidelines are recommended for PORTs, investigators will also be obligated to abide by those of their university or hospital. Because the more stringent of the two will control in any given situation, it is possible that not all PORTs would be subject to the same guidelines. This undesirable possibility is perhaps another consideration arguing for reasonably stringent standards at the funding level.
1. "Guidelines for Dealing with Faculty Conflicts of Commitment and Conflicts of Interest in Research," adopted by the Executive Council of the Association of American Medical Colleges, February 22, 1990, p. 6.
2. See, e.g., D. Blumenthal, M. Gluck, KS. Louis, D. Wise, "Industrial Support of University Research in Biotechnology," Science, Vol. 231, 17 January 1986, p. 242, and C. Sims, "Business-Campus Ventures Grow," New York Times (December 14 1987).
3. "Is Science for Sale? Conflicts of Interest vs. the Public Interest," Hearings to the House Committee on Government Operations, Subcom
mittee on Human Resources and Intergovernmental Operations, 101st Congress, 1st Session June 13, 1989.
4. "On Preventing Conflicts of Interest In Government-Sponsored Research at Universities: A Joint Statement of The Council of the American Association of University Professors and The American Council on Education," American Council on Education, December 1964.
5. See, e.g., D. Blumenthal, M. Gluck, K.S. Louis, M.A. Stoto, and D. Wise, "University-Industry Research Relationships in Biotechnology: Implications for the University," Science, Vol. 232, 13 June 1986, pp. 1361–1366.
6. The Patents and Trademark Amendments of 1980, Pub. L. 96–517, 94 Stat. 3019 et seq. and Government Research and Development Patent Policy, Pub. L. 98–620, Title V, November 8, 1984, 98 Stat. 3364 et seq., amending 35 U.S.C. Sections 200–211.
7. Letter to Thomas A. Bartlett, President of the Association of American Universities from Representatives Gore and Fuqua, House Committee on Science and Technology, United States House of Representatives, November 18, 1981, p. 1; quoted in A. Burke, "University Policies on Conflict of Interest and Delay of Publication: Report of the Clearinghouse on University-Industry Relations of the Association of American Universities," Journal of College and University Law, Vol. 12, No. 2, February 1985, p. 178 (hereinafter cited as "Clearinghouse Report").
8. Clearinghouse Report, p. 178.
9. See A. Bartlett Giamatti, "The University, Industry, and Cooperative Research," Science, Vol. 218, 24 December 1982, pp. 1278–1280.
10. Statement of Policy on Conflicts of Interest, as voted by the President and Fellows of Harvard College, March 1, 1982, and Guidelines for Research Projects Undertaken in Cooperation with Industry, as voted by the President and Fellows of Harvard College, October 3, 1983.
11. See, e.g., Utah Code Ann., Sec. 67-16-1, et seq. (1986 and 1989 Supp.), quoted in Clearinghouse Report, p. 181.
12. See, e.g., "In Re Executive Commission and Ethical Standards Re: Appearance of Rutgers Attorneys," 537 A.2d 713 (N.J. Super. 1988).
13. See, e.g., Agency for International Development Acquisition Regulation, Sec. 752.7020 (March 1985, AIDS Handbook 14), March 21, 1990, pp. 14–140, 14–141).
15. "Medical School Dean Writes to Faculty About Tseng Matter," Harvard University Gazette, November 18, 1988, p. 8.
17. See, e.g., M. Chase, "Mixing Science, Stocks Raises Question of Bias in the Testing of Drugs," Wall Street Journal (January 26, 1989); P.G. Gosselin, "Doubts Grow over Doctor's Flawed Cancer Test," Boston
Globe (June 9, 1989); and W.E. Leary, ''Business and Scholarship: A New Ethical Quandry," New York Times (June 12, 1989).
18. "Federal Response to Misconduct in Science: Are Conflicts of Interest Hazardous to Our Health?" Hearings to the House Committee on Government Operations, Subcommittee on Human Resources and Intergovernmental Relations, 100th Congress, 2d Session, September 29, 1988, pp. 56 et seq. (statement of Victor J. Marder, M.D., University of Rochester Medical Center, followed by George Bernier, Dean of University of Pittsburgh Medical School).
19. See, e.g., Rep. Ted Weiss, "Research That the U.S. Government Is Paying For Should Not Be Tainted by Any Possibility of Bias," Chronicle of Higher Education, October 4, 1989.
20. "PHS Grants Policy Statement," U.S. Department of Health and Human Services, DHHS Publication No. (GASH) 82–50,000, January 1, 1987, p. 55.
21. Notice, "Extramural Researchers' Financial Conflicts of Interest," NIH Guide for Grants and Contracts, Vol. 18, No. 2, January 20, 1989, p. 1.
22. See J. Palca, "NIH Grapples With Conflict of Interest," Science , Vol. 245, 7 July 1989, p. 23.
23. "Request for Comment on Proposed Guidelines for Policies on Conflict of Interest," National Institutes of Health and the Alcohol, Drug Abuse and Mental Health Administration, NIH Guide for Grants and Contracts, Vol. 18, No. 32, September 15, 1989.
24. See J. Palca, "NIH Conflict of Interest Guidelines Shot Down," Science, Vol. 247, 12 January 1990, p. 154.
26. E. Marshall, "When Commerce and Academe Collide," Science, Vol. 248, 13 April 1990, 152.
27. See B. Healey, et al., "Conflict Of Interest Guidelines for A Multicenter Clinical Trial of Treatment After Coronary-Artery Bypass--Graft Surgery," The New England Journal of Medicine, Vol. 320, No. 14, April 6, 1989, pp. 949–951.
28. See C. Holden, "Research Group Forswears Financial Ties to Firms Whose Drugs It Tests," Science, Vol. 244, 21 April 1989, p. 282.
29. A.S. Relman, M.D., "Economic Incentives in Clinical Investigation," The New England Journal of Medicine, Vol. 320, No. 14, April 6, 1989, pp. 933–934.
30. "Business and Scholarship: A New Ethical Quandary," New York Times (June 12, 1989).
31. Op. cit., footnote 1, pp. 9–12.
32. "Conflicts of Interest in Academic Health Centers; A Report by the AHC Task Force on Science Policy," Association of Academic Health Centers, 1990, p. 48.
33. Id., p. 50.
34. Op cit., footnote 26, p. 155.
36. Harvard University Faculty of Medicine Policy on Conflicts of Interest and Commitment, March 22, 1990.
APPENDIX D4. THE STRUCTURE AND METHODS OF PORTS: SOURCES OF BIAS
John Wennberg, M.D.1
I want to thank the Institute of Medicine for hosting this Workshop on Potential Conflicts of Interest in Patient Outcomes Research Teams. Some five years ago, the John A. Hartford Foundation provided our research group with the support to undertake an evaluation of the practice-style related reasons for variations in treatment rates for benign prostatic hypertrophy or BPH, a common condition that affects most men after age 65. We knew from previous research that the rates for one BPH treatment, namely prostate surgery, varied such that in some communities in Maine over 50% of men received a prostatectomy by age 80, while in others the rate was as low as 15%. We undertook a series of research studies to identify the causes of variation, focusing on the discovery and testing of the different treatment theories that might explain the large differences in practice patterns. It is in the evolution of our experience with the evaluation of alternative treatments of BPH we learned one version of the PORT mission.
Recognizing that at least some of our concerns over conflict of interest are idiosyncratic to the mission as we have discovered it and that they may seem foreign or remote to those who have different versions in mind—I want to raise them for your consideration, to show how important the interpretation of the mission is to understanding of the potentials for conflict of interest.
In my opinion, the PORT mission is to achieve a non-regulatory solution to the failure in the past to undertake the systematic evaluation of all treatment options available to treat a particular condition such as BPH, cataracts, stable angina or a heart attack. The Food and Drug Administration (FDA) has provided a strong regulatory focus for causing new drugs to be evaluated before they reach the medical market. But the FDA evaluation paradigm has been narrowly conceived—restricted in the theories it causes to be evaluated, in the outcomes it considers relevant, and in the pressures it places on the profession and industry to evaluate. The treatment theories actively evaluated involve only drugs versus placebos, even though in most clinical situations the options for treatment are much broader. For example, in the case of stable angina the options include use of surgery (coronary artery bypass), a device (coronary artery angioplasty), diet (low fat diets) and other life-style modifications.
The FDA approach is further limited in that it is not concerned with new uses made of drugs once they are on the market. For example, physicians have learned to use certain anti-hypertensive drugs to treat the symptoms of BPH, even though these drugs have not been evaluated for this purpose. The outcomes considered relevant by the FDA are often changes in biochemical or physiologic parameters, and they do not necessarily include those outcomes that are relevant to patients such as relief of pain, improvement in functional status, and avoidance of complications. (For example, they may include improved coronary artery blood flow or improvement in the flow of urine, but not changes in symptoms, the occurrence of side effects or changes in functional status.)
The failure to systematically evaluate all treatment theories relevant to a particular condition, to use outcome measures that capture the events that are relevant to patients and to make the results of assessments available to physicians and patients in ways useful in choosing treatment options results in systematic flaws in the scientific and ethical basis for clinical decisionmaking. The flaw in federal science policy that irrationally restricts evaluations is not likely to be repaired, however, by extending regulatory mandates. I say this not simply to echo a disfavorable attitude toward regulation, but for structural reasons. The required focus is on the evaluation of all relevant treatment theories. Much of medical theory emerges from the exigencies of clinical problem solving, as part of the ongoing practice of medicine. For example, much of surgical innovation is really a modification of traditional treatment strategies which do not involve novel equipment or new drugs that could be easily withheld from practice until evaluations have been made.
Some of the most expensive ''medical theories" emerge in the building of hospitals or other facilities. One of the major theories of interest to outcome research is whether the availability of hospital beds at the rate of 4.5 beds per 1,000 produces better outcomes to the population served than availability at a rate of 2.9 per 1,000.
The mission of the PORTs, then, is to become the focus for extending the mandate to evaluate to all relevant treatment theories for a particular condition or problem. It must do this without regulatory gate-keeping authority and without the firm (and confining) bases in administrative law that protect its members from corruption through the elaborate client-judge protocols that routinize the relationships of the FDA. PORT members must actively engage in the give and take of the marketplace of ideas. They must become known to and directly engage the originators of ideas and the innovators of new technologies. Ultimately, they must set the rules that cause ideas to be evaluated. I use the phrase "set the rules" in the sense of pointing to the examples for good research based on appropriate methods, identifying the relevant outcomes, establishing processes for evaluating and synthesizing research and reaching conclusions on the status of specific treatment theories. I also use the word "cause to be evaluated" to make clear that the PORT's mission is not to do all of the evaluation itself, but rather to move the profession (and patients) to accept the ethic of evaluation on the basis of good science.
PORTs follow a professional, not a regulatory model. I want to give you some examples of what they do based on our own experiences with the evaluation of alternative treatments for BPH, so you may see the nature of some of the interactions between PORTs, physicians and patients as well as industry. I will then turn to consider some features that could lead to conflict of interest or corruption of the process.
Experiences of the Dartmouth Assessment Team
As I mentioned, the entry into the problem of alternative treatments for BPH was the discovery and follow-up of the variations in rates of prostatectomy among communities in Maine. We met with physicians from high and low rate communities across the state of Maine. Initially, the arguments were between our group and the clinicians; very soon, however, the debate was among the clinicians in Maine, as they began to discuss their opinions on the treatment of BPH. It soon became apparent that the clinicians divided roughly into two camps, one camp subscribing to the preventive theory, the other to the quality of life theory of surgery.
Under the preventive theory, surgery early in the course of BPH is called for, because for most people the disease will progress to the point
where surgery is required to save patients from the serious complications of bladder or kidney failure. If surgery is delayed, patients will be older and sicker when they require it; overall, life expectancy was thought to be higher if one operates early. Physicians who subscribed to the quality of life theory were much less pessimistic about the natural history of the disease, believing that in most people it does not progress to the point where life is threatened. For most people, surgery is of value because it reduces symptoms and improves the quality of life. The decision to undergo surgery should therefore be based on symptoms, their anticipated relief and the willingness of patients to undergo the risks of surgery in order to secure its benefits.
We used Hartford Foundation funds to recruit an international team of urologists, epidemiologists, biostatisticians, decision analysts, social psychologists, sociologists and computer scientists to undertake a series of studies to test these two treatment theories. Putting together information from the medical literature, claims data analysis, interview studies, and decision analysis, we could show that the preventive theory was in error. Using a wide range of possible assumptions, we could find no evidence that people would live longer if patients with BPH were operated upon to prevent subsequent development of bladder or kidney obstruction. As an assessment team, we drew the conclusion that the preventive theory was not a valid reason for early prostatectomy.
The data we obtained from an interview study of patients who underwent surgery also showed a "slam bang" effect of the operation on symptoms, with outcome probabilities for symptom relief that were much more favorable than watchful waiting. However, this relief could only be obtained by risking certain complications, the probabilities for which were also documented by our study. Moreover, we could see from the data that there was no strong correspondence between patients' medical histories and their symptoms and the choices individuals would prefer to make between watchful waiting and surgery. We thus came to the further conclusion that rational choice in the decision to undergo a surgical procedure or to watchfully wait depends on patient preferences; these can only become known by actively engaging the patient in the decision process—by asking him what he wants. Values and attitudes are key because they provide the logical, rational reasons for undertaking the surgery.
We therefore could conclude that the practice-style reasons for the variations we had observed in Maine derived from an incorrect professional theory about the value of early surgery and from the entanglement of patient preferences for treatments with those of the advice-giving physician. The ethical and the scientific conclusions thus lead in the same direction. The need was to communicate the message of choice to doctors
and patients in ways that would help them base choice more firmly on the preferences of patients. I will return to describe how we have handled this dissemination process a little later because the conveying of information that effects individual choice is a focus for conflict of the PORT mission.
It was while undertaking the evaluation of surgical outcomes in Maine that we first noted differences in patient survival, depending on which type of prostate operation they underwent. Subsequently, we studied the phenomenon in Denmark, the United Kingdom and Canada, consistently finding an excess mortality in the five years following surgery among patients who underwent the transurethral prostatectomy (TURP), compared to those undergoing the open prostatectomy. A similar excess in mortality appeared when TURP was compared to other operations, such as cholecystectomy or hernia repair. We went back to the medical records, abstracting data on case severity and co-morbidity and were unable to explain the result on the basis of differences among patients. We also found a report in the literature of a small randomized clinical trial of open versus TURP that was consistent with the hypothesis that TURP, somehow, has a deleterious long-term effect on survival. However, neither our group nor consulting urologists could come up with any convincing mechanisms as to how this effect might occur. Left with this uncertainty, we concluded that a large scale randomized clinical trial would be required.
The publication of the results of the open versus TURP studies opened a new phase in the development of our assessment team. The result was not a welcomed one for either patients or urologists, who in the United States perform nearly 400,000 TURPs a year. We appreciated an editorial in the Lancet stating that someone had rolled a boulder in the quiet pond of urology. We appreciated it because it did not dismiss the result out-of-hand, but rather called for thoughtful response of the urology community to the challenge. The major breakthrough for the mission of our PORT, however, has been the response of the leadership of American urology, in particular the officers and senior membership of the American Urological Association (AUA).
We have been at work now for a year with the AUA, helping in the design of a large scale randomized clinical trial to test the hypothesis of excessive mortality following TURP. As the work has progressed, the focus of the AUA effort has widened to include a focus on new treatment theories as well. When our assessment team began its work four years ago, the field of urology was indeed like a placid pond. Other than surgery and watchful waiting, there were no serious options for the treatment of BPH. Not so today. The last few years have seen a plethora of new BPH treatment theories, some involving balloon dilation of the prostate gland, new drugs, microwave diathermy, and others involving
urethral stents and new ways of surgically affecting the prostate gland. Under traditional science policy, these new treatments would enter routine clinical practice without benefit of systematic evaluation. This is no longer the case. The AUA, working with our PORT team, is establishing a network of some 20 centers to undertake the protocol that will bring each of these new treatment theories under active assessment. Our PORT team is providing scientific direction on study design and help in managing the data collection procedures.
The recent completion of the planning and grant writing phases and the initiation of a six center pilot study funded by the AUA brings the story to the present. A PORT mission has been defined which includes (1) explicating the causes of practice variations; (2) testing the underlying conflicts in practice theory; (3) establishing relationships with the leadership of the practice community to promote systematic evaluation of new as well as widely used technology; (4) the synthesis and integration of the results of assessments, including estimates of the probabilities for various outcomes; (5) the demarcation of the role of patient values in decisionmaking; and (6) the dissemination of results of the assessments, with a focus on communication about treatment options in a form useful to patients and physicians for clinical decisionmaking.
Vulnerability of PORT Mission to Conflict of Interest and Corruption of Purpose
Let me now examine the vulnerabilities of this mission to conflict of interest and corruption of purposes. I have noted four areas of concern: economic conflict of interest, ideological corruption, existential corruption, and corruption of discursive practices.
Economic Conflict of Interest
This is perhaps the easiest to deal with because the problem is well understood and certainly not unique to the PORT. The background reading and the scenarios for discussion at this conference represent this concern well. I do not want to deal with details, but it is worth noting that the mission outlined above requires active relationships between stake holders and the assessment teams. The latter's judgments concerning the relative advantages and disadvantages of a specific stake holders particular treatment theories have obvious economic consequences.
PORT teams can expect active interest from industry in their activities and numerous opportunities for consulting fees and honoraria. PORT
teams also have the potential for access to insider information on the prospects for profit of technologies which affects their qualification to own stock in the companies they stand to influence. Policies to prevent the appearance and actuality of conflict of interest will be needed. In the case of our own group, we have chosen to forego any honoraria and consulting fees and not to own stock whose value is affected by urologic treatments.
Economic interest cut into the mission of the PORT in another way in the case of drugs or devices that have not yet received FDA approval. On more than one occasion, our group has been asked to help in the planning of the evaluation of new drugs, prior to FDA approval. While the rational interests of efficient evaluation are promoted by this involvement (for example, making certain that phase II and III drug trials include evaluations of the probabilities for each of the outcomes that matter to patients), secrecy and exclusivity requirements of the drug industry, designed to protect proprietary interests, have gotten in the way of some collaborations. These barriers not withstanding, we have been gratified by the efforts to cooperate by the drug and the device industries as well as their respect of our interpretation of the PORT mission.
We have also been impressed by the willingness of manufacturers to contribute directly to the costs of outcomes research. For example, the balloon manufacturers are contributing resources to the AUA pilot study mentioned above. However, such direct financial dealings between manufacturers and PORTs or its collaborators raises red flags and should be avoided. If we can learn to harness the energies of industry, to help them in their need to have their products developed, the outcomes research agenda can be accelerated. One suggestion is for an institute that can serve as a depository and distributor of industrial (or perhaps third party payor) funds to outcomes research which would avoid the need for a direct relationship between PORTs and their evaluatees.
PORTs may be susceptible to another kind of conflict, one I have chosen to label "ideological". This is the way I label a tendency to lose objectivity that a member of a PORT may experience if he or she becomes invested in a specific treatment theory rather than in the ethic of evaluation. A temptation we all experience as researchers is to believe that somehow we have gotten hold of the truth, that we know the answer. In our determination, we can loose sight of the role of the PORT which is to manage the process of evaluation. There is probably no specific remedy that guards against this temptation; it comes from ingrained recesses in minds and souls. However, PORTs can protect themselves
from its consequences by virtue of the fact that assessment teams are multidisciplinary, that they engage in ongoing dialogue and membership is replaced and refreshed.
The mission we have defined for the PORTs is one that challenges the conventional wisdom and raises intellectual barriers to the influx of new, underevaluated medical theory. This is an inherently conflict-ridden role, one susceptible to what, for want of a better label, I call existential corruption. It is the bad faith that comes from fear of ostracism, of career instability, and other unfortunate outcomes that befall those who are whistle blowers, who challenge optimistic assumptions that all forms of medical technology lead to progress. To understand this problem in one extreme, imagine the possible fates of the evaluative scientist who shows that it is safe and in the public interest to close 700 beds in Boston. We need to worry about how to make it possible to provide stable careers to those who want to work in the evaluative sciences, to permit intellectual freedom to reign as the primary ethic driving the evaluative processes. A good deal of the needed protection may rest in the commitment of the federal government to ongoing funding in the area of the evaluative sciences. This, in turn, works to advance the evaluative sciences into the tenured faculty ranks, to make them and the ethics of evaluation central to the mission of the academic medical center.
Corruption of Discursive Practices
These are the biases that become introduced in the act of communicating and are the least well understood of all. I first became aware of the problem when we struggled to learn how to disseminate the results of our BPH assessment in a way that could help physicians and patients make decisions that more closely reflected the patient's own preferences for outcomes and attitudes toward risk. You will recall that one conclusion was the need to ask the patient what treatment he wanted, based on a detailed sharing of information about the outcomes of watchful waiting and surgery. We worked out a strategy based on the use of interactive videodisc technology. In developing the program, we encountered a number of problems concerning the adjudication of differences in opinion on the "fairness" of a given representation of a medical decision problem. (By making one version of the communication process "objective", the debate over fairness emerged for the first time; here-to-fore,
communication on options had occurred in the black box of the physician's office, unobserved, uncriticized and uncontroversial.) We worked out these conflicts through debate and argumentation, within the context of "focus groups" that sought to narrow the range of disagreement and to arrive at an overall judgment that the presentation was "as fair as it could be." The acceptability of the version we produced has withstood several test in clinical practice where both patients and physicians have found it helpful.
I anticipate that other PORTs, as a result of their assessments, will come to similar conclusions concerning clinical practice: for many (probably most) medial conditions, there is no single "correct" or "appropriate" treatment; what is right for one patient—the treatment that best fits his or her preferences for outcomes and risks—is not the same treatment that is right for another patient faced with a similar situation. The PORTs are thus fated, if they follow this mission, to engage patients and physicians in a new discourse, the discourse of shared decisionmaking. The structuring of the institutions that produce and sustain the flow of information, and establish and maintain the rules of discursive practices are perhaps the greatest challenge of all. We are now taking steps to develop a not-for-profit corporation, the Foundation for Informed Medical Decision Making, which we hope can meet the challenge.