Review of the Fialuridine (FIAU) Clinical Trials (1995)

FDA has recently issued a series of proposed changes to existing regulations regarding adverse experience reporting in the wake of the FIAU tragedy (FDA, 1994a). The proposed changes in relation to premarketing studies are largely derived from the recommendations of the FDA Task Force report reviewed above and are described below:

• Specification of the cut-points for measures and events to be considered indicative of an adverse reaction.

• Specification of procedures for distinguishing between drug toxicity and underlying disease process.

• Specification of measures or events to be used to trigger actions regarding a trial (reduce dose, stop treating the patient, terminate the trial, etc).

• Summary of safety data at regular intervals under the supposition that all adverse events observed are drug related.

• Protocols to contain list of clinical events and laboratory measures to be reported to the sponsor; list to include events and outcome measures attributable to underlying disease or concomitant medications.

• Immediately reportable events should trigger remeasurement or challenge procedures, or interrupt treatment.

• Control groups should be used in safety studies when underlying disease likely to mimic drug toxicity.

• Sponsors required to provide estimates of underlying mortality and morbidity rates for diseased populations.

• Follow-up beyond the period of dosing or treatment (3 months suggested for Phase I and Phase II trials).

• Procedures for ongoing safety monitoring in the form of a formal monitoring committee or acceptable alternative.

• Periods for submission of written and telephone reports extended to 15 and 7 calendar days respectively; faxes acceptable in lieu of telephone reports.

• Form 3500A (same as proposed form for postmarketing reports) allowed.

• Disclaimer to apply to all safety information so as not to constitute a statement or admission of a causal link between drug and a subsequent adverse event.

• Current annual death and serious adverse experiences report to be supplemented by semiannual reports; semiannual report not to take place of annual report.

• Report all deaths, serious adverse experiences, and study discontinuations due to adverse experiences, whether or not unexpected, and whether or not caused by the study drug.

• Data required from IND studies, worldwide premarketing studies, and foreign postmarketing trials or reporting systems.

• Data to be presented cumulatively as well as for a 6-month period.

• Worst case analysis of data required—presumption against drug then attempt to refute presumption.

• Submission of all available autopsy reports and medical reports of deaths required; sponsor to clarify inconsistencies between reports and cause of death reported to FDA by the sponsor.

• FDA may modify reporting requirements, e.g., when it would interfere with a blinded (masked) study, in which neither patients nor study staff are aware of which patients are receiving control treatment until the end of the trial.

• Data on events not defined as serious adverse reactions.

• Agency to determine when it is needed and the information required.

• Should contain results of cumulative analyses in other reports as well as analyses of lesser events.

• Generally, 30 days from request to comply.

• FDA may require a final report or study summary of a study at its discretion.

CLINICAL HOLD

• FDA may suspend ongoing study of an IND or closely related drug under investigation by the same sponsor for failure to submit a special safety summary or final study report.

• IND may be terminated for failure to submit a semiannual report — nonsubmission of an annual report is currently grounds for the same.

• Failure to implement adequate safety monitoring to be grounds for clinical hold or termination of an IND.

• Investigators required to submit safety data to sponsors to allow sponsors to comply with safety reporting requirements, i.e., semiannual reports.

The time schedule of our review does not permit detailed comment on each of the proposed changes, in part because they were promulgated just prior to the committee's final meeting. The timing of the promulgation is unfortunate, if for no other reason other than that there is much in this report that bears on the regulations being proposed. The period for written comment on the proposed rule changes closed (January 25, 1995) prior to the release of this report. We find ourselves agreeing with David Challoner and David Kipnis (co-chairs of the NIH special review subcommittee) in wishing that the ''FDA had waited for another investigation [namely the present one] of the deaths to be completed before promulgating new rules'' (Hilts, 1994).

The committee has questioned elsewhere in this report the wisdom of some of the observations and recommendations contained in the FDA Task Force report. Those reservations are germane to the proposed rule changes in that many of those changes have their origins in that report. There are also elements of the proposed changes that we endorse on general principle and have so indicated elsewhere in this report. Foremost among them are the desirability of formal ongoing monitoring and the use of designed comparison groups in all stages of testing, starting with Phase I and Phase II trials.

The IOM committee believes that IRBs need to be more vigilant in ensuring that investigators have adequate plans for ongoing monitoring of trials, regardless of their size, the sponsor, or the funding source. Although the review and approval process should include assurances from the investigators of a commitment to publish the results of the trial regardless of its outcome, IRBs should insist, as well, on some form of ongoing monitoring, preferably in the form of some independent person or group, and should be satisfied that the investigator or study has an adequate system for data flow and follow-up to make such real-time monitoring practical and useful.

The committee notes in this regard that the usual case report form of data collection and reporting is not amenable to such monitoring. In that approach, investigators are given a booklet containing a set of forms to be completed for a particular patient. The entire package is retained at the data collection site until the patient has completed the study, died, or dropped out of the study. The data contained in a case report are not entered at the data center until the entire package has been completed and sent to the sponsor or data center. As a result, there is a lag of weeks or months in the flow of information to the sponsor or data center and in the conversion of the data to electronic form for processing and analysis. A real-time form of monitoring and analysis requires real-time data flow and abandoning the case report form of data collection in favor of a contact or visit-driven form of data collection and flow. This is obviously no panacea, since the FIAU studies reviewed here all had formal or informal real-time monitoring systems as well as the mandated case report forms, but it is a change that would help insure that all investigators are as alert to developing problems as these FIAU investigators were.

Among the recommendations in the FDA report, the one having the greatest intellectual appeal is their recommendation having to do with the need for controls. The use of controls and randomization in this case would have, at the very least, provided a better framework for evaluating background morbidity and mortality. It would have provided another benefit in that, assuming the same rate of recruitment as achieved but with only half of the patients being assigned to the test treatment, only half of the patients would have been exposed to a bad treatment, and the slower rate of enrollment to the test treatment would have increased the chance of detecting the late complications, with fewer patients being exposed.

These things said, however, the IOM committee is also under no illusions as to the power of such trials for detecting untoward events. The typical Phase I or Phase II trial involves 20 to 40 patients and treatment and follow-up over a short time period. Most adverse effects caused by treatments will go undetected in such settings. Hence, although there are lots of reasons to favor the notion, it would be a mistake to assume that such a provision will significantly reduce the risks of exposure in phase I and II trials, especially when those effects are heretofore unknown and unanticipated.

The absence of appropriate comparison groups in the Phase I or Phase II setting leaves those doing such trials in a quandary as to the interpretation to be given to morbid and fatal events observed during or shortly after the course of a trial. The problem of interpretation is most acute when the trial involves sick people with an increased background morbidity and mortality. One can expect, in such cases, for some patients enrolled in the trial to have morbid events or to die during or soon after the trial and for those events to be unrelated to the experimental treatment. The interpretation given to any single event in such cases is much more difficult than when the trial involves healthy subjects without any apparent risk of morbidity or mortality.

The IOM committee concurs with the view expressed in both the FDA and NIH reports arguing for follow-up beyond the formal close of a trial, especially for trials involving new compounds. It is reasonable to expect sponsors and investigators to maintain some minimal form of contact with patients (e.g., via mail or telephone and for a period of weeks or months) following separation from phase I and II trials involving new compounds. The follow-up would provide data on mortality and gross morbidity that may serve to alert to possible problems, with emphasis on may. The reason for the equivocation is that it is difficult enough to determine whether or not morbid or fatal events are treatment related when they occur while patients are under direct observation and being managed according to a specified protocol. Interpretation of such data becomes profoundly more problematic when the events occur after the patient's separation from the trial, and especially in the absence of a corresponding control group for comparison. One is forever uncertain as to whether such events are or are not related to something done weeks or months before the event in question.

The committee endorses the concept of longer follow-up on its general merit. However, we doubt that such a practice would have averted the disaster of the PPPC trial. In reality, at least one of the 3 trials done before the PPPC trial was in fact designed to provide such follow-up as a routine part of the study protocol (R91; 4 weeks of treatment and 20 weeks of follow-up), and the PPPC trial, had it gone to completion, would have provided such follow-up. Hence, only two out of the four trials in question did not make some provision for extended follow-up: R89 (FIAC; 4 weeks follow-up; n = 12) and R90 ([FIAU; 4 weeks follow-up; n = 43). The period of treatment and follow-up combined was 5 weeks for the R89 trial and 6 weeks for the R90 trial. See Table 1 of Appendix H for details.

All nine of the deaths noted in the FDA report occurred after the completion of treatment of those patients. If one assumes that the FDA report represents a full accounting of deaths up to the time of compilation of the report then the follow-up system proposed (6 months following the last administration of treatment) would have yielded just four of the nine deaths occurring before all studies of FIAU were stopped on June 28, 1993 (patients 105, 110, and 107 in trial R89 and patient 4D in trial R91).

The committee also believes that it is prudent to view all events as possibly related to the treatment, but we recognize the need for judgment in this regard. We find it unfortunate and puzzling in this regard that so much is made, in retrospect, of AST and ALT elevations that, in the actual course of activities, investigators had legitimate grounds to view as positive

indicators of treatment success. Given the grounds for that belief it is difficult to view something considered to be beneficial as an "adverse" event as well. Furthermore, the fact that they so regarded them was known to the FDA's Antiviral Drug Division, and if for no other reason than the absence of objection, investigators and sponsors had good reason to believe that their interpretation of such flares was also accepted by the FDA.

The IOM committee views the recommendations regarding stopping rules with skepticism for the following reasons. First and foremost, Phase I and Phase II trials are not large enough to be amenable to the use of statistically based stopping rules. Second, most Phase I and Phase II trials that use patients as subjects are conducted without concurrent control groups, and hence, there is no statistical basis for comparison. Third, it is virtually impossible to construct operational decision rules for the entire constellation of conditions or circumstances that one may encounter in a trial when one is dealing with events on a real-time basis. The main value in stopping rules would be in the thought processes used to focus on actions that might be taken in the presence of certain conditions or events and not in the algorithm itself.

There is merit to the process of specifying events that will trigger changes in a trial, to the extent that it serves to cause investigators to focus on the nature of results that might be expected. However, the resulting calculations are not likely to represent much more than educated guesses as to background morbidity or mortality, given the varied nature of the patients involved and their underlying disease states and medical histories. (See the following section on statistical analysis of mortality for several educated guesses about mortality in the patient population from which the FIAU/FIAC trials drew their subjects, and the predictive utility of those guesses).

Both the FDA and the NIH reports speak to the need for doing trials in sequence, with thorough analysis at each step before proceeding. The need or desirability of staging, however, must be balanced against the need for speed in trying to find and develop new treatments for existing diseases. That need, as well as pressure from the public (AIDS activists in particular) for increased efficiency in bringing drugs to approval, makes it difficult to justify a strict sequential approach for serious life-threatening or life-limiting diseases.

Both the FDA and NIH reports speak of the desirability of following persons enrolled for a period of 6 months or longer following enrollment. The FDA report speaks, in addition, to the desirability of pretrial estimates of expected mortality in the patient populations from which the subjects are to be drawn, and combining information from related trials in deciding whether or not to continue development of a drug. That being the case, it is instructive to reanalyze the available information from the FIAC/FIAU trials of 1989-1993, assuming that

each person enrolled is followed for a period of 26 weeks following enrollment and that the FDA report contains a full accounting of the deaths that occurred in that time interval.

The question of interest is whether there was sufficient death data available before the start of the PPPC trial to have caused prudent investigators not to have started that trial. The question is addressed by looking at each trial individually and by a form of ongoing reanalysis at the point of each death using all available data at that point in time for three trials (R89, R90, and R91). The analyses are performed assuming 26 weeks of follow-up for each person enrolled with censoring at that point (i.e., deaths occurring after the 26th week are not counted in the analysis) and assuming a control treated group of the same size as the number enrolled in the individual trials. For example, the R89 trial is assumed to have been done with a total of 24 patients, half of whom were assigned to treatment with FIAC and the other half of whom were assigned to ordinary medical care without FIAC.

The staging or sequencing suggested in the two reports prevailed in this case by virtue of when the trials were done. Trial R90 started 23 weeks after the last dose was given in trial R89 and, hence, all patients in trial R89 would have had at least 26 weeks of follow-up at the start of trial R90. Similarly, trial R91 allowed for at least 26 weeks of follow-up for all patients enrolled in that trial before the start of trial PPPC. The only overlap is for R90 and R91. Trial R91 started 185 days after the start of R90 and 77 days prior to the close of recruitment for R90. However, even if the start of trial PPPC would have been delayed to allow for the last person enrolled in R91 to have been followed for 26 weeks following enrollment, the staging would not have changed the information available for the start of PPPC (the reason being that the single death noted in the FDA report for trial R91 occurred in the 27th week following enrollment and would not have been noted or counted in an analysis with censoring after the 26th week of study). Hence, the information available at the start of PPPC was the same as if staging had taken place.

None of the trials was controlled, so it is necessary to hypothesize underlying mortality rates for the analyses (i.e., the rates that would have been likely in an untreated control group). Since there is no way of knowing those rates exactly, the best that can be done is to investigate a range of values (see Appendix H for details). Using hypothetical mortality rates ranging from 10 to 20 percent per year for the HIV-infected patients in the R89 trial and from 2 to 4 percent per year in the R91 and PPPC trials, the number of deaths "expected" in each group of patients was calculated and compared to the number of deaths actually observed.

The statistical evidence for harm is weak (see Tables 3, 4, 5, and 6 in Appendix H). In fact, depending on the assumed underlying mortality rate, some of the analyses actually suggest a beneficial treatment effect (i.e., death was less likely than expected). The only trial that produces any convincing evidence of harm is the PPPC trial, and all of that information accumulated after the investigators had already stopped the trial because of morbidity.

To summarize, it is unlikely that the FDA-recommended procedures of estimating expected deaths and following up for 6 months would have resulted in cancellation of trial PPPC, and might even have led the investigators to persist longer than they did when patient 2 was hospitalized.

In regard to the proposed changes requiring a marked increase in safety reporting, we remain skeptical that the benefits from such added efforts will outweigh the risks. The risks must be measured in increased costs, in terms of added time for drug development, in terms of delays in access for general use, and in terms of so-called type II errors — abandoning work on a drug because it is believed to be ineffective or harmful when, in fact, with further testing and development it would have been shown to be both effective and safe. Perhaps we would be less skeptical concerning the wisdom of proposed refinements and expansions of the existing safety reporting system had the proposal been supported with detailed analysis of the current reporting systems and its strengths and failings and with a detailed accounting of its product in terms of public safety and well-being.

A major impact of the proposed changes is likely to be in relation to the number of safety reports generated (see discussion of safety reports in Chapter 2 for current definitions and regulations). The proposed revised definition for serious adverse drug experience is

The thrust of the revision is to reduce the amount of discretion left to investigators and to broaden the scope of events to be considered reportable. That expansion is likely to lead to a flood of "safety" reports without any real increase in safety as seen from a societal perspective. Indeed, there is reason to fear the reverse. There is already anecdotal evidence suggesting that the paper flow to IRBs in regard to presumed adverse events has increased markedly in the wake of the FIAU experience. The increase is almost certainly the result, in part, of an increased defensive posture on the part of investigators that is related to their desire to avoid being faulted by FDA for having failed to report some event later judged by auditors to have been important, but not reported.

The increased flow of such reports to IRBs has consequences above and beyond the added administrative burden involved in review and written communications. The change will almost certainly result in a marked increase in false-negative reports. The long-term consequence of such a change will depend on the "personalities" of individual IRBs. One effect may be to reduce the sensitivity of an IRB to all safety reports, if the majority are viewed as unrelated to treatment. That effect will be similar to that observed by the "little boy who cried 'wolf' once too often." The other effect, more likely perhaps with inexperienced IRBs, is to make IRBs consider all reports as serious and, hence, end studies that should have been allowed to continue:

The possibility for confusion of IRBs with a flood of reports is real. As it is now, investigators are obligated to forward any safety reports received from a sponsor in relation to the IND covering their trial even if the reports relate to other trials done elsewhere and under

protocols different from their own. Each such report can result, in the case of multicenter trials, in dozens of individual communications with local IRBs.

Another impact will be the frequency of summary reports to FDA (from annual to semiannual). If there is merit at all to the increase, it should be the exception rather than the rule and should be related only to those compounds that are truly new and for which little is known regarding mode of action or safety.

There is another sort of "reporting," however, that the IOM committee would strongly endorse, even if, as seems likely, it would not have saved any lives in the present case. Trials represent a form of public trust that exists regardless of the drug being evaluated and regardless of how they are funded or sponsored. It can be argued that maintaining that trust carries an obligation on the part of sponsors and investigators to facilitate a free and open exchange of information and to do so even if at the risk of divulging trade secrets or arming would-be competitors with information on compounds under study.

The best way to maintain that trust is via free and open exchanges in regard to the design, conduct, and analysis of individual trials, to be able to do so at all stages of testing, and to do so via recognized routes of presentation and publication. We note, in this regard, that the medical community still awaits a detailed accounting in the peer reviewed literature of the events of the trials in question. Although the committee understands that such a publication is being prepared, we also note that the events in question transpired some time ago. The best defense against repeating the mistakes of the past is via publication in the medical literature.

The committee also notes, in passing, that existing regulations in regard to trials done under INDs work against free and open exchanges. The only obligation on the part of sponsors is to report to FDA. Data submitted to FDA, by law, constitute privileged communications and may not be divulged or released by FDA prior to approval of a new drug application (NDA). Even after approval, the amount of information available to the public is limited to that which can be obtained under the Freedom of Information Act (FOIA). As a result, failing publication, there is no reliable way for sponsors or investigators to obtain data on related compounds being tested under INDs held by other sponsors.

Even after a drug is approved, access to results of individual trials done under INDs supporting an NDA is problematic. Requests for the information must be made under the FOIA and, even if successful, may not result in information amenable to any reliable form of analysis or review. Fortunately, in this case all trials were done under linked INDs so the sponsor and investigators had the means of linking and combining data without constraint.

The solution to information access is not straightforward nor is it likely to be forthcoming soon, since it requires changes to existing laws. That said, however, there is nothing, so far as the committee can ascertain, to stop a sponsor from waiving its right to confidentiality of data sets supplied to FDA or of making them available themselves to interested parties on an as needed basis. Furthermore, there is nothing to stop investigators undertaking trials from publishing their results and, in the case of multicenter trials, insisting that the combined results be published as a condition for participation. In addition, there is nothing to stop IRBs from ensuring that such assurances are provided before approving a clinical trial.

Various people and groups have called for real-time systems of registration of trials as they are undertaken (Dickersin, 1988; Meinert, 1988; Chalmers, Hetherington, Newdick,

Mutch, Grant, Enkin, and Enkin, 1986; Simes, 1986). The means to do so exist. All that is needed is the collective will. Virtually all trials performed at major research institutions, regardless of the funding source, are subject to IRB review before implementation. The establishment of a system for registering trials at that point of review, along with a nationwide system for harvesting such information, would provide a means for sponsors and investigators alike to identify trials done or underway on the compound of interest or on related compounds. At present they are in the dark. At least with such a system they would have a vehicle for the identification of related investigations. Even if this did not lead to data sharing, it would at least have the potential of informing of hidden disasters.