Multi-Center Phase III Clinical Trials and NCI Cooperative Groups: Workshop Summary (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Session 1A: Organization of the NCI Clinical Trials System The NCI’s Clinical Trials System Dr. Jeffrey Abrams, associate director of the NCI’s Cancer Therapy Evaluation Program (CTEP) began the first session of the conference by describing how the Cooperative Group Program fits within the context of other NCI clinical trial programs, discussing the improvements that have been made within these programs in recent years, and outlining the challenges that still lie ahead. In addition to its intramural Clinical Center, Dr. Abrams said, the NCI has a large, multi-faceted extramural clinical trials program. The NCI has grants that can support either investigator-initiated studies or the cancer centers at which trials are conducted, as well as cooperative agreements, such as those that underlie the Cooperative Group Program and the Community Clinical Oncology Program (Box 1). The NCI’s CTEP currently supports about 250 Phase I clinical trials, 400 Phase II clinical trials, and between 100 and 150 Phase III clinical trials. In addition, the NCI’s Division of Cancer Prevention currently has 123 active trials. These provide financial and logistical support for both patients and physicians within a large national and international network for their participation in clinical trials sponsored by the NCI. Cooperative agreements also provide support for a Phase I treatment program, a program directed at treating  See http://www.cancer.gov/cancertopics/factsheet/NCI/clinical-trials-cooperative -group.   See http://ctep.cancer.gov.  

 MULTI-CENTER PHASE III CLINICAL TRIALS BOX 1 NCI Clinical Trials Program: Multi-faceted Extramural Research Activities •  rant mechanisms—R01, R03, R21, R37, and P01 grant-supported trials G in treatment, control, and prevention •  ancer Center Support (Core) grant—partial support for trials at NCI C comprehensive cancer centers • Research contracts—prevention and treatment trials •  pecialized Programs of Research Excellence (SPOREs) (P50 grants)— S treatment and prevention •  ooperative agreements—Community Clinical Oncology Program re- C search bases, cooperative groups, Phase I treatment and central nervous system tumors (adult and pediatric), Blood and Marrow Transplant Clinical Trials Network Intramural Research Activities • Clinical Center SOURCE: Abrams presentation (July 1, 2008). central nervous system tumors, and a bone-marrow transplant cancer treatment network. With the program’s participating sites, which are scattered through- out 50 states, Puerto Rico, and Canada, cooperative group trials accrued about 20 percent of all the patients at NCI-designated comprehensive cancer centers in 2007, Dr. Abrams said. This percentage is about the same as the percentages accrued by industry at these cancer centers. To further boost participation, the NCI established the Community Clinical Oncol- ogy Program (CCOP) and the minority CCOP to feed into the cooperative group trials. These CCOPs make up a large network of community-based physicians, including those based in populations with sizable numbers of minorities, which receive financial support from the NCI so that their patients can participate in cooperative group or other NCI-supported clinical trials. CCOPs provide accrual to protocols, which is the enroll- ment of qualified patients into clinical trials, and some data management and quality control, while affiliate cooperative groups or cancer centers are responsible for developing protocols, data management and analysis, and providing quality assurance.   See http://prevention.cancer.gov/programs-resources/programs/ccop. 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM  Dr. Abrams said that the Cooperative Group Program is unique among NIH-supported clinical trials programs in that it consists of researchers at institutions affiliated with the cooperative groups who jointly develop and conduct trials in multi-institutional settings. It also has a clinical trials infrastructure that is available at any time to test new therapeutic strate- gies as well as a flexible research agenda that can respond to changing scientific opportunities and new discoveries. Also unique to the Coopera- tive Group Program is the volunteerism on the part of researchers who support it. “Not everybody is getting their own particular grant to do this work,” he said. “In fact, many centers that contribute mightily only get their per-case reimbursement, which is not sufficient to support this work. So volunteerism has been a keystone of this project.” Currently there are 12 cooperative groups, which are listed in Box 2. In addition, there are related NCI-sponsored groups that provide radio- BOX 2 NCI Cooperative Group Program 2008 The NCI Cooperative Group Program is composed of 12 groups. Seven are classified as multimodality groups, while the others specialize in various cancer sites or treatment modalities. Multimodality: • Cancer and Acute Leukemia Group B (CALGB) • Children’s Oncology Group (COG) • Eastern Cooperative Oncology Group (ECOG) • European Organisation for Research and Treatment of Cancer (EORTC) • NCI of Canada-Clinical Trials Group (NCIC-CTG) • North Central Cancer Treatment Group (NCCTG) • Southwest Oncology Group (SWOG) Specialty: • American College of Radiology Imaging Network (ACRIN) • American College of Surgeons Oncology Group (ACOSOG) • Gynecologic Oncology Group (GOG) • National Surgical Adjuvant Breast and Bowel Project (NSABP) • Radiation Therapy Oncology Group (RTOG) NOTE: NCIC-CTG funding is limited to participation in intergroup trials. SOURCE: Abrams presentation (July 1, 2008). 

 MULTI-CENTER PHASE III CLINICAL TRIALS therapy or imaging quality assurance reviews and aid with imaging data management. The cooperative agreement that funds the cooperative groups is a cross between a grant and a contract, Dr. Abrams said, so that government scientists and grantees can work together to achieve the best outcome (CTEP, 2008). A financial award is given to support the infra- structure of the cooperative groups. This funding is used to support their operations, statistical analyses, offices, investigators, and committees. Some cooperative groups use this funding as a source of reimbursement of per-patient costs at their sites, once patients are accrued onto a clini- cal trial. But for many cooperative groups, the NCI provides per-patient reimbursements to individual cooperative group sites in addition to the funding for the cooperative group’s infrastructure. The NCI coordinates protocol review of all the cooperative groups and provides quality assurance and pharmaceutical management pro- grams. The NCI and the cooperative groups work jointly on data and safety monitoring boards, meeting organization, and development of intergroup relationships and arbitration procedures. Cooperative groups are also permitted to accept funds from non-government sources for research not supported by the NCI (NCI, 2000). Via this mechanism the cooperative groups can accept support for their trials from industry or from charitable contributions. Despite some industry involvement, the cooperative groups have maintained their independence. Company part- ners are not involved in the monitoring of the trials and are informed of the trial results at the same time that the public is informed. These public–private partnerships are valuable, Dr. Abrams said, especially as they assist with some of the regulatory compliance needed to pass drugs through Food and Drug Administration (FDA) review. He added, however, that private funds are usually used for specific trials and not for maintaining the infrastructure of the cooperative groups. Conse- quently, private funds cannot always compensate for insufficient public funding, as these funds often cannot be rapidly used to support trained personnel at a large number of sites. The NCI, including the cooperative groups, tends to run trials focused on the best management of disease and not on specific agents, Dr. Abrams said. Such trials might compare two or more novel approaches with an accepted standard treatment or assess predictive markers for select- ing individualized therapeutic approaches. The NCI also conducts tri- als generally neglected by industry, such as those of rare diseases or of cancer-prevention interventions, and carries out studies aimed at improv- ing upon commercially available agents or determining their safety and effectiveness at lower doses or in the pediatric community (Mauer et al., 2007). Between 1998 and 2007 the number of Phase III cooperative group 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM  trials decreased, which may reflect the fact that funding for the program declined after 2002 and has currently leveled off at around $145 million a year. This figure reflects a 20 percent decline in funding when the effects of inflation are considered, Dr. Robert Comis, president and chairman of the Coalition of Cancer Cooperative Groups and group chair of the Eastern Cooperative Oncology Group (ECOG), pointed out in a later pre- sentation (NCI, 2008b). Since 1999, Dr. Abrams said, the reimbursement for sites has remained fixed at $2,000 per patient on the treatment trials, which is about one-third to one-quarter the amount of financial support needed to support the cost of these studies (Schmidt, 2007). To counter insufficient funding the NCI increased per-case reimbursements to sites by $5 million last year. It has also set aside $1.6 million for biomarker studies run by the cooperative groups. But Dr. Comis later noted that, by way of comparison, industry often pays more than $15,000 per case in Phase III studies. “Cooperative group trials are an incredible bargain for the public. It’s almost unbelievable that we can do the work that we do at $145 million a year and still provide the kinds of data and information that continue to drive the field forward,” Dr. Comis said. Dr. Abrams described a number of improvements the NCI has made to its Phase III clinical trials programs over the past 10 years, which have made them more efficient and brought them up to date with the current state of the science. One such improvement, for example, was establishing a centralized Institutional Review Board (IRB) so as to avoid multi-center trials having to be reviewed by hundreds of IRBs throughout the country. Currently 329 institutions are enrolled in the central IRB, and 183 studies have been approved through this process, Dr. Abrams reported. But he said that although more than half of the pediatric sites participate in the central IRB, only about one-quarter of the adult sites do. Barriers to an increased use of the central IRB include individual institutional concerns about legal liability and having a separate process for cancer trials as opposed to other trials, as well as an unwillingness by these institutions to give up control in this area (McNeil, 2005). Those institutions hesitant to participate in the central IRB can participate in a “facilitated review,” which allows the local IRB to monitor the initial work of the central IRB. If the local IRB agrees with the central IRB’s review and finds it acceptable, the local IRB can then let the central IRB take over the review functions for the trial from that point on. The NCI also established the Clinical Trials Support Unit (CTSU), which is essentially a virtual administrative assistant. The CTSU created a single online menu of trials for the Cooperative Group Program, which   See http://www.ncicirb.org.   See http://www.ctsu.org. 

10 MULTI-CENTER PHASE III CLINICAL TRIALS includes all the required documents for physicians and their patients to participate in a trial, as well as data management and regulatory docu- ments. The online menu makes it possible for any cooperative group member to participate in any Phase III trial suitable for his or her patients, rather than only those trials within the specific geographical cooperative group of which they are a part. “We have gotten away from only ECOG members working in ECOG and only NSABP [National Surgical Adjuvant Breast and Bowel Project] members working in NSABP,” Dr. Abrams said. “We have really opened the system up quite extraordinarily.” CTSU also recently created a new system called the Open Oncology Patient Enrollment Network, which will be a single site for all cooperative group enrollments for Phase I through Phase III trials. Physicians can register and enroll their patients through the same system. “We think that will be a big boost for investigators to only have to deal with one system regard- less of what cooperative group trial they choose to enroll a patient in,” Dr. Abrams said. These CTSU accomplishments have speeded the accrual of coopera- tive group clinical trials, according to Dr. Abrams. It also appears to have boosted the number of non-cooperative group affiliated sites now partici- pating in clinical trials, and it may be a way for them to develop a track record and improve their clinical trials participation. In addition to creating CTSU to make participation in cooperative group trials more user-friendly and cost-effective, the NCI has also put together state-of-the-art scientific meetings and established disease- specific steering committees comprised of experts, cooperative group and CCOP members, patient advocates, and community representatives. The steering committees determine the best questions to be addressed by the cooperative group’s Phase III trials and the best ways to design those trials. Before the creation of the steering committees, CTEP reviewed the cooperative group studies. Finally, the NCI hopes to create in the near future a single remote data-capture system for all the cooperative groups. “We have developed what I think is an integrated priority-driven system,” Dr. Abrams said. “Hopefully, the pieces are now in place to make the system hum. I also think we could do a lot better and embrace Dr. Mendelsohn’s goal that we could improve by 50 percent [the time it takes to go from conception to the start of a clinical trial]. But there is a learning curve, and I think we have managed to get the pieces in place now to be able to be more efficient.” He added that in an era of flat budgets, “we do have to become more efficient and have to prioritize the trials we are doing. We cannot have hundreds of trials circulating through the system that are not of top prior- ity.” He also suggested developing and nurturing more partnerships with 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 11 industry and adding screening components to trials to meet the patient assessment needs of targeted therapies. He said that industry is increasingly outsourcing its trials to overseas institutions. While stressing that such testing will not adequately reflect how well a drug will do within the medical environment of the United States, he added that the NCI cooperative groups have to partner with international groups so as to participate in the multinational trials being run by industry. He finished his talk by mentioning the importance of the cooperative groups forming partnerships with advocacy and community organizations. “We have to make sure that advocates are included early in the process, not as an afterthought,” he said. After Dr. Abrams’ talk, Dr. Robert Califf, professor of cardiology, vice chancellor for clinical research, and director of the Duke University Trans- lational Medicine Institute, noted the disparity between the costs of run- ning cooperative group clinical trials and the amount of reimbursement from the NCI. He raised the question of whether it is ethical to attempt to do a clinical trial when those who are running it are not getting paid enough to do it well. Dr. Abrams responded that sites try to meet their cost burdens by balancing trials that are better reimbursed because of industry partnerships with those that receive lower reimbursements. He added that funds are also donated from cancer centers that see running national cooperative group trials as in their best interests. “But as times have gotten tougher, that interest has declined,” he added. Some cancer centers have capped the number of accruals that can go to cooperative group trials, because they feel it is too much of an economic burden, he said, agreeing with Dr. Califf that the economic situation is currently a crisis and is causing the numbers of cooperative group clinical trials to drop. “The costs have gone up dramatically, yet the cap on the amount reimbursed per patient has not changed since 1999,” Dr. Parkinson said (Schmidt, 2007). “So somebody is paying the price, and the issue is what trials are actually making a difference. It would be interesting to see performance metrics related to information per unit patient and whether some clinical trials are more productive than others.” Also concerned about the quality of clinical trials, Dr. Burger, associ- ate professor of clinical obstetrics and gynecology at the University of California, Irvine, School of Medicine, suggested considering manda- tory online educational course work for institutions that are not actively attending specific cooperative group meetings in cases where those trials have been initiated. He also suggested adding quality assurance repre- sentatives to each Phase III clinical trial done in the cooperative groups, in addition to the traditional study co-chairs. 

12 MULTI-CENTER PHASE III CLINICAL TRIALS Mayo Clinic and North Central Cancer Treatment Group The next speaker, Dr. Jan Buckner, professor of oncology at the Mayo Clinic College of Medicine and the group chair of the North Central Can- cer Treatment Group (NCCTG), spoke about the academic-community partnership between Mayo Clinic and the NCCTG, which has existed for the past 30 years. Although originally established as a way for Mayo Clinic to reach patients in sparsely populated nearby regions, the NCCTG now has 43 member networks that treat patients in more than 340 loca- tions within 33 states, Canada, and Puerto Rico. More than three-quarters of its patients are enrolled from community practices. Mayo Clinic serves as the research and administrative base for the NCCTG and also supports other cooperative groups via its biospecimen bank and its statistics and data center. The centralized, integrated support that Mayo Clinic provides to the NCCTG is extensive and includes: •  cientific leadership, with experts in therapeutic interventions, labo- s ratory correlative studies, statistics, epidemiology, and quality of life supporting the design, implementation, and analysis of clinical trials; •  dministrative support, including grant preparation and manage- a ment, budgeting, contracting and legal support, accounting, com- munications, and publications; •  tatistics and data management, including safety monitoring, data s analysis, statistical design of studies, data collection, quality assur- ance and control, and abstract and manuscript preparation; and •  perations support, including regulatory support, protocol develop- o ment, information technology, site and meeting management, data collection, and quality assurance and control. Dr. Buckner noted that although the quality control of the trials begins with concept and protocol review both internally at Mayo Clinic and externally by the NCI, the relevance and feasibility of a proposed trial are enhanced by having industry sponsors, the FDA, patient advocates, and community oncologists provide their input in protocol reviews. Each protocol is also reviewed by disease- and modality-specific committees at Mayo Clinic and an independent Mayo Clinic research committee. There are multiple mechanisms for quality assurance, including auto- mated web-based data monitoring that generates alerts when there are serious adverse events (SAEs), when data are overdue, or when data are questionable. On-site audits are conducted by the NCCTG, the NCI, and the FDA. Protocols are highly specified for imaging technique and assess- ment criteria, and a central review of images plus on-site audits also help 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 13 to assure image quality. Dr. Buckner noted that pathology quality assur- ance has become more important with the advent of targeted therapy. This is done with protocol-specific specimen submission kits, consensus review of problem cases, and centralized laboratory confirmation of pro- tein or gene targets. There are also guidelines for ensuring the ethical integrity of the cooperative group’s clinical trials. All investigators undergo training on the ethical conduct of human subjects research and must disclose any con- flicts of interest. Because of the pivotal role that Mayo Clinic investigators play in the process, the Mayo Clinic Conflict of Interest Review Board also reviews all Mayo Clinic participants. All these review processes ensure that the cooperative group’s clini- cal trials are high quality and scientifically rigorous, Dr. Buckner said, but they also cause protocol development times to be entirely too long. Protocols can be as long as 150 pages and sometimes require as many as 150 weeks for approval. “Certainly each cooperative group has its own internal inefficiencies that need to be addressed,” he said, “but the exten- sive external review creates a set of inefficiencies in and of itself. Often the reviews are sequential and not concurrent, and there is a lack of standard- ization of the processes.” Industry sponsors may also cause delays with their complex internal decision-making process and their contract and budget issues, while IRB reviews delay the process further still. “Whether it is an NCI central IRB review, a regional central IRB, or an individual institutional IRB, there is lack of consistency among the IRBs in terms of criteria and process,” he said. In addition to start-up inefficiencies, Dr. Buckner said, regulatory processes also impede cost-effectiveness. Site and investigator credential- ing is excessive and lacks standardization across the country. Adverse event reporting is redundant and inconsistent across organizations. Many industry and government policies add administrative costs but have ques- tionable added value, he said. Dr. Buckner ended his talk by suggesting that external inefficiencies be improved by standardizing information technology infrastructure as well as data elements, collection, and reporting. He also suggested sim- plifying and harmonizing regulatory methods, such as the adverse event reporting required by both the NCI and the FDA. He then described Mayo Clinic’s attempt to improve its internal inefficiencies by using the lean process to eliminate steps that do not add value. This process was led by the Mayo Clinic Quality Academy and involved protocol development unit staff and legal and budgeting staff. The staff focused on reducing the number of steps and time taken between drafting the first protocol and submitting it to the NCI or the IRB, because these steps were under the control of the clinic. The steps were mapped 

14 MULTI-CENTER PHASE III CLINICAL TRIALS out, and those that did not add value to the process were eliminated. For example, redundant reviews and delays caused by waiting for e-mail responses were eliminated. “We cut out a part of the e-mailing by saying you must appear on a certain date for protocol planning meetings or your protocol will not go forward,” Dr. Buckner said. “That gave some account- ability to the timeline.” A review of seven protocols submitted since the streamlining process revealed that development time from the time the protocol was first completed until it was submitted to the NCI or the IRB dropped by 75 percent. After Dr. Buckner’s talk, there was some discussion of how many clinical trials a given institution participating in the Cooperative Group Program can run simultaneously in a high-quality manner. Dr. Buckner noted that there is adequate training available for both new and estab- lished data management experts and that clinical research associates and research nurses attend semiannual meetings not only for the NCCTG but also for other cooperative groups in which they participate. In addition, because the same institutions have been running these clinical trials for 30 years or longer, there is expertise at the institutional level that is transmit- ted from one study to another. There are also a number of quality control tools that enable consis- tency and quality across participating institutions, Dr. Buckner added. These tools include remote data-capture system methods that require participants to fill out every field and the availability of quality assurance personnel to answer questions. “There are multiple mechanisms by which we can have multi-center trials of high quality if we [are diligent] in training the people and then supporting them after the protocol is open,” Dr. Buckner said. He also noted that the quality of studies tends to rise with the number of trials run at each site. “If you are talking about three or four trials at a given site, that is maybe a yellow flag that the quality may not be good. If it is 8 to 10 trials and people put on 5 to 25 patients a year, I think that generally is enough for staff to be able to do good quality research with adequate support,” he said. Radiation Therapy Oncology Group The next speaker, Dr. Walter Curran, professor and chair of the Department of Radiation Oncology of Emory University School of Medi- cine, chief medical officer of the Emory Winship Cancer Institute, and chair of the Radiation Therapy Oncology Group (RTOG), discussed how the RTOG cooperative group is organized. This multi-center cooperative group systematically tests novel radiotherapy approaches against cancer and pursues fully integrated translational research to support and further 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 15 this effort, Dr. Curran said. He added that because RTOG investigators are located at nearly every academic center in the United States and Canada, patients throughout those two countries can participate in RTOG clinical trials. Because of a new international membership initiative (Corn et al., 2008), there are several international RTOG members, including members located in China, Hong Kong, Israel, Korea, Peru, and Russia. RTOG also runs clinical trial collaborations with other organizations, including the European Organisation for Research and Treatment of Cancer, South- west Oncology Group (SWOG), Cancer and Acute Leukemia Group B (CALGB), the NCCTG, the American College of Radiology Imaging Net- work, and Specialized Programs of Research Excellence (SPOREs). RTOG is administered by the American College of Radiology. RTOG has an elected chair and both elected and appointed vice-chairs. A 15- member steering committee with elected and appointed members meets monthly. An executive committee includes those 15 members plus 13 additional members. There are also disease-site committees and working groups as well as scientific core committees, which are listed in Box 3. Some of these core committees are unique to RTOG’s mission, such as an Advanced Technology Integration Committee, which examines how to test and evaluate the available technologies specific to radiation oncology. Dr. Curran noted that a Clinical Trials Education and Recruitment Work- ing Group is embedded in every clinical trial and is particularly useful. This group consists of patient advocates and various experts who evalu- ate every trial concerning its ability to accrue and the likely difficulties of running the trial. BOX 3 Radiation Therapy Oncology Group’s Scientific Core Committees • Advanced Technology Integration • Health Services Research and Outcomes • Translational Research Program • Biospecimen Resource • Pathology • Medical Oncology • Medical Physics • Surgical Oncology SOURCE: Curran presentation (July 1, 2008). 

16 MULTI-CENTER PHASE III CLINICAL TRIALS According to Dr. Curran, RTOG is unique in that its biostatisticians, data managers, headquarters staff, quality assurance center, protocol developers, and group chair’s office are all housed in Philadelphia within an integrated office layout. “The work-flow efficiency with this model has really been outstanding,” he said. Decisions about which studies to run are aided by the disease-site committees and working groups and the sci- entific core committees, which define research priorities, Dr. Curran said. The steering committee then reviews and approves all research proposals and adjudicates among competing research priorities. Much of the research run by RTOG focuses on how to improve the physical and molecular targeting of radiation therapy, with or without chemotherapy, so that it is less toxic and more effective, Dr. Curran said. Among the innovations being tested are the use of image-guided radia- tion therapy and functional imaging to improve the physical targeting of tumors, the combination of targeted chemotherapies with radiation ther- apy, and the use of molecular biomarkers such as the epidermal growth factor receptor to target radiation therapy or identify those patients most likely to benefit from such therapy. RTOG also conducts translational and analytic research with its unique and interlinked clinical biophysical, biologic, and outcomes databases that enable powerful biostatistical and medical informatics approaches. The strategic themes of RTOG, which emphasize radiation therapy, are not duplicated by other cooperative groups, Dr. Curran said. Because of this emphasis RTOG is the lead cooperative group in studying primary and secondary brain tumors, head and neck cancer, and non-operative therapies for localized and locally advanced prostate cancer (Chung et al., 2007). About half of RTOG’s funding comes from its core cooperative group agreement, with additional funds coming from corporate foundations and other grants. Sixteen percent of its funding comes from a Pennsylvania state tobacco settlement. Dr. Curran added that RTOG relies heavily on investigator volunteerism. Recently RTOG nearly doubled the amount of funding it receives from private foundations, and it has substantially expanded its membership and volunteer member efforts. This has resulted in a 28 percent increase in accrual over the most recent grant period and an increase in the number of publications and abstracts on RTOG stud- ies, Dr. Curran said. He added that the accrual failure rate of RTOG has decreased from 33 percent in 1996 to 9 percent currently. Dr. Curran summarized his talk by saying, “We have an organization customized to our mission and strategic aims and a unique niche among the cooperative groups. I think that would be true of all of the cooperative groups, and that is why the system probably is as complicated as it is.” After Dr. Curran’s talk, there was discussion of how to handle the 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 17 regulatory, quality assurance, and funding aspects of international mem- bership. Dr. Curran said that this is a challenge and that every country is different, but he added that the initiative from the NCI on international cooperation will be helpful in this regard. “You really need someone who is an advocate at the institution,” he said. He also pointed out that there are no differences between the international and the domestic criteria for being an RTOG member, which helps with quality assurance across inter- national boundaries. But he added that the support for radiation quality in the American sites is greater than that for international sites. Improving clinical trial start-up times The next speaker, Dr. David Dilts, professor in the Owen Graduate School of Management and the Vanderbilt University School of Engineer- ing and co-director of the Center for Management Research in Healthcare, discussed organizational shortcomings in developing clinical trials and ways to overcome them. An expert in operations management, Dr. Dilts has shown with his research that it takes about the same amount of time to set up a cancer clinical trial as it does to run one. His detailed analysis found that it takes about 810 steps to open a Phase III cooperative group clinical trial, with 68 of those steps having an opportunity for looping that would involve additional revision and review steps. As many as 38 sepa- rate groups or individuals can be involved in a study before it receives its first patients, Dr. Dilts said. Dr. Dilts has mapped these steps and process loops: in 8-point font, the process map for CALGB was 35 feet by 5 feet, ECOG’s was 50 feet by 5 feet, and CTEP’s was 45 feet by 5 feet. There is no evidence that many of the steps in the development process improve the value of the study, he added (Dilts et al., 2006, 2008). “There is a ton of redundancy,” he concluded. Part of that redundancy is due to what Dilts termed scope creep, which occurs when one group or organization expands the scope of its authority or power beyond what was originally intended. An example would be an IRB reviewer that reviews not just the ethical design of a study but the scientific design as well. “Everybody wants to add something to make it a little bit better by tweaking a study, but the minute you add something, you may add months to the development process,” Dr. Dilts said. Time is also wasted, he added, when there are extraneous reviews in the develop- ment process rather than a comprehensive review. When Dr. Dilts analyzed the development process for clinical trials, it became apparent that there was not one step or one individual or group in the process that was the bottleneck. There were inefficiencies across   See http://ctep.cancer.gov/guidelines/nci_clin_intl_guidelines.pdf. 

18 MULTI-CENTER PHASE III CLINICAL TRIALS the board, and each step contributed to the delay in opening a clinical trial. Rather than a single bottleneck, there was what is referred to as a “floating” bottleneck in the system. As soon as one bottleneck is corrected, another bottleneck arises to take its place. Dr. Dilts also said it was inter- esting to note that each individual or group took about the same amount of time to complete its step in the process. The problem is not how much time each step takes, Dr. Dilts said, but how many repetitive steps there are with looping such that the same per- son or institution keeps reviewing the same study after minor alterations were made that other reviewers required. “Only by working together can we make major improvements,” he said. His computer model found that if individual cooperative groups or CTEP singly try to improve their pro- cesses, they will each cut only a few days off the trial development time- line, but if they work together to improve the entire process, the timeline will be substantially shortened. The desired outcome is to decrease the amount of time to open a study from being discussed in terms of years to being discussed in units of days, Dr. Dilts said. Dr. Gordon Bernard, professor in the Department of Medicine and assistant vice chancellor for research at the Vanderbilt University Medical Center, later added in his talk that much time is wasted, because many of the steps in clinical trial development are conducted serially rather than in parallel. He noted that his group can predict when they will be finished with a protocol and be ready for a protocol review committee to review it. But the National Heart, Lung, and Blood Institute will not set up a protocol review committee until the final protocol is in hand, and that can take an additional three or four months to set up. The median time it takes to open a Phase III cooperative group clinical trial currently is 2.5 years, but that time can vary from 1.25 years to almost 7 years, Dr. Dilts reported (Dilts and Sandler, 2006; Dilts et al., 2006, 2008). The science can change tremendously during the time it takes to approve a clinical trial. As a result, when the trial starts accruing patients, it may happen that it is no longer testing the “popular” experimental treatment or that the protocol is no longer relevant, Dr. Dilts pointed out. During the following presentation, Dr. Richard Schilsky, professor of medicine at the University of Chicago and chairman of CALGB, gave a telling example of this: A clinical trial aimed at assessing whether the addition of cetuximab to standard chemotherapy improved the survival of patients with previ- ously untreated metastatic colorectal cancer (Clinicaltrials.gov, 2008). Dur- ing the several years of protocol development and initial accrual phase of the study, bevacizumab was approved by the FDA for the treatment of metastatic colorectal cancer, and another study showed the importance of testing colorectal cancer tumors for genetic mutations (in the ras gene) that predict whether patients will respond to cetuximab. This required 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 19 rewriting the protocol twice and subjecting it twice to the same lengthy review process it went through initially so that bevacizumab could be part of the standard of care given in the trial and so that patients’ tumors were screened for the ras mutations prior to receiving treatment with cetuximab. The length of the development process for a clinical trial appears to affect the accrual success of the trial. The longer trials take to be devel- oped, the less likely it is that they will meet their minimum accrual goals, Dr. Dilts reported (Goldberg, 2008a). He stressed that the ultimate i ­ nefficiency is a clinical trial that is never completed because of insuf- ficient patient accrual, and this happens far too often with cooperative group trials. Sixty-four percent of all Phase III studies sponsored by CTEP between 2000 and 2007 did not meet their minimum accruals. Only about one quarter of all cooperative group trials accrue five or more patients, and nearly 40 percent do not accrue any patients at all. “All those 800-plus steps it takes to develop a clinical trial are wasted and useless if nobody shows up,” Dr. Dilts said. In the remainder of his talk, Dr. Dilts discussed ways to remedy the inefficiencies that cause long development times for clinical trials and affect their success in meeting minimum patient accrual levels. He sug- gested analyzing existing data—and collecting additional data—to assess how long it takes to develop various clinical trials and to determine what factors affect those development times. Redundant, non-value-added steps in the process should be eliminated. Dr. Dilts noted later during discussion that, to facilitate such data collection, a clinical trial should be identified by the same tracking number as it goes through the different review steps. “In the four comprehensive cancer centers we studied, there was an average of eight different tracking numbers for exactly the same study—you had an IRB number, a finance number, a grant number, etc., and so you could not track it,” Dr. Dilts said. “If you cannot track it, how do you know what is happening to it?” During his presentation, Dr. Bernard agreed that it is important to create metrics around the clinical trial development process in order to determine much more readily when and where in the process problems are happening. He also recommended sharing those metrics within coop- erative groups and providing comparative metrics to local organizations. “We plan to list our metrics on the CTSA [Clinical and Translational Sci- ence Awards10] website, to the embarrassment of some of our sites,” he said. Dr. Califf also recommended publicly reporting metrics during his 10  See http://www.ncrr.nih.gov/clinical_research_resources/clinical_and_translational _science_awards. 

20 MULTI-CENTER PHASE III CLINICAL TRIALS presentation. “Just the public shame of how pathetic this is will lead to improvement,” Dr. Califf said. Dr. Dilts also suggested not trying to run every feasible or scien- tifically worthy trial that is conceptualized but rather to limit trials to those prioritized as being among the top 10 per institution. Dr. Bernard concurred with this approach in his presentation, noting that otherwise “you just bury the organization in protocols that are not accruing, and they really do use up the infrastructure of the institution.” Priorities can be set based not only on scientific merit but also on operational complex- ity, which determines how likely trials are to succeed, Dr. Dilts said. The optimal studies are those that have the highest scientific merit and the lowest operational complexity, he added (Figure 1). During the discussion period, Dr. Joseph Aisner, professor of medi- cine, chief medical officer, and director of the Thoracic Oncology Program at the New Jersey Cancer Institute, raised his concern that streamlining the number of clinical trials will adversely affect research on rare or pedi- atric tumors, where there are now three protocols activated institutionally for every patient accrued. Dr. Bernard noted that this was a valid concern and suggested that another funding mechanism, perhaps one akin to that for research on orphan drugs, could be set up to make sure clinical trials on rare cancers continue. “The central protocol could become a much more automated process so that you can get a study up and running within a week as soon as you discover you have a patient to enter it,” he suggested. CAREFULLY TOP High CONSIDER PRIORITY Scientific Merit Low AVOID FILLER High Low Operational Complexity FIGURE 1 Scientific triage: A technique for determining which trials should be pursued. Prioritizing the conduct of clinical trials should take into account both scientific merit and operational complexity, according to Dr. Dilts. The most straightforward studies to undertake are those high in scientific merit, but low in operational complexity. SOURCE: Adapted from Dilts presentation (July 1, 2008). R01406 figure 1 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 21 One key to prioritizing and limiting the number of clinical trials run at each institution is eliminating what Dr. Dilts called the “entitlement cul- ture” that currently exists among the investigators of cooperative groups. This culture encourages people to think along the lines of “the cooperative group is responsible for opening my study because I am part of the coop- erative group,” Dr. Dilts said. “Suppose that the New England Journal of Medicine had that—I’m a doctor, so you should publish my paper because I am a doctor.” Clinical trial review committees or institutions are not only reluctant to “just say no” to proposed trials, but when they do say no, their denial often is not meaningful, Dr. Dilts said. He found that 14 disapproved con- cepts were still developed into protocols, and 11 of these were activated. Seventeen withdrawn concepts were developed into protocols, 8 of which were activated. “No should mean no,” he said. In addition, there should be strict adherence to review deadlines. “If the only penalty for being late is getting more time, then why do some- thing on time?” he said. He also suggested developing and using standard terminology as well as administrative standards, noting that critical scien- tific issues can vary, but administrative processes can be standardized. His final suggestion was to create focused Phase III teams composed of cooperative group and CTEP members and pharmaceutical representa- tives, which could activate a Phase III protocol within 90 days. The incen- tive for meeting the 90-day deadline could be providing the grant money to run the study, Dr. Dilts suggested. Dr. Dilts concluded his presentation with a quote from Peter Drucker, author of The Effective Executive: “Unless a decision has ‘degenerated into work’ it is not a decision; it is at best a good intention” (Drucker, 2007). He added, “We do not have time to make good intentions. We have to make the system better.” Following Dr. Dilts’s talk, Dr. Maurie Markman, vice president for clinical research at the M. D. Anderson Cancer Center, raised the possibil- ity that there was a link between the high costs of running clinical trials and low accrual rates. “The cost is somewhere between $6,000 to $8,000 per patient now for some single trials, because you are following the patients for several years,” he said. “I believe that is a large explanation for why the accrual is the way it is. You have got to pay the people to do the work, and there is not money for it. We do not have the funds at the institutional level to support this increasingly expensive enterprise.” Dr. Dilts agreed about the high expenses for running clinical trials, many of which are not reimbursed. But he reiterated that there is great expense in developing clinical trials that are never run to completion because of poor accrual. “If you really cut down the number of trials run, then perhaps there will be enough money to pay for what you need,” he 

22 MULTI-CENTER PHASE III CLINICAL TRIALS said. He added that industry studies, which provide better reimburse- ments, do not accrue better than investigator-initiated studies, and that there is no funding source–dependent time difference in how long it takes comprehensive cancer centers to open a study. “I do not know whether money makes a difference in the eventual accrual. I know it does not make a difference in the time it takes to open trials,” Dr. Dilts said. Also discussed was whether increased government involvement in cooperative group trials is causing unnecessary delays in clinical trial start-up times. Dr. John Ruckdeschel, president and chief executive officer of the Barbara Ann Karmanos Cancer Institute and Cancer Center and associate dean of cancer affairs at Wayne State University’s School of Medicine, noted that in 1981 cooperative groups stopped being funded by NCI grants and instead were funded by a cooperative agreement that boosted government oversight (CTEP, 1996). “The cooperative agreement meant that at every step of the way the government was involved,” he said. “CTEP primarily, but you can count multiple other sources and com- mittees and progress review groups. We have to reexamine whether that was the right change, because what it has imposed is a whole lot of this back and forth business. In addition, it has become just like the emergency room physician who orders test after test because he is worried about being sued. Cooperative groups spend an inordinate amount of time redesigning studies so they are more likely to get approved by CTEP and others.” In response, Dr. Dilts noted that usually in business there is what is known as disintermediation, which raises the question, Why do I need the middle layers at all? “Why do we need intermediation of both CTEP and the cooperative groups?” Dr. Dilts asked. “If they do not add value, why do we need them?” Following the discussion, Dr. Schilsky offered a presentation on how to rise to the challenge of rapid protocol activation. He began the talk by pointing out the many stumbling blocks that cooperative groups face when taking on this challenge. One of these impediments is the “all- volunteer army” of investigators that run cooperative group trials. Refer- ring to CALGB, which he chairs, he said “None of the investigators in the CALGB work for me, so I do not control how they spend their time.” This poses problems when these investigators are hard pressed to find time to review protocols, because their other responsibilities must take precedence. Dr. Schilsky also reiterated some of the problems that Dr. Dilts pointed out, such as having “too many cooks in the stew” when generating and reviewing protocols, and too much tinkering by each of these parties. “Often the study gets discussed in different venues without exactly the same people around the table,” Dr. Schilsky said. “So somebody comes in who has not seen the discussion in the last six months and says, ‘We ought 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 23 to change the eligibility criteria, or we ought to include a new drug,’ and before you know it, you have got the study being redesigned five differ- ent times, and each time it gets changed, it has to be re-reviewed.” This is exacerbated by the frequent turnover among collaborating industry staff. “We will start a discussion about study design, and a year later suddenly it is a whole new group of people, and they do not seem to have any corporate memory,” Dr. Schilsky said. He described a particularly frustrating review process for one study. After this study was approved by CTEP, it underwent a lengthy three- round review process at the central IRB, which finally approved the pro- tocol, so they assumed they could begin the trial. But then the NCI disap- proved the protocol because of one of the changes made to satisfy the IRB, which required yet another several rounds of revision and reviews at both the NCI and the central IRB. “We are striving to strike the right balance of a controlling culture versus an enabling culture,” Dr. Schilsky said, “and one might argue that we are too heavy on the control side and not heavy enough on the enabling side to allow these studies to move forward more quickly.” He noted that the NCI cooperative agreements “provide very little flexibility for the cooperative groups and how we do business. Why does the NCI provide 50 percent of the funding but retain 100 percent of the control for the cooperative groups?” Dr. Schilsky also repeated the need for communication and synchro- nization of the development team. “In my organization we have the protocol development going on in one office, and the forms development going on in a different office,” he said. “I will be first to admit that we have had many times where the protocol is ready to go, but the forms are two months behind, or vice versa.” Other impediments to speedy trial development are overburdened statistical centers and questions about who will own the data generated from the study. Of increasing importance is the bigger question of who will pay for the study. The NCI pays only for some expenses involved in running a clinical trial and often does not pay for research-related tests that health insurers are also not likely to reimburse. “Say the protocol requires a PET scan every two weeks, a research-related biopsy, or frequent ECG monitoring to assess the QT interval prolongation,” Dr. Schilsky pointed out. “You cannot bill those out as standard of care because they are not standard of care.” He noted that the last review of CALGB recommended an annual budget of $33.8 million per year, but instead it was awarded $14.4 million per year—43 percent of the recommended level of support. This has prompted CALGB to seek industry support for many of its stud- ies, but this support adds lengthy negotiations with the participating pharmaceutical companies, which do not have the best interests of the cooperative group at heart. “It is abundantly clear to me that the company 

24 MULTI-CENTER PHASE III CLINICAL TRIALS attorneys are hired to protect the company’s interest. They are not hired to negotiate favorable contracts with the cooperative groups,” said Dr. Schilsky. He finished his talk by discussing ways to improve the clinical trial development process. To counter tinkering and repetitive reviews, he suggested a “two strikes and you’re out” policy. “Our investigators send in a concept to the executive committee,” he explained. “We will either vote it up or down or table it. If we vote it down, it is done, and we do not entertain a resubmission of the concept. If we think there is a salvage- able problem, we will table it, but it then must be voted up or down the next time through.” To avoid excessive tinkering, Dr. Schilsky also asks his committee chairs not to discuss protocols once they are in develop- ment, because “the more they talk about them, the more people want to change them.” CALGB also uses Dr. Dilts’s approach to prioritizing studies based on both scientific merit and operational complexity, and Dr. Schilsky noted that they raised the bar for a priority score such that there are few concepts coming through the system. Once the executive committee approves new protocol concepts, they are added to a master priority list for the entire cooperative group. “So all parts of our organization know what are the high priority protocols at any point in time,” Dr. Schilsky said. This has helped improve synchronization because both the people in the protocol and the statistical center know, for example, what the top five protocols on the priority list are. His organization is currently also developing a web- based protocol tracking system that can easily be accessed to see where a protocol is in its development life cycle. Following Dr. Schilsky’s presentation, Dr. Scott Ramsey, full member of the Cancer Prevention Program in the Division of Public Health Sci- ence at the Fred Hutchinson Cancer Research Center, suggested that when evaluating clinical trial protocols, in addition to considering the scientific merit and operational complexity of the study, that a third category he called “clinical relevance” also be considered. Determining the clinical relevance would involve surveying the landscape of clinical trials that are currently being run in order to avoid redundancy or to delay proto- col approval based on the results of some of these trials that might affect study design. Dr. Schilsky agreed that this was important and added that CALGB tries to incorporate such information into the initial conception of the protocol. “We do have lots of experts who are involved in design- ing these trials who, generally speaking, know what the landscape is, but sometimes they do not know everything, things change, or unexpected data comes out, the impact of which no one fully appreciated,” he said. Dr. Schilsky also gave numerous suggestions for what other groups involved in reviewing clinical trials can do to improve efficiency. He sug- 

ORGANIZATION OF THE NCI CLINICAL TRIALS SYSTEM 25 gested reconsidering whether it is necessary that the NCI review protocols for which they do not hold the Investigational New Drug (IND) approval or if FDA review of those protocols might be sufficient. When the NCI does have the IND, he suggested combining NCI and FDA review within 30 days. If there are no INDs being tested in a clinical trial, he suggested that an NCI review should be sufficient and that the FDA should not have to review the protocol as well. Eliminating NCI reviews might make industry more willing to collaborate with cooperative groups, Dr. Schilsky pointed out. He suggested modifying the terms of the NCI cooperative agreements so that they have more flexibility with regard to NCI reviews of protocols. Another suggestion by Dr. Schilsky was that the FDA specify a mini- mum data set necessary for New Drug Application (NDA) submissions and that the agency assess the value added of Special Protocol Assess- ments, because they add considerable time to the FDA review process. Dr. Schilsky also suggested having the Centers for Medicare and Medicaid Services (CMS) cover all clinical care costs for patients on trials and also having them modify their physician billing codes so that doctors can bill at a higher rate for their patients on clinical trials, as these patients require more complex management. IRB issues Dr. Schilsky also said that it might be wise to reexamine the value of a central IRB. Although he thinks it is a great idea, he said it has been difficult to implement. Because just 20 percent of the cooperative group sites in the United States ascribe to only the central IRB, the remaining 80 percent are “essentially being held hostage to the central IRB review, because they have to wait for that process to be completed before they can send the protocol to their own local IRB,” Dr. Schilsky said (Ledford, 2007). In a later discussion, Dr. Califf noted that Duke University seldom uses central IRBs for its clinical trials but instead often uses commercial IRBs, which “just do the job without all the rigamarole we typically have inside the academic centers.” He continued, “I think the facilitated IRB is a much better approach, because if you turn over everything to one central IRB and if it does a stupid thing, the entire machine is shut down, whereas beaming people in by teleconference, etc., gives everybody the benefit of expert review without totally giving up local control.” During a later presentation, Dr. Renzo Canetta, vice president of oncology global clinical research at Bristol–Myers Squibb, concurred that “the central IRB is not the magic solution in terms of time saved. Some- times it helps, but sometimes it does not and is something that needs to be revisited.” But Dr. Alan Keller, chairman of clinical research at Cancer 

26 MULTI-CENTER PHASE III CLINICAL TRIALS Care Associates in Tulsa, Oklahoma, noted later in his presentation a number of reasons why central IRBs are advantageous (McNeil, 2005). During a later discussion, Dr. Markman pointed out that feder- ally funded research falls under the regulation of the Office for Human Research Protections (OHRP), but non-federally funded research, such as clinical trials sponsored by industry, does not fall under OHRP’s mandate. “So when you talk about the NCI central IRB, that is OHRP-related, but the for-profit IRBs have nothing to do with OHRP. We really are talking about different languages when we talk about the pharmaceutical com- panies that do not necessarily have to use OHRP,” Dr. Markman said. Dr. Abrams concurred and added that “the major difference between the commercial IRBs and the NCI’s central IRB model is that the commercial system comes in and takes over the entire IRB for an institution and there is not a local IRB—there is only the commercial IRB that serves that insti- tution. The NCI’s model is more of a shared model that still uses a local institution IRB, which provides local context, but the central IRB taking over certain responsibilities for the study.” Later in his talk, however, Dr. Keller deplored the fear of lawsuits and provincialism on the part of local institutions and the perceived impor- tance of “local context,” which he calls an unnecessary sacred cow. “We are a monolithic country when it comes to oncology treatment guidelines, FDA approvals, approved Medicare coding, national payor reimburse- ment standards, judging the standard of care,” he pointed out. “I am not judged by the standard of care in Oklahoma. They will bring in New York attorneys if I get sued.” Even informed consent forms are determined to a large degree by NCI specifications. To further emphasize his point, Dr. Keller pointed out that no drug has ever been denied approval because of geographic, political, cultural, religious, or ethnic differences. “So I would get rid of the local context requirement and then give some backup to our local IRBs to not do this,” he said. He suggested insisting that institutions receiving federal funds use the NCI’s central IRB. He noted that a central IRB reduces redundancy, costs, variability, and time, while increasing oversight and safety. Dr. Mendelsohn suggested requesting that OHRP have all IRBs adhere to the same standards to simplify the review process for those clinical tri- als that use several IRBs. Dr. Abrams concurred with this suggestion. Dr. Markman said later in his talk that the FDA regulations on patient safety need to be congruent with those of OHRP. “There are multiple organiza- tions and agencies at the federal and state levels that are responsible for patient safety, and they need to get on the same page,” he said.