Studies that involve multiple sites or multiple disciplines, or both, present particular problems for institutional review boards (IRBs), Robert Levine, of the Center for Bioethics Yale University, noted in his introductory remarks. A study that involves multiple sites has historically been overseen by multiple IRBs, one for each site, which raises issues of consistency and coordination. The advance notice of proposed rule-making (ANPRM) proposes requiring a single IRB for multisite studies. And a study with multiple disciplines requires IRBs to deal not only with the issues that relate to each of the individual disciplines involved but also with those issues that relate to the interactions among the disciplines. In this session, Pearl O’Rourke described her experiences serving on the central IRB of a multisite collaboration. Laura Stark discussed the way IRBs function in practice and why different IRBs sometimes come to different conclusions about identical studies. Thomas Coates described his experiences leading a long-running study that is both multisite and multinational. Levine noted that each offered unique insights into the problems that multisite and multidisciplinary studies pose to IRBs and into how these problems might be addressed.
Pearl O’Rourke, of the Partners HealthCare System in Boston and Harvard Medical School, discussed the benefits and challenges of using a single IRB for multisite research and described her experience with such a
central IRB. She began with a brief description of the possible benefits of having a single IRB handle multisite studies. These include a more efficient IRB review process, with multiple sites being approved more quickly, and continuing review and amendments being handled more effectively; less duplication of reviews; and a more consistent review because a single IRB will be seeing all of the adverse events. Together, these benefits may help studies get under way more quickly, which may allow researchers to enroll participants more easily and increase their chances of completing their studies successfully.
However, O’Rourke said, having a single IRB for a multisite study simplifies only part of the process. In addition to IRB review, there are also ancillary committee reviews, such as those for conflict of interest, radiation safety, and biosafety; grants and contracts reviews; and institutional sign-off and responsibility for the local conduct of the research. Thus, there will remain a great deal of institutional involvement at each site even with a single IRB.
Types of Central IRBs
Among central, or single IRBs, there are two types: a share model and a nonshare model. In the nonshare model the central IRB fulfills all the IRB review requirements. It is responsible for the initial review, the continuing review, amendments, adverse event reporting, and so forth. In the share model, the central IRB and the local IRBs share some review responsibilities, most frequently regarding amendments and adverse events. O’Rourke suggested thinking of them as falling on a spectrum, with nonshare, central IRBs at one end of the spectrum and the situation in which there are only local IRBs at the other end. Commercial IRBs and IRBs in the Veterans Administration system tend to fall at the nonshare end of the spectrum, she noted. The status quo for most academic multisite studies is at the other end, with each site having its own IRB. The original National Cancer Institute IRB falls somewhere in the middle, she added, because the central IRB would do the initial review, but local IRBs reviewed local site amendments and local adverse events that were not too severe. In another type of share model, the IRBshare, there is “a sharing of documentation and review of what happened at the initial review, but basically everything reverts back to the local IRB once the protocol is up and running,” she explained.
IRB Versus Institutional Responsibilities
O’Rourke also noted that there are institutional responsibilities relating to research that are not handled by IRBs. Institutions are respon-
sible, for example, for a number of the terms of the federalwide assurance for the protection of human subjects. Furthermore, IRB offices often end up with responsibilities beyond reviewing research protocols.
In the nonshare model of a central IRB, the central IRB would be responsible for all IRB review tasks, including the initial review, continuing review, amendments, and so on, and might also be responsible for the Health Insurance Portability and Accountability Act (HIPAA) issues. Meanwhile, the local institutions will be responsible for a large number of other tasks, such as ancillary reviews, HIPAA implementation, oversight of the conduct of the research itself, and required federal reporting (which is not an IRB requirement).
Thus even if a central IRB is put in place, the institution finds itself left with a number of responsibilities. For example, it has to have some way to determine whether a particular protocol is eligible for central IRB review. It must also have some internal process for following the research that is being carried out onsite. It must decide how to deal with such on-site issues as noncompliance. Similarly, the investigator is responsible for knowing the local requirements for using a central IRB and understanding the processes for completing ancillary committee reviews and completing sponsored research office sign-off.
This situation leaves a number of challenges for the institution. The institution may need, for example, to integrate different information technology systems. In many cases, O’Rourke said, the IRB serves as “the center of the wheel,” connecting the various entities involved in overseeing research. Institutions will also have to train their researchers in how to use central IRBs, and the initial negotiations involved in setting up the central IRB require a great deal of time and effort.
The relationship between a central IRB and a local site is set up with a reliance agreement, a formal document that sets forth the details of who is responsible for regulatory reviews and of how the legal, regulatory, and contractual responsibilities are assigned. It is important that the reliance agreement be very detailed, O’Rourke said. “We feel it’s very worthwhile in that if something happens, you have something to go back to and say, ‘This is what we agreed to,’” she explained.
The reliance agreements can be quite complicated, O’Rourke said, because the institutions involved tend to be very complicated. “Very few of us are single entities,” she noted. For example, there may be a primary site with three affiliates. It is important to know where the research will take place. Will it take place just at the primary site or at some of the affiliates as well? And what is the relationship among them? Do they share an IRB, or does each have its own?
Experience with NeuroNEXT Model
O’Rourke described her institution’s experience with NeuroNEXT, a network of about 25 academic medical centers funded by the National Institute of Neurological Disorders and Stroke to do Phase II research in rare neurologic diseases. It uses a central IRB located at the same site as the clinical coordinating centers, which was at Massachusetts General Hospital. NeuroNEXT uses a nonshare model, with the central IRB conducting all IRB reviews. Before any protocol, each network site had to sign a reliance agreement with the central IRB, which sets forth the process of the central IRB review and assigns legal, regulatory, and contractual responsibilities. The reliance agreement covers all NeuroNEXT studies, so when a new study begins, the agreement is already in place.
O’Rourke described the process by which the central IRB approves a research protocol. It begins with researchers submitting a protocol through the clinical coordinating center. Two IRB chairs make an initial assessment to determine whether the protocol is ready to go to a full panel. Once it is ready, they send it to the sites that have been selected for the research. These sites identify any substantive or local issues that concern them and communicate those to the central IRB, which may send the protocol back to the principal investigator for resolution. Next, the full committee of the central IRB reviews the protocol. If it is approved, it is sent back to the participating sites, which can then choose to proceed under that protocol or, if their concerns were not addressed, to drop out.
Most of the protocol and informed consent forms are fixed, but there are a few locally customizable items. Different institutions have different injury language and local contacts, for example, which must be accounted for.
O’Rourke closed with a number of challenges that must be addressed in using single IRBs for multisite studies. One is to differentiate between the tasks to be carried out by the IRB and those to be carried out by the various institutions, such as understanding and addressing the local context, dealing with the logistics of communication, and developing trust. That challenge is not surprising, but some challenges may not be expected, she said. The complexity of member sites, which had multiple subsites at which research would be conducted and myriad organizational structures, may add a challenge. “Although there are 25 member sites, we have 57 reliance agreements,” she said of her own project, adding that there was also confusion about authority. “Although everyone signed off that we would be the IRB of record for any regulatory decisions, once our first protocol went through we still had people saying, ‘It has to have
our IRB stamp on it,’” she explained. There were also diverse views on some basic issues, such as different approaches to regulations, she noted.
The member sites had their own challenges, O’Rourke said, such as if and how to provide institutional review. Questions included who should be involved? The local IRB? And what should the review include? Another challenge for the relying sites was determining the appropriate ongoing institutional oversight of the research. The institution has responsibilities for protecting the research participants, but its role once the study is under way is not clear, O’Rourke observed. There was also confusion for those institutions that were involved in a number of multisite, single IRB studies, she added. If those studies use different single IRB models, it can be difficult for the institution to keep track of the varying systems.
In closing, O’Rourke said that mandating a single IRB for all multi-site research would be “unconscionable.” The ANPRM does not provide details about how the single-IRB approach would be carried out, she said, nor does it adequately suggest the complexities involved and the resources required both for being the central IRB and for relying on a central IRB. Those who play the role of a central IRB in a multisite study should not underestimate the time required for development, the startup and long-term costs of the central IRB infrastructure, the confusion that can result from differences in the ways institutions allocate responsibilities to the institution and the IRB, and the critical role that trust and familiarity play in the development and negotiation of IRB reliance relationships.
O’Rourke does believe that single IRB review can improve the efficiency and perhaps the quality of multisite research, she commented, depending on the quality of the central IRB. But developing and using a central IRB is not easy, she added. It requires a different way of approaching the issue of research review and oversight.
When a number of local IRBs review the same protocol, they may come to very different conclusions about the level of review required for the protocol, Laura Stark, of Vanderbilt University, observed. She suggested that this is one reason for the push to have single IRBs handle multisite research. She noted that local precedent may be one reason why IRBs often reach different conclusions about the same study.
Stark, who studies IRB decision making, described three approaches institutions are using to reduce the problems associated with local precedents. Stark began by describing survey research showing that when different IRBs are presented with the same standard protocol to review, they often arrive at different judgments about how the protocol needs
to be changed before it can be approved. The survey research tends to conclude that the variable decisions are the product of uneven resources across boards, Stark said, so that with larger staffs, more time, and better training, boards would be more likely to arrive at more consistent decisions about protocols.
She believes that there is a second factor at play. Her own research has shown that IRB members rarely deliberated about specific protocols by applying broad rules, such as the Common Rule. “Instead, they made decisions about a protocol by comparing it to previous cases they had decided,” she explained. The previous exemplar cases upon which the IRBs based their decisions are what Stark refers to as local precedents, and they vary from one IRB to another (Stark, 2012). There is nothing inherently wrong with using precedents, Stark said, but in practice their use can lead to a variety of problems, such as when local IRBs use site-specific precedents to evaluate multisite studies or when IRBs use inappropriate precedents in evaluating novel areas of research or new methodologies.
Stark has studied how precedents shape IRB decision making, and she suggested that it is unlikely that any improvements will come from trying to scrub IRB judgment and discretion from the review process. However, she said, it is possible to harness local precedents and turn them into a beneficial phenomenon. “Several institutions are developing models that hold on to the advantages of local precedents while minimizing the many problems that they cause,” she noted.
Models for Addressing the Problem of Local Precedents
Stark described three models. One, the study network approach, is directly relevant to issues raised in the ANPRM. The other two—collegial review and decision repositories—are strategies that can be developed within the framework of existing regulations to address the problem of local precedents. They are practices that can improve review for social science research and research participants even if the proposed changes to the Common Rule are not carried out.
In the study network approach, large, well-funded organizations review all of the studies that are attached to common research initiatives. Examples include the NeuroNEXT collaboration that Pearl O’Rourke described, as well as networks supported by the National Institutes of Health, such as central IRBs for cancer studies, and health maintenance organization research network mechanisms. In a study network, Stark said, “hundreds of investigators who are working in one broad area and
toward one general goal but who are doing so at multiple institutions get reviewed within the same system.” Because one central meta-board makes decisions for all institutions and investigators, a study network limits the problem of local precedents. Ideally, a single set of local precedents—that of the study network—will be used in the review process for investigators conducting similar studies at different sites.
Two limitations affect study networks, Stark noted: liability and money. To reap the benefits of a network study, such as reduced administrative load and more consistent decisions, local boards must largely defer to the decisions made by study networks. (This approach is similar to the nonshare model that O’Rourke described, Stark noted.) But local boards’ liability concerns can undermine these benefits. These concerns are often grounded in the need to take local community attitudes into account in order to comply with federal regulations. Because local institutions remain legally accountable for the research their investigators conduct, she said, many are inclined to interpret those requirements strictly, dismantling many of the advantages of networked studies.
Study networks also require substantial funding. Thus it is generally only those institutions and research programs that can most easily secure funding that can afford access to study networks. There are independent study networks, which are unattached to research sponsors or research sites, Stark said, and these may remedy some of the problems of inequitable access. However, she said, review boards are more likely to be independent of financial considerations if the boards are not operated for profit. Commercial IRBs have received mixed reviews, she said, because they are paid by the very organizations—namely, study sponsors—that have an interest in seeing research move ahead quickly. In Stark’s view, not-for-profit, centrally administered review boards offer the greatest potential for addressing the contradictions between local precedents without undermining the reviewers’ freedom from financial conflicts of interest.
The second model, collegial review, is designed to minimize the application of inappropriate precedents by assigning reviewers who are from the same research tradition as the investigator(s). At universities, Stark said, collegial review is most commonly used for student projects and involves a local review by members of the student-researcher’s discipline. This model can be extended beyond student research if a system of review organized around institutions, departments, or offices is developed. Doing so would move protocol evaluation closer to the people who have expertise in the field in which the investigator is working. Reviewers
who know the implications of the proposed methodologies for the populations under study would conduct the evaluations. Some university IRBs already use department-level subcommittees to complete expedited reviews for the department’s own investigators, Stark said.
One downside of this approach, she noted, is that it could make it more likely that personal politics could influence the decision-making process. To avoid such conflicts, she said, institutions could use external collegial review mechanisms for human subjects review, sending protocols to researchers at other institutions for review.
The third approach, decision repositories, is designed to make it possible for IRB members and researchers to learn about decisions made by various institutions regarding studies similar to a particular study of interest to them. One way to do this is to create an online repository of approved protocols that have been appropriately de-identified. Such a decision repository might be more feasible than other models, Stark said, because it would not require institutions or the federal government to fundamentally restructure the review process.
This approach would require the generosity of a few institutions and agencies, she noted. “Repositories require funding, server space, time, and, importantly, the donation of protocols,” she explained. Researchers are often hesitant to share their current research protocols because of concerns about protecting their intellectual property, she said, so it might make more sense to collect records from completed projects, even though these would become outdated more quickly.
Stark pointed to two existing decision repositories to illustrate this approach. The Sunnybrook Health Sciences Centre in Toronto has created a bank of decisions with the aim of making more consistent decisions, but it is accessible only within the institution. In New Zealand, there is a publicly available digital archive of decisions called the Ethics Application Repository housed at the University of Otago.
Such repositories should help remedy the problem of local precedents, Stark said. Today, decision makers who wish to apply precedents must rely on individual memory, so IRBs with a low turnover rate in membership are likely to develop inertia in their decisions and also to narrow the set of problems they tend to identify with protocols. In contrast, IRBs with a high turnover rate are likely to make inconsistent decisions about similar protocols over time. Decision repositories can help widen the horizon of IRBs with low turnover rates and help remind IRBs with high turnover rates of what decisions have been made in the past. Publicly accessible decision repositories would also help researchers interact more
effectively with their IRBs, Stark said. “Researchers are in a position, in other words, to teach their IRBs how studies should be evaluated, to signal to their boards that a research community exists even if the topic or method is new to reviewers, and to assure IRBs that they wouldn’t be alone in approving a new methodology or new topic,” she observed.
Thomas Coates, of the University of California, Los Angeles, discussed issues that arise in international multisite research. He had recently completed a 10-year clinical trial of the effectiveness of various interventions to prevent HIV infection, which took place at five international sites and involved 14 IRBs in six different countries. The study received informed consent from approximately 200,000 people, using various forms of consent. Although international research is not addressed in the proposed changes to the Common Rule, Coates noted, the implications of these issues are vast, and they are in some ways similar to those of other issues addressed by the workshop speakers.
Coates described a number of issues that researchers face when running an international multisite study. An overarching issue is the incredible disparity in resources between the United States and many of the countries that may be involved in an international study. Potential study participants in other countries may lack access to many basic medical services, for example, and may not even be able to talk with somebody about filling out a questionnaire. The disparities are also evident in the value that participating in such a study has for local scientists and scientific organizations, who can get access to resources that otherwise would be out of reach. “This inequality is part of what needs to be acknowledged when we think about how best to structure international studies,” Coates said.
Many countries, particularly those in the less-developed parts of the world, also have less stringent regulations governing human subjects research. There may be less emphasis on informed consent, and researchers from the United States must make sure that the subjects in the studies actually are freely providing informed consent. U.S. researchers may also notice a lack of strict ethical oversight in some international research projects, in comparison with U.S. customs, as well as less concern about confidentiality and privacy among those working on a study.
Coates illustrated his point with an experience he had in a low-income
neighborhood outside a city in Peru, where a health center had been established. He discovered that the staff of the health center had developed a complete house-by-house map of the neighborhood that specified health-related problems that had been observed in each, including such health problems as diabetes or cardiovascular disease, as well as behavior-related problems such as alcoholism, depression, and partner violence.
Though most of his examples concerned biomedical research, Coates noted that many studies now blend behavioral, social, and biomedical research. The proposed changes in the Common Rule seem to be based on the assumption that the social and behavioral studies are completely separate from biomedical studies. To the contrary, he said, in many cases a combination of disciplines is essential to understanding how things work. For example, if his research group finds that a new treatment is not working in the field as well as expected, they will need to examine the reason from various angles. They will need to ask, “Is that because the agent isn’t any good? Is that because … people don’t use it the way it was intended to be used or because of health systems or other social issues?” he explained.
This blending of various areas of science raises questions about what standards will apply to a study, he said. “If it is a combined study, do the proposed changes in the Common Rule apply to the social and behavioral pieces? Do the biomedical pieces get judged by a different standard?” he asked.
Another issue is how to what extent U.S. standards and regulations should be imposed on foreign countries taking part in international studies funded by U.S. institutions. For example, Coates noted, it is not clear whether foreign regulatory bodies should be required to have a federalwide assurance in place, or whether an equivalent can be accepted. It is also not clear whether the proposed changes have the same implications in a foreign country that they do in the United States, or whether minimal risk is the same in other countries as it is in the United States. It is also important to note that the level of ethical oversight typical in the United States is very costly, and it is not clear which entities involved in multinational studies should bear the additional costs, he added.
Coates suggested that it might be necessary to develop ways to strengthen local oversight in other countries if the proposed changes to the Common Rule are enacted. “If we in the United States are interested in doing research in collaboration with local investigators in foreign countries, we also need to address the kinds of capacity building that might be necessary to ensure that the standards are adhered to,” he suggested. This would mean providing training for investigators and IRB staff and members, particularly in response to the changing expectations for oversight, he added.
Changes and Proposed Alternatives
Coates offered some suggestions for altering the proposed changes to the Common Rule to better suit international multisite studies. If it is useful to apply the proposed changes to all studies funded by federal agencies and clinical studies that seek Food and Drug Administration approval, he suggested, it might also make sense to apply the new regulations to studies that are funded entirely by other countries, multilateral agencies, or philanthropic organizations, such as the Bill & Melinda Gates Foundation, which funds much of the research in his own area. He suggested that adverse event reporting systems, even for social and behavioral studies, should be designed to address issues arising in international studies and should be multinational. He also said that enhanced and simplified consent procedures would be useful and important, particularly for written consent. In many countries, he said, “a waiver of signed informed consent is not only useful but essential.” In some parts of the world signing documents carries great significance, he explained, but there are many cases where subjects are not capable of signing their own names even if they wish to.
More thought needs to be given to the issue of minimal risk when a study spans multiple countries, he said, because the definition of minimal risk may vary from country to country. Reporting that one is homosexual is much more dangerous in a country where homosexuality is illegal than it is in the United States, for example. More thought also needs to be given to the issue of IRBs for multicountry studies, he added, to answer such questions as whether it would make sense to have a central IRB based in the United States or to have local IRBs involved. It is probably important that any study done in a foreign country have the approval of local regulatory bodies, he said, but that raises the question of how local IRBs can be held to standards of efficiency and timeliness of approval. Furthermore, it is not clear whether the U.S. IRB or the local one would make decisions about such issues as minimal risk and expedited status.
One other issue is how—or whether—U.S. guidance should be harmonized with international guidance and regulations. The U.S. Department of Health and Human Services could work with the World Health Organization or other agencies, Coates noted, but there are questions about how and to what extent the U.S. guidelines could be harmonized with local guidelines. One thing that should be carefully avoided, Coates said, is having studies funded within the United States that take advantage of lax regulations in other countries to carry out research that would not be acceptable in this country.
Stark, L. (2012). Behind closed doors: IRBs and the making of ethical research. Chicago, IL: University of Chicago Press.