2

Models of Accreditation

The committee was presented with the task of making recommendations about accreditation standards and does so in more detail in the next chapter. With the basic terminology for the committee's view of a human research participant protection program (HRPPP) defined in Chapter 1, this chapter lays the groundwork for the elements of an accreditation process. It starts by considering the various models available for accreditation systems and asks, “What is the role of accreditation in a human research protection system?” The present committee will spend another year thinking about the design and implementation of an improved system of human research protection, so it has not had the opportunity to consider the value of accreditation compared with other strategies to ensure the ethical conduct of research. However, even if one begins with the current system rather than a reconstructed one, accreditation should not be evaluated in a vacuum—it is still necessary to have a theory of accreditation and a process for carrying it out. Specifically, the value that accreditation adds to the system that already exists must be considered.

MODELS OF ACCREDITATION

Accreditation efforts in the United States have historically followed one of two models, although a third model can also be observed. The first of these is accreditation as a supplement to government regulation. Under this model, entities that are otherwise already regulated by the government seek accreditation as a mark of excellence, as it is above and beyond government regulation. Ac-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs 2 Models of Accreditation The committee was presented with the task of making recommendations about accreditation standards and does so in more detail in the next chapter. With the basic terminology for the committee's view of a human research participant protection program (HRPPP) defined in Chapter 1, this chapter lays the groundwork for the elements of an accreditation process. It starts by considering the various models available for accreditation systems and asks, “What is the role of accreditation in a human research protection system?” The present committee will spend another year thinking about the design and implementation of an improved system of human research protection, so it has not had the opportunity to consider the value of accreditation compared with other strategies to ensure the ethical conduct of research. However, even if one begins with the current system rather than a reconstructed one, accreditation should not be evaluated in a vacuum—it is still necessary to have a theory of accreditation and a process for carrying it out. Specifically, the value that accreditation adds to the system that already exists must be considered. MODELS OF ACCREDITATION Accreditation efforts in the United States have historically followed one of two models, although a third model can also be observed. The first of these is accreditation as a supplement to government regulation. Under this model, entities that are otherwise already regulated by the government seek accreditation as a mark of excellence, as it is above and beyond government regulation. Ac-

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs creditation, however, has become a mark of excellence achieved by only a fraction of regulated entities. The National Committee for Quality Assurance (NCQA) program for the accreditation of managed care organizations illustrates this model (NCQA, 2001a). Managed care organizations are regulated by state insurance departments, state health departments, or the U.S. Department of Health and Human Services (DHHS) (if they are Medicare or Medicaid managed care organizations). They also seek accreditation, however, to demonstrate their commitment to excellence, as many employers and other purchasers of managed care organization services look to accreditation as an indicator of performance above the required minimum. In a second model, accreditation substitutes private regulation for public regulation. One version of this is seen in accreditation of institutions of higher education, for which formal government regulation is (for various reasons not explored here) largely absent. Accreditation serves effectively as the only oversight system. Another variant of nongovernmental voluntary accreditation is seen under Medicare's “deemed-status” program, in which the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) hospital accreditation program serves as an alternative to state certification, which uses Medicare 's own federal regulatory standards as a basis for hospital participation in Medicare (Jost, 1994). JCAHO's accreditation standards are quite different from Medicare's own standards, but JCAHO accreditation is accepted in the place of Medicare certification. That is, a hospital or health care facility is deemed to meet federal standards by dint of being accredited by JCAHO and is thereby authorized to participate in (and be paid through) Medicare. There are significant benefits to the use of accreditation as an alternative to regulation and to the deemed-status model in particular. Accreditation reduces the cost of oversight to government, as it is effectively paid for by user fees rather than taxes. Accreditation programs, especially nongovernmental programs, tend to be much more flexible and responsive to change than regulatory programs because they are not bound by the rigidities of administrative rulemaking procedures and are more responsive to regulated constituencies. Accreditation, however, also has its costs. It is not directly accountable to the public, and there is a constant concern that the “fox is guarding the henhouse” (DHHS OIG, 1999a,b). Even JCAHO is not given unfettered authority to regulate hospitals for Medicare. The Health Care Financing Administration (HCFA), which administers Medicare, retains authority to directly assess (or “look behind”) the accreditation of hospitals. HCFA conducts its own surveys for cause, surveying a small fraction of validation surveys each year, and reviews JCAHO's “deeming” authority at least once every 6 years (Lewin Group, 1998). Furthermore, if accreditation is to be more than a pro forma exercise, it can be resource-intensive. This can be corroborated by any health care facility or educational administrator who has recently undergone accreditation.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs In a third, less common, model, the accreditation program does not create its own standards but, rather, ensures compliance with standards on the basis of interpretation of regulatory standards determined by the government or another entity. The program might also offer guidance about regulatory compliance. This is the accreditation model used by the Association for Assessment and Accreditation of Laboratory Animal Care (AAALAC),1 which does not create its own standards but which is a private voluntary accreditation system that operates in compliance with regulations from the U.S. Department of Agriculture, funding agencies, and the Animal Welfare Act, a federal statute. AAALAC standards are supplemented by the Guide for the Care and Use of Laboratory Animals, produced by the National Research Council (NRC, 1996). This volume lays out best practices and benchmarks based on science and knowledge developed from past accreditation efforts. AAALAC dates back to 1965. Until recently, the National Institutes of Health (NIH) office that had oversight over protection of humans involved in research also had responsibility for compliance with animal care regulations, so this model is familiar to both the federal officials and research centers. This is the model explicitly cited by the Association for the Accreditation of Human Research Protection Programs (AAHRPP) (see below). The analogy is not direct in one area, however, in that in research involving humans, participants can have a direct voice and those with direct experience as participants or those familiar with the concerns of human participants in research can be directly engaged in oversight of the research. The draft standards that the committee has seen to date do not fully take advantage of this possibility (see discussions in Chapter 1 and Chapter 3). On the basis of the standards shared with the committee, it appears that the framework proposed by NCQA under its contract with the U.S. Department of Veterans Affairs (VA), at least initially, is to use accreditation as a tool to implement existing regulations better, adopting this aspect of the AAALAC model (in effect, using current regulations as standards and using accreditation to bring VA facilities into compliance with them). The committee believes that this is a good way in which to get an accreditation program under way. It might also serve to supplement a regulatory program that is overburdened. Its main value is to move those being accredited into compliance with existing regulations. This strategy will improve research oversight only if noncompliance is one of the system's major problems. The same model could, however, also be used to augment regulatory standards if some accreditation standards exceed the regulatory minimum. The NCQA linkage to quality improvement programs is a step along this path (see Chapter 3). 1   For more information, see http://www.aaalac.org/.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs ELEMENTS OF AN ACCREDITATION PROCESS At the beginning of the 20th century, a private voluntary accreditation system lifted American medical education out of mediocrity. In the early 1900s, the quality and content of medical education were wildly variable. Harvard University, the University of Pennsylvania, and The Johns Hopkins University had instituted formal curricula and linked medicine to science, but “the ports of entry into medicine were wide open, and the unwelcome passed through in great numbers “ (Starr, 1982, p. 116). The American Medical Association (AMA) appointed individuals from esteemed medical schools to the Council on Medical Education, and these individuals began to grade medical schools. A 1910 report by Abraham Flexner went a step further, arguing that the strategy for improving the system of medical education was by elevating schools to the Hopkins standard, and “the AMA Council effectively became a national accreditation agency for medical schools, as an increasing number of states adopted its judgments of unacceptable institutions” (Starr, 1982, p. 121). Since then accreditation programs have been used to enhance quality in many different contexts. The improvement of care for laboratory animals involved in research has been widely attributed to the joint action of federal law, particularly the Animal Welfare Act of 1966, and the private accreditation system through AAALAC. Private accreditation has become pervasive in higher education and professional schools, hospitals and health care facilities, and managed care organizations. More recently, a long-standing and rigid regulatory framework for opioid treatment programs has begun to shift to a more flexible, clinically oriented accreditation process, even though it is still formally under federal regulation. An accreditation process is now proposed for HRPPPs. The models described above have in common several elements that are expected to be part of emerging programs for accreditation of HRPPPs: a national organization that can mediate the accreditation process; an application process and set of threshold criteria by which organizations are eligible to apply for accreditation; a process of self-evaluation; an external evaluation process, including site visits by external accreditors; an appeals process for accreditation determinations; a repeat cycle of self-evaluation and external evaluation; and a set of standards by which HRPPPs can be measured. The central focus of this report and the following chapter is accreditation standards, the benchmarks by which accreditation programs measure achievement. Standards are only part of a process, however. This chapter describes the accreditation process for which standards are a tool.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Accreditation Bodies The committee believes that the ideal accreditation body is a national independent organization that is credible among the stakeholders to be accredited but that is independent of any particular interest group among them. Independence, credibility, and intimate familiarity with stakeholders' needs are desirable attributes of any accrediting body (Hamm, 1997), and particularly so for human participant protections. As described below, both NCQA and the emergent AAHRPP appear to meet these criteria. PRIM&R and the Formation of AAHRPP Public Responsibility in Medicine and Research (PRIM&R) is a Boston-based private nonprofit organization best known for its activities in educating institutional review board (IRB) members and staff.2 It was founded in 1974, the same year in which the first bioethics commission, the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (the National Commission), began its work. The framework for IRBs was not fully in place, but IRBs were already operating at NIH and in many academic health centers. In 1999, PRIM&R formed a working group to develop accreditation standards. This grew out of discussions about the development of an accreditation process for HRPPPs (see Chapter 1), the organizational units responsible for carrying out the twin functions described by the National Bioethics Advisory Commission (NBAC) of ensuring informed consent and independently assessing risks and benefits. Under a subcontract executed for the purposes of the present committee's work, a preliminary draft of the PRIM&R standards was given to the Institute of Medicine (IOM) in December 2000 and became the focus of a January 2001 IOM public forum on the topic of accreditation standards. PRIM&R revised its draft standards after the public forum, and they appear in Appendix B. PRIM&R's proposed standards were a major input into the committee's deliberations and are discussed in greater detail in Chapter 3. The concept of AAHRPP was originally conceived by PRIM&R and was intended to provide the organizational locus for carrying out an accreditation process by using the PRIM&R standards. AAHRPP is designed to bring together diverse stakeholder organizations with the intent of implementing a voluntary accreditation process. AAHRPP was originally incorporated in Massachusetts in March 2000, but it is expected to be incorporated in Maryland in spring 2001 as a private nonprofit corporation to “provide a process of voluntary peer review and education among organizations concerned with research involving human subjects, in order to promote preservation of rights and welfare of subjects in research and 2   For more information, see http://www.primr.org.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs compliance with relevant regulatory and ethical standards” (PRIM&R, 2001b). As this report went to press, AAHRPP was supported by a consortium of interested groups, including PRIM&R, the Association of American Medical Colleges (AAMC), the Association of American Universities, the Federation of American Societies for Experimental Biology, the National Health Council, the National Association of State Universities and Land Grant Colleges, and the Consortium of Social Science Organizations (Accrediting Body for Human Subjects Research Nears Reality, 2001). The VA and NCQA Accreditation Process In March 1999, clinical research at the West Los Angeles VA Medical Facility was suspended because of noncompliance with the Common Rule (see Chapter 1). In the ensuing months four additional VA medical centers were affected by Food and Drug Administration (FDA) or Office for Protection from Research Risks (OPRR) sanctions. This shone a spotlight on Veterans Affairs, just as OPRR and FDA shutdowns had done at other academic health centers. In April 1999, the VA announced the formation of a national office, the Office of Research Compliance and Assurance. In June 1999, the General Accounting Office commenced a study of human subject protections at VA medical centers and made eight site visits (GAO, 2000). That report identified three specific weaknesses: “(1) VA headquarters has not provided medical center research staff with adequate guidance about human subject protections; (2) insufficient monitoring and oversight of local human subject protections; and (3) insufficient funds allocated for IRB operations and human subject protection oversight” (p. 5). To address these deficiencies, the VA awarded a $5.8 million, 5-year contract to establish a national accreditation system for VA medical centers engaged in research (VA, 2000). The contract was awarded to NCQA, which then began to devise and carry out an accreditation and oversight process (NCQA, 2001b). NCQA has joined with Medical Care Management Corporation (MCMC) to design the program and to recruit, credential, and schedule surveyors. NCQA and MCMC together will provide a routine external evaluation of compliance with policies. In addition, NCQA plans to convene two advisory groups and one decision-making group to help develop and implement standards and survey methods for the program. NCQA presented the rationale behind its approach at IOM's January 2001 public forum and later provided the committee with a set of its draft standards (see Chapter 3 and Appendix C). Private consulting and management firms such as Deloitte & Touche and PricewaterhouseCoopers have been hiring staff with HRPPP expertise and may assist with preparations for accreditation efforts. Other organizations may yet step forward to offer accreditation for HRPPPs.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Eligibility Criteria and an Application Process The accreditation body must specify who can be accredited, set fees to cover its costs, and establish an application process. The NCQA accreditation of VA facilities will be done, at least initially, by self-selection. Because the VA hospital system is relatively closed, the applicant pool is clear. The eligibility criteria for HRPPPs beyond VA, including the nascent AAHRPP, have not been specified in detail. It is clear that academic or independent research centers that have an operating IRB would be eligible. The stated intention is to also invite applications from private independent IRBs. It is not clear whether larger consortia of institutions that are organized as a collaborative unit would be eligible, such as cooperative clinical trials groups,3 the Multi-Center Academic Clinical Research Organization,4 independent contract research organizations, or site management organizations. Self-Evaluation Applying for accreditation requires considerable preparation. This typically involves the organization that is seeking accreditation to gather information relevant to the standards that will be used and to analyze how well prepared it is to address questions and concerns that may arise. This preparation can consume enormous efforts of a few staff members and draws on the resources of many parts of the organization. The mere process of self-study can reveal previously unknown weaknesses or sometimes strengths and can suggest administrative remedies. It can also draw the attention of senior administrators to the need for more resources, new programs, or management changes and can reveal the strengths and weaknesses of key personnel. Many organizations involved in accreditation processes regard the self-study as the most valuable element of the accreditation process precisely because it focuses the attention of senior administrators. The process of self-evaluation of HRPPPs appears to be especially promising as a way to improve the system. Self-assessment combined with systematic, continual use of quality improvement programs could, for example, identify features common to many “excellent” HRPPPs, and those features could, over 3   The Office for Human Research Protections maintains a list of cooperative protocol research programs that might be accredited, but for which a somewhat different process and set of standards would be required. (For more information see http://ohrp.osophs.dhhs.gov/humansubjects/assurance/cprp.htm). 4   Multi-Center Academic Clinical Research Organization, or MACRO, brings together five major academic health centers—Baylor College of Medicine, Harvard Clinical Research Institute, the University of Pennsylvania School of Medicine, Vanderbilt University, and Washington University School of Medicine—under a collaborative agreement that includes an agreed upon system for protocol review by IRBs among the institutions (for more information, see http://www.mc.vanderbilt.edu/ctc/macro.html).

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs time, supplant the existing regulatory standards, with their focus on IRB structure and documentation procedures. A shift from documentation-based standards to performance-based standards could not take place quickly, but it may well become possible over time (see Recommendations 6 and 7). External Evaluation The accreditation programs of NCQA and AAHRPP both intend to visit every organization seeking accreditation, at least initially. The accreditors visiting sites would review the self-evaluation; view documentation; and carry out interviews of IRB staff and members, administrators, investigators, and (if the recommendations of this IOM committee are adopted) participants. The site visit is intended to give accreditors a hands-on feel for the organization and to raise questions when they can be answered directly and immediately. The accreditors would then prepare a formal written report and make their decision to accredit the applicant, give it a probationary status, or reject the application. Launching the accreditation process is likely to encounter some capacity limits for external evaluation. The committee concurs that site visits will be necessary initially, which will limit the number of institutions that can be accredited. At the committee's December 18 open meeting, David Korn of AAMC, which is involved with helping to establish AAHRPP, estimated that AAHRPP might eventually be able to accredit as many as 650 to 700 HRPPPs, but it would take a number of years to reach this level. This is one reason that the committee believes that the accreditation process should be regarded as a pilot study rather than a fait accompli (see Recommendation 1 below). PRIM&R does have a core set of trained IRB professionals to draw upon for the initial AAHRPP site visits. This pool is limited, however, and it would be unrealistic to expect a new accreditation organization to manage more than one or two site visits per week, on average, during its first year. The minimum of potential applicants can be estimated by the 165 institutions that registered their IRBs with the Office for Human Research Protections (OHRP) as of February 5, 2001.5 It appears likely, therefore, that it would take 2 to 3 years to accredit just those institutions that registered their IRBs in the first 2 months in which they were able to do so. It would take even longer to accredit the 491 institutions surveyed in 1995 in the most recent and extensive survey of IRB operations (Bell et al., 1998). 5   The registration process began in December 2000. Most institutions have more than one IRB, so the number of IRBs registered is much larger than the number of potential applicant institutions.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Appeals Process Institutions that fail to get accredited or that are given probationary status will need a credible appeals process. This may be through the accreditation body itself or may require some involvement of FDA or OHRP.6 The NCQA standards include a standard for an appeals process by the applicant institution; the PRIM&R standards do not. Repeat Accreditation Accreditation is not permanent. The models of accreditation reviewed by the Lewin Group in 1998 involved accreditation terms of 3 to 5 years. The NCQA program plans a 3-year accreditation cycle. The AAHRPP accreditation term has not yet been firmly specified, but it is expected to be 3 to 5 years. The process for reapplication might or might not differ from that for initial accreditation. It is likely that accredited organizations with few untoward events would face a more abbreviated process, but this is likely to be decided in light of experience. APPLYING THE MODELS TO HUMAN RESEARCH OVERSIGHT Recommendation 1: Pursue Accreditation Through Pilot Testing as One Approach. Accreditation of HRPPPs should be pursued as one promising approach to improving the human participant protection system. The first step is implementation of pilot programs to test standards, establish accreditation processes, and build confidence in accreditation organizations. This effort should be evaluated for its impact on protecting the rights and interests of participants in 3 to 5 years. Accreditation as a mark of excellence—of achievement well beyond regulatory compliance—might offer an HRPPP a competitive advantage over nonaccredited competitors in seeking support from sponsors or access to participants, researchers, or students. That is, NIH or other funding review committees might look more favorably on research proposals from accredited institutions than on those from nonaccredited ones, those recruiting participants might advertise accreditation as a hallmark of quality and safety, or private drug and device firms might preferentially site clinical trials that they sponsor at accredited research institutions (or have them reviewed by accredited IRBs). 6   For example, mammography accreditation entails a two-layer appeals process, first to the private body, but if it is denied, then the private body's decision can be appealed directly to FDA (Lewin Group, 1998).

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Accreditation might also serve as an important educational tool. The process of preparing for accreditation would force institutions to attend to their HRPPPs, and that attention would necessarily entail education about the importance of protection of human participants in research. Accreditation could raise the median performance (average middle performance) of HRPPPs. It might offer HRPPPs located within research institutions, both public and private, a potent argument when asking their administrative supervisors for additional resources. (This is a major role played by accreditation of academic units within a university and is used as a tool to effect changes in, for example, library services, curricula, and services.) Accreditation could not serve these ends, however, until it became widely accepted as a mark of excellence. Any accreditation program seeking to establish its value on the basis of these terms would first need to achieve broad recognition as a credible program. All previous accreditation programs faced a similar dilemma when they were initiated, and some have succeeded in attaining credibility, but others have not.7 Accreditation that would supplant regulation (the deemed-starus model) could have several attractive features. Both OHRP and FDA have signaled that they might consider accreditation by a nongovernmental accreditation organization presumptive evidence of compliance with regulations. In the case of research institutions under OHRP oversight, accreditation could serve as a partial substitute for the assurance and compliance functions, reducing FDA and OHRP scrutiny of accredited organizations (allowing them to concentrate their scrutiny on nonaccredited organizations). FDA and OHRP would necessarily retain independent oversight authority (e.g., inspections “for cause”) and independent investigation and enforcement capacity if violations are alleged or documented and would periodically need to ”accredit the accreditors,“ as in other deemedstarus accreditation models. However, before the usefulness of this approach can be assessed in the case of HRPPP accreditation, an accreditation program(s) will need to be much further along in its development. The regulatory enforcement model is also worth considering, particularly as a starting point. It might be wise to start, as NCQA apparently proposes to do under its contract with the VA, with a focus on innovative or more effective means of evaluating regulatory compliance before moving on to a program that raises standards above the regulatory minimum. This approach could, however, have the effect of inundating HRPPPs with further paperwork if additional requirements are imposed on current ones. If the goal is to shift from a focus on such paper compliance to a focus on more meaningful performance measures, 7   Some accreditation programs fail to take root and flourish. An AMA physician certification program was announced with great fanfare in late 1996, but AMA discontinued the program in April 2000 because it had not been widely adopted. JCAHO implemented an accreditation program for managed care in 1987 but stopped in 1990, until a new managed care accreditation program was put in place in 1995 (BNA, 1996, 2000; Dimmitt, 1995).

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs then a strategy that assumes that current oversight is the baseline will not accomplish it and any additional measures will add to the regulatory burden. Again, it is important to look carefully at what value accreditation adds to the regulatory program that already exists and whether this added value justifies the added costs (financial and personnel) of such a program. Testimony that the committee heard from representatives of the FDA and OHRP left it uncertain about whether the draft accreditation standards are seen as supplementing a regulatory program that will continue largely as is or as providing an alternative means of oversight, with federal agencies “deeming” accredited HRPPPs to be in compliance and thus reducing federal inspections and audits of accredited institutions. A voluntary national accreditation system, however, could decrease the burden currently experienced by regulators, allowing them to refocus their efforts where they are most needed, and it could also increase flexibility for entities attempting regulatory compliance. An independent accreditation organization(s) could more readily modify and improve its standards than federal agencies carrying out mandatory programs. Federal agencies attempting to modify their regulatory approach are less flexible because they must follow formal rule-making procedures to do so. It took a decade to reach agreement on the federal Common Rule, and at least three agencies that conduct research with human participants did not adopt the rule,8 leaving all agencies loathe to reopen the process used to modify the regulations. The current need for multiagency concurrence is a tremendous barrier, and so short-term improvements are more likely to come from other approaches, such as nongovernmental accreditation, that do not require major regulatory overhaul.9 The most compelling argument in favor of an independent accreditation system, however, is that, if it is done right, it could move the focus of oversight from simple administrative documentation to focusing on processes and outcomes that more directly threaten the rights and interests of participants. The need to shift from paper compliance to measures that more meaningfully prevent unnecessary risks, promote sound scientific design, and ensure autonomous choice has been a consensus direction for improvement since regulations were first implemented. The call for better measures was articulated by the President's Commission for the Study of Ethical Problems in Medicine and Biomedical and 8   OPRR noted three agencies that appeared to sponsor research with human participants but that were not signatories to the Common Rule: the National Endowment for the Humanities, the U.S. Department of Labor, and the Nuclear Regulatory Commission, as cited in a report forthcoming from NBAC (NBAC, forthcoming-b). 9   The rigidity of the current regulatory framework, entailing the consensus of 18 agencies, is one major argument that NBAC offers to support its recommendation for new legislation to create a single federal agency with oversight authority for protection of human participants in research. This topic is beyond the scope of this committee's first report but will likely be taken up in its subsequent report.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Behavioral Research (the President's Commission) in reviewing regulations created in the wake of the National Commission and echoed in reports of the Advisory Committee on Human Radiation Experiments (ACHRE) in 1995 and the Office of the Inspector General (OIG) of DHHS in 1998 and 2000 (ACHRE, 1995; DHHS OIG, 1998a,b,c,d, 2000a,b,c; President's Commission, 1981, 1983). Recommendation 2: Establish a Nongovernmental Accreditation Organization(s). Organizations formulating accreditation standards and carrying out the accreditation process should be independent, nongovernmental organizations. These organizations should include within their programmatic leaderships the perspective of the relevant stakeholders in the applicant HRPPP community (i.e., institutions, investigators, sponsors, and participants). As discussed above, one of the chief virtues of a nongovernmental accreditation system is that it can evolve over time without requiring new federal regulations at each step. The regulations are demonstrably unresponsive to dramatic changes in how research is conducted; a nongovernmental accreditation system may be more responsive by comparison and would comport with Circular A-119 of the Office of Management and Budget, which urges the use of nongovernmental “voluntary consensus standards” where possible (OMB, 1998).10 The committee envisions an accreditation process that will continually evolve to update standards over time and to incorporate the variety of organizational structures through which human research programs are reviewed and carried out. The operations of organizations seeking accreditation will also evolve. The parallel evolution of accreditation standards and HRPPP operations should be an iterative process, with the formulation of standards efficiently informed by knowledge acquired in the accreditation process. The formulation of standards, the conduct of accreditation site visits, and external evaluation must therefore be intimately linked and appropriately responsive to feedback. Organizations formulating standards and conducting the accreditation process should be national in scope; be familiar with the operations of institutions that apply for accreditation; and incorporate the perspectives of research participants within their programmatic leaderships. 10   Circular A-119 was intended mainly for technical standards pertaining to products, but it also contemplates “related management systems practices” (see http://www.whitehouse.gov/omb/circulars/a119/a119.html).

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs An accreditation process should directly involve the kinds of institutions and research expertise being accredited, but an accreditation organization should not be beholden to any particular stakeholder or interest group. Accreditation bodies for HRPPPs will require input from academic health centers, organizations representing research sponsors, nongovernmental research organizations, private firms developing products and services tested in studies with humans, participants, IRB members and staff from both academic and nonacademic institutions, research administrators in both academic and nonacademic institutions, and individuals from a range of research fields appropriate to the intended range of applicant institutions. SOME ISSUES THAT ACCREDITATION ALONE CANNOT ADDRESS Some elements important to the protection of the rights and interests of those participating in research are not directly addressed in proposed programs for HRPPP accreditation. In most cases, an accreditation process could be used as an indirect means to improvement; however, further actions would be needed in parallel with the establishment of an accreditation process. The committee expects to come back to many of these topics in its second report and has discussed how to integrate some elements not currently emphasized into the accreditation process. The discussion below includes some suggestions to that effect. Accreditation is not a short-term fix. It must be viewed as one element of a long-term strategy. The VA-NCQA accreditation program will operate in a relatively circumscribed system, but it will take several years to implement the system and several more to evaluate it. The national voluntary system being developed under AAHRPP may take even longer to establish. Before a program could be granted deemed status it would need to be given time to develop and mature. Turning over regulatory authority to an untested program would be very risky, reinforcing the need for pilot testing as a first step. Identifying, Investigating, and Sanctioning Violations Accreditation cannot totally replace federal regulation. Accreditation is rarely effective in dealing with bad actors—those who intentionally flout or ignore requirements. Monitoring, investigation, and enforcement are necessary to augment an accreditation system, and under the current regulatory framework these will remain functions of OHRP and the FDA.11 The main cause of error in many prominent controversies in research ethics lies with investigators who diverge from an agreed-upon protocol. Review of protocols cannot fix the prob- 11   One recommendation of NBAC is to consolidate these functions into a single agency, as noted above.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs lem when investigators deviate from the protocols, although a more robust research monitoring capacity could reduce such deviations. Some of the most conspicuous cases in the past two decades—Martin Cline's 1980 gene transfer experiments in Israel and Italy (Thompson, 1994) and the death of Jesse Gelsinger in gene transfer experiments at the University of Pennsylvania in 1999, for example12—appear to be attributable to the conduct of principal investigators and their collaborators or to institutional decisions unrelated to the IRB, so it is not clear how accreditation of an HRPPP could prevent such cases. An accreditation body should not be expected to be the original source responsible for uncovering violations or the main body responsible for investigating or sanctioning them. Accreditation could, over time, reduce the likelihood that violations would occur as a result of changes in norms and behaviors. Accreditation could, moreover, be withdrawn or made probationary on the basis of the disclosure of infractions at an accredited institution. Reports of infractions would surely increase scrutiny by an accreditation body. An accreditation organization could also be used as part of the strategy to bring an institution back into compliance with federal regulations after infractions were detected and investigated. Therefore, accreditation is relevant to the problem of bad actors, but 12   In 1980, Martin Cline administered recombinant DNA with the hope of effecting gene transfer in two patients with thalassemia, one in Israel and one in Italy. His IRB had not approved his protocol and, indeed, rejected it just days after Cline conducted the experiments. The IRB had reviewed the protocol several times and had enlisted external expert reviewers who uniformly judged the experiment premature. Cline also deliberately misled a review panel in Israel and his collaborator in Italy, who identified the patient who was treated. The experiments had no known adverse health consequences for the patients, and after an NIH investigation, Cline had several grants terminated and was barred from seeking NIH funds for 4 years; he also resigned from his division chairmanship at the University of California at Los Angeles. IRB action in connection with this protocol was not at fault in the infractions. This case was reviewed in Larry Thompson's Correcting the Code (Thompson, 1994) and in a background paper for NBAC (Cook-Deegan, 1997). IRB action was similarly a relatively minor concern in the 1999 death of Jesse Gelsinger. The lawsuit brought by his family focuses on the actions of the principal investigator and two research institutions: James Wilson, a private company (Genovo), and the Institute for Human Gene Therapy at the University of Pennsylvania. Arthur Caplan, a bioethicist who gave advice about the trial design, was initially also named in the suit, but he was not on the IRB. Actions named in the suit, which was settled out of court on terms that have not been publicly disclosed, focus mainly on deviations from the protocol approved by the IRB and not on IRB actions. The only mention of the IRB is that it approved the protocol (for more information, see http://www.sskrplaw.com/links/healthcare2.html). The broader definition of an HRPPP could reduce the likelihood of similar events, particularly if the committee's recommendations about incorporating research monitoring were adopted.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs it is not the most direct solution to the problem and cannot replace investigation and enforcement activities. Educating Investigators In 1995 ACHRE completed a report that built on a thorough historical and ethical analysis. It concluded: It is not clear to the Advisory Committee that scientists whose research involves human subjects are any more familiar with The Belmont Report today than their colleagues were with the Nuremberg Code forty years ago. . . . No one in the scientific community should be able to say “I didn't know” or “nobody told me” about the substance and importance of research ethics (ACHRE, 1995, pp. 817–818). Many, perhaps most, of the serious problems that arise in human research arise from the actions of investigators, so policies that deal directly with investigators are at least as important as improving the review of research protocols in an HRPPP. The policies that most directly affect investigators include the following: educating them about their roles and responsibilities in the ethical conduct of research, increasing the capacity to monitor ongoing research approved by an IRB, the investigation of infractions, and the enforcement of regulations. Among these, education seems to be the one most likely to have the desired results with the least level of intrusion and the greatest direct impact on overall norms. In a background paper written for NBAC, Charles McCarthy, drawing on two decades of direct experience with federal oversight of protection of human participants in research, argued that the measure most important to improving the ethical conduct of research is education—of investigators, IRB members, IRB staff, and those working at research institutions (McCarthy, forthcoming). The devotion of resources to education led to fewer problems down the road. Incidents requiring investigation and the need for intervention increased when budgets for education decreased, and increased attention to education seemed to reduce the numbers of untoward incidents. McCarthy's observation is corroborated by the observations of ACHRE (Mastroianni and Kahn, 1998). Henry Beecher, in a seminal 1966 New England Journal of Medicine article, argued against establishing an oversight bureaucracy for medical research, asserting that the key was instead to elevate norms of research ethics among investigators (Beecher, 1966). The present committee concurs with that position. Although accreditation can reinforce education programs at accredited institutions, education on the ethical conduct of research and the ethical responsibilities of investigators are matters of central importance regardless of accreditation and will be taken up in greater depth as the committee continues its work.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Improving Research Monitoring Research monitoring has emerged as a major problem, but policies have mainly focused on administrative compliance with federal regulations that emphasize informed consent and prospective review of written protocols. One reason is that the level of administrative compliance is much easier to measure and infractions are thus easier to document. For example, every research protocol must be reviewed, and informed-consent forms and minutes of IRB meetings can reflect specific actions. This creates a trail of documentation that can be audited (or can suggest a remedy when a trail of documentation is not maintained). Research monitoring, in contrast, is mainly concerned with the prevention of rare bad events. Research monitoring may be the more important function of the system, but effective monitoring is much harder to measure. The current HRPPP system attends to the functional equivalent of maintenance records by documenting informed-consent forms and IRB deliberations, but it appears to be less adept at identifying and investigating serious breaches or systematically detecting danger signals in ongoing research. In most cases, the trigger for an investigation has come from participants who make complaints, research staff who act as whistleblowers, or public media exposure and investigative journalism.13 If the oversight processes are working well, serious violations will be rare. Learning from such rare violations, however, is essential to improving the system, and the current system appears to be deficient in this function. The elements of the protection regime most amenable to accreditation, moreover, may not be the ones most likely to first identify serious infractions or problems. The oversight system could, however, become much more systematic about detecting problems by creating feedback mechanisms by which research participants and staff can report problems (and can link those reports to IRBs), by ensuring that means for the identification and reporting of serious and unexpected adverse events are built into the research process, and by strengthening linkages between programs for HRPPP review and programs for investigation of the serious problems that do arise. The relative roles of institutions conducting research, research sponsors, accreditation bodies, and OHRP and the FDA in investigating violations are not clearly spelled out. Historical cases suggest that research institutions are sometimes delegated primary responsibility for investigation (for example, the University of California at Los Angeles for the Cline case), and at times federal regulatory agencies take the lead (for example, the FDA for the Gelsinger case). 13   The Tuskegee trial, Martin Cline's premature gene therapy experiments, and human radiation experiments were all first reported in the public media, with investigations occurring after public furor. The FDA had begun to investigate the death of Jesse Gelsinger when the case became public, but many of the details about financial conflict of interest and serious underreporting of adverse events became known mainly via investigative journalism. Investigations then followed.

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs The emergence of accreditation bodies will introduce new organizations with important roles to play in learning from lapses in the system to ensure continuous improvement, making it all the more important to spell out the roles and responsibilities of different parties when serious infractions come to light. Large multicenter clinical trials now routinely include formal data safety and monitoring boards (DSMBs). DSMBs were initially established to assist research sponsors with analysis of their data, but their importance in assessing risk and monitoring safety has become apparent. Such boards are typically composed of researchers with expertise similar to that of the principal investigators, but they come from independent research institutions and are augmented by statisticians, bioethicists, and sometimes lawyers and consumers. The only personnel requirements for NIH DSMBs are that they include expert clinicians and experts in biometrics or statistics. These monitoring boards receive reports of study outcomes, including both intended effects and adverse events. They pool findings from multiple centers (findings which the individual centers often do not receive and to which only research sponsors would otherwise have access). DSMBs may stop a trial if it appears to be causing harm or if its study objective is met early. A DSMB can also become the locus for receiving reports of mishaps and complaints, as well as adverse events and research outcomes. NIH has recently mandated that any NIH-sponsored clinical trial have a research monitoring plan and that the plan take into account the level of risk (NIH, 2000). The National Cancer Institute has mandated that any phase III trial (a large trial, typically conducted at many centers, intended to demonstrate the efficacy of an intervention) have a DSMB (NCI, 1999). The inclusion of such boards has been standard practice in most trials sponsored by private industry to test new drugs, devices, or biologics. The Good Clinical Practice portion of the International Conference on Harmonisation guidelines that govern clinical trials has an entire section (section 5.18) devoted to monitoring (International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 1996, pp. 26–29). The connections between DSMBs and IRBs are not completely consistent, however. Although all DSMBs are accountable to research sponsors for the integrity of the data, their role in ensuring safety and in protecting research participants is less well articulated. They are not always clearly accountable to IRBs, and their responsibilities to research participants or groups representing the interests of research participants are sometimes not explicit. WILL ACCREDITATION ENHANCE PERFORMANCE? The interaction between accreditation bodies and the organizations that they accredit can indicate new strategies for improving performance. Over the past three decades the constant lament of dozens of reports from a half dozen knowl-

OCR for page 45
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs edgeable commissions has been that the current HRPPP system emphasizes administrative compliance when it would do better to focus on the rights and interests of research participants, the risks that they face, and whether their choices are fully autonomous. Yet, the federal regulations governing the protection of human research subjects have been largely the same for 25 years, and it took a decade to get agreement on the federal Common Rule among 18 agencies. The arduousness of that task has itself become an argument for leaving the regulations intact, but that is a recipe for stagnation in a research enterprise that is rapidly growing and changing. Even an experiment to have a “central IRB” at the National Cancer Institute took 2 years to launch. The federal regulatory system is indeed rigid and focused on documentation rather than performance (see discussion under Applying the Models to Human Research Oversight). An accreditation process should “emphasize outcomes or performance rather than structure, process, and procedures,” and “successful accreditation bodies are flexible, future-oriented, and constantly looking at changes taking place in their fields to make sure the standards and review process are relevant to the needs of the accredited entities” (Hamm, 1997, pp. 72–73). For the first time in decades, the HRPPP system is in flux with the elevation of OHRP out of NIH and a recent shift to an IRB registration process linked to a streamlined assurance process by OHRP (OHRP, 2000b) along the lines of a recommendation by C. K. Gunsalus in a report to NBAC (Gunsalus, forthcoming). These changes were possible without a revamping of federal regulations, but flexibility beyond this will be more difficult to achieve. If a nongovernmental accreditation system could fulfill the promise of flexibility, provide an orientation toward performance, and provide adaptability, it could measurably improve the HRPPP system over time. In the immediate future, the emphasis on HRPPP accreditation, based on the draft standards and procedures proposed, appears to be bringing existing HRPPPs into compliance with existing federal regulations. The aspiration, however, is higher, and that may be possible, but the problem is difficult. In congressional testimony in 1994, Robyn Nishimi of the Office of Technology Assessment observed: The current system, while changing incrementally, has fallen short of implementing, or did not implement at all, recommendations made between 1973 and 1982 by an ad hoc committee of DHEW, a congressional report and two congressionally mandated commissions (Nishimi, 1994, p. 149). Since Nishimi made that statement, the nation has had reports from ACHRE (ACHRE, 1995), the General Accounting Office (GAO, 1996), and DHHS OIG (DHHS OIG, 1998a,b,c,d,e, 2000a,b,c). NBAC's report on those with mental disabilities and two forthcoming NBAC reports also contain many recommendations that warrant action (NBAC, 1998, forthcoming-a,b). An independent voluntary accreditation system appears to be one element that could improve the system as part of a long-term strategy and, thus, should be pilot tested and evaluated over the next several years.