3

Standards for Accreditation

Any set of standards used by accreditation organizations responsible for the protection of research participants must be flexible enough to be applicable to a variety of institutions yet rigorous enough to ensure that their enactment enhances protection of human research participants. In addition, they must be clearly written, relatively straightforward to execute, consistently applicable, and measurable. These are not easy goals.

In response to a request from the U.S. Department of Health and Human Services (DHHS), the Institute of Medicine was asked to address accreditation standards for human research participant protection programs (HRPPPs). To accomplish this task, the committee reviewed draft versions of proposed standards developed by Public Responsibility in Medicine and Research (PRIM&R) and the National Committee for Quality Assurance (NCQA), as well as the International Conference on Harmonisation Guidelines for Good Clinical Practice (ICH-GCP).

The PRIM&R standards were drafted to be used as measurement criteria for a new voluntary program for research protection. The standards are intended to guide organizations seeking private voluntary accreditation in the assessment of their human research protection programs (HRPPs) and to be used by independent site visitors during the accreditation process.

NCQA is an independent, nonprofit organization under contract with the U.S. Department of Veterans Affairs (VA) to operate an accreditation program to ensure that VA medical centers are complying with VA and other relevant federal regulations designed to protect human participants in research.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs 3 Standards for Accreditation Any set of standards used by accreditation organizations responsible for the protection of research participants must be flexible enough to be applicable to a variety of institutions yet rigorous enough to ensure that their enactment enhances protection of human research participants. In addition, they must be clearly written, relatively straightforward to execute, consistently applicable, and measurable. These are not easy goals. In response to a request from the U.S. Department of Health and Human Services (DHHS), the Institute of Medicine was asked to address accreditation standards for human research participant protection programs (HRPPPs). To accomplish this task, the committee reviewed draft versions of proposed standards developed by Public Responsibility in Medicine and Research (PRIM&R) and the National Committee for Quality Assurance (NCQA), as well as the International Conference on Harmonisation Guidelines for Good Clinical Practice (ICH-GCP). The PRIM&R standards were drafted to be used as measurement criteria for a new voluntary program for research protection. The standards are intended to guide organizations seeking private voluntary accreditation in the assessment of their human research protection programs (HRPPs) and to be used by independent site visitors during the accreditation process. NCQA is an independent, nonprofit organization under contract with the U.S. Department of Veterans Affairs (VA) to operate an accreditation program to ensure that VA medical centers are complying with VA and other relevant federal regulations designed to protect human participants in research.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs ICH-GCP represents an “international ethical and scientific quality standard for designing, conducting, recording, and reporting trials that involve the participation of human subjects” (International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 1996, p. 1). In addition to being widely accepted in the clinical trials community, the ICH-GCP standards are recognized by the Office for Human Research Protections (OHRP) and included within the Food and Drug Administration (FDA) guidance document for clinical trials. Although these are guidelines for investigators and research sponsors conducting or supporting clinical trials, they specifically address the roles and responsibilities of these parties at a level of detail not found in either the PRIM&R or NCQA standards and are thus directly relevant to the assessment of HRPPPs. As the committee struggled in a short period of time to develop a “theory” on which the standards for accreditation of HRPPPs could be based, the challenges and perhaps impossibility of developing a “one-size-fits-all ” approach became apparent. The three sets of standards were reminders of the vastness of the research enterprise and the distinctive nature of certain types of research and research settings. For example, the PRIM&R standards appear to focus on research conducted in traditional academic health care settings, the NCQA standards encompass research conducted by the VA in its own self-contained health care system, and the ICH-GCP guidelines are specific to investigators and sponsors conducting clinical trials, a specialized type of research with human participants. Even so, the three distinct research situations described above all pertain to biomedical research environments. As discussed in Chapter 1, this does not adequately represent the multiple contexts in which human research occurs. The breadth of these research contexts creates layers of complexity that are not easily absorbed when a single set of standards is being developed for the assessment of performance. An organization's scope of activities should define which standards apply. Moreover, the accreditation body must consider the degree to which an HRPPP must comply with the standards. That is, must an organization be in full compliance with every standard to become accredited? Or should the organization demonstrate overall compliance with the full set of applicable standards? The answers to these questions might dictate the magnitude and scope of a set of standards and the level of detail that is necessary to support them. If the goal is to develop a single set of standards, such standards must accommodate several types of organizations engaged in the review and conduct of research with human participants. STANDARDS FOR STANDARDS At a minimum, standards should address an organization's level of performance in specific areas and, some would argue, not just what the organization is capable of doing but what it actually does (JCAHO, 2000). In theory, stan-

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs dards should set forth maximum achievable performance expectations for activities that affect the protection of human research participants. Perhaps most importantly, they should be based on widely accepted ethical principles that form the norms for research behavior. In the United States, the principles embodied in The Belmont Report have served as the foundation for the ethical requirements in human research (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979). The three basic ethical principles in The Belmont Report are (1) respect for persons, (2) beneficence, and (3) justice. The first principle, respect for persons, encompasses two ethical concepts: first, “individuals should be treated as autonomous agents” and their decisions respected; and second, “persons with diminished autonomy are entitled to protection ” (p. 4). The second principle, beneficence, incorporates the rules of “do no harm” and “maximize possible benefits and minimize possible harms” (p. 4). The third principle, justice, refers to a fair and equitable distribution of benefits and burdens, fair selection of participants, assurance that participants receive what is deserved or due, and ascertainment that equals are treated equally (p. 5). In the United States, these principles strongly influenced the development of federal regulations—in particular, regulations governing research sponsored by the federal government or regulated by the FDA—via the Federal Policy for the Protection of Human Subjects (45 CFR 46, subpart A, also known as the “Common Rule”) or parallel FDA regulations (21 CFR 50, 56; international studies of devices are covered by 21 CFR 312.120). The ethical principles found in The Belmont Report are also found in many international documents, including the Declaration of Helsinki and guidelines promulgated by the Council for International Organizations of Medical Sciences, a source on the ethics of international research involving human subjects (CIOMS, 1993; World Medical Association, 2000). The ethical principles should be accompanied by procedural requirements, which then form the basis of the standards. Thus, standards should have an explicit rationale that is consistent with the goal of protecting individuals or populations that participate in research. The committee 's “standards for standards” are contained in two recommendations. Recommendation 3: Articulate Sound Goals Within Acreditation Standards The goals of accreditation standards should be to ensure that the proposed research promises to contribute knowledge sufficient to justify research involving human participants; independent review of research by a board knowledgeable about protection standards and the fields of research being reviewed;

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs that the perspectives of participants are represented on institutional review boards (IRBs), on research monitoring bodies, and throughout the research oversight system; that IRB members do not review protocols with which they have financial or nonfinancial conflicts of interest;1 that investigator and institutional conflicts of interest, both financial and nonfinancial, are disclosed to IRBs and participants and are managed responsibly by research institutions; a review process that balances risks and potential benefits, keeps risks to the minimum necessary, and monitors research on a continuing basis; that an effective process for obtaining voluntary informed consent of participants is in place; that policies and procedures to assess the quality of HRPPP operations, enhance accountability, and improve performance are in place; there is fairness in the recruitment and selection of participants; that the privacy and confidentiality of research participants are protected; and that the HRPPP is transparent so that participants can judge the research process to be trustworthy. Recommendation 4: Establish Flexible, Ethics-Based, and Meaningful Standards Accreditation standards should meet the following minimal criteria: They should be based on sound and widely accepted ethical principles. 2 1   The committee does not mean that any member who could have a conflict with any conceivable protocol coming to an IRB for review should be excluded from service on an IRB but, rather, that the individual should recuse himself or herself from reviewing such protocols. 2   The principles laid out in The Belmont Report are one foundation (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979). Accreditation standards, however, should also incorporate the recommendations of the President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research (President's Commission, 1981, 1983), the recommendations of the Advisory Committee for Human Radiation Experiments (ACHRE, 1995), recommendations presented in reports of the National Bioethics Advisory Commission (NBAC, 1997, 1998, 1999a,b, forthcoming-a,b), the recommendations of the Office of the Inspector General of DHHS (DHHS OIG, 1998b, 2000b), and the recommendations of the General Accounting Office (GAO, 1996). In addition, recommendations from reports and declarations of private bodies and independent scholars should be incorporated. This presupposes that an advisory apparatus is available to cull this literature.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs They should be flexible and adapted to different kinds of research and different research institutions. They should encourage accredited organizations to shift from a culture that relies on external compliance checks to a culture that puts safety and voluntary participation foremost. They should facilitate compliance with federal regulations but should aim to move an organization toward having stronger protection of human research participants. To the extent possible, they should focus on the use of meaningful measures of how well the rights and interests of research participants are being protected rather than simple determination of whether informed-consent statements have been signed or IRB meetings were duly constituted. Measurement of an organization's compliance with the procedural requirements set forth by standards serves as a proxy for ascertainment of the organization's level of compliance with the ethical principles that underlie the standards. In its early discussions, the committee noted that beyond the primary aspiration of protecting those who participate in research, institutions seeking accreditation will be motivated by other aims as well, for example, enhancing the qualities and reputations of their research programs (and, as a result, potentially improving their financial status or prestige), attracting faculty and students to their graduate research training programs, and facilitating the recruitment of individuals as research participants. A successful system of accreditation must offer incentives for participation, such as enhancing the likelihood that a program in compliance with the standards will attract these resources. In addition, a successful accreditation system must have realistic and enforceable mechanisms by which to deter noncompliance with the standards (e.g., suspension from the program or loss of accreditation). DEVELOPING MEASURES TO ACCOMPANY STANDARDS Standards must be developed with consideration of the measures that will be used to evaluate an organization's level of compliance. The processes of developing standards and designing a set of tools that can be used to measure compliance (i.e., accreditation) cannot generally be uncoupled. The measures must address areas in which performance is likely to have a significant impact on the protection of human research populations. In addition, they must be precisely defined and specified, that is, standardized with explicit predefined requirements for data collection and for calculation of the value of the measure or the score for the measure. Furthermore, for the purpose of accreditation, there must be documentation for the measure that includes defined data elements,

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs corresponding data sources, and allowable values. Such measures must be reliable—that is, the measurement should be able to identify consistently the events that it was designed to identify across multiple HRPPPs over time—and they must be valid, that is, they must capture what they were intended to measure. The tools used to measure compliance with standards should be easily interpreted by those who use the resulting data, including accreditors, research participants, and those conducting or overseeing the research. Finally, determinations of the levels of compliance with the standards must be based on data. HRPPPs seeking accreditation will be required to provide evidence of compliance. This evidence must be supported by a reasonable data collection effort. In the development of standards, accreditation bodies must be mindful of the availability and accessibility of the required data elements and the effort and cost of abstracting and collecting data. In general, standards should help HRPPPs and accreditation bodies identify exemplary performance and best practices, thus serving as a benchmarking service for the organizations seeking accreditation. In addition, ideal standards would provide the content for publicly available comparative reports on the performance of the accredited organization. This view of accreditation standards is reflected in Understanding Accreditation (Young et al., 1983), which notes four trends in the accreditation process: (1) it has moved from a more quantitative to a more qualitative system of assessment, with more general rather than specific standards; (2) it has placed less emphasis on making institutions look alike and more emphasis on a stance of recognizing and encouraging individuality; (3) it has evolved from a system based more on external review to a system of self-evaluation and self-regulation; and (4) it has moved from a focus on the institution to a focus on encouraging and assisting the organization in its efforts to improve quality. In this light, standards should describe important functions related to the protection of research participants, and they should be framed as performance objectives that are unlikely to change substantially over time. Because standards aim to improve outcomes, they should place minimal emphasis on how to achieve these objectives. In addition any set of standards should make clear which standards are cores, that is, those that must be applied across programs and that are essential to an HRPPP. Some standards, such as those that directly relate to the protection of human research participants, should carry more weight than others. It is especially important that clear measurement tools be available for core standards and that guidance on how the measurement will be interpreted is available.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs NEED FOR STANDARDS TO ENCOMPASS MULTIPLE RESEARCH SETTINGS AND METHODS Recommendation 5: Accommodate Distinct Research Methods and Models Within Accreditation Programs The accreditation process should accommodate other research organizations in addition to the tradtional models provided by academic health centers and VA facilities. The accreditation process should also cover research other than clinical research. Standards must accommodate the distinct natures of several types of organizations, including research institutions, educational institutions, independent IRBs, academic medical centers, nongovernmental organizations, and private interests. A set of standards can make clear the scope of institutions to which they apply in several ways: (1) state explicitly in the preamble the intended focus of the standards; (2) include flexible language, such as “where applicable” or “as appropriate” to certain standards so that institutions not engaged in particular activities (e.g., nonmedical, low-risk research) could be exempt from certain standards (e.g., reporting of adverse events); or (3) organize the standards so that institutions and accreditation bodies can quickly ascertain which sections apply to them and which ones do not. If standards were structured in a manner that requires the existence of a single entity with exclusive authority over all parties involved in the research process, then the three requirements listed above would not apply. It must be recognized, however, that certain organizations, such as independent IRBs and some private sponsors of research, would then not be eligible for accreditation. This would be an unfortunate consequence, as it would exclude organizations that play an increasing role in the research enterprise. Accreditation of an independent IRB, for example, might use only the subset of standards pertinent to IRBs, but doing so would also require formal assurance regarding the functions covered by proposed standards that pertain to investigators, research institutions, and research participants, as well as standards that pertain to sponsors but that are not yet incorporated into NCQA or PRIM&R standards (but covered by ICH-GCP guidelines) (see discussion below). Another approach would be to accredit the organization that directly controls all the relevant elements of an HRPPP (e.g., a contract research organization that has a formal agreement with an independent IRB to review all its protocols, the research unit of a private firm, the unit of a federal agency that performs research, or a clinical trials cooperative group). One of the virtues of a nongovernmental voluntary accreditation process is its flexibility, and nongovernmental accreditation bodies should not find it difficult to accommodate disparate organizational structures. It is not yet clear, however, how the current proposed standards or accreditation processes would do so.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Although there is a natural tendency to develop standards and review procedures around a specific model, accommodation of innovative or unique organizations is central, and although “basing development on a commonly accepted template may benefit the accrediting organization, there is a danger that innovative structures or processes undergoing accreditation will encounter additional challenges or problems in the review process” (Hamm, 1997, p. 31). In addition to accommodating distinct types of research infrastructures, the language of standards should acknowledge that even though the principles that underlie them apply to all human research, the criteria and mechanisms for review must be adaptable and must be based on the nature of the research being conducted and the context within which the research is to be performed. The committee heard strong, consistent comments that the proposed standards (in this case, those of PRIM&R) do not fully recognize either the diversity of institutions or the full range of research (AAU, COGR, NASULGC, 2001; Kulakowski, 2001; Ryan, 2001). The standards proposed by NCQA under contract with the VA, however, are necessarily limited in scope to VA facilities. Although the committee believes that the same principles for protection of the rights and interests of research participants apply to all research—for example, biomedical, behavioral and social, public health, and outcomes research—it is likely that the processes needed to comply with the standards will differ depending on the nature of the research. Thus, it is an open question whether the best accreditation strategy would be to use one set of operational standards for all research. That might well prove viable, but it also might prove better to encourage the evolution of different specific standards for different kinds of research institutions. Those in the best position to make this determination will be organizations devising the nongovernmental accreditation processes, not this committee or the federal government. Whether to develop one set of standards or a few sets of standards specific to a few different classes of research organizations should not be decided by fiat but should be decided in light of experience gained through pilot accreditation programs that include medical and nonmedical sites. Accreditation pilot programs can begin by focusing on the research institutions for which they were designed, but they might evolve in many different ways. In the future, there could be one or a few accreditation bodies and one or a few sets of accreditation standards, and many different kinds of organizations will certainly be involved in research with human participants.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs RELATION OF THE STANDARDS TO THE EXISTING REGULATORY REQUIREMENTS Recommendation 6: Base Standards on Existing Regulations Accreditation standards should start from federal regulations for the protection of human research participants but should augment those regulations. The process should be iterative and continual, with evolution of both accreditation standards and the operations of accredited organizations, creating incentives for accredited organizations to improve. Institutions that receive federal funds, that hold an assurance from OHRP, or that seek FDA approval must comply with the Common Rule or parallel FDA regulations. Therefore, it is important that any standards be considered in relation to the regulatory requirements; that is, are they consistent, supplemental, or contradictory? Many commentators at the committee's public forum, as well as committee members themselves, expressed concern that new standards for accreditation could impose another layer of bureaucracy on a system that is already sagging under the weight of paperwork, but would add little to the protection of human research participants (AAPP, 2001; Cornblath, 2001; Oakes, 2001). Three issues to be considered in this context: (1) If the standards are identical to federal regulatory standards, both the institution and the accreditor are performing redundant tasks (presumably largely paperwork, assuming that the institution is already in compliance with the federal regulations) unless some simple means is found for the certification of compliance; (2) If the standards are inconsistent with federal regulations, confusion is likely to result; (3) If the standards are more demanding than federal regulations, a question must be raised: are the additional expectations likely to strengthen protections at a reasonable cost? Accreditation standards should start from the base of regulations governing research with humans. These regulations, in turn, are based on a set of principles for the ethical conduct of research (see Recommendation 4). By the use of standards that emphasize processes of continuous quality improvement instead of an exclusive focus on regulatory compliance (see below), the way may be open to the development of future standards that center on HRPPP performance, in addition to the current focus on documentation. For example, an HRPPP that demonstrates that it can ensure informed consent because it has data showing that participants understand the protocols in which they are enrolled, could begin to supplant or augment paper audits of signed informed-consent forms. This strategy therefore has the potential to introduce the desired flexibility and focus on outcomes into the oversight system. Furthermore, this goal that standards continuously evolve supports the committee 's recommendation (Recommendation 2) that HRPPP accreditation bodies be nongovernmental organizations, as the

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs federal regulatory process does not possess the sensitivity and responsiveness to maintain pace with opportunities for improvement. STANDARDS FOR QUALITY IMPROVEMENT AND SELF-STUDY Recommendation 7: Incorporate Continuous Quality Improvement Mechanisms into Standards Accreditation organizations should emphasize the process of self-study, evaluation, and continuous quality improvement among applicants. They should move beyond documentation of informed consent and protocol review, which, although essential, do not of themselves protect the rights and interests of research participants. Standards provide an HRPPP with the opportunity for benchmarking, a continuous, systematic process used to make improvements. By periodically examining activities, policies, procedures, support functions, organizational performance, and the status of data collection and processing, an HRPPP can develop an approach to quality improvement. A sound system of self-assessment can identify the best practices in an organization and target areas in need of improvement. Compliance with regulatory requirements, in contrast, provides an important but irregular approach to ensuring that protections are in place. Thus, standards not only provide the basis for a system of self-study and improvement but also should incorporate the expectation of such a quality improvement system. This is not to say that self-study alone is sufficient. To maintain the integrity of the accreditation process, an HRPPP must conduct self-study as well as be subjected to external review (whether by an accreditation body or a regulatory agency). Standards should aim to improve outcomes and should not overly prescribe how to achieve the specified objectives. Rather, they should focus on the core standards that apply across programs and that are essential to a quality HRPPP. Current proposed standards generally reinforce the documentation practices required by federal regulations but do not yet go beyond the regulations. In general, both entities seeking accreditation and accreditation bodies should identify exemplary performance and best practices, providing benchmarks for the research community at large and making information on organization performance openly available to the public and policy makers. In this way, for example, an HRPPP demonstrating a particularly reliable system for the monitoring of participant safety or the reporting of problems in ongoing research, might have an advantage over nonaccredited competitors in seeking support from sponsors or having access to participants, researchers, or students.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs NEED FOR STANDARDS TO ENHANCE THE ROLE OF RESEARCH PARTICIPANTS Recommendation 8: Directly Involve Research Participants in Accreditation Programs and HRPPPs The formulation of accreditation standards, the accreditation process, and HRPPP operations should directly involve research participants. 3 Current regulations lay a foundation for and even invite stronger involvement of those representing the interests of those participating in research. Yet, some “noninstitutional” members of IRBs have little experience as participants in research; they may be independent of the institution, but it does not follow that they represent the perspective of research participants. The regulations are necessarily nonspecific about the involvement of research participants in the review process and set a low standard for qualification. When HRPPPs are regularly judging the benefits and risks of studies that involve particular populations, there should be evidence that the review process directly involved those who genuinely understand and represent the perspective of those populations. This requirement could be incorporated into accreditation standards. Practices regarding membership on data safety and monitoring boards (DSMBs) are even more diverse. The only stipulated expertise on DSMBs is technical: a clinician familiar with the medical aspects and a statistician familiar with data analysis. In instances in which they attend explicitly to safety and the ethical conduct of research, DSMBs are more apt to include a bioethicist or a lawyer than someone who brings the perspective of research participants. Accreditation standards—and even more so, the guidance documents that accompany them by giving examples of good practices—can improve the HRPPP system to ensure stronger representation of the interests of the research participants. Given the primacy of the concepts of autonomy in research ethics and the training of IRB members, the relative lack of attention to standards and measures that would systematically cultivate these concepts in both the PRIM&R and NCQA proposed standards is somewhat surprising (see the discussion in the What's Missing section below). Several measures can be taken to bolster these concepts to improve the ethical conduct of research involving human participants. IRBs, DSMBs, research design teams, and merit review committees should increase their level of attention to the involvement of research participants or those who genuinely represent participants' perspectives in the design, selection, review, and monitoring of research involving human participants. In 3   By “participants,” the committee refers to those whose background and expertise are credible to a lay constituency external to the research institution and who are knowledgeable about the research process and research protections. The term is further defined in Chapter 1.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs TABLE 3-1 Elements in Three Sets of Standards and Guidelines   Organization Developing the Standard or Guideline Key Elements PRIM&R NCQA ICH Intended use Standards Standards Guidelines Targeted sites or bodies Research institutions (U.S.) VA facilities Organizations conducting clinical trials of drugs Foundational principles The Belmont Reporta The Belmont Report Declaration of Helsinkib Regulatory relevance Implied 45 CFR46, 21 CFR 50 and 56, and VA regulations are the starting points (cross-referenced) Drug approval regulations in the European Union, Japan, and the United States Components affected Organizations IRBs Investigators and other personnel HRPPs Institutions IRBs Investigators IRBs or ethics review committee Investigators Sponsors Link to quality improvement program? No Yes No Standards for participant involvement (beyond consent)? No No No Standards for sponsors? No Noc Yes Standards for monitoring? Limited, one mention in one documentation standard Yes Yes

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Specific guidance for inter-preting standards? No Yes Yes Data source identified? For some documentation standards Yes Yes Methods for measuring provided No, except documentation standards Yes Partial Thresholds established for compliance? No Yes No Appeals process No Yes Yes aNational Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (1979). bWorld Medical Association (1964). cNCQA standards are written for research conducted in VA facilities. For research conducted at VA facilities but sponsored by external sources (e.g., National Institutes of Health; the U.S. Department of Defense; or pharmaceutical device, or biotechnology firms), additional sponsor provisions, such as written agree-ment to abide by ICH-GCP guidelines, would be needed.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs roles and responsibilities of investigators and the “organization” (e.g., institutional officials, administrative offices, personnel, existing compliance programs, or oversight mechanisms), there is far less attention to these parties than to IRBs, and little to no mention is made of the roles and responsibilities of research sponsors, despite the central role that sponsors play in much of the privately funded research. NCQA Standards The standards developed for the VA by NCQA (Appendix C) are distinct in that they are applicable to a defined system. The VA conducts biomedical, health services, and rehabilitation research to improve the health care delivered to the nation's veterans. The VA has developed policies, consistent with the Common Rule and FDA regulations, to safeguard human participants in research and has established the Office of Research Compliance and Assurance (ORCA) to support the field operations in protecting human participants and to assess their compliance with regulations that protect human research participants. The standards will be applied to VA hospitals and VA employees. In that sense, the standards do not face the same level of complexity in the field as the proposed PRIM&R standards do. Nonetheless, they appear to be potentially applicable, with some additions and modifications, to research conducted in other, non-VA, nonmedical settings (see Table 3-1). The draft NCQA standards are notable in several respects. First, they are not overly prescriptive, although they do begin (as do the PRIM&R standards) from the base of federal regulations (see below). Second, the NCQA standards specifically rely on institutional policies and procedures as the methods by which standards are met. The explicit “data source” for several of the standards is the policies and procedures documentation on file at the institution or the quality improvement document maintained by the institution (see Recommendation 7). This is noteworthy because although the standards will apply to a system that is far more homogeneous than the general research environment, they allow variations in procedures, perhaps recognizing that even within the VA health care system there will be institutional variations. Third, the standards provide thresholds for compliance in each core area: the IRB, informed consent, institutional accountability, privacy and confidentiality, recruitment and subject selection, and risks and benefits. Thus, to receive full compliance with a requirement, a site must achieve compliance with specified “critical elements. ” The site may still receive partial compliance with the requirement if those elements are not met6 (see previous discussion in the section Developing Measures to Accompany Standards). 6   For more information on this process, see http://www.ncqa.org/Pages/Programs/QSG/vastandards.htm.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs Relation to Existing Regulatory Requirements As suggested in Recommendation 6, both the PRIM&R and the NCQA draft standards use the current regulatory standards as the “starting point” for the development of their accreditation programs (Chodosh, 2000; Goldschmidt, 2001). In fact, Standards 1.2 and 1.3 in the PRIM &R standards state that “the organization must uphold ethical principles underlying the protection of individuals studied in research” and that “the organization must assure compliance with applicable legal requirements, including state and local laws ” (see Appendix B). However, in the PRIM&R document, there are some instances in which consistency with the federal regulations could be more explicit and concise, such as the reporting of adverse events to the National Institutes of Health, research sponsors, the FDA, IRBs, and institutional biosafety committees. The relationship of the standards to additional regulatory requirements, such as DSMBs and emerging medical privacy regulations, should be considered and made clear. A notable aspect of the NCQA standards is that they cross-reference the federal regulations. This is a useful approach and one that will be welcomed by administrators facing competing guidelines, regulations, and standards. In addition, because they rely on the regulations to establish which research must be reviewed by an IRB and which research requires retrieval of informed consent, they provide the flexibility that is needed to exclude some types of minimal-risk research from full review and also possibly the requirement to obtain informed consent. Extent to Which the Standards Can Be Implemented, Measured, and Enforced To be measurable, there must be some objective means through which the extent to which a program is in compliance with accreditation standards can be gauged. Put another way, if an institution was denied accreditation or had its accreditation revoked, are the standards sufficiently well defined and consistently applied that the accreditor could defend its decision in court? The need for objective measurement tools is critical to ensuring consistency and diminishing arbitrary subjectivity in the accreditation system. What is considered independent and credible in one institution might not be considered so in another. In the material provided by PRIM&R, some of the standards seem largely hortatory.7 Some committee members found it difficult to envision how these standards could be implemented, measured, or enforced (except perhaps retrospectively, after egregious noncompliance). For instance, the language directed toward investigators in Standard 3.1 and 3.2 ( Appendix B) is very important, as investigator conduct is essential to the realization of ethical research. It is not clear, however, how one would ensure in an objective way that investigators are 7   For example, Standards 1.1, 1.7, 3.1, and 3.2 (Appendix B). Documentation standards are more specific, but many other standards are similarly hortatory.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs meeting the PRIM&R standards. Data collection from even a sample of investigators at an accreditation site would be overwhelming, and sample bias would be a very serious concern. The committee therefore had a difficult time conceiving of how these standards could be effectively enforced, even if a useful measurement approach could be devised. On the other hand, several standards do indeed seem measurable but appear to depend on the production or appropriate filing of pieces of paper (or other bits of data) and may have little to do with the quality of research or protecting the rights and interests of participants. For example, standards for IRB minutes and record keeping are fairly prescriptive and provide some measure of activity for individuals inspecting or accrediting a site. Although the ability to keep accurate records is necessary, it is insufficient to guarantee an effective human research protection program. Similarly, the NCQA standards also possess a reliance on documentation already called for in federal regulations. However, the NCQA program is based on the assumption that an institutional quality improvement program exists at the organization seeking accreditation (in this case VA facilities). The quality improvement documentation is an important source of data for the accreditation body, serving as a measure of performance at a particular point in time but also as a measure of change over time. This strategy provides the opportunity within the NCQA HRPP accreditation standards to become less reliant on documentation and more reliant on performance (Recommendation 7). The NCQA standards clearly articulate the data source and measurement method to be used by the accreditation organization. As noted above, this is a real strength because clear indications of the data source to be tapped and an unambiguous method for the measurement of compliance with the standards must be developed in conjunction with the standards if they are to be workable. In contrast, evaluation of the level of compliance with the PRIM&R standards has not been thoroughly described in the materials reviewed by the committee. It is not enough for the institution to just have policies. It must also follow them. In the absence of clear guidance on how outcomes should be measured, determination of whether an institution meets these standards could be daunting for both accreditors and the organizations that they are accrediting. What Is Missing The committee identified a few topics that do not appear to be explicitly included in the current drafts of the PRIM&R and the NCQA standards. Both lack standards for improving participant involvement in the local research review and decision-making processes. There is little to no mention of the rights and responsibilities of research participants or the need for subject participation in the functions of the HRPP (except for those that are required by regulation). In addition, the standards might better address some procedural ap-

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs proaches to the inclusion of research participants in the HRPP (see discussion following Recommendation 8). However, members of the Program Advisory Committee for the NCQA accreditation system will be selected from research stakeholder groups, including participant advocates, and will consider programmatic issues to advise the Program Accreditation Committee (the decision-making group for this program). As mentioned earlier, the roles and responsibilities of research sponsors are important omissions from both sets of standards that should be addressed. In the case of the NCQA draft standards, it is possible that VA headquarters, through ORCA, is developing standard operating procedures that establish standards when the VA is the sole sponsor. However, for externally sponsored research conducted at VA facilities, HRPP standards or assurance that sponsors are abiding by ICH-GCP or other accepted external standards is needed. INTERNATIONAL CONFERENCE ON HARMONISATION GUIDELINE FOR GOOD CLINICAL PRACTICE The ICH-GCP was developed as a handbook for researchers conducting clinical trials, particularly drug trials conducted by sponsors and researchers from more than one country (International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, 1996, 1997). Although the guidelines presented in the ICH-GCP are not actually standards, they provide a clear and explicit set of best practices for those conducting clinical trials (see Box 3-1). The committee looked to the ICH-GCP because it includes defined goals for sponsors and investigators. However, it does not address, per se, the institutions or the setting in which the research will be conducted. As such, the ICH-GCP is “portable” and is therefore an important contribution to enhancing the protection of research participants, wherever the clinical trial is conducted. Aspects of the ICH-GCP serve as clearly delineated models for investigator and sponsor behavior, and, thus, the responsibilities contained within these models should be included in the development of guidelines for HRPPPs. The ideals or norms that the document espouses, however, would need to be translated into standards, and such standards would have to be applicable beyond clinical trials and biomedical research methods.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs RECOMMENDATION FOR INITIAL STANDARDS TO BEGIN PILOT TESTING Recommendation 9: Use Modified NCQA Standards to Initiate Pilot Programs Pilot accreditation programs should start from the accreditation standards and processes proposed by NCQA for VA facilities, as adapted for use in other organizational contexts. In expanding the draft NCQA accreditation standards for use beyond VA facilities, the standards should be strengthened in six specific ways as pilot testing commences. The PRIM&R standards were prepared for a broad set of potential applicant organizations, which would include but not be restricted to academic health centers. The NCQA standards were explicitly prepared for accreditation of VA medical facilities. In this instance, the applicant pool is defined, and, in fact, pilot tests that will use those standards are being planned as this report goes to press. As noted throughout this discussion of report recommendations, the committee regards the NCQA standards as an excellent starting point for accreditation of VA facilities. The committee recommends, however, that the NCQA standards be strengthened in six areas, to specify (1) how investigators will be reviewed beyond the review of the protocols that they submit for IRB approval;8 (2) whether and how research sponsors will be assessed in the accreditation process;9 (3) how participants will be involved in setting standards and accrediting HRPPPs;10 8   For research programs involving only a small set of investigators, accreditors might contact all of them; for most programs, however, accreditors would need to sample investigators in a way that is independent of control by the IRB or the institution's research administration. How to do this will likely vary by institution and will have to be specified in advance by the accreditation body. The sampling procedure is likely to evolve during the pilot testing phase. 9   Some organizations do little or no externally sponsored research so they would be exempt from this aspect of accreditation review. Organizations that do sponsored research will vary widely in the number of protocols and the kinds and numbers of sponsors. For programs with extensive externally sponsored research portfolios, accreditation bodies will need to develop sampling methods that are credible and independent of the organization's IRBs and research administration. Standards for this aspect of review could initially start from the ICH-GCP guidelines noted in Table 3.1. 10   Accreditation bodies will need to develop methods to sample participants in a manner that is credible and independent of IRBs and research administrators of the organizations seeking accreditation. Participants were not surveyed in the 1998 survey of IRBs and investigators commissioned by National Institutes of Health (Bell et al., 1998), yet the committee believes that participant perspectives are essential to judging whether an HRPPP is operating effectively.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs BOX 3-1 The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use is a project that brings together the regulatory authorities of Europe, Japan, and the United States and experts from the pharmaceutical industry in the three regions to discuss scientific and technical aspects of product registration (or, in the United States, approval for marketing). The purpose is to make recommendations on ways to achieve greater harmonization in the interpretation and application of technical guidelines and requirements for product registration to reduce or obviate the need to duplicate tests carried out during the research and development process for new medicines. The objectives of such harmonization are the more economical use of human, animal, and material resources and the elimination of unnecessary delay in the global development and availability of new medicines while maintaining safeguards on quality, safety and efficacy, and regulatory obligations to protect public health. The Guideline for Good Clinical Practice is an international ethical and scientific quality standard for the design, conduct, recording, and reporting of trials that involve the participation of human subjects. Compliance with this standard provides public assurance that the rights, safety, and well-being of trial subjects are protected, consistent with the principles that have their origin in the Declaration of Helsinki, and that the clinical trial data are credible. The objective of the International Conference on Harmonisation Guideline for Good Clinical Practice (ICH-GCP) is to provide a unified standard by which the European Union, Japan, and the United States can facilitate the mutual acceptance of clinical data by their respective regulatory authorities. The guideline was developed with consideration of the current good clinical practices of the European Union, Japan, and the United States, as well as those of Australia, Canada, the Nordic countries, and the World Health Organization. Investigators should follow this guideline when they are generating clinical trial data that are intended to be submitted to regulatory authorities. The principles established in ICH-GCP may also be applied to other clinical investigations that may have an impact on the safety and well-being of human subjects. SOURCE: http://www.ifpma.org/ich5.html.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs (4) how oversight mechanisms can ensure participants' safety in ongoing research;11 (5) the steps that research institutions and their leadership can take to cultivate a culture that puts the safety and interests of research participants foremost;12 and (6) mechanisms by which research institutions and, where applicable, research sponsors can be held accountable for ensuring sufficient funding, structural support, and professional rewards for HRPPPs. 13 The NCQA standards, if improved as recommended, could also be used —by NCQA, the Association for the Accreditation for Human Research Protection Programs (AAHRPP), or other accreditation organizations —as the basis for the development of accreditation standards for non-VA research organizations. Accreditation will not be successful until it is widely accepted as a mark of excellence. To accomplish this, it should serve as an educational tool to raise the median overall performance of an accredited organization. To do this, accreditation standards and the processes in which they will be used must incorporate consistent feedback from the parties involved in the various aspects of an HRPPP. As discussed above, the local aspects of this issue (i.e., aspects that apply to individual applicant institutions) should be enhanced in the NCQA standards. The committee is encouraged that both NCQA and AAHRPP include stakeholder representatives in their programmatic leaderships (see Recommendation 2). Those who encounter problems in the research protection system, irrespective of the perspective that they represent in that system, need simple, con- 11   Chapter 3 describes some options for research monitoring and feedback. When organizations applying for accreditation conduct research that is monitored by DSMBs, for example, details of how those boards interact with investigators, IRBs, and research administrators would need to be evaluated for all or a representative sample of DSMBs. Reporting mechanisms for severe or unanticipated adverse events would similarly be necessary to evaluate all protocols or a representative sample of protocols. Ombudsman programs and reporting mechanisms for concerns, complaints, and other feedback mechanisms would be included. Pilot testing will likely reveal a wide variety of monitoring and feedback methods that will have to be accommodated in the accreditation process. 12   PRIM&R's Standard 1.16 calls for assessment of quality improvement programs, and NCQA's standards presented in Table C-3(B) do so with even more specificity. The committee believes that procedures for evaluating the informed-consent process in particular deserve special attention and will be both the foundation of effective protections and the best hope of shifting from documentation to performance measures. 13   Budget and staffing for IRB operations, monitoring and ombudsman programs, and other HRPPP components are not sufficient to evaluate quality and effectiveness. Insufficient budgets and staffing, however, would be clear indications of deficiencies. The committee sought information about budgets and staffing, but found few data. (The 1998 report by Bell and colleagues contains some data on IRBs and investigators at 491 institutions; it does not, however, include data on IRBs regulated only by the FDA, monitoring bodies, or administrative costs.) Extant data were insufficient for the committee to develop benchmarks for different kinds of organizations seeking accreditation. Such benchmarks will thus have to be established in light of experience from pilot testing.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs sistent ways to bring their concerns to light and to bring relevant information into the procedure for the review of the process at the level of both the HRPPP and the accreditation process. It is the committee's understanding that the NCQA standards will be tested in a pilot study beginning in the spring of 2001.14 This is an important step in gauging the feasibility of the use of these standards for the accreditation process, and the committee encourages similar pilot testing with appropriately modified standards in non-VA research environments. 14   As this report went to press, NCQA made their draft standards available for public comment. See http://www.ncqa.org/Pages/Programs/QSG/VAHRPAP/vahrpapdraftstds.htm for further information.

OCR for page 63
Preserving Public Trust: Accreditation and Human Research Participant Protection Programs This page in the original is blank.