Click for next page ( 144


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 143
6 Enhancing the Effectiveness of Review: Minimal-Risk Research ' N THIS CHAPTER we address the operation of institutional review I boards (IRBs) with respect to review procedures for minimal-risk social, behavioral, and economic sciences (SBES) research. The available evidence, although limited (see Chapter 2), supports a con- clusion that IRBs often decide not to use the flexibility in the Common Rule that allows them to exempt some types of SBES (and biomedical) research from review or to review other kinds of minimal-risk SBES (and biomedical) research with an expedited procedure. Some IRBs do not exempt or expedite any research, even when they agree that it is minimal risk, and others do not do so for certain categories of el- igible research (see Boxes 1-1 and 1-2 in Chapter 1 for categories of exempt research and categories of research that can be expedited, re- spectively). Furthermore, although there is only anecdotal evidence on this point, IRBs in some instances may overestimate the risks of harm to participants in SBES (and biomedical) research. In such cases, they may use a time-consuming full board review when it is not needed and perhaps request changes in research design that compromise the sci- entific validity of the study without necessarily increasing protection for participants. At the other extreme, a few IRBs may underestimate risk given that they never or hardly ever conduct full board reviews. The primary goal of the IRB system is to conduct reviews that pro- tect participants to the extent possible. The Common Rule sets out alternative procedures for minimal-risk research that are adequate for protection and can free up scarce resources of IRBs and investigators to devote more attention to higher risk research when needed and to other activities that are important for participant protection, such as the development of improved guidance and training for IRB members, researchers, and research institution officials (see Chapter 7~. The chal- lenge in the difficult environment in which IRBs operate today is to encourage them to make appropriate use of the flexibility in the regu- lations. 143

OCR for page 143
144 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH We begin this chapter by explaining why we think that the Office for Human Research Protections (OHRP) in the U.S. Department of Health and Human Services (DHHS) should develop detailed guidance for IRBs on the designation and treatment of minimal-risk SBES (and biomedical) research. We then discuss guidance that addresses each of the provisions of the Common Rule that are designed to help the ef- fectiveness of the review process for minimal-risk research, beginning with definitions of "research" and "human subject" and proceeding to criteria for exempting research and for using an expedited review pro- cedure. We also consider guidance on effective procedures for ongoing review of minimal-risk research. The last two sections of the chapter discuss the need for better data of two types. First, there is a need for data about perceived risks and actual harms encountered by participants in SEES research to better inform classification of research protocols. Second, there is a need for data on the operation of the IRB system. At present there is no way to track changes in the central tendency or variability of IRB opera- tions, such as how many IRBs are using expedited review procedures appropriately. Our focus on IRBs in this chapter is not meant to imply that they are the only source of concern in the operation of the U.S. human research participant protection system. All actors in the system can and should do more to reinforce commitment to participant protection. In par- ticular, we believe it is incumbent upon the research community to do more than raise concerns about IRB review procedures. Researchers need to be proactive in developing knowledge that will help inform as- sessments by IRBs and investigators of harm and risks for various types of research and help determine appropriate procedures for participant protection in research protocols. GUIDANCE ON THE REVIEW PROCESS OHRP should develop detailed guidance for IRBs on using the mini- mal-risk provisions of the Common Rule for four reasons. First, no IRB can be expected to have the expertise among its members to un- derstand the particular issues and requirements for all of the kinds of research it is likely to encounter, even if the IRB is charged to review research that is clearly in one domain (see Chapter 71. Enlisting other researchers with appropriate expertise to help review individual proto- cols can help (the 1995 Bell survey found this to be the practice of many high-volume IRBs see Chapter 21. But we also see a role for OHRP to assist IRBs to handle a diverse workload by providing specific guid-

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH I45 ance for a variety of research topics, methods, and study populations on such matters as when it is appropriate to exempt a protocol or to use the expedited review process. Second, there is evidence that greater specificity of guidance leads to a greater likelihood that IRBs will in fact follow the guidance. For example, the 1995 Bell survey found that IRBs were most likely to use an expedited procedure, as a matter of standard practice, to review minimal-risk protocols that involved such methods as collecting nail and hair clippings and scraping dental plaque. Taking high-volume and low-volume IRBs together, 75 percent had such a practice for nail and hair clippings, and 66 percent had such a practice for dental sam- ples. In contrast, IRBs were least likely, as a matter of standard prac- tice, to use an expedited procedure to review minimal-risk protocols that involved a drug or deviceonly 26 percent had such a practice (Bell, Whiton, and Connelly, 1998:30~. We believe one likely reason for this difference is that, in the regulatory listing of research eligible for expedited review (provided the research is minimal risk), nail and hair clippings and dental plaque are very specific items, while drugs and devices cover a wide range of (unspecified) items (see Box 1-2~. Hence, we expect that greater specificity regarding minimal-risk re- search involving surveys, interviews, observations, and other typical SEES methods could encourage more IRBs to use an expedited review procedure for such research than do so at present. A third reason for providing more detailed guidance on such mat- ters as exemptions and expedited review is that greater use of the guid- ance could reduce variability in IRB procedures (see Chapter 21. His- torically, reasons for the decentralized structure of the IRB system, in which there are many individual IRBs, each with the power to deter- mine its own procedures (so long as they are at least as rigorous as the Common Rule), were to facilitate local community input to the research review process and promote responsiveness to community norms and practices. Local input is important, particularly when re- search involves community residents as participants. However, we do not believe it likely that differing community norms explain the current wide variation in such practices as exemption and expedited review policies. Reducing variability in IRB practice would not, in our view, undermine the principle of community input and would greatly assist many researchers who move from one institution to another and the growing number who are involved in multisite studies. Finally, at present, only limited, scattered guidance is available to IRBs on ways to use review procedures that are appropriate to the level of risk. Some research sponsors have provided specific guidance to IRBs. For example, the Centers for Disease Control and Prevention

OCR for page 143
146 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH has a document that provides examples of public health investigations that do and do not constitute "research" (Centers for Disease Control and Prevention, 19991. The recently issued National Science Founda- tion (2002) guidance also has some useful examples, but this document is aimed at researchers primarily and at IRBs only indirectly. The IRB Guidebook (Office for Human Research Protections, 1993, available http://ohrp.osophs.dhhs.gov/irb/irb guidebook.htm t4/10/031) provides useful general guidance but gives very few specific examples. We be- lieve that for IRBs to move toward greater use of Common Rule pro- visions that are appropriate for minimal-risk research in the current climate of intense scrutiny of IRB actions will require detailed author- itative federal guidance from a body with broad oversight powers. In our view the appropriate body to prepare and implement such guid- ance is OHRP. GUIDANCE FOR INITIAL REVIEW Recommendation 6.1: To promote review appropriately tailored to risk, the Office for Human Research Protections should develop detailed guidance for IRBs and researchers (with clear examples for a variety of methods) on what kinds of social, behavioral, and economic sciences (SBES) re- search protocols qualify as "research" with "human sub- jects." OHRP should also develop detailed guidance, in- cluding examples, regarding SBES research that IRBs are strongly encouraged to exempt from review and research that IRBs are strongly encouraged to review with an expe- dited procedure. Below we consider some of the issues affecting each decision in the process whereby IRBs determine the type of review to afford a newly proposed protocol and offer some examples of possible guidance for different kinds of SEES research. We look forward to discussion in the research and IRB communities to inform action by OHRP on guidance that includes a range of useful examples. OHRP should issue such guidance as soon as possible and add to it and modify it as appropriate in future years (see Chapter 7 for discussion of a process for developing OHRP guidance). What Is Research? The Common Rule (45 CFR 46.102e) defines "research" as "a sys- tematic investigation, including research development, testing and eval-

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH 147 nation, designed to develop or contribute to generalizable knowledge." Research activities that are part of demonstration and service pro- grams are included, as well as stand-alone research. The Common Rule also states that research may be conducted by a professional or student investigator. This definition has two aspects. First, it refers to a set of activities that involve a human participant in a research experience, that is, a "systematic investigation." Second, it states that, regardless of the na- ture of the participant experience, that experience is research only if it is intended to produce or contribute to generalizable knowledge. Be- cause the line between a systematic investigation and exploratory ac- tivities preparatory to research may be unclear, and because the same investigation may or may not be research depending on the intent, dis- agreements may well arise in specifying what is and is not research. Ate agree that the definition of research should include such devel- opmental activities as pretests and structured focus groups when they are integral to the design and planning of a larger research project (e.g., one or more focus groups to test the psychological sensitivity of survey questions). However, not all planning activities constitute research. Purely exploratory activities, such as contacting represen- tatives of employers to determine their policies for cooperating with surveys of their employees, or asking a few colleagues to react to a questionnaire format, do not, in our view, fit any definition of review- able research. A type of activity involving a "systematic investigation" that may or not be "research" under the Common Rule is research with human participants conducted by undergraduate students in SBES courses in political science, economics, psychology, and other subjects. For exam- ple, students in a course on American politics may be asked to design and conduct a small survey of community residents or fellow students on a current public policy issue. If the sole intent of the exercise is to teach survey construction and not produce generalizable knowledge, the exercise, by the definition above, is not research. Yet if the exact same survey is conducted to gain generalizable knowledge related to the public policy issue (as might be the case in a senior thesis), then the survey qualifies as research. Finally, if the data are collected simply to teach the method but, once obtained, are deemed of sufficient qual- ity and interest to submit for publication, then the same survey would change from not being research to being research. Not surprisingly, IRBs differ in their approaches to reviewing un- dergraduate research. From our review of IRB websites for 47 major research institutions, it appears that 75 percent of these IRBs require that all undergraduate research projects be submitted for IRB review.

OCR for page 143
148 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH Six percent do not have such a requirement, and the remaining 20 per- cent require review of selected projects (e.g., senior theses) or of the instructor's plans for student research for the course as a whole (see Appendix D). We agree with the importance of instilling an awareness of human research participant protection issues among undergraduates, and there is also always the possibility that an undergraduate may design a project that poses a serious concern. For example, a student might design a laboratory experiment to test differences in speed of percep- tion between sober and inebriated fraternity members. Yet as our web- site review reveals, there are likely more effective and efficient proce- dures for needed ethical review of undergraduate research projects whether or not they meet the strict definition of "research"than in- dividual project review by IRBs. For example, the IRB could review instructors' course plans for student projects and have the instructor, who is the responsible principal investigator, provide the individual project reviews (see Appendix D). Alternatively, an IRB under its man- date to educate the research community about ethical requirements for human participant protection could provide a school or department with guidelines for review of student projects which should be of no more than minimal risk. It would then be the relevant department's obligation to conduct the appropriate review in accordance with the guidelines. Projects falling outside of the guidelines would be subject to IRB review. Who Are Human Subjects? The Common Rule (45 CFR 46.1 02f) defines a "human subject" (par- ticipant) as a "living individual about whom an investigator (whether professional or student) conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable pri- vate information." The regulations go on to stipulate that: Intervention includes both physical procedures by which data are generated (for example, venipuncture) and manip- ulations of the subject's environment for research purposes. Interaction includes communication or contact between the investigator and subject. Private information includes in- formation about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and information which has been provided for specific purposes... which the individual can reasonably expect will not be made public (for example, a

OCR for page 143
ENHANCING THE EFFECTIVENESS OF ~VIEW: MINIM^-RISK SEARCH medical record). Private information must be individually identifiable. . . in order for obtaining the information to con- stitute research involving human subjects. Litalics in the original] 149 Deciding when research involves human subjects or participants under the regulations is not always straightforward. In some instances, the sought-after information might not seem to be "about" the indivi- dual for example, asking survey respondents about world events or governmental programs. Yet such responses can inform the researcher about attributes of the individual, such as the person's political stance, attitudes toward authority, or awareness of current events, and, hence, the survey does constitute research with human participants. In other instances, humans may contribute importantly to research projects in ways that do not and should not involve them as participants under the regulations. One example is when humans are asked to pro- vide data about organizations but are not themselves an object of study. A second example is when research is conducted using data that were originally obtained from human participants but that are provided in an aggregate or tabular form from which information about individu- als cannot be recovered. We elaborate further on these two examples below. Research on Organizations SEES researchers study not only individuals and groups, but also organizations, such as businesses and governments. For example, busi- ness economists may seek to relate firm size, measured by number of employees or value of products sold, to measures of innovation, such as spending on research and development or patent applications. Public finance researchers may seek to relate spending by local governments to changes in the business cycle. For such projects, it is usually nec- essary to ask one or more individuals to provide needed information about the organization from relevant organizational records (e.g., ac- counting or personnel files). When individuals are interviewed in their capacity as knowledgeable agents of an organization and not as the di- rect objects of inquiry, and no attempt is made to include any of their characteristics in the study analysis, then the analysis is not likely re- search with "human subjects." Organization research often raises issues that are similar to re- search with human participants in regard to protecting confidentiality (of organizations in this case) and providing sufficient information for an informed decision by the organization about participation. How-

OCR for page 143
150 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH ever, the IRB system is designed to address such issues for human par- ticipants and not organizational entities, which are presumed to be able to protect their own interests. Consequently, IRBs should not use their scarce resources to review research that involves humans solely as agents of an organization. Secondary Analysis with Aggregate Data SBES researchers in many fields conduct secondary analyses of ag- gregate data collected by another researcher, research organization, or statistical agency. An example of aggregate secondary analysis is re- search conducted by geographers and sociologists on migration flows between states, counties, and cities from tabulations of the U.S. decen- nial census long-form sample. Such analysis is at the level of groups, not individuals. As we describe in Chapter 5, prior to publication the Bureau takes steps to ensure that manipulation of the individual cells of a tabulation will be extremely unlikely to reveal the identity of indi- vidual respondents. When aggregate data are obtained for geographic areas or popu- lation groups from a source that does not have a known track record of confidentiality protection (e.g., an individual investigator), then re- searchers may need to seek IRB review before proceeding to use such data in analysis. However, when analysis is planned of publicly avail- able tabular data from such sources as the Census Bureau and other federal statistical agencies, there should be no need for researchers to seek IRB review. The research does not involve individual human par- ticipants and no issues of protection of humans arise. Exemption After determining that a protocol clearly covers human subjects, the next stage in the IRB decision process is to determine if the protocol qualifies for and should be exempted from IRB review. There are six categories of research activities that are eligible for exemption, as spec- ified in the Common Rule (see Box 1-11. The Common Rule does not specify that exempted research must be of minimal risk; however, the categories are clearly focused on minimal-risk research or, in the case of public service evaluation programs, on research that is under the direct review of department or agency heads. As of 1995 (from the Bell survey; see Chapter 2), sizable propor- tions of IRBs were not exempting eligible research in one or more of these categories as a matter of standard practice; 35 percent reported never giving an exemption. Most commonly, IRBs were reviewing re-

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH 151 search that was eligible for exemption with an expedited procedure (Bell, Whiton, and Connelly, 1998:281. Our review of IRB websites of 47 major research universities in late 2002 showed that most will consider granting an exemption, although a few (9 percent) will not. ~ We acknowledge that it can be difficult to judge the appropriate- ness of exempting a particular research project from review even when it appears to be eligible for exemption. For example, the consumer telephone survey example in Box 2-5 (in Chapter 2) could qualify for exemption under category (21: it does not collect identifying or poten- tially damaging information. Yet an IRB might want to review such a survey with an expedited procedure to satisfy itself that the interviewer script provides adequate information to obtain respondents' informed consent. Whether an IRB decides to exempt such research might well de- pend on how effective the IRB believes it has been in providing guid- ance and training on ethical research practices to investigators at its institution. We argue that the development by OHRP of guidance for IRBs on exemption practices could encourage higher percentages of IRBs to exempt eligible research and thereby conserve scarce resources to use on protocols that merit IRB attention and on such activities as development of improved training. IRBs may also want to systemat- ically evaluate their policies and procedures for determining exempt status, such as rotating the duty among members to assess the consis- tency of members' implementation of the IRB's exemption policy, or sampling a subset of exempted protocols to assess the decisions made. Exemption relieves an investigator from the obligation to undergo IRB review; however, it does not relieve him or her of the duty to ad- here to the ethical principles of the Belmont report. Investigators are not empowered to make decisions themselves on whether a research project with human participants qualifies for exemption; they are re- quired to submit the appropriate information to request an exemption. However, some IRBs (33 percent in the 1995 survey) routinely accept an investigator's declaration of eligibility for exemption;2 other IRBs (67 percent in the 1995 survey) independently determine exempt status (Bell, Whiton, and Connelly, 1998:28-291. Below we discuss two kinds of research that we believe IRBs should exempt from review as a matter of standard practice: observational studies of public behavior when the investigator has no contact with participants and secondary analyses of public-use data for individuals 1 Of this 9 percent, half stated that the IRB does not grant exemptions; the other half made no mention of and provided no way to seek exemption. 2It is not clear from Bell, Whiton, and Connelly (1998) whether the chair or the IRB administrator routinely determines exempt status.

OCR for page 143
152 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH (microdata), when the data are obtained from suppliers, such as federal statistical agencies and data archives, that regularly follow good prac- tices to minimize the risk of identification of individuals. Given that confidentiality of participants is protected, these types of research are prima facie of minimal risk because they do not involve any interven- tion or interaction that could pose a risk to an individual participant. Observational Studies of Public Behavior Many studies by social psychologists and cultural anthropologists involve observing the behavior of people in public places going about their ordinary business who are not aware of the observation. In such situations people should be able to expect that they will remain anony- mous, but they should not have an expectation of privacy: they are in a public place (e.g., on a street, in a building lobby, in a govern- ment building, in a public park), which may be observed not only by researchers, but also by journalists, civil or criminal investigators, and casual loiterers or passers-by. This type of observational research generally qualifies for exemp- tion from IRB review under category (2) of the Common Rule listing of exempt research categories (see Box 1-11. For example, the obser- vational study of pedestrians crossing a street (described in Box 2-3) meets the requirements for exemption: the investigator takes notes of what occurs but makes no attempt to interact with the observed pedes- trians. The investigator's notes may record such inflation as the sex, race, age, and type of dress of pedestrians (as determined by ob- servation), but this information would scarcely permit identification of individual participants if it became known to others, and disclosure would not likely place the participants at risk of any legal, economic, or social harm. If the investigator were to photograph or videotape the participants as an aid to analysis, then the photographs or videos could possibly identify individuals. Yet photographic records should not automati- cally exclude the possibility of exemption. Exemption can be granted under category (2) when respondents are identifiable so long as disclo- sure outside the research team could not "reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' fi- nancial standing, employability, or reputation." Secondary Analysis of Public-Use Microdata Much SBES research requires microdata containing numerous vari- ables on individual people or households to use in multivariate analy-

OCR for page 143
ENHANCING THE EFFECTIVENESS OF ~VIEW: MINIM~-RISK SEARCH 153 sis. For understanding such behaviors as voting, labor force participa- tion, and welfare program participation, aggregate data are generally not richly detailed enough to enable estimation of relationships. The problem for the researcher is that collection of microdata from large samples involves substantial up-front costs. The researcher can avoid these costs if he or she can gain access to microdata that someone else has already collected. Beginning in the late 1960s, when computers first became widely used for data analysis, the number and richness of available public- use microdata files from federal statistical agencies and other sources has grown enormously (see Chapter 51. These files, which the source agency processes to minimize the risk that individual respondents can be re-identified, add significantly to the infrastructure for cost-effective research in many SEES fields. Technological developments in particular, the widespread avail- ability of administrative records and other data on the Internet that can possibly be linked with microdata sets, especially when the microdata are richly detailed (e.g., longitudinal surveys) are making it harder to construct public-use microdata files that minimize disclosure risks. These developments may make it necessary in some instances to scale back the richness of the data that are included on a public-use file or to use statistical techniques to perturb the data in ways that may make it harder to estimate relationships. We discuss these challenges and po- tential solutions to them in Chapter 5. Here we argue that individual IRBs should not second-guess the confidentiality protection measures that have been implemented to minimize the risk of disclosure from a public-use microdata file. Instead, IRBs should accept proper certifi- cation from researchers that the file they propose to use is a public-use microdata file as we have defined it and, therefore, that the proposed research (assuming it does not use other, identifiable data) is exempt from further IRB review under category (4) of the regulations. We describe the kind of certification procedures we propose in Chap- ter 5. They are similar to those recommended by the National Human Research Protections Advisory Committee, which acted in response to a presentation from its Social and Behavioral Science Working Group at a committee meeting, January 28-29, 2002 (see http://ohrp.osophs. dhhs.gov/nhrpac/documents/dataltr.pdf t4/10/031~. Some might argue (as the working group did initially) that the use of certified public-use microdata files is non-human-subjects research on the grounds that human participants are not identifiable. We argue instead that such re- search qualifies as research with human participants because the risk of individual identification on public-use microdata files, while close to zero, is never zero. Therefore, it is appropriate for IRBs to be in-

OCR for page 143
154 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH formed of proposed studies of public-use microdata files from a certi- fied source, but they should exempt such studies as a matter of standard practice because participants have already been protected to the extent possible. Expedited Review If a human subjects research protocol does not meet the criteria for exemption, the next stage in the IRB decision process is to deter- mine if it qualifies for expedited review. Expedited review means that the IRB chair or a subset of other members (or both) conducts the re- view, rather than the full board at a meeting. An expedited review is as comprehensive as full board review in the sense that the IRB chair (or subgroup) follows all of the applicable requirements of the Common Rule. However, it can be conducted in a much more expeditious man- ner with regard to elapsed time, and it saves on the time required for IRB business totaled over all IRB committee members (see Chapter 21. The Common Rule authorizes publication in the Federal Register of a list of categories of research that, if the research is minimal risk, may be reviewed by an expedited procedure. The list may be amended, as appropriate, from time to time. The original list of such research was published in 1981, and the list was revised in 1998 (see Box 1-21. To qualify, the proposed research must be minimal risk, must fall into one of the approved categories, must incorporate steps to minimize the risk of breach of confidentiality in instances "where identification of the subjects and/or their responses would reasonably place them at risk of criminal or civil liability or be damaging to the subjects' finan- cial standing, employability, insurability, reputation, or be stigmatiz- ing" (see Box A-5 in Appendix A), and must not be classified. We focus our attention on category (7), which relates most directly to SEES research. It covers "research on individual or group char- acteristics or behavior (including, but not limited to, research on per- ception, cognition, motivation, identity, language, communication, cul- tural beliefs or practices, and social behavior) or research employing survey, interview, oral history, focus group, program evaluation, hu- man factors evaluation, or quality assurance methodologies." At the time of the 1995 Bell survey, fully 51 percent of IRBs, as a matter of standard practice, did not expedite any minimal-risk SBES research of the types included in category (71; 15 percent of IRBs did not expe- dite research in any category. Our review of IRB websites of 47 major research universities in 2002 showed that 13 percent did not expedite research in any of the eligible categories.

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH 155 We present examples of laboratory behavioral research, interview research (specifically, oral histories), and survey research that we be- lieve most likely qualify for expedited review and are candidates for OHRP to include in guidance to IRBs. The intent of OHRP guidance should be to provide a level of specificity comparable to that provided for biomedical research in categories (1) through (4) of the current list of research eligible for expedited review (see Box 1-21. Laboratory Behavioral Research Box 2-1 in Chapter 2 provides two examples of types of laboratory research that readily qualify for review with an expedited procedure. The first example is an experiment about economic decision making (e.g., reaching agreement on terms of exchange). A small number of participants are brought together and given precise, detailed instruc- tions about how they are to interact and how they will be rewarded on the basis of their decisions and the decisions of other participants. They are informed they may leave at any time. Their decisions are recorded, and they are rewarded accordingly (in private, anonymously, and after the experiment). Reward amounts are small. No personal identifiers are kept. There seems to be no reason for full board re- view of such a study except when the proposed reward amount exceeds some thresholdwhich the IRB could specify- that could reasonably be viewed as inappropriate. Indeed, perhaps the only reason for the IRB to review such studies at all is that they commonly involve stu- dents, and the IRB may want to assure itself that students are not pres- sured to participate. The second example is a social psychology experiment with decep- tion. The research question is the extent to which people engage in ethnic stereotyping. A small number of participants are brought to- gether and told that the purpose of the experiment is to determine how fast people can associate characteristics (e.g., good, bad) with lists of names (which differ in cues about ethnic origin). They are told they may leave at any time. Their results are recorded, and they are told at the conclusion of the experiment about its true purpose. No per- sonal identifiers are kept from the experiment. Such experiments, even though they involve deception, do not require full board review. The stimulus (list of names) is not threatening, the only deception involves the purpose of the stimulus, and participants are fully debriefed. More- over, the risk of breach of confidentiality is not an issue. In contrast, if the stimulus were threatening (e.g., aggressive behavior from inves- tigators masquerading as participants), then the study would warrant full board review.

OCR for page 143
156 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH Interview Research Oral histories constitute a type of research using unstructured or semistructured interviews that often qualify for review with an expe- dited procedure. Most such histories are designed to obtain informa- tion from an individual participant that the participant agrees will be released in full (or in part) at a specified time according to a legal agree- ment. The participant expects confidentiality only until the date when he or she has agreed that the interview may be made publicly avail- able for historical research and other specified purposes (e.g., use in a television documentary). In the 1998 revision of the list of expeditable research, oral history was specifically included in the list of types of SBES research in category (7) that IRBs could review with an expe- dited procedure.3 The Oral History Association has developed extensive guidelines for the conduct of oral histories on such topics as informed consent, archiving and protection of confidentiality, legal releases governing when interviews may be made available, and many other aspects of ethical research using this method (see "Principles and Standards" and "Oral History Evaluation Guidelines" at http://www.dickinson.edu/oha t4/10/200311. We believe that OHRP guidance could draw from these guidelines to encourage IRBs to develop a checklist that would allow for expedited review of many types of oral history as a matter of stan- dard practice. The checklist could focus on the major sources of risks to human participants from oral histories the risk of breach of con- fidentiality during the period before authorized release, the risk that some participants may not fully understand they have agreed that their personal histories eventually will be publicly available, and the risk that third parties who have not given consent may be adversely affected- and responses from investigators used accordingly to determine the type of review. Survey Research Many SEES surveys contain financially or psychologically sensitive content or raise other issues (e.g., third-party consent- see Chapter 4) that merit full board review by an IRB. However, many surveys could just as effectively be reviewed with an expedited procedure, qualifying for such review under category (7) of the current list of expeditable research (see Box 1-2~. 3 Oral history qualifies for exemption when it is conducted with elected or appointed public officials or candidates for public office.

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH 157 Box 2-5 in Chapter 2 provides an example of a minimal-risk survey that qualifies for review with an expedited procedure (or even exemp- tion from review). The survey is a short telephone survey of a sample of one adult in each of 1,000 households selected by random digit dialing. The content pertains to expectations about the state of the economy. Minimal content is collected about the household itself (e.g., number of members), and no identifiers are obtained. Even longer and more complex surveys on such topics as attitudes and expectations could be effectively reviewed with an expedited pro- cedure, if they do not retain identifiers and the questions being asked are not embarrassing or threatening. Examples are the University of Michigan Survey of Consumers and the Conference Board Survey of Consumer Attitudes and Buying Plans. The Michigan survey (see http://www.sca.isr.umich.edu t4/10/031) interviews 500 people by tele- phone each month, drawing the sample from a list of household tele- phone numbers. Households are recontacted one more time 6 months after the original interview. The Conference Board survey (see http: //www.conference-board.org [4/10/031) sends a mail questionnaire to 5,000 people each month. The two surveys ask similar questions. The Michigan survey includes about 50 core questions. The majority of the questions are about the respondent's expectations on such topics as where the economy is heading and whether the household's in- come is likely to go up or down; a few questions ascertain basic demo- graphic characteristics, including household size, number of children, and marital status, race, and education of the respondent. Were it not for the fact that the survey has a re-interview procedure (to produce stability in the estimates), which necessitates retaining telephone num- bers, and is an ongoing survey that is subject to change, it could well be exempted from IRE review; in fact, it receives an annual expedited review (Rebecca McBee-Bonello, Survey Research Center, University of Michigan, January 28, 2003, personal communication). CONTINUING REVIEW Recommendation 6.2: Institutional review boards should use efficient procedures to review minor changes to minimal- risk research protocols that arise during the period of autho- rization. When appropriate, IRBs should approve protocols that allow researchers flexibility in making specific design decisions during the course of their research without the need to seek further review. (An example would be one of two forms of a questionboth minimal risk to be decided on the basis of a pretest.)

OCR for page 143
158 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH The Common Rule requires IRBs to "conduct continuing review of research covered by this policy at intervals appropriate to the degree of risk, but not less than once per year" (45 CFR 46.109c). Another provision (45 CFR 46.110b) permits IRBs to use an expedited proce- dure to review "minor changes in previously approved research during the period (of one year or less) for which approval is authorized."4 Re- searcher concerns about these provisions are similar to those about the functioning of IRBs in general: that continuing review is perfunc- tor~r and not sufficient for human participant protection and that the requirement for IRB approval of changes is cumbersome and imposes needless delays, even when the changes are minor and the review is expedited. Below we address some issues and suggest some ways to facilitate review of minor changes to minimal-risk research. In our discussion of common SEES research methods in Chap- ter 2, we note that one such method involves unstructured (or semi- structured) interviews in which the content and scope of the question- ing evolves over the course of the project and may differ for different participants. It would be highly disruptive to the research to bring ev- ery such change back to an IRB for approval. What is needed, instead, is for an investigator to provide the IRB with information at the time of initial review about contingencies that may arise. Perhaps the informa- tion provided could include illustrative interview paths that the inves- tigator anticipates may be taken by different kinds of participants. If the research is minimal risk and the anticipated contingencies are also minimal risk, the IRB should approve the research without requiring the investigator to bring back every change in measurement for review. Instead, the investigator should be charged to come back to the IRB if the measurement evolves in unexpected ways that could raise the risks to participants. A similar situation arises in a survey when the investigator is un- certain about which version of a question will better elicit the desired information and wants to conduct a pretest or focus group analysis before making a decision on how to word the question. Often, it will be reasonable for an IRB to require review of a pretest and a sepa- rate review of a final survey questionnaire. However, in the case of a minimal-risk survey in which the scope of the pretest is limited (e.g., a few questions) and both versions of the questionks) being tested are minimal risk, then the IRB could approve the protocol in one review operation. That is, a reasonable option would be to approve not only 4IRBs may also use an expedited review procedure to conduct annual reviews of research that is in the data analysis phase and of research that does not fall under cat- egories (2) through (8) see Box 1-2 but that the IRB has previously determined to be minimal risk.

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH 159 the pretest, but also the survey, with the understanding that the sur- vey may use either version of the questionks) that is supported by the pretest results. Finally, IRBs should seek to learn from the practices of other IRBs that have particularly efficient methods for handling the review of changes to research protocols. For example, some IRBs allow re- searchers to inform the IRB of a change and, in turn, for the IRB to approve the change, entirely by electronic interchange. Electronic communication methods can undoubtedly facilitate IRB initial reviews as well, to reduce the time from submission of a protocol to a decision about the type of review and the time required in negotiation to pro- duce an acceptable protocol. DOCUMENTING RISKS AND HARMS Recommendation 6.3: In order to build knowledge of re- search risks, OHRP and funding agencies should encourage researchers to build into their studies such steps as debrief- ing participants to learn about types, incidence, and mag- nitude of harm encountered in social, behavioral, and eco- nomic sciences research. Researchers should seek publica- tion of their results. As we discuss in Chapter 2, there is very little evidence on the distri- bution of harm experienced by participants in SBES research. Some types of SBES research, such as surveys, are believed by some inves- tigators to have been conducted for decades with no serious physical or psychological harm to respondents. There are examples of stud- ies in which psychological harms occurred (see Box 3-1 in Chapter 3), but whether the research caused long-lasting harm is not known. There are also examples of research that demeaned participants and did not respect their autonomy. Yet there is hardly any quantitative data on risks and harm of SBES research that could inform policy on human research participant protection or contribute to improved re- search practice. We believe it is incumbent on SBES researchers, as part of sound, ethical research practice, to adopt procedures to debrief respondents (or samples of them) on their perceptions of risk and experiences of harm and to include the results in published accounts of the research. For example, survey respondents could be asked whether they felt co- erced to participate, whether they found any questions to be troubling, whether any adverse reactions they experienced to questions were in- tense or expected to linger, and so on. Such debriefing could be part

OCR for page 143
160 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH of the study itself or conducted as a follow-on. Post-study interviews or debriefings would need careful design to minimize response bias (e.g., participants might feel compelled to minimize or, alternatively, exaggerate their experience).5 Debriefings and post-study interviews would be respectful to partic- ipants. If results are collected and disseminated, they would also con- tribute to knowledge about risks and harm in SEES research. In turn, such knowledge could help OHRP formulate guidance for IRBs on such practices as exemption and use of expedited review and thereby facilitate research by reducing the variability in how IRBs handle these aspects of review. Toward this goal, we urge funding agencies and OHRP to encourage researchers to build participant assessments of risks and harm into their study designs. ONGOING DATA SYSTEM Recommendation 6.4: The Office for Human Research Pro- tections should establish an ongoing system for collecting and publishing data that can help assess how effectively IRBs protect human research participants, how efficiently they review research, and how commensurate review is with risk. There is astonishingly little hard information about the operation of the IRB system for human research participant protection in the United States today and how IRBs are interpreting provisions of the Common Rule. Research institutions are required to provide informa- tion about IRB membership to OHRP in order to obtain a federal-wide assurance (EWA) that authorizes the IRB to operate.6 IRBs must also retain their meeting minutes and other records for a specified period, and they must report any instances that investigators report to them of harm to a research participant (adverse impact reports). However, IRBs are not required to submit regular reports of activities, such as meetings held, protocols reviewed, disposition of protocols, or other 5 participant debriefings are not meant to substitute for the reports of specific harms that investigators are required to make to IRBs upon occurrencefor example, if a sur- vey respondent were to experience a panic attack in response to questioning, the inves- tigator should report that harm immediately to the IRB. In turn, the IRB is required to report specific harms to OHRP. 6 previously, institutions had to provide extensive documentation of IRB procedures in order to obtain a multiple project assurance. The FWA procedure is designed to reduce the burden of assuring that an institution will comply with the Common Rule.

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH 161 information that would provide a basis for monitoring their workloads and decisions.7 Furthermore, over the 35-year life of IRBs (and predecessor re- view committees), only a handful of major surveys have examined their characteristics, performance, and effects on human participants and research projects. Findings from these surveys are not often compara- ble, and most findings are not reported by type of research. The most recent comprehensive survey of IRB operations was in 1995, and it has significant deficiencies, including a restricted study universe and the absence of such important information as compar- isons of informed consent procedures and forms before and after IRB review; comparisons of IRB and investigator assessments of harm and risk; breakdowns of results by specific field of research; and any harm reported for participants and how it relates to type of IRB review, such as full board or expedited review. Moreover, and, in our view, inex- cusably, complete documentation for the Bell survey is not available because of lack of funding. Other surveys and case studies reported in the literature are even more out of date and limited in focus (e.g., sur- veying particular types of IRBs, research, or investigator disciplines; see Appendix D). The limited available evidence, when carefully analyzed, is suffi- cient in our view to support a conclusion that IRBs are indeed over- loaded and, furthermore, that some of the overload is the result of unnecessary resources being spent on minimal-risk research. Yet sub- stantially better information would be required to monitor and refine the operations of the IRB system in the future. We see a need for an ongoing federally funded data collection pro- gram that will provide regularly updated useful information on IRB characteristics, operating procedures, and outcomes. (Such a pro- gram is in addition to the systematic research that we recommend re- searchers undertake about perceptions and experiences of risks and harm by research participants.) Our sister Committee on Assessing the System for Protecting Human Research Participants also recog- nized this need. It asserted, "a fact that has repeatedly confounded this committee's deliberations is the lack of data regarding the scope and scale of current protection activities" (Institute of Medicine, 2002:41. It called for DHHS to "harmonize safety monitoring guidance for re- 7Some agencies have captured basic information, such as dollar funding and num- ber of participants, on research they support that involves human participants. For ex- ample, the U.S. Department of Energy has a database on 300 such projects conducted or sponsored by the department since 1994 (see http://www.eml.doe.gov.hsrd [4/10/03]). However, there is no systematic database on the thousands of projects using human par- ticipants that are sponsored by DHHS.

OCR for page 143
162 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH search organizations, including standard practices for defining and re- porting adverse events," "issue a yearly report summarizing the re- sults of research monitoring activities in the United States, includ- ing OHRP and FDA findings from inspections conducted the previous year," "commission studies to gather baseline data on the current na- tional system of protections for research participants," and "assemble data on the incidence of research injuries and conduct economic anal- yses of their costs." We believe an OHRP data system on IRBs should include several types of indicators, including basic descriptive characteristics (e.g., number of members, disciplines represented, number of initial exemp- tions, expedited reviews, full board reviews); measures that could be used to assess performance (e.g., average elapsed time to complete ac- tion on protocols by type of research and type of review, percentage of expeditable and exemptible research that is expedited and exempted); and outcome measures (e.g., reported harms and reasons for their oc- currence). This data system would have its greatest positive effects on human participant protection and researcher conduct if it were viewed as the monitoring device for a large production processin this case, the pro- cess of reviewing protocols by IRBs. By design, CHIRP seeks some variation, reflective of community standards variation, and it does not want to increase threats to human participants by overly loose review nor to impede the progress of science by overly harsh prohibitions of research. The data system might provide a tool to measure both the average behavior of IRBs and the extent to which some IRBs are de- parting from that typical behavior. When the averages suggest too loose or too harsh reviews are occurring, OHRP might seek to clarify guide- lines. When the variation in behavior across IRBs becomes too large to be reflective only of differences in community standards, OHRP might seek to learn why the IRBs showing unusual behavior are doing so. For example, imagine the simple display of the distribution of the proportion of protocols that receive expedited review that appears in Figure 6-1. The figure shows a hypothetical distribution of expedited review at two time points. At time 1, most IRBs are expediting between 50 and 80 percent of eligible protocols (the boxed area denotes this), but there are some IRBs that expedite as few as 20 percent and some that expedite as many as 90 percent. OEIRP might view this variation as problematic, examine the outliers through informal conversations with IRB chairs, and clarify guidelines. At time 2, following the inter- vention by OHRP, the variation in IRB behavior is much lower, with most IRBs expediting between 60 and 70 percent of eligible protocols and the outliers expediting 30 and 80 percent. Such a change would

OCR for page 143
ENHANCING THE EFFECTIVENESS OF REVIEW: MINIMAL-RISK RESEARCH o o _ o 0- = 1 1 o _ l o _ . o- 1 1 lime 1 Time 2 Figure 6-1 Boxplots for Hypothetical Proportion of Expedited Reviews Across IRBs NOTES: These are hypothetical distributions for the proportion of protocols that would receive expedited review before and after a proposed change; see text for discussion. 163 signal to OHRP that the intervention led to more consistency across IRBs. We recognize that it would be easy to interpret our recommenda- tion as requiring the immediate imposition of burdensome paperwork requirements for every IRB. That is not our intention. At this time we propose that ORHP develop a relatively small-scale system which collects a limited set of data from a sample of IRBs, stratified by re- search volume at the institution. OHRP should consult with IRBs, re- searchers, and other relevant organizations to determine data priori- ties and the most feasible methods for obtaining the needed data items from sampled IRBs. As experience is gained with the system, it may be possible to expand its sample size and scope by taking full advantage of electronic communication technology. IN-DEPTH STUDIES Recommendation 6.5: Federal research funding agencies, including the National Science Foundation and the National Institutes of Health, should fund in-depth studies to better understand the operations and effects of the IRB system and to develop useful indicators of IRB performance. It is not easy to devise performance measures for IRBs and other indicators that are feasible to collect, easy to define in a consistent

OCR for page 143
164 PROTECTING PARTICIPANTS AND FACILITATING SOCIAL AND BEHAVIORAL SCIENCES RESEARCH manner among IRBs, and provide useful information. Moreover, for cost and feasibility reasons, an OHRP data system will necessarily be limited in the indicators that are regularly collected. For these reasons, we see a need for federal research funding agencies to fund in-depth studies on IRBs. Such research might use anthropological techniques, reviews of documents, and surveys (possibly including a longitudinal component) to examine various aspects of IRB functioning for differ- ent types of IRBs and kinds of research. The expectation is that a sys- tematic research program would, in time, lead to better understanding that could inform the development of regularly collected performance measures and help guide policy for the operation of the IRB system. As we noted in Chapter 4, the National Institutes of Health recently had a research program to study aspects of IRB operations, such as informed consent. This program should be continued and expanded. The National Science Foundation should similarly fund IRB-related in- depth research with a focus on SEES research.