Click for next page ( 69


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 68
APPENDIX C PANEL 1 Emit OF IlIE PANEL CN IA1~R~ORY UNIT AND PRACrI~ Ihe parer was aomrened to discuss ways to inprc~re Arrant laboratory ream practices ~ the academic ~viro~nt. me namers indicated that two goals were the Hit desirable: ill!praving the overall quality of research as currently practiced by university investigators and also remedying conditions that mav fail to discourage Misconduct in the rearm press. Sixteen members participated in the panel digressions, most of whom were practicing scientists (primarily Ph.Ds) covering a wide range of fields, including molecular biology, biophysics, biochemistry, and science and law. The group also included several senior research administrators, including two academic research center directors, two vice-president/vice- chancellors for research and graduate deans, a chief scientific officer of a major private research funding agency, a laboratory chief frown the National Institutes of Health, and staff members frog the Academy. The panel addressed a broad set of topics during its discussion. These topics included the handling, storage, and archiving of research data; existing models for quality control in science, including regulations governing the use of animals, the handling of hazardous materials, and experiments with recombinant DNk technologies; the applicability of the Good Laboratory Practices (GLPs) guidelines used by the Food and Drug Administration and the Environmental Protection Agency; the value of external admits in basic research; training and supervisory practices; the relevance of lab size and structure to research ~ntPoritv authorship practices; the significance of formal and informal rules in achieving good laboratory practice; the communication of high standards of research practice; and the relevance of good laboratory practices to fields of research beyond the biomedical sciences. _ _ =_ _ _, , In these discussions, the panel numbers identified problems in evaluating the effectiveness of current mechanisms designed to promote integrity and quality in academic research. participants thought that instances of fraud in science are rare but that sloppy research practices are more common than would be desired. The . A majority of the pane' members expressed concern that the relatively rare instances of scientific fraud might provoke regulatory reforms that would be inappropriate for improving the conduct of responsible research practices and that would also threaten the vitality of creative research. Most of the panel members agreed that substantial improvement in laboratory practices could be achieved mast readily through mechanisms that improve oversight of the work of those directly involved in experimentation. An appreciable member of the participants thought that certain reforms Should be initiated by the universities and professional 68

OCR for page 68
organizations to improve He effectiveness of existing oversight i~. A mirror expressed Sambas that these reform were redry, indicatir~g that the monitoring In worked well, that peer refries was highly su~:ful, are that evic3~e of serials prciblems in the conduct of r~r~ was insufficient to justify Changes;. A smaller minority support Ire emotive reforms, including the use of vernal auditing as a Ned; of assuring Cavity in academic i. (300D IA=R~O~ PRACTICE GUIDELINES AND 1~ USE OF AlJDlI~ ~ of the part participants express serials reservations abet the need for and in act of legislative or r ~ atory actions In improving the quality of basic research practices. For example, the panel considered the rules associated with the FDA and EPA GLP guidelines that regulate c hemio=1 and drum testing. Thea- GLEs describe in detail the _ _ procedures to be followed during the tests and include periodic agents by a regulatory agency to ensure adherence to the guidelines and the study protcool agreed upon at the inception of an experimental study. Federal regulations governing the handing of radioactivity and other hazardous materials, He manipulation of recombinant He, arm the use of animal and human research subjects also rely extensively upon oversight of research protocols. me panel believed this rotatory approach was impractical and injuries to He conduct of good science in fields ir~rolving basic or discovery res ~ and noted that the agencies that use GLPs had rejec their application to basic research. A regulatory approach based on oversight of the research protocols requires that a detailed canon of _~ a, research practices He establlsned, that detailed research plans be drawn up at the initiation of the study, that deviations from the seedy protocol be justified in advance, and that adherence to these practices and protocols be monitored through periodic agents by responsible parties outside the research unit. m e agents might be conducted by granting agencies, by governmental ablating agents, or by institutional officials empowered to conduct such procedures. These plans were viewed as unworkable by the panel. such of the research in question concerns basic discovery research, and it necessarily involves constantly changing protocols and research procedures. Adherence to a preplanned search trajectory would hamper basic science a ~ foreclose the pursuit of unexpected but promising findings. The auditing bodies will be unlikely to acquire the n cessary expertise to monitor effectively the activities of a large and diverse array of specialized research enterprises. Perhaps most important, the concept of rout~nized institutional auditing of the details of basic research procedures undermines the principles of trust and personal integrity that currently govern the conduct of academic research and the professional status of its practitioners. 69

OCR for page 68
me panel explored a second regulatory strategy involving a sbr~ of existing peer review presses. me pass of peer review oars at several distinct levels, arm a majority, of me panel agreed Cat each of these cold benefit Frau Potential i~r~rer~nt. Peer Review within the Pram laid me most intense surveillance of ~a~ prac~ti~: Occurs on a fr~ent and ongoing Uris within me confines of the r~i ~it. review of this work cams on a ~ of practices, many of math are implicit or intrinsic in the correct of scientific iT~restigation arm experimentation. mese include: 0 instruction of learn pE=sorme1 in how to primely design experiments and collect experimE~n~cal eta; o insis ~ :e that raw data be preserved within no ~ books; and offer repositories so that they are clear and readily infer pret=d by qualified researchers beyond those directly responsible for conducting the retrench ~ In ques ion; o frequent interaction between the principal investigator of the research unit and the individuals conducting the research. Effective interactions require frequent review of raw data and interpretation of their implications; and o frequent interactions between the experiment~1ists within a research unit involving those not directly concerned with the research in question. The panel thought it desirable that research units conduct frequent, periodic semurars in which investigators and trainees are able to scrutinize and critique each others' work. Conversely, laboratory practices that encourage corpartment~lization, secrecy, or isolation within the research unit were viewed as incompatible -with the conduct of good research. Moreover, the panel sought to discourage practices in which experimenta~ists would describe their work only to a supervisor and not routinely discuss findings with their peers within the laboratory. Peer Review Within the Department Most research units are affiliated with a larger administrative unit, usually termed a "department." The panel thought it important to recognize that departments have an essential role in monitoring the integrity and quality of the science within their constituent research units. Intradep}rtmenta1 review may not be invested with the same degree of specialized expertise that resides within the research unit. itself, but it should play a useful and essential part in assuring quality. The panel members indicated that departmental affiliation implies interest and 70

OCR for page 68
willingness to subject one's research to the scrutiny of other departmental membered including those outside one's own research group. A variety of mechanisms can be used to achieve this end, including: 0 ~rn:a1 Irk Emit, 0 joint research meetings involving: multiple research uniter 0 annual or semi-annual reports of investigators' work, a ~ o other means of engaging contact between members of different research units, including participation in joint training programs. This approach assumes that such review and scrutiny should be undertaken as a matter of common practice and not only on rare occasions and in response to suspicion of implications of poor quality or misconduct. Md~Na1 review performed in a constructive and collegial manner was viewed by the pane] as a normal, routine element of good science and a characteristic of an active and productive research department. Review of the Department by Outside Peers Because the departments exercise a central role in the existing peer review produce' the panel considered mechanisms to ensure that departments and their component research units function in an appropriate manner. One useful mechanism is the use of periodic review, often performed every several years, by peers outside a department. mese peers may cam_ from related departments within a university or fort other universities in the form of "visiting committees." mese departmental reviews often include: 0 surveys of representative research projects, o surveys of the entire output of the department, arid 0 interviews with trainees, supervisors, and principal investigators, o preparation of candid reports for department heads and higher administrative personnel, including deans, provosts, and the president of the university. Review of Published Work by Peers The three levels of peer review discussed above monitor the everyday process of science within The laboratory. These activities include data acquisition and interpretation. The panel felt that the end point of this works the formalization of results in a published report also requires an increased level of peer monitoring. 71

OCR for page 68
m is monitoring of published work begins at The stage of drafting reports for event publication. me panel thought it essential that all parties contributing to the report and appearing as authors be involved in its creation. In certain Hasp=, this sight involve only a detailed reading of the report prior to submission. In others, co-authors ~ ght redraft portions of the text. In all cases, the panel staled that participation as author implies a detailed understanding and agreement with the report and a certification that the results contributed be an author are properly represented by the finished text. _ . , _ ~ ,~ ~ . A subsequent level of manuscript review occurs by qualified peers following submission for publication. me panel did not micros= changes in this prone== as presently constructed. mey did focus attention on the post-publication prices the scrutiny of a published report by peers throughout the world. me central effectiveness of this scrutiny relic= on the ability of peers to reproduce or replicate published results. In the end, the panel believed that independent reproduction represents the ultimate and most credible measure of scientific veracity. SHARlNiG I, ME~)D6, AND Rat the panel indicated that authors of published work have a traditional obligation to aid scientists interested In independent replication. m is obligation includes the duty to assist in providing raw data, if requested, and access to the methods and reagents necessary for reproduction. the panel further noted that this obligation is usually an unstated assumption in academic research and that there may be a need to develop written policies to clarify and strengthen traditional practices. the pane] observed that whether or not written guidelines were developed, certain practices needed to be addressed to improve the fulfillment of data retention and sharing duties. Raw Data In certain cases, questions frail peers may arise that can only be adequately addressed by exam m mg the raw data that underlie a published report. These data should be written in a form that is subject to independent reexar mation, readily interpret~hle by qualified practitioners in the field. Moreover, the data need to be preserved for reexamu nation for a period of time after publication. Several panel members suggested that five years seemed to be an appropriate period for data retention. Equally important is the implied readiness to supply these data to a qualified peer. The panel indicated that customary practice attaches two conditions to this access: o The party requesting arc=== to raw data should have a well-articulat=d justification for reexamination of the raw data and a history of experience in the scientific speciality in question or in the 72

OCR for page 68
examination of closely related problems in the field. This experience ensures qualification to interpret critically the data requested. O The data requested should directly underlie a scientific conclusion that has keen questioned. Requests for data not associated with published reports are seen as improper and may intrude on the rights of an investigator to man individual control of preliminary results. Moreover, requests for raw data not directly related to the narrow issue in question are seen as inappropriate. The panel recognized the need for safeguards to prevent requests for wide-ranging, blanket surveys of data records stored In a laboratory's archives. These surveys are seen as an unreasonable imposition on an investigator's time and resources and represent unjustifiable and arbitrary intrusions into the workings of a scientist's activities. ~ pn-==c to Methods and Reagents Publication of methods in a fashion enabling independent replication is a well~ablished procure and requires little elaboration. A more difficult problem is the customs that govern access to reagents mercury for ~qprc~uction of the work. In most instances these are readily available commercially or through wide distribution among laboratories; In some instances they are not. The panel suggested that investigators have a responsibility to share with qualified peers attempk~ng reproduction of published work the reagents that are ~C=pntial to the independent reprc~uction of the work when these reagents are not generally available. It is believed that many big; research journals, such as Cell, have formal guidelines requiring this practice as a condition of publication. Panel members identified several considerations that may govern the sharing of reagents. o m e reagents should be made available within a reasonably short time after publication, such as several months. O The reagents may be distributed to academic research for their own `~=, but investigators should not be expecbel to share them with industrial concerns. O The amount requested should be minimally essential for reprc~uction of published results. O The request should not impose an unreasonable supply burden on The original laboratory. O Certain requests for distribution may be unreasonable. These would include reagents available only in small amounts, such as unique antiserum or animal strains bred in small number, or otherwise unique and not susceptible to distribution. Certain animal populations, as well as human clinical sty pupations, for example, cannot be ~ to be shad - . 73

OCR for page 68
Rainier are S~;lsory Lactic The panel participants did training ark supervisory practices and rate that many principal investigators may be inad~at~ly trained to manage a Heard group. This was thought to be Ire camnan in bile case of ~ restigal;ors who receive research train) ~ In a ~ ical school rather than a graduate school program. There was also significant discussion about the effect of size of research grc ups on the quality of academic research. Recognizing that individual investigators are competent to direct the activities of a limited number of people, the pane] hesitated to suggest an effective limit because of the wide variance of managerial competence among investigators. The panel notes with concern, however, that when an investigator heads a group that exceeds his or her competence, these circumstances heighten the possibility for inadequate control, sloppy research, and fraud. He panel suggested that each laboratory employing trainees and students have a set of generally accepted good laboratory practices defined for the members of the research staff. There was consensus that it would be important and useful for a national research organization to describe the essential elements of these practicer in a document that could be provided to all investigators and trainees involved in discovery research. INSTTTUTION~L OVERSIGHT me panel observed that various institutions place significant pressures on invesigators which, if carried to the extreme, threaten the responsible ~nduct~of research. mese pressures include the perception that administrative officers value numbers of publications and the accumulation of research dollars from external funding agencies as the most important indicators of research quality. me panel also noted that spots oring instigations and universities have a right to expect mechanisms to be in place to ensure and improve the reproducibility of published work. In particular, panel members suggested that the instibution needs to recognize the crucial role an] influence of The laboratory director. Academic officers need to provide guidance to Their lab chiefs about the probe ems of overcommitment, distraction by other activities, and how to resolve individual variations In the quantitative arcs of laboratory direction. Finally, the panel stated that universities, medical shoots, hospitals, and other ~a~ aerate; have a Pica re~ponsibili~ for He r~ ~viro~nt of Heir faculty and staff. There was general agreement that the officers of th ~ institutions need to assure themselves and others, when questioned, that the environment for their research faculty, staff, and students is intellectually, structN rally, and administratively supportive of the conduct of good science. Chair: Robert Weinberg 74 Rapporteur: R. William Butcher

OCR for page 68
PANEL 2 hi OF THE PANEL ON CLINICAL REFLEX ~Ac]'lCF~ AND ~ lee panel e~i~ issues file rearm practice and training, with specific relevar~ce to clinical hi. hue main topics Aged ding its digressions: the diverse nature of clinical r~z=, the adequacy of training, and the efficacy of existing monitoring Anise;. m e panel included research directors and scientists frum academic, industrial, and government research centers; the chairman of medicine and faculty members from major medics schools; the director and staff members from professional associations representing academic health centers and clinical investigators; and the editor of a journal on institutional review boards. In addition to medical specialties, the participants included two sociologists an] a bio6~atistician. IHE DIVERSE NATURE OF CIINIC~L RESEARCH m e panel examined the diverse nature of clinical research, which may include both physicians conducting original research and those conducting therapeutic drug trials. m e former group represents the focus of the discussion; the latter often have technical rather than scientific responsibilities. Ire panel indirect that scientists To conduct reseal in a clinical setting should be guided bar the same principles in research as their colleagues In the university. me rest of the clinical scientist, hammer, may be subject to additional regulations associated with human subjects research. Given this distinction, clinical rearm on human subjects is still c~xr~licat~l by diversity. For ex ~ le. clinical trials sc~reti ~ ; involve multiple institutions and Large numbers of subjects. These studies are quite different in scope and execution from more conventional studies within a single institution on a limited number of individuals. Behavioral research on humans is inherently different from trials of Therapeutic agents in a specialized medical unit. The principles and practice of chaining "informed consent" are more straightforward in some experimental circumstances than in others. In the conduct of research on humans, conflicting pressures can arise if the established experimenter design entails a risk to the welfare of the patient. This prospect is minimized by the careful design of experiment=] proboools so that guidelines for behavior of the investigator will prevent loss of scientific Objectivity. Nonetheless, despite the best experimental design, DeSeBrCh on humans often has to strike a riddance between patients' rights and welfare on the one hand and the scientific endeavor on the other. m e panel discussed a third feature of research on humans: it is often conducted in a "fishbowl," in full view of many parties with diverse 75

OCR for page 68
it. Ihis is er~tirely app' - riate. lever, it Is sciatic difficult to transmit to all i~t~ parties, professional arx] lay, ff`e nature of he r~ and he balance between rip art benefit. ha: ADB~CY OF RESET A: Not intently, those Crouching research on hens are physicians trained In a postdoctoral pa ~ n within a m ~ Iran school. As a rule, this training IS supervised by an experienced individual, often termed a 'mentor,' rather than a department. As a result, the nature of training for research within a specialty can vary widely. Panel members suggested that the sponsoring institution often plays only a minor role in the training of a physician-scientist. The background of the physician-scientist before undertaking research training also bears upon the behavior of the clinical investigator. Premedical education is a highly competitive undertaking. Inevitably, the habits and attitudes acquired before entry into medio~1 school and hospital staff overflow into the D~SearCh experience. Collaborative efforts are uncommon. The combination of competitiveness and lack of opportunity for collaborative underba kings is an obstacle to the team research requirements that are often mandated by clinical investigation studies. Scientists trained in a medic school setting oft en do not receive the same exposure to courses in statistics and research techniques as their colleagues trained in a graduate school curriculum. The panel numbers suggested that the role of the mentor is central to responsible research training for human research, as may also be the case in other fields of basic research. In recent years, mentors have become increasingly remote from junior personnel, depriving emerging investigators of necessary moral and professional as well as scientific guidance. Responsibilities for training have become diffused, while the number of fellows has increased. As a result, ccmmunications between the scientific head of the laboratory and those in the initial phases of training have become increasingly casual and sporadic. The physician-scientist also operates under different conditions than the basic scientist. The physician-scientist is often required to commit extensive time to activities that are peripheral to research. For example, patient care is a demanding pursuit that can disrupt ra search schedules. Committee assignments and administrative responsibilities in the clinical- departments are time consuming. These distractions from research generate stresses and may impede proper professional development in research. Rarely is the developing independent investigator protected frail the pressure to achieve independent funding and to participate in teaching and administration during the formative period of a physician- scientist. Some panel members observed that the absence of a written set of research guidelines exacerbates the dependence of the young physician- scientist. Consequently, much of the prof visional training of the 76

OCR for page 68
clinical SCimltist is influence by me merrier In With the Hearth Filet ~ this} ~ trainee ~ I ~ iamb ~ ~ practices, include He tedhniq~s and its of r~r~ deign; the har~lir~ am Waring of r~r~ data; the emulate of e~peri~ntal results; am the general sense of has science is priorly cold. ~ e panel believed ~ at, as a rule, there ~ little explicit at ~ cion to the ethics of good scientific research on humans or to the responsibility of the indiviflu~] scientist to the scientific community at 1ar - . NORM; Clinical research is often mare intensely monitored than other types of research. Institutional review boards are central to the review process. Moreover, collaborative trials generally have built- m review - In research universities, institutional review beards are quite busy Men working at full capacity. the panel ban ieved that, In considering strategies for improving r~h monitoring, ~ oversight re~ponsib;1 ities ~ Id not be it on these boards. `3 panel;suggest" that effective in develc~ping proper reseal axes and practices are the Manias each to brain yaw scientists. Of particular concern is the Craig of the EShysician- sci~ist as a responsible indulgent investigator. knee no; exp~ Morn abarc the lack of attention by the tea<~ir~ institutions to Negro ark their roam an. This clefici~y affects bag the ~ainir~ Of physician scientists and also influerxx; their subagent behavior as indulgent irwestigators. ~air: Alfred P. Finn 77 Rappor~ur: Leonard Barter

OCR for page 68
PANEL-3 REPORT OF THE PANEL ON INSTITUTIoN~L OVERSIGHT The panel focused on the role of the university or academic research center, as an institution, in ensuring and promoting scientific responsible ity. Recognizing that goverurenlal institutions, including the National Institutes of Health and the Congress, have emphasized the role of the universities in detecting and preventing scientific misconduct, the panel sought to discern institutional practices that would support and promote good science. The 17-member panel included government and university attorneys, academic a,chinistrators,,government,officials, and professional society officers. Many of the members had direct experience in handling or investigating allegations of scientific misoonduct. The pane] acaep~P~ the premise that peer review at the institutional leaved is the cornerstone of good science. They explored whether a system of incentives and structures can be devised to enhance,responsible science and to discourage the impulses and practices that contribute to scientific misconduct. In discussing actions to enhance good science, the panel sought to identify areas where initiative by university officers, rather than action at the level of The laboratory or professional association or journal, would have a oo~parative~advantage. , The panel identified five areas where institutions could profit from increased attention: o, training and Hectoring; o handling, storage and maintenance of research data; o the meaning and responsibilities of authorship; O definition and as _ nt of scientific acacmplishment; and o processes to assure full and fair review of allegations of . . . - r~prle~. A common theme for all five areas was the need for each institution to clarify and communicate its expectations for individual investigators about the environment In which scientific inquiry is to be conducted. A striking example of one institution's efforts to articulate these expectations is the set of guidelines recently adopted by the Harvard M~di~1 School, which address many of theme five areas. ~N~ AND MEN Effective training of young scientists is an essential duty for the perpetuation of good science. Training is teaching, and the primary responsibility for carrying out this duty ne~-c=~rily rests with individual faculty members and research supervisors. The academic institution's role is seoon~ary and largely supportive. 78

OCR for page 68
To or ~t is this institution? ~iviclua~ are collective faulty are administrative officers cerise the university. the Collective face y, overseer He individual faculty ~rbe:r, is tile core of a modern anemic instillation. Ire collective faculty holds the primary r~nsibility for He ~n~ri~ are quakier of the institution, although He degree of care with Rich they Anise this r~nsibility is often influenced by the administration. He administration ~nicat~ with the a~tsicle world arxl serx3s into nal nYY;~ges as well, supervises the expenditure of grants and contracts, and prods the faculty to provide the oversight and verification duties described above. In exploring the often-discussed issues of academic advancement and the reward structure of science, the panel identified some urexpectel problems on first taking up the initial steep of an academic career: appointment. The panel considered whether individuals were adequately reviewed with regard to their ethical as well as their scientific qualifications. AM - Personality defects of varying seriousness usually play a significant and perhaps dominant role in cases of scientific misconduct. However, it is difficult to screen for deviant personalities at the stage of faculty or staff appointments because the interview process is unreliable and is usually not aided toward this particular goal. The communication of relevant information four previous employers and associates is restricted by federal state and privacy laws. Frank and candid answers may often be obtained only by asking specific questions directed tc ward the detection of imprcger behavior, and it is rare that this behavior would be suspected at the time of recruitment. The panel suggested that a formal waiver of privacy by academic applicants could help overcome the legal barriers to sharing information about prior force of misconduct. They proposed that this waiver could be modeled on the statement used as a standard for obtaining clinical privileges to avoid deleterious efforts. The panel also Suggested that it would be useful have models for responsible reference letters, highlighting the questions that should be routinely asked in the course of recruitment and evaluation of applicants. They proposed that state privacy laws could be surveyed to determine the legal standards governing communication of misconduct or deviant behavior. Pane] members suggested that there may be a need for legal protection for institutions who wish to request or report confidential information relevant to academic performance in cases involving scientific m~saon~uct or when resignation has occurred during the course of an investigation. Several panel members questioned the legality as sell as the desirability of incorporating gag orders into private agreements or settlements between academic institutions and faculty or research staff members act of misconduct. 88

OCR for page 68
AD~NCE~ENT Panel Is coved mat maw people perceive an onrer~asis on publication as a source of aft bow by instib~ions and no furlers. Is "publish or perish" pe~pti=1 has been s~est"1 as a safe of Motive pressure on fawner ant the cause of major ark minor abuses of row standards. Better au Cow hip practice== might en ~ rage the fat to talk more individual and collective responsibility for the integrity of published material. These practices could be achieved by.sbep6 designed to resume gift co-authorship and the fragmentation of data. However, the pane] was not certain that reducing the number of publications by each author will significantly relieve the "publish or perish" pressure. A more important consideration, in seeking to reduce the reality of this pressure as well as the perception of its importance by the faculty and young scientists, may be the nope for institutions to structure their performance evaluations to reflect the value of quality aver quantity. The criteria for advancement outlined in recent guidelines issued by the Harvard Medical & hoof were considered by The panel, particulary the criteria in the guidelines that limit the number of Dublications used to support a promotion derision. needed to be further developed: The panel thought ~ at these criteria Should The candidate be allowed to select the papers to be considered? Should the candidate justify the value of the selected papers? Should the entire biblicg~aphy be available to the Section committee, automatically or on request? What are effective methods of distinguishing competent individuals of differing productivity? The panel suggested that the faculty of a selected institution should develop the criteria that affect their 'appointment Fond advancement decisions. If different institutions adopt significantly different guidelines, a broader consensus will need to be achieved to change the existing criteria used by funding agencies to predict productivity and quality. The issue of size of bibliography stimulated debate within the panel, partly because of a paucity of information about the significance of either unusually long or short bibliographies of research scientists. The identification of correlative factors and other assessments of creativity were thought to be desirable, and the panel suggested that this area be designated for further study. The panel also oonsidered problems in obtaining genuinely critical peer reviews of performance or even quality of publications by colleagues or referees. Some obvious reasons for these difficulties include the time-consuming and analytical nature of careful reviews and the lack of incentive to devote significant time to this area. Many faculty do not wish to risk faulty judgment that may hurt someone else's chances, and they are lens likely to write negative than positive comments when there 89

OCR for page 68
is un~a~n=. Collegial relationships are ~ntial to ff,e snow furx~ioni~ of the academic unit and thee may be threat by coo frank criticism. ~ E;mall specialities or grows, If nay be taco few individuals willing to provide the newsy reviews. Faculty engaged clinical work and teaching may have limit eviderxx of creative scholarship and may active hatch criticism at Off ti~ of academic adva~t reviews by peers to are often tenure faculty, primarily engaged in starch. lee pant suggests bat we area of group procures and psychosocial Imitations on peer review needed f ~ her sandy. Research In this area could lead to significant improvements in the quality of journal or personal referee systems. There is also a need to define and popularize alternatives to counting papers in evaluating faculty performance. Many young fat develop unrealistic expectations based on lack of knowledge of and experience with actual performance evaluations. The panel members thought this was a problem of uncertain dimensions, but it may refle ~ an underlying disorder and it can be agy=avat^~ by the competitive, averprc~uctive atmosphere of same disciplines. Many young faculty are uncomfortable about their perceptions of what is expected of them, although even accurate perceptions may not diminish pressures to cut corners in order to advance quickly. The panel suggested that closer supervision and restoring of junior faculty would help them to understand prafe~ciona] standards and would provide an early opportunity to detect aberrant. ideas or behavior. The panel also considered the special problems of M.D. rather than Ph.D. investigators. These often include little or no formal research training before faculty appointment and inadquate research time following appointment. The panel believed these problems promote sloppy or inaccurate reearch' bad authorship practi ~ ~ and even misconduct. One approach that may alleviate these problems is extending the probationary period, with subsidized research e ~ fence, for M.D. investigators. Another may be stricter requirements for appointment of mea; Bad school Graduates to research investigator positions. Neither approach may be realistic in many specialities. A more feasible approach may be The promotion of support for part-time clinical investigators by experienced researchers through special grant mechanisms. The periodic peer review of Ph.D. prc grams in many gra Mate schools (often through the use of 'visiting commit') is a common mechanism for assuring The quality of graduate school training and research programs. A common feature of this review is the use of outside expert scholars to evaluate The strengths and weaknesses of an individual graduate program. It is not certain whether these reviews also ~ the experience of postdoctoral fellows or the sectoring of junior faculty. The panel observed that research and trainer program conducted in the ~di~1 ~ ols usually do not receive treble Cereal peer reviews. They surest that a sty of the mesons use to evaluate academic swears art training practices, and their aE1plicahility tO the Id school viro~nt, deserved consideration. 90

OCR for page 68
ASS[~CF~ me part spent substantial time disa~ssir~ the need for Aims to demonstrate He performance and Maturity of acadenic Cart. Ike panel did not suborn He net for broader institutional involvement In pr~iding assurer At Art performance and -alrthonihip contributions. Whey did ire a need for more e~censive tinseling and red iation services within academic institutions, with cony and confidential adepts to debit ated institutional officers. m e panel oomsidered one suggestion that an individual outside of a research group be designated to sign off on manuscripts but concluded that this approach was exo~=ively intrusive and overly ~ aucratic for the academic environment. A similar institutional sign-off mechanism was explored in the context of confirming attribution of authorship. The consensus of the panel was that this confirmation should be made by the research group itself, not a supervisor or administrator. m e panel considered with favor a proposal that each co-author should sign a statement confirming approval of a final manuscript and concurring in the designation of authors as part of the submission of the manuscript to scientific journals. The value of an additional confirmation statement by a university official seemed superfluous. These propceals a ~ directly aimed at remedying the problem of co-authors failing to take collective responsibility for the Integrity and quality of a research project or publication. Panel members observed that some faculty seeking to build their bibliographies would not be inhibiter by these procedures. Faculty willing to put their name on anything may also be willing to sign anything. In the final analysis, the criteria pro posed by many organizations and journals for entitlement to authorship are no more useful than the scrupulousness of those who use the criteria. Some persons who have not taken any responsible action may claim to have performed several functions, including the planning and the writing or rev ~ ion of a paper, without objections by co-authors. Part Is expressed concern that coauthors to are not satisfied with Me ~.=cig~t of credit have limited Irks ~ racist then in selling disputes. it chairs Should racist in mediating these conflicts in theory, but in; an era of ~a~epar~rents and multia~nter collaboration, they Tnay not be able to play an effective role. Junior co-authors may be unwilling to approach senior faculty to avoid being characterized as troublemakers. The panel perceived a need for improved consultation and mediation services within the academic institution and expre~cPO particular interest in the omtodsperson positions used by several universities and medical schools (including MIT and Stanford). In many cases, the owbuisperson provides a confidential advisory service and has the resources to assemble a panel of experts to mediate disputes on an ad hoc basic. 91

OCR for page 68
The panel suggested that authorship guidelines may be helpful in resolving disputes by clarifying in advance of the submission of a manuscript the criteria to be observed in recognizing indivi~rmn contributions. Professional societies and journals play an important role in defining these criteria, since authorship is often influenced more by disciplinary customs than institutional traditions. AREAS FOR FURTHER STUDY the panel emphasized the increasingly important influence of commercial or industrial ties by university research laboratories and academic scientists. There has been concern about the effects of these affiliations on graduate education and the open exchange of ideas and research materials, but there has been less discussion of their Am pact on row City. Though research fraud cases have been extensively disr~=P~, there is still much to learn about patterns of misconduct, their pr~-ipitatir~g factors, and ultimate altcome5 of Me corrective pates. SCAR swim of deviance have indicated that these incidents have a significant impact on the "victims" or colleagues of the perpetrator. Me panel bought it wand be useful to sway research chars where fraud Is ~ to end ore how this incident has affect the S-lC~1~5, practioes, arm relations trips; of the research ~virorment. For example, ~t happens to Me ~istle-bic~wers and to those guilty of m~scor~uct? Ire panel identified a row for a comprehensive review arx] follaw~up on randy d~ cam. The panel also reviewed the subject of data ~hip, cushy, and access. Ones members recognized this as a complex area ark express ~ a lack of knowledge about the status of intellectual property law and government regulations and their Am pact on research data-handling practices. The panel suggested that this important area deserved careful analysis, separate from the study of other research practices, and that this analysis should precede the development of institutional guidelines and 5t3ndar~5. Finally, the panel considered the importance of collaborative efforts in modern research. Multiple authorship practices suggest that collaborative efforts have been increasing for years. The problems discussed above take on a new complexity when considered in the context of interdepartmental or multicent~r research efforts. There may be unique problems in assigning credit when first authorship must be limited' although these problems may be aileviat=d when multiple papers are produced. Data access and storage issues acquire greater significance when the research data result from a unique, time-consuming, and valuable collaborative effort. Institutional assuranc s of validity and integrity require new forms of Interinstitutional cc operation. The panel suggested that the particular influences of interdisciplinary and multicenter collaboration deserve priority in further investigations of this topic. Chair: William Anlyan Rapporteur: Paul Friedman 92

OCR for page 68
PANEL 6 REPORT OF THE PANEL ON AUTHORSHIP, REFEREE, AND SCION ~ The panel addressed current authorship practice=, publication procedures in institutions and journals, and post-publication procedures that may affect the integrity and reliability of published results, with particular attention to Neal science and clinical research in the Uniter States. The 14 panel members included editors from basic science and clinical journals (including one British journal), faculty face. basic research departments, and government agency officials (including the National T.;hrary of Medicine). The panel discussed an array of topics concerning authorship and publication standards. These topics iw luded repetitive publication, supernumerary authorship, institutional responsibilities, peer review, documentation of reported research results, errors and corrections, frequency of fraud and other misconduct, the role of government, and pressures to publish. RE;~1'1W ~m The panel members recognized that.publication of more than one report of the same research material (repetitive publication) may be advantageous to authors for their personal interests and to readers who use the journal literature for llawareness needs.ll Members indicated that these interests needed to be weighed against the economic costs and intellectual confusions caused by this practice. The panel attempted to define repetitive publication. This practice includes identical republication; rere porting observations in one or more additional papers with different emphasis and detail; and reporting a small portion of data from a single study, often referred to as "salami science" or the "least publishable unit." Authors may justify the practice of repetitive publication to address different audiences or to meet requirements for Participation in a sYmc06ium or con-tress. Editors e view repetitive publication as causing excessive costs in manuscript handling, peer reviewing, and journal space and distribution. Readers may have difficulty deciding on the uniqueness of the content of a particular paper. The panel considered several possible solutions to this practice, including the need for editors to define repetitive publication explicitly, to state penalties for violations, to find means of informing readers of occurrences, and to consider other possible means of prevention such as maintenance of a "blacklist" for a jourr~l's own use. 93

OCR for page 68
SUPERNUMERARY AUTHORSHIP Are new criteria for authorship needed? Panel members indicated that the criteria ado pte1 by some journals may not be wellknown to the community of authors. Should a category "contributing author" be established and added to present definitions of true authorship? Should readers assume that all listed authors can take responsibility for a paper? The panel agreed that proper authorship practices protect the scientific community against Mets conduct in research and its reporting. m e trend of an increasing number of authors per paper may be explained by the increasingly complex nature of research, but there is also evidence of not infrequent gratuitous or '~gift" authorship. Authorship assignments tend to be based on traditional practices. Professional societies, editors' associations, and a few institutions have published detailed criteria for authorship, but panel members believed these criteria are often violated. It is difficult for the editors to obtain proof of violation of authorship guidelines. They could ask for more detailed and explicit justifications of authorship, but these requests would not necessarily be more reliable than present authorship statements or copyright agreements. The panel indicated that editors would prefer to trust authorship assignments. The members believed that the proper locus of authorship control is institutions. They proposed alternate means for institutions to screen papers prior to their submission so that authorship criteria could be properly applied. The panel indicated that existing authorship criteria should be disseminated more widely. There may also be a need for new collective efforts to develop a national standard of authorship criteria for academic institutions. New categories of authorship, such as a secondary level like Contributing author." or wider use of collective , - ~ or corporate authors hip may be needed, and a few individual editors have established criteria for these categories. INSTTTUTION~L RESPONSIBILITIES Should universities screen papers prior to their submission for publication? Should this screening seek to prevent repetitive publication? What disadvantages Bight result form institutional screening? Panel members indicated that a few academic departments and research units do screen manuscripts of their faculty and staff. m ey suggested that institutional screening could reduce undesirable publication practicer. The members recognized that screening could add a new administrative burden in academic departments and Bight pose risks of censorship. Although screening manuscripts may not be the best solution, the editors on the pane] thought that academic institutions near to est~h~ish some means of controlling irresponsible publication practices. 94

OCR for page 68
Ed REVIEW The panel also examined aspects of the pear review process including: Can peer review detect fraud? How sight adequate peer review be facilitated? How desirable is anonymity of reviewers an4/or authors in peer review? The panel concluded that research fraud is unlikely to be detected by peer review. Data that are "too good," logically impassible, or from unspecified locations can be clues to fraud ~ a particular paper, but well-oonstrucbed fraud may readily escape detection. Large numbers of papers attributed to a particular author could be a signal to an institution of possible fraud, if it maintains manuscript records. Peer review is expensive. It might be improved if journals conduct more vigorous screening and reject submitted papers immediately. me process It also be improved by providing reviewers with mow explicit criteria for making critical judgments. At present, He pared concluded that He advantages of using anonymity for reviewers outweighs the disadvantages. It is more difficult to ensure the anonymity of authors, and the editors believe they can easily detect the biases often attributed to author non-anonymity. The panel indicated that there is a need for more research on the peer review process to develop firmer answers to these questions. DCCLMEN~ATION OF REPORIEu RESEARCH RE SUITS The panel examined the question of whether authors should be expected to supply data to editors and peer reviewers beyond those presented in the caner itself. A Darer should carry all the data needed to support its . ~ at. I. - ~ ~ ~ ~ . ~ . ~ . analyses and conclusions. Authors should be expected' however, to submit a~;tior~al data for an editor or pew reviewer if resect for critical joints. The panel believed oft routine relents for submission of s~l~ntary data with papers would put Inns administrative }dens on Me journal editors. RESPONSES TO FRAUD, REIRAC1'IONS, AND CORRECTIONS How should editors respond to charges or suspicions of fraud? should journals publish retractions or other How should they handle errors and correctional How statements concerning fraud? - Some differences of opinion emerge] ~ the panel regarding the appropriate procedures for editors ~ addressing the first issue. Mast panel members believed that editors have a limited ability to resolve questions of fraud and that allegations of wrongdoing are more properly addressed at the institutional level. Others suggested that editors should respond to these incidents by initially approaching the authors and Sheen approaching their institutions to investigate the claims if the initial approaches have failed. A differing view is that the problem should be taken to the institution immediately. 95

OCR for page 68
me parcel noted that me I:nternational C~i~ee of Medical Journal Editors Is issued a policy sextant on editors' responsibilities for r~porxti~ to Barges of scientific m~scorx~uct, With require the appropriate publication of relevant notices. me National Library of Medicine has Revels a Dyson for tagging refed; in MED=NE ~ alert bibliographic seen Hers to notices of fraud or scientific e m or. With this system in place, the panel believed that editors have a responsi- bility to publish notice== of retraction, investigatory conclusions of fraud, and errors and corrections. m is responsibility includes a duty to publish these notice= in a standard format and location. m e panel agreed that there is a need to publicize this res,oonsibility to editors and Editors' associations. EMINENCY OF FRAUD AND TIER ms~cr How frequent are fraud and other kinds of substantial misconduct? How does this question affect editorial and institutional responses to scientific misconduct? The panel had no evidence of an increase in the frequency of fraud and other kinds of misconduct in publication, but members indicated an impression that there is an increase in the number of incidents. This perception may simply be the result~of wider awareness of the problem. Two members of the panel favored the use of research audits as a means of determining the extent of the misconduct in science. Other members felt that the lack of evidence of incidence should not preclude the development of measures to control or reduce the problem. While serious research misconduct must be controlled, most pane] members felt ~ at other publication abuses, such as irresponsible authorship and repetitive publication, are probably far more frequent and present a greater threat to the efficiency of research and the integrity of the publication pass. rHE ROLE OF Go~r What should be the rGle of government ~ supporting the responsible conduct of research? Should federal agencies require institutional controls as part of grant-supporbed or contract research? Should government exert more direct controls, such as audits? m e panel members believed that government's interest in reliable authorship stems free,` a need to verify reported research results, especially when the products of research have economic and regulatory importance (e.g., patents and copyrights). In addition, government agencies are publicly account he for proper use of tax funds, and maintaining the integrity of authorship is one avenue of demonstrating accountability. Misconduct in research can suggest evidence of governmental mismanagement of public funds. The panel suggested that government can provide incentives and apply pressure for the responsible conduct of research. At a minimum, government research furriers new assurer that the research is conduct 96

OCR for page 68
~ a responsible namer. me ~ ~a <~ fear institutional and journal pros related to authorship ark publication practices. SO TO ~5H me panel discos" the variers ply to publish and ways to Rae =~c pro. the econc~nic rewards fmn publication are seen as driving }=th ~ ary authorship ark rep etitive publication. Ihe rewards include nck only direct monetary gains but also the professional prestige derived four appointment, promotion, and tenure decisions. One aLtr-ac.tive dinero} on pressures to publish might be the creation of institutional limits on the number an] frequency of publications to be considered for decisions on faculty status. One panel member questioned whether photo Patrols wound reduce unjustified authorship, as there are substantial personal gains other than academic status associated with the high visibility resulting from frequent and widespread publication. Some control might come from cantors limiting the numbers of authors and more frequently requiring collective authorship, but these controls may require wide adoption to be effective. As noted above, the panel come to general agreement on a number of steps that academic institutions, individuals, and government might take to reduce the severity, number, and frequency of at uses of scientific publication. These actions could demonstrate the research communi~y's cc~nitment to assuring the integrity of scientific restore. The panel's portals were format with t:he following criteria In mid: 0 In order to remain highly productive, the research cc~nity must take resEx>nsibili~ for the conduct of its work and its relations with the public arm gaver~nt. O Appropriate conduct in research Should be develops arm supports as far as possible bar the Iran il:u:tions of science it~If. o Recomb nations should identify measures that avoid excessive control, address present concerns, and not expect a utopian state of scow. O Reccmmen~ations should not suggest measures likely to increase the burdens of documentation. O Clear distinctions should be drawn between misrepresentation, such as fraud and data-trimming, and errors of fact and interpretation. Chair: Marcia E. Angell 97 Rapporteur: Edward Ruth

OCR for page 68