An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures.
—Jonathan Buckheit and David Donoho (1995), paraphrasing Jon Claerbout
The promotion of responsible research practices is one of the primary responses to concerns about research integrity. Other responses include the development of policies and procedures to respond to allegations of misconduct (covered in Chapter 7) and education in the responsible conduct of research (covered in Chapter 10). Exploring best practices in research helps to clarify that promoting these practices is not only a moral imperative but is also essential to good science.
Over the past three decades, government agencies, advisory bodies, scientific societies, and others have issued reports, educational guides, and other materials that address the topic of research practices. For example, the 1992 report Responsible Science points to a number of factors that affect research practices, including general scientific norms, the nature and traditions of disciplines, the example of individuals who either hold positions of authority or command respect, institutional and funding agency policies, and the expectations of peers and the larger society (NAS-NAE-IOM, 1992). That committee’s review of research practices focused on four areas: data handling (including acquisition, management, and storage); communication and publication; correction of errors; and research training and mentorship. The report explained how commonly understood practices in each of these areas promote research integrity.
A number of other documents and codes of conduct from around the world have specified good or appropriate research practices (CCA, 2010; DCSD, 2009; ESF-ALLEA, 2011; ICB, 2010; IOM-NRC, 2002; MPG, 2009; NHMRC-ARC-Singapore Statement, 2010; TENK, 2002; UA, 2007; UKRIO, 2009). In addition, responsible research practices have constituted the primary subject matter for responsible conduct of research education activities, as illustrated by various educational guides (Gustafsson et al., 2006; Steneck, 2007; NAS-NAE-IOM, 2009b; IAP, 2016). These materials address the topics covered in Responsible Science—
data handling, publication, correcting errors, and mentoring. Some add other topics, including research collaboration, peer review, conflicts of interest, and communicating with the public. Formulations of responsible research practices specific to certain fields address additional requirements, such as protection of human research subjects, care of laboratory animals, and prevention of the misuse of research and technology. For example, the National Institutes of Health (NIH, 2009) has specified nine core areas of responsible conduct of research instruction.
Given the extensive effort to formulate responsible research practices, what does this report hope to add to the discussion? One goal is to reexamine the primary elements of responsible research practices in light of current conditions for doing scientific and scholarly work. A key conclusion of this study is that significant threats to research integrity exist in the United States and elsewhere, arising from a combination of factors present in the modern research environment. As discussed elsewhere, determining the incidence and trends of research misconduct and detrimental research practices is difficult or impossible with the existing data. However, failure to respond effectively, or in some cases an apparent tolerance for detrimental research practices by researchers, research institutions, journals, and funding agencies, has clearly contributed to delays in uncovering misconduct in several well-publicized cases. In some instances, this misconduct occurred over many years, and fabricated results were reported in many papers. And while survey data have limitations, a growing number of studies indicate that the prevalence of detrimental and questionable practices is too high and that the adherence to responsible practices is too low, both in general and in particular fields that are facing problems with irreproducibility of reported results (John et al., 2012).
One reason that holding to best practices is such a challenge and is ultimately so important is that researchers, research institutions, journals, and sponsors may face incentive structures that are not completely aligned with the responsible practice of research. While individual researchers have long been recognized and discussed as potentially conflicted, it is reasonable to apply this perspective as well to other actors. For example, externally funded research is a revenue stream for research institutions and plays a business function in those settings, in addition to providing the necessary funding for scientists to conduct research. The need for institutions to maximize such funding streams may sometimes detract from their ability to uphold best practices. Institutions may not exercise the necessary degree of skepticism and oversight toward researchers who are very successful and valuable to the institution in terms of securing resources or enhancing its reputation.
Likewise, journal publishers and the editors who work for them may have incentives to take actions that are not consistent with best practices for fostering research integrity. In particular, the rise of bibliometric indicators such as the journal impact factor may pose difficulties as journal editors seek to publish the best research but also have an incentive to see the impact factor of their journals
rise as far as possible. The inappropriate practice known as coercive citation, in which authors are pressured by journals to cite other papers from the journal, is an example (Wilhite and Fong, 2012).
Finally, sponsors of research and users of research may be subject to pressures or incentives of their own that are not completely aligned with maintaining the integrity of science.
One element of this committee’s task was to address the question of whether the research enterprise itself is capable of defining and strengthening basic standards for scientists and their institutions. A critical aspect of this question is that the integrity of the research enterprise is achieved not solely through the integrity of individual researchers and their research practices but through the integrity of the system of which they are a part—the combination of participants and processes that constitute the system as illustrated in Figure 1-1. The best practices outlined here aim to reflect best practices in the context of the entire system of research and the interdependence of researchers, research institutions, funding agencies, journals, societies, and other participants. Developing this updated framework of responsible research practices will help the research enterprise identify particular practices that should be better understood and adhered to and how such understanding and adherence might be promoted and fostered.
As described in Chapter 2, the values of objectivity, honesty, openness, accountability, fairness, and stewardship underlie the effective functioning of research. These values are realized through the norms that apply to research practices. For example, honesty requires that researchers do not alter the data an experiment has produced, and openness means that researchers share the methods they used.
Norms permeate research. Some are formal and explicit, such as the regulatory requirements for treatment of animal and human subjects. Others are informal and sometimes implicit. For example, although there may be no policy that explicitly prohibits practices such as taking undeserved credit for the work of graduate students or postdocs that one is supervising or not extending deserved credit to them, researchers who exploit those who they supervise for personal ends are working against the norms of science.
Norms can be descriptive as well as aspirational. Descriptive norms are those that are generally adhered to and are expected of members of the enterprise. Sanctions may be attached to serious violations of descriptive norms; for example, all those involved expect that researchers will accurately report the results of their research. Aspirational norms are ideals that members of the research enterprise hold and attempt to achieve; for example, researchers seek excellence in the design and execution of their research and seek results that will make significant contributions to the body of knowledge in a field (Anderson et al., 2010).
The best practices described here are aimed at individuals and entities serving different roles within the research system, including researchers, reviewers, institutions, journals, and funders. The committee uses the term best practices here to refer to prescriptive and aspirational norms. The committee has drawn these best practices from the relevant literature, from the experts that it has consulted, and from the accumulated knowledge and experiences of its members. The practices identified encompass principles, strategies, modes of behavior, and activities that preserve the integrity of research and avoid the pitfalls that impede scientific progress. Except where noted, these practices do not require significant additional resources to implement and are indeed practiced in a variety of locations and settings. For most of these practices, the necessary conditions for implementation are recognition on the part of the identified stakeholders that the integrity of research is central to the practice and progress of research, and willingness to act on that recognition. One of the major impediments to such recognition and willingness, of course, is that these practices may not be completely aligned with the perceived self-interests of some stakeholders.
These best practices do not cover every possible ethical situation encountered in research. Nor do they include matters of science and technology policy that are largely administrative, procedural, or discipline specific, such as data retention policies in particular fields or the distribution of research funds. However, the ethical and the administrative overlap in many areas, especially in areas involving obligations of stewardship to the research system as a whole (e.g., in workforce policies), and these overlapping areas are addressed in what follows.
These best practices apply across all areas and forms of research. In contrast, specific codes of conduct are more prescriptive than best practices and can vary from discipline to discipline, such as the number and order of authors on a paper. The application of best practices may also vary in some particulars depending on whether research is undertaken in academia, industry, or government laboratories. The following compilation will strike many readers who are experienced in research as self-evident. These responsibilities are delineated here in part to demonstrate the dense web of relationships and obligations that characterize the research enterprise.
The committee has aimed to describe best practices that are specific enough to be implemented but that may also encompass a number of detailed components. Responsible research practice checklists are provided to enumerate these components.
Principal investigators and other scientists (including technicians, undergraduate and graduate students, and postdocs) are the foundation of the research enterprise. The research record begins with their work, and researchers are the primary evaluators and verifiers of work done by others in their respective
fields. Every scientific finding a researcher reports contributes to progress in the discipline, and failings made in the conduct or reporting of the research can immensely harm the progress of the field. Every researcher has the responsibility to ensure that these tasks are carried out to the best of his or her ability.
Researchers may play a number of roles during their careers, often simultaneously, including student, trainee, young investigator, principal investigator, department head, reviewer, editor, and administrator. The research process itself includes planning research, performing research, and disseminating results, and researchers have responsibilities at all points during the process. In planning research, they need to consider the effects of research, both positive and negative, on the broader society. It is especially important that they be vigilant about the possibility of unanticipated and potentially dangerous consequences of research, whether on a local or global scale. In interdisciplinary or international research collaborations, investigators may need to engage in continuing discussions about the standards that apply to such efforts.
As they perform research, scientists are expected to maintain high standards of proof and scientific credibility through validation of methods and rigorous confirmation of findings. They should keep clear and accurate records. They should follow the rules and procedures of their institution and laboratory regarding the physical and electronic security of data and the devices on which they are stored. They need to adhere to policies and regulations on the conduct of research related to personal safety. They should be open with supervisors and funders regarding progress, including positive and negative results.
Disseminating research entails responsibilities as well. Researchers should give credit to colleagues for help in completing work, whether in a presentation or a manuscript. They should reveal all methods and corresponding experimental findings that support conclusions as well as any unexplained outlying data that do not fit with the conclusions, allowing others to decide whether the conclusions are still valid despite the outliers.
Best Practice R-1: Research Integrity. Uphold research integrity with vigilance, professionalism, and collegiality.
According to one formulation, integrity for the researcher “embodies above all the individual’s commitment to intellectual honesty and personal responsibility” (IOM-NRC, 2002). The duty of researchers to uphold research integrity is multifaceted. Fulfilling this duty starts with a broad understanding of scientific methods and the research enterprise as a human institution. Research requires the constant exercise of judgment and is subject to bias, whether conscious or unconscious. Researchers need to be aware of their own personal potential sources of bias in designing, carrying out, evaluating, and reporting their own work. They need to understand that knowledge advances over time, although errors and mistaken interpretations can occur along the way. Researchers who acknowledge and correct their own errors or misinterpretations with equanimity contribute to
the progress of science. Likewise, researchers should be fair and generous when critiquing the work of others. Criticisms should focus on errors in the work and disagreements about interpretation, but not on the person.
In addition to meeting their field’s standards of integrity and quality in their own work, as specified in the best practices on data handling and authorship, researchers need to promote high standards among colleagues. They should take careful and timely action when a concern about research integrity arises. As a prerequisite, they should understand the definitions of, and policies to address, research misconduct adopted by their institutions and funding agencies. They should be familiar with the appropriate formal procedures for expressing concerns and making allegations, as well as informal rules and steps to help ensure that such concerns and allegations are made responsibly (Gunsalus, 1998a). These informal rules include accounting for one’s own biases, appreciating that one’s knowledge of a situation may be incomplete or incorrect, and getting confidential perspectives on possible misconduct from a trusted advisor before making a formal allegation.
Researchers should maintain an active commitment to openness in research as the essential foundation of academic freedom, not just the integrity and credibility of science. A commitment to openness means both acting and advocating for openness.
Best Practice R-2: Data Handling. Manage research data effectively, responsibly, and transparently throughout the research process. This includes providing free and open access to research data, models, and code underlying reported results to the extent possible, consistent with disciplinary standards, funder requirements, employer policies, and relevant laws and regulations (such as those governing intellectual property).
Effective record keeping and data management while undertaking research, and complete sharing of data, models, and code when publicly reporting results, are fundamental to research integrity. The importance of updating knowledge and practices related to data is increasingly recognized around the world (NAS-NAE-IOM, 2009a; KNAW, 2013). The pitfalls that can occur when dishonest, closed, or ineffective data management practices are employed are illustrated by the translational omics case and other examples discussed in Chapter 7 and Appendix D.
Researchers need to understand and follow the data collection and analysis standards of their own fields. For example, research data will often contain potential outlying results. While refining data to remove outliers is appropriate, any data refinements should be made to the entire dataset and should similarly improve subdatasets as it does the entire set. The refinement should also be well documented wherever the dataset appears. Some data refinements made after an experiment may be acceptable, since the types of noise that will show up in a dataset may be unclear until after the data are collected, but should be based on an analytic principle that provides an explicit rationale for exclusion. Researchers should guard against the temptation to use a post hoc rationale to make undocu-
mented refinements that strengthen support for a favored hypothesis. Such behavior is a detrimental practice or could even cross the line and become falsification.
In some settings and some cases, data, models, and code may not be made available, or sharing may be delayed due to legal or regulatory restrictions, including those related to privacy, intellectual property protection, and national security classification. For research that does not result in publicly reported results, such as some work performed by industrial or government labs, sharing of data and code is not a requirement but should be undertaken where possible.
In the 21st century, many novel findings and published works are based on nonobvious analysis of large datasets. How to effectively manage these datasets and properly provide them or refer to them during review and publication are challenging issues that are being considered across many fields and disciplines. Internal curation of large datasets may be expensive for research groups, and many journals do not have resources to host the datasets. However, examples of falsification, fabrication, or error discussed in Chapter 7 illustrate that posting of data and code can enable researchers to identify problematic conclusions and correct the research record.
Researchers need to ensure that appropriate statistical and analytical expertise is utilized in the project. The use and misuse of statistical tests such as p-values are current topics of discussion in a number of fields; the American Statistical Association recently released a statement listing six principles on the misconceptions and misuse of the p-value (Wasserstein and Lazar, 2016). Researchers should avoid detrimental practices such as p-hacking, in which statistical and analytical parameters are adjusted until a desired result is achieved (Nuzzo, 2014). Supervisors should stay close to the primary data even if they lack the technical skills to generate those data themselves.
Best Practice R-3. Authorship and Communication. Follow general and disciplinary authorship standards when communicating through formal publications. Describe the roles and contributions of all authors. Be transparent when communicating with researchers from other disciplines, policy makers, and the broader public.
Decisions about authorship of research publications are an important aspect of the responsible conduct of research. Although many individuals other than those who conceive of and implement a research project typically contribute to the production of successful research, authors are considered to be the person or persons who made a significant and substantial intellectual contribution to the production and presentation of the new knowledge being published.1
1 In Recommendation Five, this report calls for the development and adoption of authorship standards and suggests a framework that if adopted would formally codify several of the best practices discussed here, such as describing the roles of all authors. See Chapter 8 for the rationale underlying the recommendation and Chapter 11 for the recommendation text.
As discussed in Chapter 3 and Chapter 7, authorship is also the “coin of the realm” in science—the mechanism through which scientists receive credit for intellectual work. Authorship, particularly lead authorship, carries with it credit that affects careers and promotions. Because of this, authorship often becomes a fraught topic and can invite misconduct and detrimental research practices.
In addition, authorship carries responsibilities. For example, authors are responsible for the veracity and reliability of the reported results, for ensuring that the research was performed according to relevant laws and regulations, for interacting with journal editors and staff during the publication, and for defending the work following publication (Smith and Williams-Jones, 2012). The article or paper presented by researchers “should be complete, and, where applicable, include negative findings and results contrary to their hypotheses” (NHMRCARC-UA, 2007). Publication bias, selective reporting, and poor reporting are serious problems that damage the research record. Authors also need to follow discipline-specific reporting guidelines, such as those covering the registration and reporting of clinical trial results. They are responsible for ensuring that previous work is appropriately and accurately cited. In all fields, responsible authorship involves avoiding detrimental practices such as honorary authorship and duplicate publication, as well as the affirmative responsibility to ensure that all who deserve credit on a paper receive it.
As discussed in Chapter 3, authorship practices vary among disciplines and within research groups and may change over time; professional and journal standards and policies on authorship also vary (journal best practices are discussed below). Technological changes in how research is done and the prevalence of multidisciplinary and even global research teams have raised challenges for authors, such as an increase in the number of authors per paper and more limited knowledge by all authors of the methods used by other contributors.
Authors should clearly identify which portion of a research project each coauthor performed (see the section on best practices for journals below). Even in cases where this is not required, this information can help readers interpret the work and may also avoid blanket condemnations if the work is later shown to be flawed. If responsibility for an article or other communication is not specified as clearly as possible, all authors can be held accountable for its contents.
Researchers may also need to communicate with specialists from other fields in interdisciplinary studies or may have opportunities to explain their work to policy makers and the broader public. Similar standards of accuracy and transparency should apply. For example, “any attempt to exaggerate the importance and practical applicability of the findings should be resisted” (ESF-ALLEA, 2011). The authors of a research article or other communication have a responsibility to ensure that press releases and other institutional documents describing that work are accurate and unexaggerated. Researchers should work with their institutional media affairs office to avoid unfounded claims and reveal both the positive and the negative aspects of research results. Researchers should also become more
sophisticated in distinguishing between reporting research results and advocating policy positions related to their research. Issues of advocacy can be complex, and no hard-and-fast rules cover all situations.
Best Practice R-4: Mentoring and Supervision. Know your responsibilities as a mentor and supervisor. Be a helpful, effective mentor and supervisor to early-career researchers.
The 1992 report Responsible Science defines a mentor as “that person directly responsible for the professional development of a research trainee” (NAS-NAE-IOM, 1992). In this report, the term supervisor is used to describe the person directly responsible for the professional development of a trainee. Here, the term mentor refers to a broader group that includes supervisors as well as other more senior researchers who are in a position to contribute to the professional development of trainees and junior researchers. Professional development encompasses the development of technical expertise, socialization in research practices, and adherence to the highest standards of research integrity. The 2002 report Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct outlines the responsibilities of supervisors as including “a commitment to continuous education and guidance of trainees, appropriate delegation of responsibility, regular review and constructive appraisal of trainees, fair attribution of accomplishment and authorship, and career guidance, as well as help in creating opportunities for employment and funding” (IOM-NRC, 2002).
Since supervisor-trainee relationships are often complex, it is important that supervisors and trainees clarify their mutual expectations for the relationship (NAS-NAE-IOM, 2009b). Conflicts can sometimes occur over the time and opportunities allocated to trainees, credit for and ownership of results, and other issues related to research practices. Supervisors should make sure that trainees are aware of the risks of misrepresenting data, should be aware that subordinates can have an overzealous concern to meet expectations, and should recognize that periods of heightened stress may impair their judgment.
In the context of this report, ensuring that trainees understand and follow best practices in research is an important element of mentorship. This includes checking the work of trainees, particularly work that is being submitted for publication. In several of the individual cases that the committee examined during the study, failures and deficiencies in mentorship and supervision were factors contributing to significant delays in addressing serious problems with data underlying reported results.
Supervisors and other mentors should ensure that trainees receive high-quality instruction in, and appropriate socialization into, the responsible conduct of research. This may involve incorporating activities within the lab as well as institutional and other instruction. A potentially useful practice is to set aside portions of group meetings to discuss issues of research integrity, including group analysis of current examples of detrimental practices. Supervisors should
be certain that all persons working under them understand their commitment to responsible research and their expectation for responsible conduct. Students, researchers, and staff should be encouraged to be open about results. Constructive skepticism serves a valuable function in research. “Show me the data” is always a legitimate request. Supervisors should cultivate the expectation that others in the group may be asked to confirm complex experiments or unexpected findings, not as a check on the individual competence or integrity of research group members, but as needed to ensure validity.
In addition to the formal supervisory relationships discussed above, mentoring occurs informally in many cases. Individuals may have multiple mentors, both formal and informal, and all have some responsibility for the appropriate socialization of those they mentor. Mentors should be sensitive to the challenges that mentees belonging to underrepresented groups may be facing. Mentors need to avoid the reality and even the appearance of exploitative practices, such as asking graduate students to babysit or house sit. Although the responsibility of avoiding hypercompetitive research environments characterized by intense resource competition lies mainly with institutions and sponsors, as described below, individual supervisors should do what they can to prevent competitiveness in the lab from reaching the point where it becomes harmful.
Best Practice R-5: Peer Review. Strive to be a fair and effective peer reviewer who provides careful reviews, maintains confidentiality, and recognizes and discloses conflicts of interest.
Peer reviewers of grants and journal submissions provide the guiding and corrective machinery that enables the research enterprise to progress. As in other contexts of their work, researchers who serve as reviewers are expected to be honest, objective, and accountable and to preserve confidentiality and protect the ideas of others during the review process. In the context of grant review, peer reviewers are responsible for determining whether a research direction is worthy of funding based on novelty, importance, available data, and whether the proposed methods are suitable for the investigation. For journal submissions, the reviewer’s responsibility is to carefully evaluate the experimental design, presented data, and analysis techniques to determine whether they cumulatively support the presented interpretation and conclusions from the data.
Potential reviewers should completely disclose conflicts of interest to the program office for a grant proposal or to the editor for a journal submission. Upholding fairness as a research value, as discussed in Chapter 2, requires that reviewers be aware of their own biases so as to avoid critiques that are motivated by a desire to defend their own work. The program officer or editor has the responsibility to decide whether a bias or conflict of interest affects a potential reviewer’s eligibility.
Reviewers also need to uphold the confidentiality of the review process by
not sharing materials or ideas from grants or manuscripts under review. Appropriating ideas from grants or manuscripts under review is a form of plagiarism.
Best Practice R-6. Research Compliance. Understand and comply with relevant institutional and governmental regulations governing research, including those specific to a given discipline or field.
Research often involves risks to human subjects and animals, to those in the lab, or to those in the buildings where the research takes place. Because research has a potential for harm, it is regulated by local, state, or federal laws, and human and animal studies are governed by Institutional Review Board and Institutional Animal Care and Use Committee rules, respectively, and regulations imposed by the federal government. Failure to comply with governing rules and regulations can lead to civil—or in some cases criminal—penalties for researchers. Moreover, compliance failures undermine public confidence in the researcher, the institution, the field, and the broader research enterprise.
Researchers have the responsibility to determine what the governing rules are for a designed experiment before the work is conducted. Most institutions have offices that specialize in safety, human experiments, and animal use. These offices should be consulted fully to ensure safety—of the researchers and participants in the experiment or the larger community—and that all governing rules and regulations are satisfied. In some fields, researchers also need to be aware of the risks inherent in doing science, understand the possibilities of harmful consequences that could arise accidentally or through misuse, and take steps to reduce those risks as much as possible.
Finally, researchers need to disclose personal financial interests that might reasonably appear to be related to the research for review by institutional officials at the appropriate time. In many cases, the conflict can be managed through the actions of the researchers involved and through oversight. In some cases, the conflict may not be manageable and must be eliminated or the project may have to be abandoned. Personal financial interests related to the research may have the effect of undermining a reader’s view of the credibility of the results, but honesty and objectivity require that they be listed so that others can draw conclusions about the possible effects.
A best practices checklist for researchers is provided in Box 9-1.
As the employers of researchers and the institutional stewards of financial and other resources that support research, universities and other research institutions in the United States have a number of responsibilities (both formal and informal) for ensuring integrity. According to the Institute of Medicine and the National Research Council, “Each research institution should develop and implement a comprehensive program designed to promote integrity in research, using
multiple approaches adapted to the specific environments within each institution.” (IOM-NRC, 2002) Specific responsibilities include the maintenance of policies and procedures to investigate and address research misconduct—including the responsibility to notify the appropriate federal agency of misconduct investigations involving that agency’s funds—and the provision of educational and training programs for students and faculty to raise awareness of research integrity (IOM-NRC, 2002; NAS-NAE-IOM, 1992; NSF-OIG, 2013; OSTP, 2000).
In addition, research institutions carry a range of research-related legal and regulatory compliance responsibilities, such as administering regulations governing research on human subjects and laboratory animals; acting as stewards, as
required, of data from federally funded research (see NAS-NAE-IOM, 2009a); enforcing environmental and hazardous substance regulations; ensuring proper financial accounting of research funds; and implementing general workplace laws and regulations in areas such as discrimination and harassment. The challenges presented by these myriad, often overlapping regulations are many. Institutional leadership must take a role in seeking a responsible compliance environment that is designed to facilitate and support a quality working and learning environment for all.
Some specific policies and practices of research institutions may differ according to whether they are controlled and operated by public or private universities, other nonprofit entities, for-profit companies, or government bodies. Presentations to the committee by corporate representatives indicated that some multinational companies take a very thorough and systematic approach to training and mentoring young researchers (Williams, 2012).
As experience has accumulated over the past several decades, new perspectives have appeared regarding how research institutions can best foster research integrity. For example, the practice of assessing the climate for research integrity in an institution has emerged and is becoming more widely adopted, and its benefits are becoming more clearly understood (CGS, 2012; IOM-NRC, 2002). Around the world, more attention is being paid to the role of universities and research institutions in ensuring integrity (ESF-ALLEA, 2011; UUK, 2012). The responsibilities of universities and research institutions may change over time due to the challenges raised by new technologies and collaborations (IOM, 2009, 2012).
Best Practice I-1: Management. Integrate research integrity considerations into overall approaches to research, education, and institutional management.
Changes in the funding, structure, and organization of research in the United States and the possible effects of these changes on the incentives of researchers to uphold best practices are discussed in several places in this report. In fulfilling their responsibilities to create an environment where the fundamental values of research are valued and reinforced, institutions need to consider organizational and management issues that have not traditionally been associated with research integrity and have not been traditionally seen as organizational responsibilities. In this regard, institutional leaders and others with research administration responsibilities need to demonstrate through their approach to oversight and implementation of policies that fostering research integrity is a central priority that supports the quality of research. It would be a mistake for institutional and faculty leaders to observe that the institution has basic policies and administrative procedures in place and assume that research integrity issues do not require their attention.
While this is a broad exhortation compared with other best practices presented here, the committee identified several areas for particular focus during the course of the study. To begin, institutions should explicitly evaluate mentoring as
part of their evaluation of faculty. Mentoring and supervision of young researchers at U.S. institutions needs systematic attention and improvement. A review of closed Office of Research Integrity (ORI) cases found that almost three-quarters of supervisors had not reviewed source data with trainees who committed misconduct and two-thirds had not set standards for responsible conduct (Wright et al., 2008). Another recent survey of research faculty found that less than a quarter have had opportunities to participate in faculty training to be a better mentor, advisor, or research teacher, and about one-third of faculty did not or could not remember whether they had guidelines related to their responsibilities to PhD students (Titus and Ballou, 2014). Recent work by the InterAcademy Partnership indicates that the need for improved mentoring of young researchers is a global issue (IAP, 2016).
Another imperative is to regularly communicate relevant institutional policies—such as the definition of research misconduct—as well as the rights and responsibilities of researchers directly to young researchers. Compacts between institutions and postdocs, students, and faculty are one mechanism for such communication. The American Association of Medical Colleges has developed several sample compacts, including one between graduate students and their research advisors and one between postdocs and their mentors (AAMC, 2006, 2008). These are documents of several pages that include bullet points outlining the responsibilities of both parties, such as the responsibility of graduate students to seek regular feedback and the responsibility of graduate advisors not to require students to perform duties unrelated to training and professional development. A particularly important and sometimes vulnerable group is postdocs (Phillips, 2012). Postdocs are formally trainees but are often called upon to be mentors of students or younger postdocs. A 2005 survey of postdocs found that less than half of respondents were aware of institutional policies toward determining authorship, defining misconduct, resolving grievances, or determining the ownership of intellectual property (Davis, 2005).
A related responsibility is for institutions to collect data on career outcomes for recent science and engineering graduate cohorts and postdocs and to provide these data to incoming students and trainees at the front end of their training programs so they are better informed. Providing this information is one indication that the institutions have the students’ best interests at heart. To the extent that students have a realistic perspective of their career prospects and the likelihood of being able to pursue research as a career, they will be better equipped to make decisions about how to proceed with their graduate training
Further, institutions might benefit from keeping track of such organizational and funding issues as the number and proportion of soft-money positions in various departments, as well as trends. As explored elsewhere in the report, the combination of increasing emphasis on soft-money positions and declining success rates for grant applications at agencies such as the National Institutes of Health may have a negative impact on researcher incentives to uphold high standards.
Finally, the committee has noted a trend toward institutions and researchers undertaking more aggressive public relations efforts on behalf of their research activities. Institutions and researchers should impose careful quality control on such efforts. One recent study indicates that the quality of media reporting on discoveries is directly related to the quality of press releases (Schwartz et al., 2011). Well-known cases over the years of aggressively promoted results that turned out to be based on fabricated data, such as the Hwang stem cell case, or were otherwise irreproducible, such as the Fleischmanm-Pons “cold fusion” discovery, provide cautionary tales (Appendix D; Goodstein, 2010). Overhyping may ultimately be both a cause and a consequence of a “winner take all” culture in research where disincentives to cutting corners, or even worse behaviors, are weakened over time (Freeman and Gelber, 2006; Freeman et al., 2001a,b). It may also damage public trust in researchers and in the research enterprise.
Best Practice I-2: Assessment. Perform regular assessments of the climate for research integrity at the institutional and department levels and address weaknesses that are identified.
A baseline expectation is that institutions should create a climate for research integrity and institute supportive policies and practices. The 2002 report Integrity in Scientific Research explains that research organizations “engage in activities that help establish an internal climate and organizational culture that are either supportive of or ambivalent toward the responsible conduct of research” (IOM-NRC, 2002). That report recommended that institutions utilize ongoing self-assessment and peer review in order to evaluate their climate for research integrity and guide continuous improvement. At that time, instruments for that purpose had not been developed.
In recent years, an instrument to assess the organizational climate for research integrity has been developed and validated (Crain et al., 2013; Martinson et al., 2013). A recent Council of Graduate Schools (CGS) project worked with a group of universities to integrate “research ethics and the responsible conduct of research (RCR) into graduate education” (CGS, 2012). The participating universities utilized climate assessment as an important tool to identify areas for improvement and to track progress. One participating institution reports that the data produced by the assessment tool helped efforts to improve research integrity approaches gain traction among the faculty (May, 2013).
Institutions can also assess the effectiveness of their own efforts to promote research integrity. Are allegations or concerns addressed in an appropriate and timely way? Are policies related to transparency and data sharing well understood and followed?
Strengthening education and training in the responsible conduct of research, discussed below, is an important approach to addressing issues uncovered in assessment exercises and improving local research climates. As illustrated by several of the cases discussed in Appendix D and in other parts of the report, if
detrimental research practices are tolerated at the laboratory or department level, it can lead to a vicious circle where young researchers perpetuate these practices in the belief that they are behaving appropriately. In response, institutions might look for other proactive approaches such as placing succinct posters on bulletin boards to encourage best practices. ORI has produced an infographic on how research supervisors can foster integrity that provides an example of the sorts of information that might be communicated (ORI, 2016). The Singapore Statement on Research Integrity (2010) produced by the Second World Conference on Research Integrity is also available as a single-page pdf. Such posters would perhaps be more effective if they were locally produced by labs or departments.
Best Practice I-3: Performing Research Misconduct Investigations. Perform regular inventories of institutional policies, procedures, and capabilities for investigating and addressing research misconduct and address weaknesses that are identified.
Universities and other research institutions are responsible for undertaking fair, thorough, and timely investigations into allegations of research misconduct. A comprehensive assessment of how U.S. research institutions are performing in the area of addressing research misconduct is not possible, because most investigation results and reports are never made public due to confidentiality rules. Over the course of the study, experts who briefed the committee pointed to considerable unevenness in the capabilities of universities to investigate and address allegations of research misconduct (Garfinkel, 2012). In addition, the examples described in other parts of the report, particularly Chapter 7 and Appendix D, illustrate that even the most highly regarded institutions can fail in the performance of basic tasks, such as following appropriate investigation procedures, ensuring that internal committees have the right knowledge and expertise, and ensuring that investigation processes avoid the pitfalls that can result from institutional conflicts of interest.
Regular inventories of institutional policies, procedures, and capabilities can help to ensure that the minimum requirements needed to comply with existing regulations are met, but universities should aim for more than compliance. The requirements of ORI and the National Science Foundation (NSF) should be a floor, not a ceiling.
Ensuring that institutions have the appropriate policies and resources in place to address research misconduct allegations starts with the support and involvement of institutional leaders. Often, concerns can be addressed and questions can be answered at an early stage, obviating the need for formal investigations (Gunsalus, 1998b).
Elements that should be part of institutional capabilities include a trained Research Integrity Officer or other professional who can act on allegations, involvement of the institution’s general counsel’s office, clear policies and procedures that are understood and followed, and support from institutional leadership. In
research universities, faculty leaders play a critical role in the effective communication and implementation of these policies and procedures. Institutions should also protect good-faith whistleblowers and prevent negative career consequences for young researchers who become whistleblowers. This demonstrates the institution’s moral commitment to its students and employees. As illustrated by the Goodwin case, young researchers who do the right thing by raising concerns or making allegations against superiors may find that their research careers are effectively over, even when they uncover misconduct.
Maintaining confidentiality during an investigation, protecting the accused, and minimizing the negative consequences of investigations for those who are cleared are also essential. Institutions need to communicate with federal agencies such as ORI and the NSF Office of Inspector General, sponsors, and journals, as appropriate, to ensure that these entities can fulfill their responsibilities related to the stewardship of funds and correcting the research record.
Institutions also need to have policies and mechanisms in place that allow them to call in external sources of expertise, particularly when their financial, reputational, or other interests may be affected by an allegation. Incorporating external members on the institutional committees that undertake research misconduct investigations is one mechanism for accomplishing this. In some particularly serious or problematic cases, an institution may decide that all members of such a committee should come from outside the institution, although considerations of logistics and cost would make it difficult to institute this as a normal practice. The University of Illinois requires that all investigation committees should include at least one external member (University of Illinois, 2009). In addition, institutions may ask external experts to review the mission statements of investigation committees at the start of the process and the draft reports of committees to help ensure that the appropriate questions and issues are addressed. It is not clear how common external review is currently.
Regular evaluations of capabilities, incorporating perspectives external to the institution, can also help institutions improve their systems and processes over time. For example, in addition to designated institutional points of contact for allegations of misconduct, such as Research Integrity Officers, some institutions have found additional resources, such as ombudsmen and hotlines, to be helpful. In managing a system with multiple entry points, it is necessary to clearly define roles and coordinate responses so that those who are bringing their concerns to the institution do not receive incorrect or conflicting advice. Mediation mechanisms can be put in place for disputes that arise between colleagues or between subordinates and superiors. Ideally, enhanced communication and related interventions will allow many issues and concerns to be addressed before research misconduct occurs. Ensuring that this information is widely disseminated through posting on bulletin boards in labs and through other mechanisms is also important.
Best Practice I-4: Training and Education. Strive for continuous improvement in RCR training and education.
The development of RCR training and education programs and related issues—including funder mandates, content, delivery mechanisms, and assessment—are covered in detail in Chapter 10. The 1992 report Responsible Science noted that institutional RCR education programs were not very common at that time and that the research enterprise was ambivalent about such programs (NAS-NAE-IOM, 1992). Although there is still much to be learned about the effectiveness of particular educational approaches, recognition that institutions have clear responsibilities has grown over time, both in the United States and around the world. The report Integrity in Scientific Research recommended that “institutions should implement effective educational programs that enhance the responsible conduct of research” (IOM-NRC, 2002). The Australian Code for the Responsible Conduct of Research states that
Each institution must provide induction and training for all research trainees. This training should cover research ethics, occupational health and safety, and environmental protection, as well as technical matters appropriate to the discipline. (NHMRC-ARC-UA, 2007)
As is the case with institutional policies and resources to address allegations of research misconduct, the formal requirements of funders should constitute the floor, not the ceiling, for institutional efforts. NIH mandates participation in RCR education for all persons receiving NIH support. This requirement includes instruction in nine core areas: (1) data acquisition, management, sharing, and ownership; (2) mentor/trainee responsibilities; (3) publication practices and responsible authorship; (4) peer review; (5) collaborative science; (6) human subjects; (7) research involving animals; (8) research misconduct; and (9) conflict of interest and commitment (Steneck, 2004). A 2009 update on the Requirement for Instruction in the Responsible Conduct of Research requires RCR training to be provided in person, noting that online instruction is a helpful supplement but is insufficient as the sole provider of RCR training (NIH, 2009). The guidance suggests at least a semester-long series of RCR instruction from faculty on a rotating basis to ensure full faculty participation and that instruction recur through the different levels of a scientist’s career (NIH, 2009). The CGS project discussed below produced a number of possible approaches for institutions aiming to improve RCR education, such as engaging faculty in developing discipline-specific content, holding lunchtime workshops for graduate students, integrating RCR content into courses, and developing courses that escalate in complexity (CGS, 2012). The Integrity in Scientific Research report also recommends RCR instruction be provided by “faculty who are actively engaged in research related to that of the trainees” (IOM-NRC, 2002). The CGS project made recommendations for institutional leaders to demonstrate engagement in RCR education through public
endorsement from the university president and by assembling a steering committee of institutional leaders and a project director to oversee a plan to integrate RCR education into the curriculum (CGS, 2012).
Institutions can participate in and take advantage of other RCR education development efforts. Recently, RCR training has shifted emphasis from the traditional focus on imparting knowledge, specifically of regulations and compliance requirements, toward the potential value of imparting skills in ethical decision making (see Appendix C). The effectiveness of techniques such as team-based learning is also being explored (McCormack and Garvan, 2014). An organization involved in RCR is the National Postdoctoral Association, which oversaw a project aimed at developing RCR educational approaches specifically for postdocs (NPA, 2013).
Box 9-2 provides a best practices checklist for research institutions.
Journals and Other Scholarly Communicators
This section and the associated practices are addressed to journals—editors, governing bodies, and publishers—and other individuals and groups involved with scientific publishing and other forms of scholarly communication, including university librarians, digital archivists, and academic presses.
The basics of responsible publishing include ensuring that a journal’s existing rules and guidelines have been followed, such as those related to data sharing and research involving human subjects (Gustafsson et al., 2006). Editors are also responsible for the scientific quality of the journal. Journals should clearly articulate their publication criteria and evaluate submissions based on those criteria. They should provide the authors of proposed publications with a fair and full account of reviewers’ comments and ensure transparent communication in the event of disputes, questions, or difficulties in the publication process. Journals should make their principles and processes visible to authors, readers, librarians, and peer reviewers. As an example, publishers should disclose sources of funding or other issues that may affect the choice of work to disseminate.
The 1992 report Responsible Science mentions scientific journals and editors and contains a general recommendation that journals and societies support research integrity. Journal concerns and responsibilities related to research integrity have grown and shifted in recent years, as article retractions have increased, a series of high-profile cases of fabricated research published in several high-profile journals has come to light, and relatively new challenges such as image manipulation have prompted journals to develop new policies and approaches. The fact that detecting fabrication often requires specialized technical and analytical tools makes it unlikely that it will be uncovered in the normal peer review process (i.e., before publication).
Although it is sometimes assumed that journal peer review processes are or should be effective mechanisms for uncovering fabricated data and other re-
search misconduct, history and recent experience indicate that this is not the case (Ioannidis, 2012; Stroebe et al., 2012). Most misconduct is uncovered through revelations by whistleblowers or by other scientists who have tried and failed to replicate fabricated research.
Over the years, a number of individual journals and publishing groups, journal associations, and other groups have developed ethical codes and good practice guidelines for scientific publishing (COPE, 2011; CSE, 2012b; ICMJE, 2013; SfN, 2010). Some publication executives and boards regard the Committee
on Publication Ethics (COPE) principles and recommendations as directive and more or less adhere to them. Others regard them as informative and suggestive while holding independent views on responsible publishing that occasionally vary from COPE’s advice. COPE promulgates a mandatory code of conduct for journal editors and a more aspirational set of best practices. COPE has also published a number of guidelines and monographs intended to assist editors and publishers in the course of their work.
Digital innovation has been a major source of disruption in science, engineering, technology, and medical research and publishing, and this has implications for responsible research. Predicting the directions and extent of progress in information technologies is difficult, yet principles and best practices in publishing should be flexible enough to be applied as innovations in research practice arise. The Society for Neuroscience’s recently revised ethics policy and guidelines for responsible conduct in scientific publishing are useful examples (SfN, 2010). The set of guidelines put forward for authors is notable for the detailed specifications given for describing the intellectual contribution of authors.
Some journals have introduced technical checks to detect plagiarism and image manipulation. These tools have been useful in detecting misconduct and detrimental practices in proposed papers. In addition, a recent trend among biomedical journals has been to hire ethics officers. It should be noted that these sorts of steps contribute to rising costs that are passed on to university libraries, other subscribers, and, in the “open access” arena, the authors of research. Still, these costs need to be balanced against the costs incurred in editorial time when a journal has to retract a paper.
Best Practice J-1: Practicing Transparency. Practice transparency in journal policies and practices related to research integrity, including publication of retractions and corrections and the reasons for them.
Openness is fundamental to the success of the entire chain of processes and relationships involved in scholarly communication. This principle translates directly into best practices in publishing, with just a few exceptions. The one obvious exception is that of peer review, in which the identity of peer reviewers has traditionally been hidden so that undue influence on reviewers is minimized, pre- or postpublication, thus creating an environment enabling direct and frank critical commentary for authors and editors by reviewers. As discussed in Chapter 3, improving peer review policies and practices and considering other models—such as unblinded review—are issues currently facing journals and disciplines.
Following this best practice begins with maintaining an up-to-date set of author instructions, as well as ethical policies for authors, reviewers, and editors. The policies should include procedures to be followed when allegations of misconduct arise. Journals should communicate retractions (including the reasons for retractions or why a reason cannot be provided), corrections, clarifications, and apologies promptly and openly to ensure that the published record of research is
as free of bias, error, and falsehoods as possible. New means of electronic communication provide new and potentially powerful ways of correcting the research literature. There is great value in putting retractions in the place of the target article and in tables of contents. Metadata—which is information about a dataset embedded within it—associating each with the target article should be included for ongoing observation and analysis.
In addition, data and code that support an article should be published with the article (or chapter or book) or made otherwise available (e.g., through linking) in its original position in an issue (or edition) as well as a separate issue- or title-level section with its own explicit entry in the table of contents. Publishers and editors should provide for postpublication review and commentary attached to scientific, technical, and medical articles. Such commentary can be helpful in uncovering problems with published work and in exploring promising areas for research that would confirm or extend the reported results.
Journals should have policies in place to prevent conflicts of interest on the part of editorial staff from affecting editorial decisions. One way of handling this would be for editorial staff to provide conflicts of interest in narrative form in articles and as metadata for systematic observation and analysis. Alternatively, the journal might define what constitutes a conflict of interest for any editor, and then state that if an editor has a conflict of interest with any of the authors of a paper, he or she is excluded from handling the paper. Journals would have on hand declarations from their editors that are updated annually or more often as circumstances change. Addressing conflicts of interest of other participants in the publication process is covered below.
Throughout the publishing process, journals should negotiate fairly and as transparently as possible in author, author-reviewer, and author-reader disputes.
While not as directly supportive of research integrity as the other steps outlined above, journals contribute to the effective functioning of the research enterprise by providing open access to publications, perhaps after an embargo period so as not to interfere with a publisher’s business viability.
Best Practice J-2: Requiring Openness. Require openness from authors regarding public access to data, code, and other information necessary to verify or reproduce reported results. Require openness from authors and peer reviewers regarding funding sources and conflicts of interest.
As described in other parts of this report, including Chapter 7, requiring authors to share data and code for purposes of verification, replication, and reuse is an important step that the research enterprise can take to help ensure research integrity. Journals are in a powerful position to implement this step, and some are developing new policies and procedures aimed at ensuring access to data and code (Nature, 2013). Although making data available with the article is the traditional approach in many disciplines, linking to a specialized database or repository will likely be the preferred way to provide access to data in most cases.
One example of efforts to expand the availability of data is a 2016 proposal by the International Committee of Medical Journal Editors that in order for an article to be considered for publication authors should be required to commit to publish “deidentified individual-patient data underlying the results” of clinical trial research within 6 months of the corresponding article for reproducibility purposes (Taichman et al., 2016).
The data to be made available should include outlier data and negative results if appropriate. Alterations to images should be specified. In cases where regulatory, legal, or technological constraints prevent authors from providing full access to data, an explanation should be published along with the paper.
Journals should work with sponsors, authors, and research institutions to ensure long-term access to data, code, and other information supplementary to the article. Archiving of articles and supplementary information by third parties is the ultimate goal, although securing the necessary resources and developing the appropriate mechanisms remain challenging tasks in some fields and disciplines.
It is also important for full method descriptions to be included in every publication. Currently, references to method sections in previously published work are common in some fields, but this may cause ambiguity as to what was actually done. With the availability of electronic supplements, there is no reason why full methods cannot be included, even if this means reprinting what the same author published previously. Good practice should not be discouraged by concerns about self-duplication if this increases transparency and reduces ambiguity.
Financial conflicts of interests, other relevant financial relationships, and relevant nonfinancial interests should be identified by all authors and included in print and as metadata (PLOS Medicine Editors, 2008). For example, “publishing relevant competing interests for all contributors and publishing corrections if competing interests are revealed after publication” is a best practice listed in COPE’s guidelines (COPE, 2011). This disclosure should include an explicit citation of support from funders, whether corporate or not for profit.
Journals should also take steps to safeguard the integrity of the peer review process. COPE’s guidelines for peer reviewers include submitting a declaration of potential competing interests, respecting the confidentiality of the process, and not intentionally delaying the process (Hames, 2013). Journals might ask reviewers to explicitly commit to these guidelines by signing a statement.
Best Practice J-3: Authorship Contributions. Require that the contributions and roles of all authors be described.2
2 In Recommendation Five, this report calls for the development and adoption of authorship standards and suggests a framework that if adopted would formally codify the requirement that the roles of authors be disclosed across all fields and disciplines. See Chapter 8 for the rationale underlying the recommendation and Chapter 11 for the recommendation text.
Article authors are the researchers who have contributed significantly to the article and are listed in the article byline. Authorship determines who receives credit for the work and fixes responsibility if or when mistakes or misconduct is uncovered. While guidance on authorship is provided by journals, institutions, societies, and other groups, specific practices vary by discipline. Although detrimental authorship practices other than plagiarism have not been included in the U.S. government’s definition of research misconduct, practices such as honorary authorship and unacknowledged ghost authorship, as well as authorship disputes, pose challenges to research integrity. The Council of Science Editors points out that “problems with authorship are not uncommon and can threaten the integrity of scientific research” (CSE, 2012b). A recent review of research on authorship across all fields found that 29 percent of researchers in several separate studies reported that they or others they know had experiences involving the misuse of authorship (this figure could be inflated by multiple reports of the same behavior in some of the reviewed studies) (Marušić et al., 2011).
In an environment of increasing collaboration across institutions and borders, it may be more difficult to determine who is responsible for mistakes or fabricated work. In some cases of fabricated or falsified research, senior researchers have claimed that they were merely honorary authors and therefore were not responsible for the integrity of the reported work.
These issues pose challenges to journals, which have responded by paying increasing attention to authorship. One journal practice that has become fairly widespread is to require authors to describe their individual contributions, which are published in a designated place in the article. Journals such as the Lancet began adopting this practice in the 1990s (Yank and Rennie, 1999). The Nature Publishing Group journals, which had requested that authors provide contribution disclosures beginning in 1999, made them mandatory in 2009 (Nature, 2009). At the same time, Nature had considered requiring corresponding authors to sign a statement that they had taken some integrity assurance steps, but there was significant skepticism about this proposal.
Most current contribution disclosures tend to be fairly broad. For example, the Proceedings of the National Academy of Sciences provides an example list of contributions that includes research design, research performance, contribution of new reagents or analytic tools, data analysis, and writing (PNAS, 2013). Advances in technology hold out the possibility that such contribution disclosures can become more detailed and useful in the future, providing the underlying tools for researchers to maintain up-to-date, verified accounts of their work (Frische, 2012).
For now, journals should require contribution disclosures at as detailed a level as practical and be open to adjusting these requirements as technologies and tools evolve. For peer-reviewed papers, all authors should be identified along with the sources of funding for their work. To avoid questions of duplication, previously published materials should be identified and cited.
Best Practice J-4: Training and Education. Facilitate regular training and education in responsible publishing policies and best practices for editors, reviewers, and authors.
Best practices for research institutions and mentors in RCR training and education are described above. Journals can play an important role in focused areas of RCR education as well. It is particularly important for editors to be knowledgeable about responsible publishing practices, requirements that need to be communicated to authors and reviewers, and what to do if problems arise. Some aspects of responsible writing, reviewing, and editing may not be covered in RCR training provided to graduate students. A recent review indicates that many writers, reviewers, and editors lack the necessary training to play their roles effectively, but little is known about the availability and effectiveness of such training (Galipeau et al., 2013). The Council of Science Editors, which has provided training for editors for some time, recently launched a certificate program in scholarly publication management (CSE, 2012a). A 2006 paper recommended that an international online training and accreditation program for peer reviewers should be established (Benos et al., 2007).
Journals have varied capabilities and resources to encourage training or to undertake their own educational programs. They should take what steps are appropriate to their own circumstances to help ensure that authors, reviewers, and editors are well prepared to perform their tasks.
Best Practice J-5: Collaboration. Work with other journals to develop common approaches and tools to foster research integrity.
As described elsewhere in this section, the work of groups such as the Committee on Publication Ethics, International Committee of Medical Journal Editors, and Council of Science Editors has been of great value to the research enterprise in developing policies, tools, and approaches to ensure research integrity. While individual journals and other scholarly communicators need to maintain the independence to adopt policies and practices that are appropriate to their circumstances, continued collective efforts by journals can contribute to improvements in standards and practices across the enterprise. Uniform policies reinforce the norms of research integrity.
Box 9-3 provides a best practices checklist for journals and other scholarly communicators.
Research Sponsors and Users of Research Results
Sponsors and users of research occupy particularly important positions in the research enterprise. In general, researchers and research institutions rely on funding from government and private-sector sponsors such as industry and foundations to perform their work. The incentive structures created by sponsors
can have a significant influence on the motivations and behaviors of researchers and institutions. The changing environment for research funding and the resulting pressures on researchers are described in Chapter 3 and Chapter 6. While specific recommendations to sponsors are developed in Chapter 11, this section identifies several specific best practices that research sponsors and users of research results can adopt to ensure research integrity.
The 1992 report Responsible Science recommended several roles for government research sponsors related to integrity, including adopting a common framework of definitions of research misconduct and common policies, adopting policies and procedures that ensure appropriate and prompt responses to allegations of misconduct, and providing support for institutional efforts to discourage questionable research practices (NAS-NAE-IOM, 1992). The 2002 report Integrity in Scientific Research recommended that research sponsors support work to increase understanding of the factors that influence research integrity, including monitoring and assessing those factors (IOM-NRC, 2002). As discussed in
Chapter 6, the Office of Research Integrity and the National Science Foundation maintain programs to support such research.
U.S. government research sponsors such as the National Institutes of Health and the National Science Foundation have imposed several mandates and other regulatory requirements on research institutions and researchers over the past several decades covering RCR education and training. The Office of Research Integrity also requires institutions to file an assurance that they have developed and will comply with policies for addressing allegations of misconduct in Public Health Service–sponsored research that meet Public Health Service policies.
The need for research sponsors to take an active role in fostering research integrity is becoming more recognized around the world. The Irish Council for Bioethics report Recommendations for Promoting Research Integrity (ICB, 2010) provides a useful overview of various approaches. The Global Research Council’s Statement of Principles on Research Integrity is a succinct list of funding agency responsibilities that includes promotion of education, leading by example, and conditioning support on upholding research integrity (GRC, 2013). The InterAcademy Council and InterAcademy Panel (IAC-IAP, 2012) have also described the responsibilities of funding agencies in Responsible Conduct in the Global Research Enterprise: A Policy Report.
Best Practice RS-1. Research Integrity and Quality. Align funding and regulatory policies with the promotion of research integrity and research quality.
Aligning funding and regulatory policies with the promotion of research integrity and research quality has several distinct aspects. For example, as described in Chapter 4, some funding agencies and regulatory bodies maintain policies on research misconduct and exercise oversight over how institutions address allegations of misconduct. Private foundations such as the Howard Hughes Medical Institute also have research misconduct policies (HHMI, 2007). As discussed in Chapter 9, agencies require grantee institutions to provide RCR education. Funders that play these roles should ensure that their policies are clear and implemented consistently. Additional commentary on the policies and practices of U.S. government agencies is provided in Chapter 7 in support of the committee’s recommendations in this area.
A second aspect of aligning policies and practices with the promotion of research integrity is to increase awareness of how funding policies affect research integrity and to make adjustments when possible and necessary. This may involve support for research that illuminates issues related to research integrity. For example, in recent years the Office of Research Integrity has responded to evidence that the institutional environment has a major impact on research integrity by supporting efforts to study, assess, and strengthen those environments. Some policy initiatives might be based on direct understanding of a situation rather than the results of sponsored research—ORI has also sought to address unevenness
in institutional capacity to respond to allegations of misconduct by supporting professional training for research integrity officers.
A recent international report has pointed out that funders have a responsibility to ensure that funding policies not cause researchers and research institutions to emphasize quantity over quality (IAC-IAP, 2012). Chapter 6 explores whether changes in the level and structure of research funding might be associated with detrimental research practices or misconduct. As explained there, this is a complex issue. Evaluating the extent of possible problems and recommending solutions are beyond the scope of this committee’s task. Nevertheless, agencies may already be collecting relevant data on how changes in funding and organization are affecting research environments (NIH, 2012a). Sponsors should look for opportunities to develop evidence on possible impacts of funding policies on the researchers and institutions that are supported, including impacts on integrity, and take appropriate actions. One example is the NIH policy that limits the number of publications that can be listed in the biosketch submitted in grant and cooperative agreement applications, which may help reduce incentives for researchers to maximize the number of publications (NIH, 2014).
Finally, research funders can take steps to coordinate and harmonize their activities within their own domestic contexts as well as internationally. Examples of international cooperation include NSF’s participation in the Global Research Council and Organisation for Economic Co-operation and Development Working Group activities to develop common approaches to dealing with research integrity issues across member countries (GRC, 2013; OECD, 2009, 2007). The Fogarty International Center, part of NIH, supports capacity building in bioethics and research integrity in the developing world.
Best Practice RS-2. Data and Code. Promote access to data and code underlying publicly reported results.
The importance of ensuring access to data and code for research integrity and quality is covered above with reference to journal practices and policies. Funders have important roles to play as well. The America COMPETES Reauthorization of 2010 called on federal agencies to ensure access to publications and data resulting from work that they support, and the Office of Science and Technology Policy began working with agencies on implementing the legislation in early 2013 (Holdren, 2013). Federal sponsors can also play a role in providing resources to cover the costs borne by researchers and institutions in making data and code available. Funders will play a critical role in supporting the development of necessary infrastructure, such as data and sample repositories, efforts to develop metadata standards, and the development of applications that facilitate the direct deposit of data to the repositories complete with the metadata. Without those efforts and tools, compliance for data deposition will be low, and the ability of others to use the data for reproducibility will be hampered.
Industry research sponsors also have important contributions to make in
this area. Clinical trial data constitute a prominent specific example. Over the years, the share of clinical trials funded by industry has grown (Buchkowsky and Jewesson, 2004). At the same time, pressure has grown to make the clinical trial process more transparent through mechanisms such as public registration of all trials and encouraging the release of all results, including negative results. A recent report states that there are “compelling justifications for sharing clinical trial data to benefit society and future patients” (IOM, 2015). There is a need to ensure that data sharing is done responsibly and protects privacy. Lack of timely reporting of clinical trials is not solely or even primarily an issue in industry-performed or industry-sponsored work; clinical trials performed at academic medical centers and sponsored by federal agencies and other nonindustry sources also need to improve their practices (Chen et al., 2016). Still, since clinical trials are an important component of industry-sponsored research that is published in peer-reviewed journals, industry sponsors can make an important contribution by registering all of their trials, reporting all results in a timely way, and sharing data responsibly.
In September 2016, NIH issued a final policy to promote broad and responsible dissemination of information from NIH-funded clinical trials through ClinicalTrials.gov. Under this policy, every clinical trial funded in whole or in part by NIH is expected to be registered on ClinicalTrials.gov and have summary results information submitted and posted in a timely manner, whether subject to section 402(j) of the Public Health Service Act or not (NIH, 2016).
Best Practice RS-3: Utilizing Research. Practice impartiality and transparency in utilizing research for the development of policy and regulations.
As discussed in Chapter 3, scientific evidence and inputs are increasingly important to numerous areas of policy making—public health, environmental protection, economic development, criminology, food safety, education, and many other areas. The interpretation of research results is a central part of many contentious policy debates, which often feature accusations that science is being manipulated or distorted by powerful interests.
One recent report identifies the five “tasks” that science has in relation to policy: “(1) identify problems, such as endangered species, obesity, unemployment, and vulnerability to natural disasters or terrorist acts; (2) measure their magnitude and seriousness; (3) review alternative policy interventions; (4) systematically assess the likely consequences of particular policy actions—intended and unintended, desired and unwanted; and (5) evaluate what, in fact, results from policy” (NRC, 2012b). The report also develops a framework for understanding how science is used in policy and points to areas where better knowledge could improve the utilization of science in policy making.
The utilization of science as an input to policy is a broad, complex field that this report cannot cover in detail. It raises questions and issues of global concern that scientists, policy makers, and citizens of nations around the world
will be wrestling with for years to come (Gluckman, 2014). At the same time, the responsible communication of results to policy makers and the public by researchers, and the adoption of best practices by governments in utilizing that input, are important components of scientific integrity that are closely related to other issues discussed in this report.
Recent efforts to define and implement best practices in utilizing science for policy making have focused on the development of clear policies and procedures and the utilization of transparent processes. For example, a 2009 report of the Bipartisan Policy Center explored the need for clearer policies governing the disclosure of relevant relationships by potential members of federal advisory committees, including expert testimony and consulting relationships, to prevent conflicts of interest in these activities (BPC, 2009).
As discussed in Chapter 3, the Obama administration launched an initiative in 2010 to require all federal agencies to develop and adopt scientific integrity policies (Holdren, 2010). Although an analysis by the Union of Concerned Scientists concluded that the efforts of a number of agencies fell short of what is needed to “promote and support a culture of scientific integrity,” the universal adoption of such policies is certainly an important step (Grifo, 2013).
Box 9-4 provides a best practices checklist for research sponsors and users of research.
According to one perspective on the role of scientific societies in fostering research integrity, “As visible, stable, and enduring institutions, scientific societies serve as the custodian for a discipline’s norms and traditions, transmitting them to their members and helping to translate them into accepted research practices” (Frankel and Bird, 2003). The focus here is on disciplinary societies, although it should be noted that the largest general professional association of scientists, the American Association for the Advancement of Science, has been active over the years in a number of areas related to research integrity. Several members of the committee met with a large number of scientific society representatives as part of this study, discussing the concerns and issues facing societies and learning about what they are doing to foster integrity. Many societies publish journals as one of their core activities, and best practices associated with publishing are covered above.
Honorific academies can also play a constructive role in fostering research integrity in their national contexts, and interacademy networks can contribute at the international level by developing and disseminating guidelines and educational materials (ESF-ALLEA, 2011; IAP, 2016; NAS-NAE-IOM, 2009b).
Best Practice S-1. Standards and Education. Serve as a focal point within their disciplines for the development and updating of standards, dissemination of best practices, and fostering RCR education appropriate to the discipline.
The specific areas where many societies are active, apart from those related to publication, are the formulation of codes of conduct and educational efforts (Macrina, 2007). Responsible Science asserted that societies should play a key role in developing guidelines for research conduct appropriate to their specific fields (NAS-NAE-IOM, 1992). Many societies developed codes of conduct when research misconduct became a prominent issue in the late 1980s and 1990s, covering issues such as data handling, authorship, mentoring, and research misconduct. An American Association for the Advancement of Science survey undertaken in 2000 reported on the content and subject matter coverage of society ethics codes (Iverson et al., 2003). The American Society for Microbiology, for example, developed its first code of conduct in 1988, and it has been revised several times since (Macrina, 2007). This points to the importance of regularly updating codes of conduct in order to keep pace with changing research practices within disciplines and new ethical issues.
Societies have been active in fostering RCR education. One mechanism for doing this is through workshops or symposia held during the society’s annual meeting (Iverson et al., 2003). ORI has provided support for these efforts (Macrina, 2007). Societies can also develop case studies and other educational materials that illustrate ethical issues that can arise in their disciplines. One example is