Problems of scientific freedom and responsibility are not new; one need only consider, as examples, the passionate controversies that were stirred by the work of Galileo and Darwin. In our time, however, such problems have changed in character, and have become far more numerous, more urgent and more complex. Science and its applications have become entwined with the whole fabric of our lives and thoughts. . . . Scientific freedom, like academic freedom, is an acquired right, generally accepted by society as necessary for the advancement of knowledge from which society may benefit. Scientists possess no rights beyond those of other citizens except those necessary to fulfill the responsibility arising from their special knowledge, and from the insight arising from that knowledge.
Synopsis: The integrity of research is based on adherence to core values—objectivity, honesty, openness, fairness, accountability, and stewardship. These core values help to ensure that the research enterprise advances knowledge. Integrity in science means planning, proposing, performing, reporting, and reviewing research in accordance with these values. Participants in the research enterprise stray from the norms and appropriate practices of science when they commit research misconduct or other misconduct or engage in detrimental research practices.
The core values and guiding norms of science have been studied and written about extensively, with the work of Robert Merton providing a foundation for subsequent work on the sociology of science (Merton, 1973). Merton posited a set of norms that govern good science: (1) Communalism (common ownership of scientific knowledge), (2) Universalism (all scientists can contribute to the advance of knowledge), (3) Disinterestedness (scientists should work for the good of the scientific enterprise as opposed to personal gain), and (4) Organized Skepticism (results should be examined critically before they are accepted). Research on scientists and scientific organizations has also led to a better understanding of
counternorms that appear to conflict with the dominant Mertonian norms but that are recognized as playing an inherent part in the actual practice of science, such as the personal commitment that a scientist may have to a particular hypothesis or theory (Mitroff, 1974).
More recent work on the effectiveness of responsible conduct of research education, covered in more detail in Chapter 9, explores evidence that at least some scientists may not understand and reflect upon the ethical dimensions of their work (McCormick et al., 2012). Several causes are identified, including a lack of awareness on the part of researchers of the ethical issues that can arise, confidence that they can identify and address these issues without any special training or help, or apprehension that a focus on ethical issues might hinder their progress. An additional challenge arises from the apparent gap “between the normative ideals of science and science’s institutional reward system” (Devereaux, 2014). Chapter 6 covers this issue in more detail. Here, it is important to note that identifying and understanding the values and norms of science do not automatically mean that they will be followed in practice. The context in which values and norms are communicated and transmitted in the professional development of scientists is critically important.
Scientists are privileged to have careers in which they explore the frontiers of knowledge. They have greater autonomy than do many other professionals and are usually respected by other members of society. They often are able to choose the questions they want to pursue and the methods used to derive answers. They have rich networks of social relationships that, for the most part, reinforce and further their work. Whether actively involved in research or employed in some other capacity within the research enterprise, scientists are able to engage in an activity about which they are passionate: learning more about the world and how it functions.
In the United States, scientific research in academia emerged during the late 19th century as an “informal, intimate, and paternalistic endeavor” (NAS-NAE-IOM, 1992). Multipurpose universities emphasized teaching, and research was more of an avocation than a profession. Even today, being a scientist and engaging in research does not necessarily entail a career with characteristics traditionally associated with professions such as law, medicine, architecture, some subfields of engineering, and accounting. For example, working as a researcher does not involve state certification of the practitioner’s expertise as a requirement to practice, nor does it generally involve direct relationships with fee-paying clients. Many professions also maintain an explicit expectation that practitioners will adhere to a distinctive ethical code (Wickenden, 1949). In contrast, scientists do not have a formal, overarching code of ethics and professional conduct.
However, the nature of professional practice even in the traditional professions continues to evolve (Evetts, 2013). Some scholars assert that the concept of professional work should include all occupations characterized by “expert knowledge, autonomy, a normative orientation grounded in community, and
high status, income, and other rewards” (Gorman and Sandefur, 2011). Scientific research certainly shares these characteristics. In this respect, efforts to formalize responsible conduct of research training in the education of researchers often have assumed that this training should be part of the professional development of researchers (IOM-NRC, 2002; NAS-NAE-IOM, 1992). However, the training of researchers (and research itself) has retained some “informal, intimate, and paternalistic” features. Attempts to formalize professional development training sometimes have generated resistance in favor of essentially an apprenticeship model with informal, ad hoc approaches to how graduate students and postdoctoral fellows learn how to become professional scientists.
One challenge facing the research enterprise is that informal, ad hoc approaches to scientific professionalism do not ensure that the core values and guiding norms of science are adequately inculcated and sustained. This has become increasingly clear as the changes in the research environment described in Chapter 3 have emerged and taken hold. Indeed, the apparent inadequacy of these older forms of training to the task of socializing and training individuals into responsible research practices is a recurring theme of this report.
Individual scientists work within a much broader system that profoundly influences the integrity of research results. This system, described briefly in Chapter 1, is characterized by a massive, interconnected web of relationships among researchers, employing institutions, public and private funders, and journals and professional societies. This web comprises unidirectional and bidirectional obligations and responsibilities between the parts of the system. The system is driven by public and private investments and results in various outcomes or products, including research results, various uses of those results, and trained students. However, the system itself has a dynamic that shapes the actions of everyone involved and produces results that reflect the functioning of the system. Because of the large number of relationships between the many players in the web of responsibility, features of one set of relationships may affect other parts of the web. These interdependencies complicate the task of devising interventions and structures that support and encourage the responsible conduct of research.
The integrity of research is based on the foundational core values of science. The research system could not operate without these shared values that shape the behaviors of all who are involved with the system. Out of these values arise the web of responsibilities that make the system cohere and make scientific knowledge reliable. Many previous guides to responsible conduct in research have identified and described these values (CCA, 2010; ESF-ALLEA, 2011; IAC-IAP, 2012; ICB, 2010; IOM-NRC, 2002). This report emphasizes six values that are most influential in shaping the norms that constitute research practices and relationships and the integrity of science:
This chapter examines each of these six values in turn to consider how they shape, and are realized in, research practices.
The first of the six values discussed in this report—objectivity—describes the attitude of impartiality with which researchers should strive to approach their work. The next four values—honesty, openness, accountability, and fairness—describe relationships among those involved in the research enterprise. The final value—stewardship—involves the relationship between members of the research enterprise, the enterprise as a whole, and the broader society within which the enterprise is situated. Although we discuss stewardship last, it is an essential value that perpetuates the other values.
The hallmark of scientific thinking that differentiates it from other modes of human inquiry and expression such as literature and art is its dedication to rational and empirical inquiry. In this context, objectivity is central to the scientific worldview. Karl Popper (1999) viewed scientific objectivity as consisting of the freedom and responsibility of the researcher to (1) pose refutable hypotheses, (2) test the hypotheses with the relevant evidence, and (3) state the results clearly and unambiguously to any interested person. The goal is reproducibility, which is essential to advancing knowledge through experimental science. If these steps are followed diligently, Popper suggested, any reasonable second researcher should be able to follow the same steps to replicate the work.
Objectivity means that certain kinds of motivations should not influence a researcher’s action, even though others will. For example, if a researcher in an experimental field believes in a particular hypothesis or explanation of a phenomenon, he or she is expected to design experiments that will test the hypothesis. The experiment should be designed in a way that allows the possibility for the hypothesis to be disconfirmed. Scientific objectivity is intended to ensure that scientists’ personal beliefs and qualities—motivations, position, material interests, field of specialty, prominence, or other factors—do not introduce biases into their work.
As will be explored in later chapters, in practice it is not that simple. Human judgment and decisions are prone to a variety of cognitive biases and systematic errors in reasoning. Even the best scientific intentions are not always sufficient to ensure scientific objectivity. Scientific objectivity can be compromised acci-
dentally or without recognition by individuals. In addition, broader biases of the reigning scientific paradigm influence the theory and practice of science (Kuhn, 1962). A primary purpose of scientific replication is to minimize the extent to which experimental findings are distorted by biases and errors. Researchers have a responsibility to design experiments in ways that any other person with different motivations, interests, and knowledge could trust the results. Modern problems related to reproducibility are explored later in the report.
In addition, objectivity does not imply or require that researchers can or should be completely neutral or disinterested in pursuing their work. The research enterprise does not function properly without the organized efforts of researchers to convince their scientific audiences. Sometimes researchers are proven correct when they persist in trying to prove theories in the face of evidence that appears to contradict them.
It is important to note, in addition, Popper’s suggestion that scientific objectivity consists of not only responsibility but freedom. The scientist must be free from pressures and influences that can bias research results. Objectivity can be compromised when institutional expectations, laboratory culture, the regulatory environment, or funding needs put pressure on the scientist to produce positive results or to produce them under time pressure. Scientists and researchers operate in social contexts, and the incentives and pressures of those contexts can have a profound effect on the exercise of scientific methodology and a researcher’s commitment to scientific objectivity.
Scientific objectivity also must coexist with other human motivations that challenge it. As an example of such a challenge, a researcher might become biased in desiring definitive results evaluating the validity of high-profile theories or hypotheses that their experiments were designed to support or refute. Both personal desire to obtain a definitive answer and institutional pressures to produce “significant” conclusions can provide strong motivation to find definitive results in experimental situations. Dedication to scientific objectivity in those settings represents the best guard against scientists finding what they desire instead of what exists. Institutional support of objectivity at every level—from mentors, to research supervisors, to administrators, and to funders—is crucial in counterbalancing the very human tendency to desire definitive outcomes of research.
A researcher’s freedom to advance knowledge is tied to his or her responsibility to be honest. Science as an enterprise producing reliable knowledge is based on the assumption of honesty. Science is predicated on agreed-upon systematic procedures for determining the empirical or theoretical basis of a proposition. Dishonest science violates that agreement and therefore violates a defining characteristic of science.
Honesty is the principal value that underlies all of the other relationship val-
ues. For example, without an honest foundation, realizing the values of openness, accountability, and fairness would be impossible.
Scientific institutions and stakeholders start with the assumption of honesty. Peer reviewers, granting agencies, journal editors, commercial research and development managers, policy makers, and other players in the scientific enterprise all start with an assumption of the trustworthiness of the reporting scientist and research team. Dishonesty undermines not only the results of the specific research but also the entire scientific enterprise itself, because it threatens the trustworthiness of the scientific endeavor.
Being honest is not always straightforward. It may not be easy to decide what to do with outlier data, for example, or when one suspects fraud in published research. A single outlier data point may be legitimately interpreted as a malfunctioning instrument or a contaminated sample. However, true scientific integrity requires the disclosure of the exclusion of a data point and the effect of that exclusion unless the contamination or malfunction is documented, not merely conjectured. There are accepted statistical methods and standards for dealing with outlier data, although questions are being raised about how often these are followed in certain fields (Thiese et al., 2015).
Dishonesty can take many forms. It may refer to out-and-out fabrication or falsification of data or reporting of results or plagiarism. It includes such things as misrepresentation (e.g., avoiding blame, claiming that protocol requirements have been followed when they have not, or producing significant results by altering experiments that have been previously conducted), nonreporting of phenomena, cherry-picking of data, or overenhancing pictorial representations of data. Honest work includes accurate reporting of what was done, including the methods used to do that work. Thus, dishonesty can encompass lying by omission, as in leaving out data that change the overall conclusions or systematically publishing only trials that yield positive results. The “file drawer” effect was first discussed almost 40 years ago; Robert Rosenthal (1979) presented the extreme view that “journals are filled with the 5 percent of the studies that show Type I errors, while the file drawers are filled with the 95 percent of the studies that show non-significant results.” This hides the possibility of results being published from 1 significant trial in an experiment of 100 trials, as well as experiments that were conducted and then altered in order to produce the desired results. The file drawer effect is a result of publication bias and selective reporting, the probability that a study will be published depending on the significance of its results (Scargle, 2000). As the incentives for researchers to publish in top journals increase, so too do these biases and the file drawer effect.
Another example of dishonesty by omission is failing to report all funding sources where that information is relevant to assessing potential biases that might influence the integrity of the work. Conversely, dishonesty can also include reporting of nonexistent funding sources, giving the impression that the research
was conducted with more support and so may have been more thorough than in actuality.
Beyond the individual researcher, those engaged in assessing research, whether those who are funding it or participating in any level of the peer review process, also have fundamental responsibilities of honesty. Most centrally, those assessing the quality of science must be honest in their assessments and aware of and honest in reporting their own conflicts of interest or any cognitive biases that may skew their judgment in self-serving ways. There is also a need to guard against unconscious bias, sometimes by refusing to assess work even when a potential reviewer is convinced that he or she can be objective. Efforts to protect honesty should be reinforced by the organizations and systems within which those assessors function. Universities, research organizations, journals, funding agencies, and professional societies must all work to hold each other to honest interactions without favoritism and with potentially biasing factors disclosed.
Openness is not the same as honesty, but it is predicated on honesty. In the scientific enterprise, openness refers to the value of being transparent and presenting all the information relevant to a decision or conclusion. This is essential so that others in the web of the research enterprise can understand why a decision or conclusion was reached. Openness also means making the data on which a result is based available to others so that they may reproduce and verify results or build on them. In some contexts, openness means listening to conflicting ideas or negative results without allowing preexisting biases or expectations to cloud one’s judgment. In this respect, openness reinforces objectivity and the achievement of reliable observations and results.
Openness is an ideal toward which to strive in the research enterprise. It almost always enhances the advance of knowledge and facilitates others in meeting their responsibilities, be it journal editors, reviewers, or those who use the research to build products or as an input to policy making. Researchers have to be especially conscientious about being open, since the incentive structure within science does not always explicitly reward openness and sometimes discourages it. An investigator may desire to keep data private to monopolize the conclusions that can be drawn from those data without fear of competition. Researchers may be tempted to withhold data that do not fit with their hypotheses or conclusions. In the worst cases, investigators may fail to disclose data, code, or other information underlying their published results to prevent the detection of fabrication or falsification.
Openness is an ideal that may not always be possible to achieve within the research enterprise. In research involving classified military applications, sensitive personal information, or trade secrets, researchers may have an obligation not to disseminate data and the results derived from those data. Disclosure of results
and underlying data may be delayed to allow time for filing a patent application. These sorts of restrictions are more common in certain research settings—such as commercial enterprises and government laboratories—than they are in academic research institutions performing primarily fundamental work. In the latter, openness in research is a long-held principle shared by the community, and it is a requirement in the United States to avoid privileged access that would undermine the institution’s nonprofit status and to maintain the fundamental research exclusion from national security-based restrictions.
As the nature of data changes, so do the demands of achieving openness. For example, modern science is often based on very large datasets and computational implementations that cannot be included in a written manuscript. However, publications describing such results could not exist without the data and code underlying the results. Therefore, as part of the publication process, the authors have an obligation to have the available data and commented code or pseudocode (a high-level description of a program’s operating principle) necessary and sufficient to re-create the results listed in the manuscript. Again, in some situations where a code implementation is patentable, a brief delay in releasing the code in order to secure intellectual property protection may be acceptable. When the resources needed to make data and code available are insufficient, authors should openly provide them upon request. Similar considerations apply to such varied forms of data as websites, videos, and still images with associated text or voiceovers.
Central to the functioning of the research enterprise is the fundamental value that members of the community are responsible for and stand behind their work, statements, actions, and roles in the conduct of their work. At its core, accountability implies an obligation to explain and/or justify one’s behavior. Accountability requires that individuals be willing and able to demonstrate the validity of their work or the reasons for their actions. Accountability goes hand in hand with the credit researchers receive for their contributions to science and how this credit builds their reputations as members of the research enterprise. Accountability also enables those in the web of relationships to rely on work presented by others as a foundation for additional advances.
Individual accountability builds the trustworthiness of the research enterprise as a whole. Each participant in the research system, including researchers, institutional administrators, sponsors, and scholarly publishers, has obligations to others in the web of science and in return should be able to expect consistent and honest actions by others in the system. Mutual accountability therefore builds trust, which is a consequence of the application of the values described in this report.
The purpose of scientific publishing is to advance the state of knowledge through examination by peers who can assess, test, replicate where appropriate, and build on the work being described. Investigators reporting on their work thus
must be accountable for the accuracy of their work. Through this accountability, they form a compact with the users of their work. Readers should be able to trust that the work was performed by the authors as described, with honest and accurate reporting of results. Accountability means that any deviations from the compact would be flagged and explained. Readers then could use these explanations in interpreting and evaluating the work.
Investigators are accountable to colleagues in their discipline or field of research, to the employer and institution at which the work is done, to the funders or other sponsors of the research, to the editors and institutions that disseminate their findings, and to the public, which supports research in the expectation that it will produce widespread benefits. Other participants in the research system have other forms of accountability. Journals are accountable to authors, reviewers, readers, the institutions they represent, and other journals (for the reuse of material, violation of copyright, or other issues of mutual concern). Institutions are accountable to their employees, to students, to the funders of both research and education, and to the communities in which they are located. Organizations that sponsor research are accountable to the researchers whose work they support and to their governing bodies or other sources of support, including the public. These networks of accountability support the web of relationships and responsibilities that define the research enterprise.
The accountability expected of individuals and organizations involved with research may be formally specified in policies or regulations. Accountability under institutional research misconduct policies, for example, could mean that researchers will face reprimand or other corrective actions if they fail to meet their responsibilities.
While responsibilities that are formally defined in policies or regulations are important to accountability in the research enterprise, responsibilities that may not be formally specified should also be included in the concept. For example, senior researchers who supervise others are accountable to their employers and the researchers whom they supervise to conduct themselves as professionals, as this is defined by formal organizational policies. On a less formal level, research supervisors are also accountable for being attentive to the educational and career development needs of students, postdoctoral fellows, and other junior researchers whom they oversee. The same principle holds for individuals working for research institutions, sponsoring organizations, and journals.
The scientific enterprise is filled with professional relationships. Many of them involve judging others’ work for purposes of funding, publication, or deciding who is hired or promoted. Being fair in these contexts means making professional judgments based on appropriate and announced criteria, including processes used to determine outcomes. Fairness in adhering to explicit criteria
and processes reinforces a system in which the core values can operate and trust among the parties can be maintained.
Fairness takes on another dimension in designing criteria and evaluation mechanisms. Research has demonstrated, for example, that grant proposals in which reviewers were blinded to applicant identity and institution receive systematically different funding decisions compared with the outcomes of unblinded reviews (Ross et al., 2006). Truly blinded reviews may be difficult or impossible in a small field. Nevertheless, to the extent possible, the criteria and mechanisms involved in evaluation must be designed so as to ensure against unfair incentive structures or preexisting cultural biases. Fairness is also important in other review contexts, such as the process of peer reviewing articles and the production of book reviews for publication.
Fairness is a particularly important consideration in the list of authors for a publication and in the citations included in reports of research results. Investigators may be tempted to claim that senior or well-known authors played a larger role than they actually did so that their names may help carry the paper to publication and readership. But such a practice is unfair both to the people who actually did the work and to the honorary author, who may not want to be listed prominently or at all. Similarly, nonattribution of credit for contributions to the reported work or careless or negligent crediting of prior work violates the value of fairness. Best practices in authorship, which are based on the value of fairness, honesty, openness, and accountability, are discussed further in Chapter 9.
Upholding fairness also requires researchers to acknowledge those whose work contributed to their advances. This is usually done through citing relevant work in reporting results. Also, since research is often a highly competitive activity, sometimes there is a race to make a discovery that results in clear winners and losers. Sometimes two groups of researchers make the same discovery nearly simultaneously. Being fair in these situations involves treating research competitors with generosity and magnanimity.
The importance of fairness is also evident in issues involving the duty of care toward human and animal research subjects. Researchers often depend on the use of human and animal subjects for their research, and they have an obligation to treat those subjects fairly—with respect in the case of human subjects and humanely in the case of laboratory animals. They also have obligations to other living things and to those aspects of the environment that affect humans and other living things. These responsibilities need to be balanced and informed by an appreciation for the potential benefits of research.
The research enterprise cannot continue to function unless the members of that system exhibit good stewardship both toward the other members of the system and toward the system itself. Good stewardship implies being aware of
and attending carefully to the dynamics of the relationships within the lab, at the institutional level, and at the broad level of the research enterprise itself. Although we have listed stewardship as the final value in the six we discuss in this report, it supports all the others. Here we take up stewardship within the research enterprise but pause to acknowledge the extension of this value to encompass the larger society.
One area where individual researchers exercise stewardship is by performing service for their institution, discipline, or the broader research enterprise that may not necessarily be recognized or rewarded. These service activities include reviewing, editing, serving on faculty committees, and performing various roles in scientific societies. Senior researchers may also serve as mentors to younger researchers whom they are not directly supervising or formally responsible for. At a broader level, researchers, institutions, sponsors, journals, and societies can contribute to the development and updating of policies and practices affecting research. As will be discussed in Chapter 9, professional societies perform a valuable service by developing scientific integrity policies for their fields and keeping them updated. Individual journals, journal editors, and member organizations have contributed by developing standards and guidelines in areas such as authorship, data sharing, and the responsibilities of journals when they suspect that submitted work has been fabricated or plagiarized.
Stewardship also involves decisions about support and influences on science. Some aspects of the research system are influenced or determined by outside factors. Public demand, political considerations, concerns about national security, and even the prospects for our species’ survival can inform and influence decisions about the amount of public and private resources devoted to the research enterprise. Such forces also play important roles in determining the balance of resources invested in various fields of study (e.g., both among and within federal agencies), as well as the balance of effort devoted to fundamental versus applied work and the use of various funding mechanisms.
In some cases, good stewardship requires attending to situations in which the broader research enterprise may not be operating optimally. Chapter 6 discusses issues where problems have been identified and are being debated, such as workforce imbalances, the poor career prospects of academic researchers in some fields, and the incentive structures of modern research environments.
Stewardship is particularly evident in the commitment of the research enterprise to education, both of the next generation of researchers and of individuals who do not expect to become scientists. In particular, Chapter 10 discusses the need to educate all members of the research enterprise in the responsible conduct of research. Education is one way in which engaging in science provides benefits both to those within the research system and to the general public outside the system.
Making judgments about definitions and terminology as they relate to research integrity and breaches of integrity is a significant component of this committee’s statement of task. Practicing integrity in research means planning, proposing, performing, reporting, and reviewing research in accordance with the values described above. These values should be upheld by research institutions, research sponsors, journals, and learned societies as well as by individual researchers and research groups. General norms and specific research practices that conform to these values have developed over time. Sometimes norms and practices need to be updated as technologies and the institutions that compose the research enterprise evolve. There are also disciplinary differences in some specific research practices, but norms and appropriate practices generally apply across science and engineering research fields. As described more fully in Chapter 9, best practices in research are those actions undertaken by individuals and organizations that are based on the core values of science and enable good research. They should be embraced, practiced, and promoted.