6
Evaluation by Self-Assessment

Various elements of programs intended to enhance the integrity of institutional research were described in Chapter 4, and many of these elements have demonstrated at least a measure of success in some circumstances and by some, often vague benchmarks. The committee concluded, however, that the principal mode for evaluation of the effectiveness of an integrated program should be based on self-assessment and peer review, particularly when undertaken in the context of institutional accreditation. The other elements of an effective program—performance-based assessment, education, and attention to regulatory compliance— are generally components of rigorous institutional self-assessment. Use of self-assessment as a principal tool directly extends, with greater specificity, the first two recommendations of the 1992 National Academy of Sciences (NAS) Panel on Scientific Responsibility and the Conduct of Research (NAS, 1992):

  • Individual scientists in cooperation with officials of research institutions should accept formal responsibility for ensuring the integrity of the research process. They should foster an environment, a reward system, and training processes that encourage responsible research practices. (p. 13)

  • Scientists and research institutions should integrate into their curricula educational programs that foster faculty and student awareness of concerns related to the integrity of the research process. (p. 13)



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct 6 Evaluation by Self-Assessment Various elements of programs intended to enhance the integrity of institutional research were described in Chapter 4, and many of these elements have demonstrated at least a measure of success in some circumstances and by some, often vague benchmarks. The committee concluded, however, that the principal mode for evaluation of the effectiveness of an integrated program should be based on self-assessment and peer review, particularly when undertaken in the context of institutional accreditation. The other elements of an effective program—performance-based assessment, education, and attention to regulatory compliance— are generally components of rigorous institutional self-assessment. Use of self-assessment as a principal tool directly extends, with greater specificity, the first two recommendations of the 1992 National Academy of Sciences (NAS) Panel on Scientific Responsibility and the Conduct of Research (NAS, 1992): Individual scientists in cooperation with officials of research institutions should accept formal responsibility for ensuring the integrity of the research process. They should foster an environment, a reward system, and training processes that encourage responsible research practices. (p. 13) Scientists and research institutions should integrate into their curricula educational programs that foster faculty and student awareness of concerns related to the integrity of the research process. (p. 13)

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct The committee endorses the principle of self-assessment as an antecedent to formal appraisal of the performances of academic departments and individual faculty members. This chapter discusses in further detail the processes of self-assessment at the levels of both the institution and the research unit (at the level of the department, the research group, and the individual investigator), and it offers some initial steps that might be taken in the application of self-assessment to evaluation of the environment for integrity in research. SELF-ASSESSMENT AND ACCREDITATION IN HIGHER EDUCATION State and federal governments mandate accreditation of institutions of higher education as a requirement for the recognition of the degrees they grant, but a process of peer review is used almost exclusively to grant accreditation. Many different accrediting bodies exist, and these are based either on geography or, for professional schools, on the degree granted. In virtually every case, the heart of the accreditation process is self-assessment (Borden and Owens, 2001; Ewell and Lisensky, 1988; Middle States Commission on Higher Education, 2000). Process of Self-Assessment in Higher Education Self-assessment begins with instructions from the accrediting body regarding the criteria for evaluation. These instructions generally provide a template for self-assessment that enables the institution to respond to a series of “must” and “should” standards. The issues and questions posed are usually of a general nature so that institutions can present their solutions in different ways. These responses are then judged by external reviewers and provide the basis of an institution’s case for accreditation. Because institutions of higher education vary markedly in their histories, cultures, curricula, and human and physical resources, accreditation is not based on presumptions as to particular “right” answers. In fact, within very broad boundaries, institutional diversity is valued and encouraged. The process of self-assessment in institutions of higher education is lengthy, costly, and difficult (Ewell, 1991; Middle States Commission on Higher Education, 2000). In the process, institutions critically evaluate their strengths and weaknesses and consequently develop new ideas for self-improvement. Continuous quality improvement is the goal. The intent is to accomplish this by associating the process of self-assessment with anticipated improvements in the desired output (e.g., better-educated students). Periodic reaccreditation provides a formal process for evaluation of the results. It cannot be accomplished by simple completion

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct of a mandated checklist, although attention to inputs and processes, as well as outcomes, is required. Determining the effectiveness of any program of continuous quality improvement is largely qualitative, although some measures can be quantified. For example, to evaluate an outcome of creating better-educated students, institutions may look to improvements in scores on examinations taken by their students interested in pursuing advanced degrees (e.g., the Graduate Records Examination and the Medical College Admission Test), or they may monitor the postgraduation careers of their students to determine what kind of postgraduate programs or professional positions they enter and how well they were prepared (National Center for Higher Education Management Systems, 2001). Institutions that have recently completed an accreditation cycle often serve as models for other institutions as the latter prepare for a similar process, thereby encouraging a culture of quality improvement (Ewell and Lisensky, 1988). The self-assessment process is organized around a set of faculty committees, each responsible for analysis and recommendations concerning an element or program of the institution considered important in determining institutional effectiveness (Middle States Commission on Higher Education, 2000). Both the openness and the accuracy of the reports produced by these committees are furthered by the inclusion of large numbers of faculty in their membership. The reports are assimilated and integrated into a master document, the self-study report. A template provided by the accrediting body may guide both the format and the general content of the report. This self-assessment is submitted to the accrediting body and is then provided to volunteer peer reviewers, who complete their evaluations with a visit to the institution. The reviewers generally interview members of the faculty, administration, and student body, and then prepare a detailed report that addresses perceived strengths, weaknesses, and areas for desirable or necessary improvement. This report constitutes the basis of a recommendation to the accrediting body regarding the continuation of accreditation. The committee appreciates that in an open society like that of the United States, different people look at many measures of the quality of institutions of higher education and research (e.g., rankings of institutions according to the amount of support for research they receive from the National Institutes of Health, and U.S. News and World Report rankings of schools and educational programs). Nevertheless, governments have traditionally relied upon processes of accreditation to define those entities qualified to provide education at a particular level or in a specific field. Despite this reliance, however, the federal government has generally avoided specific involvement in accreditation processes, largely entrusting them to private-sector accrediting agencies, consistent with the

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct nation’s tradition of limited government. The effectiveness of this approach to rigorous institutional accreditation is reflected, at least in part, by the fact that the U.S. system of higher education is admired around the world, as well as by the public’s continuing general acceptance of self-monitoring as a means of assessing professional quality. Promotion of Integrity in Research Promotion of integrity in the research environment is about institutional culture and behavior, as well as the professional performance of individuals. It is about the system in which research is done. Research trainees learn about the culture of science through experience, mentoring, and formal educational processes. Even the best institutional climate and programs for researchers and trainees, however, will not preclude either research misconduct or nonprofessional behavior. Still, an educational environment that makes clear what is expected of scientists and their teams, combined with systems and institutional behaviors and policies that encourage accurate and careful pursuit of scientific ends, can beneficially influence researchers and trainees. The committee has defined integrity in research more broadly than an absence of research misconduct. Integrity in research embraces the aspirational standards of scientific conduct rather than simply the avoidance of questionable practices. There is a role for education of students, faculty, and staff regarding not only professional behavior but also the common culture of science that as a whole promotes a research environment of high integrity. It can focus on the joy of rightful discovery and recognition by one’s peers. Such education should be viewed in its broadest sense, however, occurring not only through formal instruction but also through the institutional atmosphere, policies, and guidelines, as well as through the quality of mentoring (King, 1999; Swazey et al., 1993). A Role for Institutional Accrediting Bodies The committee believes that assessment of the effectiveness of institutional efforts to ensure integrity in the research environment can best be accomplished by incorporating evaluations of integrity in research into existing accreditation processes for institutions of higher education. Benefits that flow from the systematic evaluation of institutional behavior and policies associated with a process of self-assessment and accreditation include the following (Ewell and Lisensky, 1988): highlighting a need for change without impairing institutional autonomy or uniqueness;

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct efficient coordination of data gathering; creation of a permanent capacity for analyzing institutional effectiveness; and changing of the attitudes of faculty and staff regarding institutional assessment from negative to positive. Although peer review of the self-assessment process used to ensure integrity in the research environment could be accomplished by establishing a new accrediting body for the purpose the committee believes this is not the preferred approach. A more attractive alternative is for research institutions to work with established accrediting bodies to incorporate research integrity into overall accreditation processes. The processes of established accrediting bodies should be more effective and more cost-efficient than those of a new entity, whose establishment would constitute one more administrative burden, and thus would encourage cynicism. Moreover, creating a separate body for assessment could easily communicate an undesirable message that an environment that enhances integrity in research can readily be distinguished from one that promotes high-quality education and research more generally. The committee believes that the research mission should be considered as a whole. Thus, it seems reasonable that entities charged with accrediting the quality of education at institutions of higher learning that conduct scientific research might also be charged with reviewing the data from the institutions’ self-assessment of their climate and procedures for promotion of integrity in research. The committee is aware, however, that adoption of this recommendation must begin with a substantial commitment from the institutions themselves. Accrediting bodies respond to priorities established by schools and universities in determining the issues to be addressed in the process of accreditation. Consequently, if institutional cultures are to be enhanced, then both the call for change and its implementation must come from research institutions. An important next step will be for universities and university associations, working together, to acknowledge the importance of conducting research and education in research in an environment of high integrity. Certainly the strong governmental and public interest in research integrity will prove ample encouragement to support initiating the peer review process, Universities and research sponsors should urge accrediting bodies, especially those charged with accrediting education programs with a substantial research mission, to include an evaluation of the environment for promotion of integrity in research in the overall processes of accreditation. This objective might first be accomplished through collaboration with specialized accrediting bodies for science-based professions, such as medicine and engineering. Although not always realized (Bayles, 1981), a

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct hallmark of the concept of professionalism and an integral factor in the social contract of professions with society is a presumption of integrity and self-policing (ABIM, 2001; Camenisch, 1996). Schools of engineering have led in explicitly tying this responsibility to accreditation requirements. Recently, the Accreditation Board for Engineering and Technology has established criteria which mandate that graduates demonstrate an understanding of professional and ethical responsibility (Engineering Accreditation Commission, 2000; Rabins, 1998). Additionally, research in schools of medicine is largely supported by the U.S. Department of Health and Human Services, in which the Office of Research Integrity (ORI) resides. Thus, if ORI were to support the development of pilot programs to evaluate this approach, the Liaison Committee on Medical Education, which is charged with accreditation of programs for medical education (LCME, 2001), and the Accreditation Board for Engineering and Technology might be effective places to begin. The heterogeneity of the accreditation process among major accrediting bodies is acknowledged. For example, the process of self-study leading to accreditation of research universities may be less prescriptive than that for professional schools in regard to the specific areas mandated for evaluation. Universities are commonly allowed substantial leeway and choice in determining the elements examined in their self-assessment process. Accrediting bodies, however, respond to the demands of their client communities. Thus, if institutions of higher education regarded the integrity of research as an element essential for their accreditation, then it is likely that assessments of integrity in research would be incorporated into both the self-study and peer-review phases of accreditation. It should also be noted that self-assessment of an institution’s environment for integrity in research does not depend inextricably upon formal ties to a process for institutional accreditation. The need for independent self-assessment has already been recognized by a number of prominent institutions absent a mandate (Center for Academic Integrity, 2001). Other mechanisms for peer review can be developed, but the committee concludes that explicit and public processes for external peer review help to ensure credibility and public confidence. In institutions where accreditation is not available or cannot for some reason be incorporated into institutional processes of accreditation, other approaches to the provision of external validation should be explored. This is particularly important in the large number of private research institutes and industrial research groups that offer formal programs of education in research at either the predoctoral or the postdoctoral level. By encouraging accrediting bodies to develop their own designs for review of research integrity, the committee anticipates improvements and a confluence of processes over time as these bodies learn from each other

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct and from feedback from the institutions. Although this will be an evolutionary process, it could begin with, and be facilitated by, such agencies as ORI. Implementation should take place as quickly as possible, given the increasing demands for accountability in the use of public funds. The committee’s focus on the quality of an environment that promotes integrity in research within institutions of higher education is not to imply disinterest in such integrity in other contexts. However, virtually all investigators begin their research careers in a university setting; therefore, the university research group can be considered the crucible for education in research. Additionally, as the principal recipients of public research funds, academic research groups have been the major focus of concern for integrity in research. Self-assessment as a first step in the accreditation of educational programs is predicated on a judgment that the process can lead to important changes in the environment. Experience with the process in other facets of the institutional milieu supports this notion. Desirable changes have successfully been introduced in many universities, in such areas as personal harassment, teaching about sexuality in medical schools, treatment of minorities, addition of instruction in ambulatory care to medical education, and addition of instruction in ethics to law school and business school curricula. The impetus for these changes has come from within the university community and has occurred as a consequence of government initiatives. A more difficult challenge, but one that warrants a substantial research effort, will be to determine, as evidence of effectiveness, whether long-term changes in behavior have been achieved as a consequence of any interventions intended to promote integrity in research coupled with processes for self-evaluation (Davis et al., 1999; Parochka and Paprockas, 2001). A Role for Governments Governments have an appropriate interest in the effectiveness of programs in self-assessment and accreditation, since they rely on such programs in a number of ways. For example, state licensing boards for many professions require graduation from an accredited school as a condition of licensure. They can also influence curricula through their licensing powers. Similarly, only accredited schools of medicine are eligible to receive federal education grants and to participate in federal loan programs. Additionally, federal agencies sometimes review and approve the accrediting bodies themselves. Such a review, based on a set of federally determined standards, becomes a condition for accepting the findings of these bodies in determining eligibility for the commitment of federal funds (Middle States Commission on Higher Education, 2000). Thus, the federal

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct government may oversee accreditation efforts that are based on private-sector self-assessment and peer review, even though government does not participate directly in the review and accreditation process. These government policies, which restrict recognition of graduates and extension of certain educational programs exclusively to accredited schools, are well accepted by the higher education community. The nongovernment coordinating agency for accreditation of postsecondary education is the Council on Higher Education Accreditation (CHEA, 2001). The U.S. Department of Education maintains a listing of recognized regional, national, institutional, and specialized accrediting bodies (DoEd, 2001). Since the nation’s economy and national security, as well as the health of its people, are heavily dependent on continuing reliable research findings, the federal government and the public rightfully place a high priority on integrity in research. The federal government deserves support in its call for more effective strategies to encourage changes in the environment for the promotion of integrity in research and the evaluation of the outcomes of such changes. However, the committee judges that a direct role per se for the federal government in such programs that assess and accredit institutions for their integrity in research is neither necessary nor desirable. The committee believes instead that self-assessment coupled with peer review of the environment for the promotion of integrity in research is more likely to have a positive impact on those programs and cultures. Funding Funding is needed in several important areas: to support research aimed at developing new methods for fostering integrity in research (including research on assessing the effectiveness of such approaches), and to support the everyday operation of assessment programs already in place at numerous institutions. The federal government (through the Office of Research Integrity and various research agencies) and private research foundations can play a role in supporting these efforts. One way for them to encourage this process would be to augment grant programs to provide support for research into the enhancement of integrity in research and assessment of the effectiveness of alternative approaches. In principle, the costs of conducting programs to gauge the integrity of research conducted under federal sponsorship could be covered by the reimbursement of facilities and administrative costs (so-called indirect costs) associated with federal research grants and contracts. However, administrative costs on grants and contracts to educational institutions (but not to other research entities) have been capped for more than a decade at 26 percent of direct costs (Goldman and Williams, 2000; OMB,

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct 2000). When coupled with the additional fact that most research-intensive universities at present have documented administrative costs in excess of 26 percent (personal communication, T. DeCrappeo, Council on Government Relations, March 18, 2002), the consequence is that universities alone bear any additional costs associated with the development or enhancement of programs that evaluate integrity in research. If further encouragement of an environment for integrity in research is truly a priority for research sponsors, then the sponsors should work cooperatively with educational institutions to share in the funding of such programs, particularly if the intent is to develop best practices rather than simply require minimal compliance with applicable regulations and policies. A Role for Professional and Scientific Societies Professional and scientific societies have a key role in developing, promoting, and inculcating codes of research ethics within their memberships. The common culture of science within academic departments and professional and scientific societies provides the peer pressure that promotes professionalism. To date, however, the activities of these societies related to these efforts have been limited. Among their actions, professional and scientific societies should examine the requirement for membership and standing relative to integrity. Are there organizational sanctions that apply to members who have been shown to engage in misconduct, for example? In addition, the organizations should ensure that content relative to the responsible conduct of research is included at their annual meetings, in their journals and other publications, and in other organizational venues for communication. Well-crafted society codes of research ethics (ACS, 2001; ASBMB, 2001) could be used as guides in the development of both specific objectives and processes for the development of institutional self-assessment programs. Each scientific society should express its standards for the responsible conduct of research and take steps to ensure that its members know these standards. It is important to note that major research universities are homes to a large number of different disciplines. The issues should be regarded as equally important across the entire spectrum of general and discipline-specific scientific societies, including societies involved in the life and earth sciences; social sciences; chemical and physical sciences; and mathematical, computational, and computer sciences. INDIVIDUAL SELF-ASSESSMENT Self-assessment for accreditation is an institutional process. It may involve global accreditation of the educational environment and programs

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct or evaluation of particular institutionally imbedded structures, such as programs for evaluation of animal care and use or the protection of human research subjects. A second useful level of self-evaluation is directed at the individual researcher or academic unit as an initial step in periodic appraisals of performance. Many institutions and departments have formal processes for assessment of faculty members and academic leaders (deans and chairs). Faculty members are usually evaluated annually. For deans and chairs the interval is usually longer, and the evaluation serves as a component of in-depth school or departmental reviews. The academic community accepts these processes on the basis of a presumption that they promote and reflect individual, departmental, and institutional excellence. Department chairs commonly regard annual self-evaluation of faculty members, followed by a formal discussion with the chair, as an important aspect of faculty mentoring. The committee concurs, and it advocates the inclusion of questions in the self-assessment process that evaluate behaviors that promote integrity in research. Faculty members should be asked to evaluate not only their own behaviors as researchers but also interpersonal relations and their research environments. Similarly, as chairs and deans are formally evaluated, questions in self-assessment forms should address the extent to which their leadership within the institution promotes a culture of integrity in research. Responses to such questions could then be incorporated more formally into institutional self-assessment as a component of the institutional accreditation process. There are few data regarding the effectiveness of individual self-assessment in altering behavior, particularly behavior that is deviant or lacking in integrity, and this is another area in which research is needed. However, individual self-assessment does provide a useful framework for a formalized process of evaluation by institutional superiors. Additionally, such a process may have salutary effects, at least in terms of interpersonal relationships within a laboratory or department, particularly when it includes input from subordinates to the individual being evaluated. SUMMARY Evaluation of the institutional environment for the promotion of integrity in research should be based on processes of self-assessment and peer review. Evaluation of integrity in research should be incorporated into existing processes for accreditation of educational and research institutions. Creation of a new entity specifically for accreditation of an environment

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct that promotes integrity in research is probably unnecessary; it would be costly and likely impose burdens disproportionate to the benefits. Assessment of integrity in research for research groups should be a component of regular performance appraisals for faculty and academic leadership. Effective self-assessment will require the development and validation of evaluation instruments and measures. Federal research agencies and private foundations should support the development of programs to integrate self-assessment of the environment for integrity in research into accreditation processes through interactions with and among stakeholders, and they should fund research into the effectiveness of such programs. Federal research sponsors should work with educational institutions to develop funding mechanisms to support programs devoted to promoting the responsible conduct of research. REFERENCES ABIM (American Board of Internal Medicine). 2001. Project Professionalism. [Online]. Available: http://www.abim.org/pubs/default.htm [Accessed January 22, 2002]. ACS (American Chemical Society). 2001. Academic Professional Guidelines. [Online]. Available: http://chemistry.org/portal/servlet/resources/org/chemistry/avercom/display/ContentRetrievalServlet/ACS/ACSContent/careers/apg_Jan01_final.pdf [Accessed November 5, 2001]. ASBMB (American Society for Biochemistry and Molecular Biology). 2001. ASBMB Code of Ethics . [Online]. Available: http://www.asbmb.org/ASBMB/ASBMBSiteII.nsf/MenuHomePage/PublicAffairs [Accessed November 5, 2001]. Bayles MD. 1981. Professional Ethics. Belmont, CA: Wadsworth Publishing Co. Borden VMH, Owens JLZ. 2001. Measuring Quality: Choosing Among Surveys and Other Assessments of College Quality. Washington, DC: American Council on Education. Camenisch PE. 1996. The moral foundations of scientific ethics and responsibility. Journal of Dental Research 75:825–831. Center for Academic Integrity. 2001. Academic Integrity Assessment Guide. [Online]. Available: http://www.academicintegrity.org/assessGuide.asp [Accessed November 5, 2001]. CHEA (Council on Higher Education Accreditation). 2001. Good Practice Database. [Online]. Available: http://www.chea.org/good-practices/index.cfm [Accessed March 18, 2002]. Davis D, O’Brien MAT, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. 1999. Impact of formal continuing medical education. Journal of the American Medical Association 282:867–873. DoEd (U.S. Department of Education). 2001. Overview of Accreditation. [Online]. Available: http://www.ed.gov/offices/OPE/accreditation [Accessed March 18, 2002]. Engineering Accreditation Commission. 2000. Criteria for Accrediting Engineering Programs. [Online]. Available: http://www.abet.org/images/Criteria/2002-03EACCriteria.pdf [Accessed March 18, 2002 ]. Ewell P. 1991. Benefits and Costs of Assessment in Higher Education: A Framework for Choice Making. Boulder, CO: National Center for Higher Education Management Systems.

OCR for page 112
Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct Ewell PT, Lisensky RP. 1988. Assessing Institutional Effectiveness. Washington, DC: Consortium for the Advancement of Private Higher Education. Goldman CA, Williams T. 2000. Paying for University Research Facilities and Administration. Santa Monica, CA: RAND. King PM. 1999. Why Are College Administrators Reluctant to Teach Ethics? Synthesis: Law and Policy in Higher Education 10(4):756-757. Asheville, NC: College Administration Publications, Inc. LCME (Liaison Committee on Medical Education). 2001. Accreditation Procedures. [Online]. Available: http://www.lcme.org/procedur.htm [Accessed March 18, 2002]. Middle States Commission on Higher Education. 2000. Designs for Excellence: Handbook for Institutional Self-Study, 7th ed. Philadelphia, PA: Middle States Commission on Higher Education. Pp. 1–4, 13–21. NAS (National Academy of Sciences). 1992. Responsible Science: Ensuring the Integrity of the Research Process. Panel on Scientific Responsibility and the Conduct of Research. Washington, DC: National Academy Press. National Center for Higher Education Management Systems. 2001. Comprehensive Alumni Assessment Survey (CAAS). [Online]. Available: http://www.nchems.org/Surveys/caas.htm [Accessed November 5, 2001]. OMB (Office of Management and Budget). Circular A-21, Cost Principles for Educational Institutions. 2000. [Online]. Available: http://www.whitehouse.gov/omb/circulars/a021/a021.html [Accessed March 18, 2002]. Parochka J, Paprockas K. 2001. A continuing medical education lecture and workshop, physician behavior and barriers to change. Journal of Continuing Education in the Health Professions 21:110–116. Rabins MJ. 1998. Teaching engineering ethics to undergraduates: Why? what? how? Science and Engineering Ethics 4:291–302. Swazey JP, Anderson MS, Lewis KS. 1993. Ethical problems in academic research. American Scientist 84:542–553.