This report recommends a set of strategies to evaluate undergraduate teaching and learning in science, technology, engineering, and mathematics (STEM1). It is based on a study conducted by a National Research Council (NRC) committee charged with synthesizing relevant research in pedagogy and practice as a basis for developing resources to help postsecondary STEM faculty and administrators evaluate and reward effective teaching. The study committee was a subcommittee of the NRC’s Committee on Undergraduate Science Education.
The committee’s principal goal was to determine whether fair and objective methods exist for the evaluation of teaching and learning, and if so, how such methods could be used as a basis for the professional advancement of faculty. The committee found that many such methods exist, and that their utility deserves wider appreciation and application in the evaluation of both individuals and departments.
The committee found that summative evaluations of teaching, such as those used in some faculty promotion and tenure decisions, often do not rely on evidence of student learning, and this relationship needs to be strengthened and formalized. The committee also found that formative evaluations (e.g., ongoing informal feedback from students and colleagues) can serve several important educational goals: (1) coupling candid teaching evaluation with opportunities for ongoing professional development; (2) supporting faculty
who wish to explore the scholarship of teaching and learning; and (3) applying such formative evaluation techniques to departmental programs, not only to individual faculty.2
Four fundamental premises guided the committee’s deliberations:
Effective postsecondary teaching in science, mathematics, and technology should be available to all students, regardless of their major.
The design of curricula and the evaluation of teaching and learning should be collective responsibilities of faculty in individual departments or, where appropriate, through interdepartmental arrangements.
Scholarly activities that focus on improving teaching and learning should be recognized as bona fide endeavors that are equivalent to other scholarly pursuits. Scholarship devoted to improving teaching effectiveness and learning should be accorded the same administrative and collegial support that is available for efforts to improve other research and service endeavors.
Faculty who are expected to work with undergraduates should be given support and mentoring in teaching
throughout their careers; hiring practices should provide a first opportunity to signal institutions’ teaching values and expectations of faculty.
Underlying these premises is the committee’s recognition that science, mathematics, and engineering instructors face a number of daunting challenges: the need to apply principles of human learning from research in cognitive science to the assessment of learning outcomes, to teach and advise large numbers of students with diverse interests and varying reasons for enrolling, to prepare future teachers, to provide faculty and students with engaging laboratory and field experiences, and to supervise students who undertake original research. Simultaneously addressing these challenges requires knowledge of and enthusiasm for the subject matter, familiarity with a range of appropriate pedagogies, skill in using appropriate tests, ease in professional interactions with students within and beyond the classroom; and active scholarly assessment to enhance teaching and learning.
Yet the committee found that most faculty who teach undergraduates in the STEM disciplines have received little formal training in teaching techniques, in assessing student learning, or in evaluating teaching effectiveness. Formal programs aimed at improving
Detailed definitions of formative and summative evaluation can be found in Chapter 5.
teaching are still rare. A firm commitment to open intradepartmental communication about teaching effectiveness is therefore critical to any convincing evaluation of teaching based on these premises. And because considerable variation exists across institutions and disciplines, there is no single formula or pathway to effective evaluation of teaching.
The research literature suggests that some combination of the following kinds of formative and summative evidence about student learning can be helpful in evaluating and improving a faculty member’s teaching:
Departmental and other colleagues can provide informed input about teaching effectiveness through direct observation, analysis of course content and materials, or information about the instructor’s effectiveness in service and interdisciplinary courses. Undergraduates and graduate teaching assistants could offer useful information based on their experiences in the instructor’s courses and laboratories, the instructor’s supervision of research, and the quality of academic advising. Additionally, graduate students could comment on the supervision and mentoring they have received as they prepare for teaching. The faculty member being evaluated could provide self-assessment of his or her teaching strengths and areas for improvement; this assessment could be compared with the other independent evidence. The instructor’s willingness to seek external support to improve teaching and learning also is evidence of her or his commitment to effective undergraduate teaching.
Effective evaluation also emerges from a combination of sources of evidence. Current students, those who had taken a course in previous years, and graduating seniors and alumni could provide evidence about the instructor’s role in their learning. Graduate teaching assistants could discuss the instructor’s approaches to teaching, levels of interactions with students, and the mentoring that they receive in improving their own teaching skills. Departmental and other faculty colleagues, both from within and outside the institution, could evaluate the currency of the materials the instructor presents and his or her level of participation and leadership in improving undergraduate education. The faculty member being evaluated can provide critical information about his or her teaching challenges and successes through self-reflection and other evidence of effective teaching, student learning, and professional growth. Institutional data and records offer insights about changes in enrollments in a faculty member’s courses over time, the percentage of students who drop the instructor’s courses, and the number of students who go on to take additional
courses in the discipline and related subject areas.
Each of these criteria is subject to multiple interpretations and should be viewed with care. For example, research suggests that grade distributions are not as useful an indicator of teaching effectiveness as other types of indicators and should be used cautiously, if at all.
A central idea behind formative evaluation of teaching and learning is a two-way feedback system known as “outcomes assessment.” Faculty need to set clear goals for their students and ascertain whether students are meeting those goals throughout the course. Students need to have a clear idea of what is expected of them and whether they are meeting those expectations. Chapter 5 describes in detail a variety of procedures that close these feedback loops, providing faculty with credible information about what students know and can do as a result of instruction while giving students information about how well they have mastered the course material. Whatever the means of outcomes assessment that are employed, measures of students’ conceptual understanding are critically important in judging the success of a course.
Implementing such processes can be time-consuming and involve faculty other than the instructor in charge of the course. Departmental commitment to the shared goal of improving undergraduate education is critical to the success of such approaches. Improving summative evaluation also requires that the faculty at-large, academic administrators, and committees on promotion and tenure have confidence in the credibility of the process.
An undisputed strength of American higher education is that each institution has a unique mission. It is very unlikely that any general model for evaluating teaching and learning could pertain to all schools. Several broad recommendations, however, may be generally useful when adapted to local goals and visions.
1. Overall Recommendations
(1.1) Teaching effectiveness should be judged by the quality and extent of student learning. Many different teaching styles and methods are likely to be effective.
(1.2) Scholarly activities that focus on improving teaching and learning should be recognized and rewarded as a bona fide scholarly endeavor and accorded the types of institutional supports aimed at improving scholarship generally.
(1.3) Valid summative assessments of teaching should not only rely on student evaluations, but should include
peer reviews and teaching portfolios used for promotion, tenure, and post-tenure review. Such assessments should be designed to provide fair and objective information to aid faculty in the improvement of their teaching. Building consensus among faculty, providing necessary resources, and relying on the best available research on teaching, learning, and measurement are critical for this approach to evaluation.
(1.4) Individual faculty—beginners as well as more experienced teachers— and their departments should be rewarded for consistent improvement of learning by both major and nonmajor students. All teaching-related activities—such as grading, reporting of grades, curriculum development, training of teaching assistants, and related committee work—should be included in evaluation systems adopted for faculty rewards.
(1.5) Faculty should accept the obligation to improve their teaching skills as part of their personal commitment to professional excellence. Departments and institutions of higher education should reinforce the importance of such professional development for faculty through the establishment and support of campus resources (e.g., centers for teaching and learning) and through personnel policies that recognize and reward such efforts. At the same time, institutions should recognize that disciplines approach teaching differently and that such differences should be reflected in evaluation procedures.
Much of this report offers recommendations to faculty about how they can use evaluation to improve their teaching. Accordingly, the following set of recommendations is directed toward policy makers, administrators, and leaders of organizations associated with higher education.
2. Recommendations for Presidents, Overseeing Boards, and Academic Officers
(2.1) Quality teaching and effective learning should be highly ranked institutional priorities. All faculty and departmental evaluations and accreditation reviews should include rigorous assessment of teaching effectiveness. University leaders should clearly assert high expectations for quality teaching to newly hired and current faculty.
(2.2) Campus-wide or disciplinary-focused centers for teaching and learning should be tasked with providing faculty with opportunities for ongoing professional development that include understanding how people learn, how to improve current instruction though student feedback (formative evaluation), and how educational research can be translated into improved teaching
practice. Such centers should provide equipment and facilities required for innovative teaching.
(2.3) At least one senior university-level administrator should be assigned responsibility for encouraging departmental faculty to adopt effective means (as proven by research) to improve instruction.
(2.4) Faculty who have excelled in teaching should be publicly recognized and rewarded. Endowments should be established to recognize the serious contributions of faculty who have made a sustained contribution to quality teaching.
(2.5) Faculty should be encouraged to develop curricula that transcend disciplinary boundaries, through a combination of incentives (including funding), expectations of accountability, and development of standards for disciplinary and interdisciplinary teaching.
(2.6) Willingness to emphasize student learning and to make allocations of departmental resources in support of teaching should be an essential requirement in appointing deans, department chairs, and similar administrative positions.
(2.7) Graduate school deans should require that departments that employ graduate students in fulfilling their teaching mission should show evidence that their faculties are effectively mentoring graduate teaching assistants and advising them about their duties to undergraduate students.
3. Recommendations for Deans, Department Chairs, and Peer Evaluators
(3.1) Departments should periodically review a departmental mission statement that includes appropriate emphasis on teaching and student learning. These reviews should address not only the major curriculum, but also service offerings—such as courses designed for nonmajors and prospective teachers.
(3.2) Individual faculty members should be expected to contribute to a balanced program of undergraduate teaching. Participation of established faculty in lower-division, introductory, and general-education courses should be encouraged. Faculty who are most familiar with new developments in the discipline can provide leadership in departmental curricular review and revision. Not all faculty must contribute equally to instruction at every level, but it is a departmental responsibility to ensure that the instructional needs of all students are met by caring, responsible faculty.
(3.3) Departments should contribute to campus-wide awareness of the premium placed on improved teaching. They should build consensus among
their own faculty about the suitability of the institution’s procedures for summative evaluation of teaching, recognizing that the way that practitioners of a specific discipline approach learning will affect the ways that teaching should be evaluated.
(3.4) In addition to numerical data from end-of-course student evaluations and on participation in specific courses, effective peer reviews of teaching should provide a subjective assessment of a faculty member’s commitment to quality teaching. Generally, this should include evaluation of a faculty member’s knowledge and enthusiasm for the subject matter; familiarity with a range of appropriate pedagogical methods; skills in using appropriate tests and laboratory experiences; quality of advising and other professional interactions with students within and beyond the classroom; and active scholarly commitment to enhancing top-quality teaching and learning.
(3.5) Department heads, in submitting personnel recommendations, should provide separate ratings on teaching, research, and service, each with supporting evidence, as key components of their overall rating and recommendation.
(3.6) Normal departmental professional development activity should include informing faculty about research findings that can improve student learning.
(3.7) As appropriate for achieving departmental goals, departments should provide funds to faculty to enhance teaching skills and knowledge and encourage them to undertake or rely upon educational research that links teaching strategies causally to student learning. Additional funds should be made available to departments that adopt this strategy.
(3.8) Departments should recognize that in the course of their careers, some faculty may shift the balance of their departmental obligations to place a greater emphasis on instruction or educational leadership. These shifts should be supported, consistent with a departmental mission, so long as active engagement with innovative teaching is being addressed.
4. Recommendations for Granting and Accrediting Agencies, Research Sponsors, and Professional Societies
(4.1) Funding agencies should support programs to enable an integrated network of national and campus-based centers for teaching and learning. An important goal of such a network is to conduct and disseminate research on approaches that enhance teaching and learning in STEM. The network can also provide information on the use of formative and summative assessment
for improving teaching and learning. To the extent possible, these investments should not be made at the expense of sponsored research.
(4.2) Funding agencies and research sponsors should undertake a self-examination by convening expert panels to examine whether agency policies might inadvertently compromise a faculty member’s commitment to quality undergraduate teaching.
(4.3) Accreditation agencies and boards should revise policies to emphasize quality undergraduate learning as a primary criterion for program accreditation.
(4.4) Professional societies should offer opportunities to discuss undergraduate education issues during annual and regional meetings. These events might include sessions on teaching techniques and suggestions for overcoming disciplinary and institutional barriers to improved teaching.
(4.5) Professional societies should encourage publication of peer-reviewed articles in their general or specialized journals on evolving educational issues in STEM.