7
Evaluation of Departmental Undergraduate Programs

The discussion in this report thus far has focused primarily on attributes of effective teaching by individual faculty members. The central theme has been that evidence of high-quality student learning should be the major criterion for measuring a faculty member’s teaching effectiveness. The report has also emphasized the importance of using multiple indicators and different kinds of evaluators (e.g., students, alumni, graduate assistants, colleagues), as well as increasing reliance on ongoing formative evaluation, to provide a more holistic view of an individual’s teaching effectiveness.

The committee believes that similar expectations can and should apply to academic departments and colleges. Departments should regularly evaluate their current undergraduate programs and their commitment to fostering an environment that recognizes and reinforces effective teaching practices and student learning. This position is consistent with the National Science Foundation’s report Shaping the Future (NSF, 1996, pp. 63–64), which recommends that college and university governing boards and academic administrators:

  • Accept responsibility for the learning of all students and make that clear not only by what the institution says but also by putting in place mechanisms to discharge that responsibility at the institutional and departmental levels.

  • Hold accountable and develop reward systems for departments and programs, not just individuals, so that the entire group feels responsible for effective STEM (science, technology, engineering, and mathematics) learning for all students.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 108
7 Evaluation of Departmental Undergraduate Programs The discussion in this report thus far has focused primarily on attributes of effective teaching by individual faculty members. The central theme has been that evidence of high-quality student learning should be the major criterion for measuring a faculty member’s teaching effectiveness. The report has also emphasized the importance of using multiple indicators and different kinds of evaluators (e.g., students, alumni, graduate assistants, colleagues), as well as increasing reliance on ongoing formative evaluation, to provide a more holistic view of an individual’s teaching effectiveness. The committee believes that similar expectations can and should apply to academic departments and colleges. Departments should regularly evaluate their current undergraduate programs and their commitment to fostering an environment that recognizes and reinforces effective teaching practices and student learning. This position is consistent with the National Science Foundation’s report Shaping the Future (NSF, 1996, pp. 63–64), which recommends that college and university governing boards and academic administrators: Accept responsibility for the learning of all students and make that clear not only by what the institution says but also by putting in place mechanisms to discharge that responsibility at the institutional and departmental levels. Hold accountable and develop reward systems for departments and programs, not just individuals, so that the entire group feels responsible for effective STEM (science, technology, engineering, and mathematics) learning for all students.

OCR for page 108
Provide resources to ensure that faculty, particularly new faculty, have the opportunity to both learn how to and have the time to design effective instruction, use technology appropriately, foster inquiry-based and collaborative learning, and assess learning achieved. Make sure that the faculty reward system, in practice as well as in theory, supports faculty who effectively help students learning in hospitable environments that recognize individual students’ differences and that provide reasonable opportunities to address those differences. Academic departments serve many roles, including general education of nonmajors, professional preparation of majors, contributions to interdisciplinary or honors programs, and professional preparation of teachers and health professionals. Departments can encourage and support their members to work collectively to integrate courses and curricula and improve teaching and learning. They also can redirect their physical and financial resources to encourage continual improvement in teaching and learning. In summary, academic departments can become both the primary units for catalyzing change in undergraduate education and true learning communities (American Association for Higher Education [AAHE], 1993; Wergin, 1994; Wergin and Swingen, 2000; Wyckoff, 2001). Because the organization and roles of academic departments vary so widely within and among institutions, and especially among disciplines (Diamond and Adams, 1995, 2000), the task of performing any kind of systematic evaluation of these entities would appear to be nearly insurmountable. However, a number of reports have suggested how members of academic departments might assume collective responsibility for developing a coherent set of courses, programs, and other educational experiences that can enable all participating students to maximize their opportunities to learn (e.g., Shulman, 1993; Wergin, 1994; Wergin and Swingen, 2000). In addition, some disciplines have developed guidelines for evaluating undergraduate programs (e.g., for chemistry, American Chemical Society, 1992; for earth sciences, Ireton et al., 1996; for engineering, Accreditation Board for Engineering and Technology, 1998; for mathematics, Mathematical Association of America, 2000). However, many of these guidelines focus primarily on defining what is expected of students who will major in those subjects. Little attention has been paid to defining a quality education for other students who enroll in courses primarily to fulfill graduation requirements for

OCR for page 108
future teachers (see McNeal and D’Avanzo, 1997). There is growing consensus on the characteristics of effective undergraduate programs in STEM but too little effort has been expended to date on determining how measures of quality might be made more consonant and consistent with national efforts to improve undergraduate STEM education (e.g., Boyer Commission, 1998; National Research Council [NRC], 1995a, 1996a, 1999a; NSF, 1996, 1998; Rothman and Narum, 1999) or to align such programs more closely with national standards and benchmarks in these disciplines for grades K–12 (American Association for the Advancement of Science [AAAS], 1993; International Technology Education Association [ITEA], 2000; National Council of Teachers of Mathematics [NCTM], 1989, 2000; NRC, 1996b). Members of academic departments, in conjunction with the principal academic and executive officers on their campuses, need to examine critically the criteria they currently use to evaluate the efficacy of their approaches to undergraduate education. The first step in accomplishing this task is for each department to adopt a mission statement on improving teaching and student learning. Other issues on which departmental members might focus include classroom teaching, academic advising for students, and the roles of teaching laboratories and independent research opportunities in enhancing student learning. Faculty and administrators also need to reach consensus on the underlying assumptions, guidelines, and metrics they will use to improve undergraduate programs. Many of the issues surrounding the evaluation of teaching for individual faculty also apply to the collective performance of academic departments. The principles set forth in this report for evaluating the teaching effectiveness of individuals can easily be reshaped to apply to academic departments. This chapter lays a foundation for such discussions. Unlike the rest of the report, this chapter offers no findings or recommendations. Instead, it articulates a series of questions that members of departments might ask themselves and each other as they examine their unit’s role in fostering the improvement of undergraduate education. These questions are organized in accordance with the major responsibilities of departments in the STEM disciplines.

OCR for page 108
EVALUATING A DEPARTMENT’S ABILITY TO ENHANCE TEACHING AND LEARNING IN CLASSROOMS AND OTHER VENUES Engaging student interest in the department’s curricular offerings: Does the department encourage faculty members to discuss how to employ the most effective teaching techniques and educational experiences for students with various educational backgrounds and aspirations? Are the department’s programs designed to engage and excite students about the discipline specifically and about STEM generally? Does the department evaluate the effectiveness of courses for nonmajors and for preparation of students pursuing other science or engineering majors, especially for prospective elementary and secondary teachers? Applying research on human cognition and learning: Does the department encourage faculty to base instructional techniques on modern research on human cognition and learning (e.g., NRC, 1997a, 2000e)? Does the department sponsor seminars, workshops, or other activities to help faculty members become familiar with this research and its implications for improving teaching and learning? Employing effective pedagogy: Has the department examined ways in which teaching effectiveness and student learning can be enhanced in large classes, especially large sections of introductory courses? Has the department established protocols for evaluating teaching based on the kinds of criteria described in this report? Have members of the department been trained to undertake evaluative procedures such as peer review of teaching (e.g., Bernstein and Quinlan, 1996; Huber, 1999)? Has the department developed expectations regarding the teaching expertise of new hires? Does the department support faculty who become engaged in active scholarship on teaching and learning? Have guidelines been established for evaluating such work for personnel decisions? Does the department encourage and support graduate students to pursue future faculty programs designed to introduce them to issues and scholarship in teaching and learning?

OCR for page 108
Assessing student learning: Does the department encourage faculty to discuss ways of optimizing the assessment of student learning and provide sufficient time and resources to support such efforts? Are student learning outcomes considered a primary criterion when assessing the success of the department’s curriculum and programs? Emphasis on improving teaching and learning in introductory and lower division courses: Have members of the department agreed on the role and mission of introductory courses for both majors and nonmajors? Does the department encourage faculty members to work together in structuring the subject matter of and approaches to teaching introductory courses? Do introductory courses meet the educational needs of those who will become the next generation of students in STEM, future teachers, and nonmajors in the discipline? Incorporating advances in the discipline and related subject areas: Do the department’s introductory and advanced courses and other educational programs incorporate cuttingedge topics and skills of the discipline and present them to students in ways that are pedagogically appropriate? Does the department encourage colleagues to focus some of the coursework at both the introductory and upper levels on real-world applications and on connections between STEM and other disciplines? Do members of the department seek ways to provide students who will never again have formal exposure to the sciences, mathematics, or engineering with the intellectual skills and background needed to appreciate and engage in lifelong learning in these disciplines? Does the department offer encouragement and funding to purchase, maintain, and integrate into undergraduate courses cutting-edge tools and technologies (e.g., information technology, real-time data acquisition and processing, remote sensing) so that students can better appreciate and experience how advances in the discipline are achieved? Given the increasing proliferation of always-available databases, real-time data acquisition through remote sensing and instrumentation, and similar advances, is the department finding ways to extend the teaching and learning of STEM beyond traditional classroom and laboratory settings?

OCR for page 108
Does the department encourage faculty members to integrate the curriculum of lower and upper division courses? Providing academic advising and career planning: Does the department view academic and career advising as central to its mission? Does the department encourage faculty members to become more effective academic and career advisors and provide the necessary resources and time for the purpose? Does the department encourage undergraduate students to undertake real-world work and academic experiences through summer and academic-year internships? Does the department bring people to campus for presentations to students about career options and opportunities? EVALUATING DEPARTMENTAL EFFORTS TO IMPROVE TEACHING LABORATORIES AND OTHER UNDERGRADUATE RESEARCH EXPERIENCES Emphasizing the role and importance of teaching laboratories: Have members of the department collectively established criteria for assessing the role and nature of teaching laboratories in the department’s curriculum? For example, is there general agreement on whether laboratory exercises should parallel coursework, provide students with learning experiences not directly related to work in classrooms, or some combination of the two? Does the department encourage faculty to develop inquiry-based laboratory exercises that encourage students to develop their own hypotheses, design original experiments, and analyze data? Have members of the department discussed the criteria for assessing students’ work in laboratories? Is the department familiar with the use of virtual laboratories and the current status of research comparing real and simulated approaches to laboratory teaching and learning? Encouraging students to engage in independent research: Does the department encourage faculty to oversee and support students who wish to engage in independent, original research either on campus or off site (e.g., cooperative arrangements with other universities, private and government research establishments, or industry)? Does the department take advantage of undergraduate research as a way for its graduate students to grow

OCR for page 108
professionally by helping to supervise such work? Are venues available for providing academic credit or financial compensation to students and teaching credits, time, equipment, and rewards to faculty who undertake such supervisory responsibilities? Has the department discussed what the role of undergraduate research should be in relation to advancing its mission of teaching, research, and service? Has the department considered how it might offer opportunities to engage in short- or long-term research experiences to both current and prospective teachers (especially those who will teach in the primary grades) and students who will not major in STEM? EVALUATING INTERDEPARTMENTAL COOPERATION IN IMPROVING UNDERGRADUATE SCIENCE, TECHNOLOGY, ENGINEERING, AND MATHEMATICS EDUCATION Has the department established dialogues with other departments about the suitability and usefulness of its introductory courses as prerequisite or corequisite requirements for other STEM disciplines? Is the department’s curriculum structured in ways that offer gateways for students from other departments, including those who will not major in the sciences, to continue studies within the discipline? Has the department worked with other STEM departments to discuss ways in which the presentation of topics common to courses in several disciplines (e.g., energy) might be better coordinated and the connections between disciplines emphasized (see NRC, 1999a, p. 36)? Has the department worked recently with other STEM departments and the institution’s college of education to improve the preparation and continuing professional development of K–12 teachers in STEM (especially those students who plan to teach in the primary and middle grades)? Given the recent national emphasis on partnerships between higher education and local schools, has the department discussed with other STEM departments and local schools ways to establish such partnership programs and to recognize and reward faculty colleagues who undertake such efforts? Has the department worked with counterparts in local community colleges and 4-year institutions to establish policies and agreements that allow students to move more seamlessly between institutions?