1
Introduction

RATIONALE

What students learn and how they are taught in college science, technology, engineering, and mathematics (STEM) courses are issues that have occupied educators for many years (Dwyer, 1972; Arons, 1983) and have been the focus of previous National Research Council (NRC) studies (e.g., 1997, 1999, 2003). These studies point to the growing body of empirical research showing that learning can be enhanced when college instructors incorporate teaching strategies that are student-centered, interactive, and structured around clearly stated measurable learning outcomes.

A crucial question, then, is why introductory science courses in many colleges and universities still rely primarily on lectures and recipe-based laboratory sessions where students memorize facts and concepts, but have little opportunity for reflection, discussion, or testing of ideas. Do instructors have readily available information about instructional techniques shown to be more effective in eliciting students’ understanding and in helping them develop useful knowledge? Are they afforded opportunities to learn about alternative teaching strategies? Do barriers and disincentives at the institutional/departmental levels discourage faculty from adopting such strategies?

These were some of the questions that prompted the NRC’s Committee on Undergraduate Science Education (CUSE) to organize the present workshop. A steering committee (biographical sketches in Appendix E) was established to develop a workshop with invited presenters and small working groups that were asked to explore three related issues: (1) how appropriate measures of undergraduate learning in STEM courses might be developed; (2) how such measures might be organized



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
1 Introduction RATIONALE What students learn and how they are taught in college science, technology, engineering, and mathematics (STEM) courses are issues that have occupied educators for many years (Dwyer, 1972; Arons, 1983) and have been the focus of previous National Research Council (NRC) studies (e.g., 1997, 1999, 2003). These studies point to the growing body of empirical research showing that learning can be enhanced when college instructors incorporate teaching strategies that are student-centered, interactive, and structured around clearly stated measurable learning outcomes. A crucial question, then, is why introductory science courses in many colleges and universities still rely primarily on lectures and recipe-based laboratory sessions where students memorize facts and concepts, but have little opportunity for reflection, discussion, or testing of ideas. Do instructors have readily available information about instructional techniques shown to be more effective in eliciting students’ understanding and in helping them develop useful knowledge? Are they afforded opportunities to learn about alternative teaching strategies? Do barriers and disincentives at the institutional/departmental levels discourage faculty from adopting such strategies? These were some of the questions that prompted the NRC’s Committee on Undergraduate Science Education (CUSE) to organize the present workshop. A steering committee (biographical sketches in Appendix E) was established to develop a workshop with invited presenters and small working groups that were asked to explore three related issues: (1) how appropriate measures of undergraduate learning in STEM courses might be developed; (2) how such measures might be organized

OCR for page 1
into a framework of criteria and benchmarks to assess instruction; and (3) how departments and institutions of higher learning might use such a framework to assess their STEM programs and to promote ongoing improvements. Workshop participants would focus on four questions regarding undergraduate STEM education at the classroom, departmental, and institutional level: (a) what characteristics and indicators should be included in a comprehensive evaluation instrument that could serve as the basis for recognizing exemplary STEM courses and academic programs; (b) what are the desired student outcomes of such STEM courses that can indicate course effectiveness; (c) what qualities of organization, governance, and incentive structures can be identified at the departmental and institutional levels that promote quality STEM education; and (d) how can such qualities be used as the basis for creating indicators and benchmarks for the evaluation of institutions and departments? To sharpen and focus these questions, breakout groups at the workshop were asked to define “appropriate measures of undergraduate learning” by developing a list of desired student learning outcomes for each science discipline. The logic here was that student success in achieving defined learning outcomes could serve as an indicator of the effectiveness of a particular course or an instructional approach. Further, still using student learning outcomes as a criterion of success, workshop participants were challenged to identify characteristics and indicators that should be included in a comprehensive evaluation instrument or framework for recognizing a hypothetical “exemplary” STEM course. To investigate how departments and institutions of higher learning might use such a framework to assess their STEM programs, workshop participants were instructed to identify qualities of organization, governance, and incentive structures at the departmental and institutional levels that promote quality STEM education, and to consider how such qualities could be used to create a set of indicators and benchmarks for the evaluation of institutions and departments. As an initial step in thinking about appropriate measures of undergraduate learning in STEM disciplines, workshop participants were asked first to identify a few “exemplary” programs that were known by reputation to be effective in achieving desired learning outcomes. Participants then outlined characteristics that would enable an observer to classify these programs as effective. These characteristics, which are summarized in Chapter 3, could be included in a comprehensive evaluation instru-

OCR for page 1
ment that would serve as the basis for assessing STEM courses and academic programs. Broken into groups by discipline (physics, chemistry, life sciences, geosciences), the participants were then asked to enumerate the desired learning outcomes that indicate course effectiveness. The groups chose not to emphasize any content-specific lists of outcomes, but instead focused on the cross-disciplinary outcomes. The reported outcomes were remarkably similar across groups. These outcomes are summarized in Chapter 2. The process of developing appropriate learning outcomes as well as measures, which includes designating working teams, asking appropriate questions, and collaborating to answer these questions, was described by several workshop presenters (see Chapters 2 and 4) and exemplified by participants during the workshop. The organization of a framework of criteria and benchmarks, as specified in issue 2, was not accomplished at the workshop. A recent NRC report Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics (2003) points out that development of a universal evaluation instrument is difficult if not impossible since academic institutions vary greatly in mission and demographics. The workshop participants represented a variety of institutions and found reaching consensus on a set of criteria suitable for each institution difficult. Readers of this report may choose to organize the characteristics of effective programs and instructional strategies outlined in Chapter 3 into a framework of criteria and benchmarks suitable for their own institutions. Instead of deliberating on how institutions and departments would use such a framework (issue 3), workshop participants focused their discussions on the personality traits of faculty as well as the qualities of organization, governance, and incentive structures at departmental and institutional levels that promote effective STEM education. They also discussed institutional characteristics that are barriers to the implementation of effective instruction. These qualities, which are summarized in Chapter 4, can serve as the basis for creating indicators and benchmarks for the evaluation of institutions and departments. Several presenters described approaches that faculty, departments, and institutions can take to develop and incorporate qualities that promote quality STEM education (see Chapter 4). Throughout the workshop, participants voiced many concerns that have been raised in earlier studies; however, they also presented new efforts, arguments, and evidence. Buttressed by recent studies, this report documents more convincingly than earlier reviews

OCR for page 1
that instruction based primarily on lectures may be useful for transferring factual information but is much less effective at achieving more complex conceptual learning outcomes. It provides a new approach to promoting effective STEM instruction by examining the personality traits of faculty— those characteristics that allow faculty to respond to institutional/departmental changes as well as those that directly or indirectly affect the culture of the department and institution. The report advances the argument for increased collaboration among faculty and administrators and provides illustrative examples of effective collaborative efforts. The rest of this section provides background to the body of research supporting the concerns of the steering committee and raises additional questions that prompted the development of this workshop. Student Learning Outcomes Starting with A Nation at Risk (National Center for Excellence in Education, 1983), with its “Imperative for Educational Reform,” hundreds of reports—at the rate of almost one per week according to Tobias (1992)—by national associations, blue-ribbon committees, commissions, and accrediting boards have produced visions of improved STEM education. These new visions, remarkable in their agreement, focus less on what instructors teach and more on what students learn and are able to do with their new knowledge. They also focus less on terms and facts that students memorize and more on students’ conceptual understanding and their ability to apply knowledge in novel contexts. For higher education, a primary objective is that “all undergraduates have learning experiences that motivate them to persist in their studies and consider careers in these fields” (Project Kaleidoscope, 2002, p. 1). The requirements of the Accreditation Board for Engineering and Technology (2002), for example, state that students should gain an ability “to apply knowledge of mathematics, science, and engineering; to design and conduct experiments as well as to analyze and interpret data; to function on multidisciplinary teams; and to communicate effectively.” The American Psychological Association standards for undergraduate education in that discipline expect that students “will understand and apply basic research methods…including research design, data analysis, and interpretation; will respect and use critical and creative thinking, skeptical inquiry and when possible the scientific approach to solve problems…; [and] will be able to communicate effectively in a variety of formats” (2002). A panel convened by Sigma Xi, The

OCR for page 1
Scientific Research Society, concluded that introductory STEM courses should enable students “to understand science, mathematics and engineering as processes of investigation—as ways of knowing; to have hands-on experience with investigations and to discover the joy and satisfaction of discovery; to understand the powers and limitations of science mathematics and engineering; [and]…to understand the synergisms among science disciplines and the synergisms among science, mathematics and engineering” (1990, p. 9). With each passing year, the need for faculty to define and learn how to elicit appropriate learning outcomes among students in undergraduate STEM courses has gained in importance. The percentage of high school graduates who choose to enter college has increased dramatically in the past two decades—from about 50 percent to 66 percent between 1980 and 1998 (National Center for Education Statistics [NCES], 2000). During those same years the number of fall semester enrollees represented by minorities increased from 16.5 percent to 26.6 percent (Snyder, 2001, pp. 295–296). Spurred in part by an increasing “college wage premium” that promises far higher earnings from degrees in specific fields (NCES, 2000, p. 144), many of these students declare their intentions to pursue an aspect of science or technology in their postsecondary education. Between 1970 and 1998 enrollment in computer and information sciences as well as agriculture and health-related sciences rose by an average of 250 percent (although physical sciences and mathematics actually lost students over that period) (Snyder, 2001, pp. 295–296). As recently noted in a related NRC report (Hudson, 2002, p. 37), “In general, the shift in the past three decades appears to be away from the humanities and hard [physical] sciences toward business, technical, and health fields.” These students, those still selecting science and technology and many who pursue fields outside of science or technology, are required to take college science courses at the introductory level. The rise in number and diversity of students intensifies the need for faculty to see introductory courses for both majors and nonmajors as a critical part of the undergraduate curriculum. Such courses can serve not only as vehicles for providing students with the facts and concepts of science but also as opportunities to develop their understanding and appreciation of the processes of science as well as cognitive skills such as posing and solving problems, making sense of data, and reasoning and arguing from evidence—all of which are crucial to decision making no matter what field of endeavor a student enters

OCR for page 1
after graduation. Introductory science courses can also be designed to properly prepare those who wish to continue in science as a profession, those who will affect the science education of future students as K–12 teachers, administrators, or policy makers, and those who desire to be informed citizens in this increasingly scientific and technological world. Unfortunately, this vital education in science is reaching too few of today’s undergraduates (Seymour and Hewitt, 1997). Many students leave science-rich fields for other areas of interest after their first lower-division college science courses or drop out of higher education completely. According to Seymour and Hewitt’s (1997) comprehensive study, a loss of over half of the students who enter college intending to pursue majors in the natural sciences occurs within two years of taking their first college science or mathematics classes, a problem of wastage that affects both minority and majority students. Students reported being dissatisfied with what they perceived as poor teaching and other negative experiences in “weed-out” science courses. Frequent complaints were heard about courses and textbooks that are filled with facts that students are expected to memorize with little opportunity for conceptual development, and tests that only assess students’ abilities to remember such facts from recent lectures and chapters. An important result of this shift away from science and math courses during the upper division undergraduate years is a striking decline in students choosing advanced or graduate courses leading to STEM professions. From 1993 to 2000, enrollment in STEM graduate programs decreased by more than 14 percent, with three areas, math (32 percent), engineering (25 percent), and the physical sciences (18 percent) suffering the most prominent losses (Zumeta and Raveling, 2003, p. 37). Effective Instruction Reacting to reports indicating that new knowledge is assimilated through interaction with existing knowledge (summarized in NRC, 2000), workshop participants considered how an instructor might be encouraged to provide opportunities for students to become actively involved in creating new understandings. Recent evidence suggests that students who sit passively in lectures for an entire course may fail to replace their prior misconceptions with new knowledge; the conceptual difficulties they have when they enter a course are likely to persist if instruction does not address their difficulties specifically (King, 1994; Mestre, 1994; Loverude et al., 2002; Marchese, 2002). For many students the traditional didactic lecture, when applied as the primary instruc-

OCR for page 1
tional method in science courses, fails to provide opportunities for integrating new and old knowledge. Lectures may lead to memorization of factual information but often do not succeed well in eliciting comprehension of complex concepts (Terenzini and Pascarella, 1994; Honan, 2002; Loverude, Kautz, and Heron, 2002). Despite such evidence, according to a broad survey of 123 research-intensive (Research I and II) universities nationwide by The Reinvention Center at Stony Brook (2001), only about 20 percent of R-I and R-II universities provide opportunities for active learning or real-world problem solving for their students in a substantial number of introductory science courses. On a majority of campuses the instructor as a didactic lecturer remains typical practice in STEM courses. As noted by Alison King (1994), “Much of what transpires in today’s college classrooms is based on the outdated transmission model of teaching and learning: the professor lectures and the students take notes, read the text, memorize the material, and regurgitate it later on an exam” (p. 15). Role of Academic Departments in Improving Teaching Effectiveness The personal experiences of a number of workshop participants confirmed that the current culture of many science and engineering departments is one that values the productive investigator more than the effective teacher. In an effort to counteract that emphasis, the NRC report Transforming Undergraduate Education in Science, Mathematics, Engineering, and Technology (1999) presents six vision statements and multiple strategies for implementing these visions. The vision statements are designed to assist academic officers, faculty members, and departments in their efforts to improve STEM education. Vision two of that report calls for the development of introductory college courses that would present content information in ways that engage undergraduates in exploring the fundamental and unifying concepts and processes of science. These courses would emphasize real problems, applications to related areas of knowledge, and the evolving processes of scientific thought and inquiry. Vision three calls on all colleges and universities to continually and systematically evaluate the efficacy of their STEM courses and programs. The NRC report Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics (2003) recommends that evidence of student learning be used as a benchmark for evaluating teaching effectiveness. That report also stresses the utility of ongoing self-study and evalua-

OCR for page 1
tion by STEM departments and suggests a series of questions for departments to use in this process. OBJECTIVES AND ORGANIZATION OF THE PROJECT Against the background outlined in the preceding paragraphs, CUSE established a steering committee to develop the present workshop. Eleven experts in STEM education and/or institutional reform accepted invitations to present at the workshop as a means of informing the committee and catalyzing discussion among attendees. Additional experts were assigned as facilitators to two breakout sessions. The breakout sessions were planned to capture the interactions among the attendees, who brought with them much relevant experience. The facilitators, two per group, were asked to keep their groups focused on the questions, to make sure that everyone’s voice was heard, to promote a supportive atmosphere that evoked creative ideas without overorganizing the conversation, and to record the discussions in order to report a summary during the plenary session. Additional authorities on STEM education were sent workshop announcements and encouraged to attend. In addition, three of the presenters were asked to prepare short papers in advance of the workshop to further discussion of the guiding questions, and were then asked to modify and expand their papers based upon the discussion at the workshop (revised papers are provided in Appendix A). The workshop was held in Washington, D.C., November 19–20, 2002, at the National Academies. (The Workshop Agenda can be found in Appendix C.) Commissioned papers were distributed to registered participants within the week before the workshop. Fifty-one invited participants, including presenters and facilitators, attended the workshop along with NRC staff and other interested parties. Names and institutional affiliations of registered participants and steering committee members are listed in Appendix D. ORGANIZATION OF THE REPORT This report is based on the presentations and papers commissioned for the workshop and on the discussion that emerged from the workshop itself. The commissioned papers as revised by the authors following the workshop are reprinted intact in Appendix A, so they are not summarized in the report. However, authors also formally presented material found in their papers as part of the plenary sessions, and those aspects of the papers appear in that

OCR for page 1
context in the following chapters. Participants in this workshop were charged with examining student learning outcomes in the sciences, exemplary instructional practices, and the barriers as well as the enablers to instructional reform at the institutional level. Through presentations by plenary speakers and the discussions that followed, reports from breakout sessions, and general discussions throughout the workshop, major themes emerged. In this report, a summary of the workshop presentations and subsequent discussion, participants’ statements, and the resulting themes are organized around the workshop’s guiding questions into three related areas: identifying desired learning outcomes, evaluating effective instruction, and promoting effective instruction at institutional and departmental levels. The following chapters focus on the four guiding questions (a-d) regarding undergraduate STEM education presented at the beginning of this chapter. Chapter 2 addresses question b—what are the desired student outcomes of such STEM courses that can indicate course effectiveness—by examining the process of developing such learning outcomes. It outlines some of those outcomes defined as most important by the workshop participants. Chapter 3 identifies characteristics and indicators that can be included in a comprehensive evaluation instrument for rating exemplary STEM instructional programs (question a) and tools for assessing the quality of faculty instruction. Chapter 4 examines characteristics of organization, governance, and incentive structures identified at the personal, departmental, and institutional levels that promote quality STEM education (question c). It also considers qualities that serve as barriers to implementation of effective instruction and describes approaches to promote such instruction at the institutional/departmental level. Chapter 5 summarizes the general discussion that occurred at the end of the workshop, highlighting qualities that could be used as the basis for creating indicators and benchmarks for the evaluation of institutions and departments (question d). In an Epilogue (Chapter 6), overriding concerns that participants voiced repeatedly serve as a summary of the major issues in the report. References to specific programs and initiatives that were discussed by workshop participants are included throughout this report. These programs are cited for information purposes only; such citation does not imply endorsement by the NRC.