National Academies Press: OpenBook
« Previous: 2 Identifying Desired Student Learning Outcomes
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 25
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 26
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 27
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 28
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 29
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 30
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 31
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 32
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 33
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 34
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 35
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 36
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 37
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 38
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 39
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 40
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 41
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 42
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 43
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 44
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 45
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 46
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 47
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 48
Suggested Citation:"3 Evaluating Effective Instruction." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 49

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Evaluating Effective Instruction To develop a list of criteria and In this chapter, evidence is consid- benchmarks for evaluating effective ered that defines the characteristics of science, technology, engineering, and effective instruction. In the workshop, mathematics (STEM) instruction, participants were asked to enumerate educators must agree on the character- features that could be included in a istics of such instruction. If the aim is comprehensive evaluation instrument as not only to teach science content but indicators for rating exemplary STEM also to foster inquisitiveness, cognitive programs. While workshop breakout skills of evidence-based reasoning, and groups specified characteristics of an understanding and appreciation of programs of effective instruction, two the processes of scientific investigation, presenters offered instructional tech- then—as noted in Chapter 2—courses niques that exemplify many of these that consist solely of traditional lectures characteristics. Paula Heron, University and laboratory sessions may be inad- of Washington, illustrated the tutorial equate. Moreover, if introductory program at her university designed to science courses are expected to play address students’ preconceptions and even broader roles—such as increasing details evidence that students gain the likelihood that nonmajors and improved conceptual understanding. preservice teachers will choose to take Brian Reiser, Northwestern University, additional science courses, and expand- described a scaffolding tool and class- ing the number of students who go into room environment that gradually builds science and engineering careers— students’ skills for multidirectional evaluation of introductory science instruction (i.e., teacher to student, instruction must be improved and student to teacher, student to student) appropriate criteria developed. and independent learning. Other work- 25

shop participants imparted additional Pascarella, 1994; Honan, 2002; instructional strategies that would Loverude, Kautz, and Heron, 2002). achieve desired learning outcomes. Although direct instruction is useful in To begin to shape how criteria would some settings (e.g., Klahr, Chen, and be used to evaluate such instruction and Toth, 2001), and the lecture format can programs, two presenters offered be improved by allowing learners to examples of assessment strategies and grapple with an issue on their own tools. Gloria Rogers, Rose-Hulman before they are provided with answers Institute of Technology, described the (Schwartz and Bransford, 1998) or by fundamentals of evaluation for any other modifications that add an element educational program. Anton Lawson, of interactivity (NRC, 2000; Laurillard, Arizona State University, illustrated an 2002), accumulating research indicates instructional assessment tool and the that the traditional approach with no contexts in which it was used as an additional cognitive assistance leads to example of how to measure effective- memorization of facts rather than ness of instruction. Expanded summa- understanding of concepts for a majority ries of the presentations, the learning of students (Wright et al., 1998; outcomes proposed by workshop Loverude et al., 2002; see Appendix B, participants, as well as additional ideas this volume). The evidence indicates, and cautions put forward by participants moreover, that most students who sit during plenary discussions are detailed passively in lectures for an entire course within this chapter. are unlikely to appropriately link their prior conceptions to the new knowledge being presented. The conceptual misun- CHARACTERIZING EFFECTIVE derstandings they have when they enter INSTRUCTION WITH RESEARCH a course are likely to persist if instruc- EVIDENCE tion does not address their difficulties specifically (King, 1994; Mestre, 1994; Recent evidence suggests that, for Loverude et al., 2002; Marchese, 2002). many students, traditional didactic Even students who receive good grades lectures that promote memorization of and persist in science courses often gain factual information may be unexpect- little understanding of the basic science edly ineffective for eliciting learning of concepts (see Appendix B, this volume; more complex concepts when applied as Sundberg, 2002). the primary instructional method in The broadened roles of introductory science courses (Terenzini and science courses, plus recent gains in 26 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

understanding how people think and apply knowledge in new situations learn, have forced a reconsideration of (Stephans et al., 1988). what is meant by effective science When implemented properly, the teaching. Two recent NRC volumes evidence suggests that inquiry-based mentioned earlier, How People Learn: instruction and problem-solving strate- Brain, Mind, Experience, and School gies engage the learner in developing (2000) and Evaluating and Improving the mental models required for concep- Undergraduate Teaching in Science, tual understanding (NRC, 2000, pp. 239– Technology, Engineering, and Mathemat- 241). These strategies assist students in ics (2003), describe numerous research assimilating new information through findings that have powerful implications interaction with their prior concepts and for how instruction might be organized knowledge of the world outside the and implemented to elicit learning. One classroom. When combined appropri- of the hallmarks of the emerging sci- ately with direct instruction and other ence of learning is its recognition that teaching approaches, students are students bring with them diverse motivated to identify and gather relevant learning styles and that they learn in factual content and integrate that new different ways under different circum- knowledge with their preconceptions stances. This finding suggests that there (Loverude et al., 2002; Marchese, 2002). is not likely to be one best mode of With such instruction, these studies instruction for all purposes. Precollege show that learners can be helped to educators recognize this need by absorb new facts and concepts, to devise utilizing a “learning cycle” (Stephans, and carry out scientific investigations Dyche, and Beiswanger, 1988) that that test their ideas, and to understand engages students’ intellectual curiosity why such investigations are uniquely before introducing formalisms. To be powerful as a way of learning. effective, undergraduate teaching faculty must also have at their command an aggregate of instructional strategies DEFINING CHARACTERISTICS OF and be prepared to use combinations of EFFECTIVE INSTRUCTIONAL inquiry-based, problem-solving, infor- PROGRAMS mation-gathering, and didactic forms of instruction under appropriate classroom One of the goals of the workshop circumstances that promote conceptual participants was to determine if it was understanding and students’ ability to possible to reach agreement on the E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 27

characteristics of an effective instruc- tics. They then set out to define explic- tional program. In the first breakout itly the characteristics of these courses session, the disciplinary groups were or course materials in physics that assigned three tasks: (1) on the basis of justified their designation as “exem- the knowledge and expertise of the plary.” The identified characteristics are assembled group, develop a list of described below, following the summa- highly regarded courses/programs ries of the breakout groups, and reflect within the discipline; (2) consider the the importance of developing students’ characteristics of each entry that justi- understanding of underlying concepts. fied its selection for the list; and (3) As noted above, many of these cur- extract from those characteristics a list ricula utilize a process known as learn- of criteria or indicators that would ing cycles (Kolb, 1984; Healey and enable an observer to assess programs Jenkins, 2000). A learning cycle is in that discipline. designed to engage students’ curiosity about a phenomenon, to elicit their The Summaries of Breakout thoughts and preconceptions about the Groups phenomenon, then to provide them with Priscilla Laws, Dickinson College, opportunities for direct observations summarized the discussions of the and problem-solving experiences so physics working group. The group they may judge the validity of their made a list of thirteen curricular materi- ideas, and finally to provide opportuni- als and approaches in physics1 that the ties to resolve any discrepancies be- members believed from personal tween their preconceptions and canoni- experience had exemplary characteris- cal concepts. The group pointed out that 1 The exemplary programs and approaches identified by the physics working group included context-rich cooperative problem solving (Heller, 1997), Physics by Inquiry (McDermott, Shaffer, Keith, and Anderson, 1992; Heller and and Rosenquist, 1996), Powerful Ideas in Hollabaugh, 1992), Explorations in Physics Physical Science (http://www.psrc-online.org/ (http://physics.dickinson.edu/~EiP_homepage/ classrooms/papers/layman.html), Real Time html), Interactive Physics (http:// Physics (http://physics.dickinson.edu/ www.interactivephysics.com), Just in Time ~wp_web/Introduction/FAQ/ Teaching (http://webphysics.iupui.edu/jitt/ Real_Time_Physics), Studio Physics (http:// jitt.html), lecture demonstrations (Sokoloff and www.rpi.edu/dept/phys/education.html), Thornton, 1997), Models in Physics Instruction Tutorials in Introductory Physics (McDermott, (http://www.physics.umd.edu/perg/papers/ Shaffer, and the Physics Education Group, 2002), redish/jena/jena.html), Peer Instruction (Mazur, and Workshop Physics Activity Guide (Laws, 1997). 28 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

many students of different majors and David Gosser, City College of New diverse needs are required to take York, and Ishrat Khan, Clark Atlanta physics. The physics community gener- University, summarized the discussions ally requires students to learn what the of the chemistry group. This was a experts feel is important for them to small group, with most members cur- know without seeking input from rently involved in the same projects— students. All too often, this is achieved New Traditions (http:// by simply telling students what to newtraditions.chem.wisc.edu) and Peer- memorize through a series of traditional Led Team Learning (PLTL) (http:// didactic lectures. In contrast, Laws www.pltl.org). The noteworthy charac- noted “One of the things most of the teristics of these programs are that they reform curricula have in common is…a engage faculty in teaching, require great emphasis on [the need for] stu- incremental changes, and take into dents understanding the account forces that drive institutions. underlying…concepts [while recogniz- The group identified additional charac- ing that] one of the real weaknesses of teristics that make these programs traditional [instruction] is that students exemplary and discussed other meth- …memorize how to solve categories of ods to engage faculty, outside of those problems…without understanding the immediately invested in educational fundamental underlying concepts.” reform, to adopt effective instructional Marshall Sundberg, Emporia State strategies. These characteristics and College, summarized the life sciences methods are described below following group’s discussions of a range of ex- the summaries. amples, such as those listed in the NRC Khan cited New Traditions and PLTL report BIO2010 (2002a) in which as efforts that are designed to be readily instructors provide class-based investi- adaptable to different settings but which gative opportunities, both for students also allow faculty ownership of material majoring in the sciences and for and assessment. Gosser added that with nonmajors. The characteristics repre- PLTL, faculty are asked to evaluate sentative of a number of successful students and report findings to the programs are described below following parent program; he believed this data the summaries. In all cases, an impor- collection convinces faculty that the tant feature was that the instructor model is working and provides them sought to build learning communities with resources to support the program that engaged students and encouraged at their institutions. He also noted that further study. students are involved in efforts to E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 29

disseminate PLTL. “They…actually • Provide experiences for stu- conduct workshops and show faculty dents to develop functional under- how poised and how capable they can standing. These programs place be. This is more of a selling point than if emphasis on students’ understanding of I talk about it for an hour.” In response science concepts and ability to apply to questions by Richard McCray, Uni- these concepts to new situations. Less versity of Colorado, Gosser and Khan emphasis is placed on end-of-the-chapter confirmed that the PLTL program could problems and exams as the bottom line be adapted for other disciplines. Susan in grading, to minimize students memo- Singer, Carleton College, added that the rizing and pattern matching without program has already been modified and developing functional understandings. is being used in history and introduc- The programs often outline the con- tory biology. cepts they expect students to learn as Bonnie Brunkhorst, California State explicit learning outcomes (see the University, San Bernardino, summa- partial list of concepts for biology rized the discussions of the geosciences education in Table 2-1 for an example). group that included representatives Opportunities for undergraduate re- from the fields of geosciences, space search were identified as appropriate science, and astronomy. The group experiences to develop an understand- chose to look at NSF-funded projects ing of the scientific process. and examined the criteria that had been • Have strategies for iterative developed out of those projects. McCray evaluation. These strategies should added that since the group had a hybrid include self-assessment by faculty of representation, their discussions were instruction and program effectiveness, not discipline specific. mechanisms for identifying instructor expertise both conceptually and peda- The Characteristics of Effective gogically, assessment of student learn- Programs ing, and procedures for learning from The characteristics of exemplary failure through formative evaluation. programs identified by the geosciences Exemplary programs often take risks, group and other breakout groups are learn from failure, reevaluate, and try described below. The characteristics again. Summative evaluation is needed were very similar across the disciplin- to demonstrate effectiveness to develop- ary groups. Recognized exemplary ers or institutions. instructional programs can be charac- • Invest in training and terized as follows. They: mentoring of instructors, both 30 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

faculty and teaching assistants. departments to take group ownership of Training should assess in a nonthreaten- effective programs. Institutionalizing ing manner the instructors’ competen- effective programs is critical to sustain cies and comfort levels for the program. the programs in the event that the Training should encourage instructors individuals driving the programs retire to take ownership of the program and its or otherwise leave. materials and to adapt it as necessary to suit their own contexts. • Include efforts to collect and ACHIEVING DESIRED OUTCOMES disseminate information about the WITH EFFECTIVE INSTRUCTIONAL program, applying accepted prin- TECHNIQUES ciples of research on teaching and learning. As noted earlier, there is accumulat- • Make interdisciplinar y connec- ing evidence that new knowledge is tions. Instructors should make stu- shaped by interaction with existing dents aware that methods or concepts knowledge (NRC, 2000). Instructors from other sciences are often used in need to pay attention to students’ the context of one discipline. beliefs, incomplete understandings, and • Foster independent learning the naïve versions of concepts they skills. These programs strive for the bring to a given subject. An important outcomes regarding learning skills (i.e., element of effective instruction involves “learning to learn”) defined in Chapter building on these preconceptions and 2. prior beliefs in ways that help each • Promote students’ ability to student achieve a more mature under- work cooperatively and to communi- standing. If students’ initial ideas and cate orally and in writing. beliefs are ignored, the understandings • Address materials’ relevance to that they develop can fall far short of the students. The working group agreed goals of the instructor (Minstrell, 1989; that material should be relevant to Mestre, 1994; NRC, 2003). Striking students’ lives, but they pointed out that evidence for this comes from a study in sometimes the topics that have the which undergraduates in a leading biggest impact are not perceived as university who took a traditional geom- relevant initially but become surpris- etry course that ignored their entering ingly significant. misconceptions represented and visual- • Become institutionalized and ized three-dimensional forms more self-sustaining. Strategies exist for poorly than did a comparison group of E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 31

elementary children whose prior ideas The “baseball bat problem” (see about space were engaged (Lehrer and Figure 3-1) tests whether students Chazan, 1998). This discussion of the recognize that both the amount of mass resistance of students’ preconceptions and its distribution determine the to change was the setting for Paula location of the center of mass, or the Heron’s contribution. balancing point. The problem was administered to students at different Instruction Designed to Help stages of instruction in different Students Overcome Conceptual courses. Responses from students in the Difficulties introductory calculus-based physics Paula Heron, University of Washington course at the University of Washington In her presentation entitled Research (UW) were compared before and after as a Guide to Improving Student Learn- traditional instruction in the course and ing in Undergraduate Physics, Heron after additional Tutorials (McDermott, illustrated how misinterpretations or Shaffer, and the Physics Education problems that students have with Group, 2002). Tutorials at UW meet specific ideas can persist and adversely once a week, while lectures are held affect desired learning outcomes. She three times a week. (Students partici- described how the university’s Physics pate in a weekly lab as well.) In the Education Group (PEG) conducts tutorials, students work with materials research to develop effective instruc- and complete worksheets tailored tional strategies to address the difficul- specifically to guide them through the ties students commonly have with development and application of impor- specific physics topics. Heron stressed tant concepts in the course, and to the importance of education research address specific difficulties that have that is conducted within the discipline. been uncovered through research. “What physics education research Responses to the bat problem were constitutes is an approach to improving also gathered from students in an instruction that is objective [and] introductory calculus-based physics efficient and allows for cumulative course at Purdue University, where the progress to be made in the teaching of tutorials developed at UW have been the discipline.” She illustrated this incorporated, and from students in an perspective with a specific example of introductory engineering statics course research that focused on student under- at UW. These students, who are re- standing of the concept of center of quired to take an introductory calculus- mass (Gomez, 2001). based physics course prior to the statics 32 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

A student balances a baseball bat of uniform mass density by placing a finger directly beneath point P. P The bat is cut into two pieces at point P. mA P mB Is the mass of the left piece greater than, less than, or equal to the mass of the right piece? Explain. FIGURE 3-1 The baseball bat problem. SOURCE: Gomez (2001). Reprinted with permission. course, revisit material on the center of compression of ideal gases (Loverude et mass in the statics course. Responses al., 2002). Cumulatively, the group’s were also obtained from prospective and research findings indicate that on practicing K–5 teachers in special certain types of qualitative questions, physics courses at UW that are de- student performance remains essen- signed to prepare them to teach physics tially unchanged before and after and physical science as a process of instruction in either calculus- or algebra- inquiry. Table 3-1 presents the results based courses, with or without standard from the bat problem: the percentages laboratory, with or without demonstra- of students answering the question tions, in large and small classes, and correctly sorted by type of instruction regardless of perceived effectiveness of received. the lecturer (see Appendix B, this The results from the bat problem volume). were consistent with those obtained in Heron summarized the group’s other studies by PEG related to student interpretation of their data with a ques- conceptual difficulties with the wave tion: “Is…good quality standard instruc- properties of light (Ambrose, Heron, tion, through a lecture, textbook, and Vokose, and McDermott, 1999) and laboratory, sufficient to develop a E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 33

TABLE 3-1 Student Response to Baseball Bat Problem (Percentages by Course) Percentage Percentage Course and Responding Responding Relative Time of Test N mA < mB (correct) mA = mB Introductory mechanics, 152 5 90 University of Washington, before instruction Introductory mechanics, 455 15 80 University of Washington, after instruction Engineering statics, 71 15 85 University of Washington, after all instruction Introductory mechanics, 255 55 40 University of Washington, after traditional instruction plus tutorial Introductory mechanics, 1,160 50 45 Purdue University, after traditional instruction plus tutorial Graduate TAs, 30 70 30 University of Washington, after traditional instruction (presumed) Physics by Inquiry course 30 100 0 for preparing K–5 teachers, University of Washington, after instruction SOURCE: Gomez (2001). Reprinted with permission. 34 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

functional understanding of an impor- masses of bat “halves” by weighing the tant concept or principle? By functional pieces obtained from cutting the bat at understanding, what we mean is the the balance point, Heron described an ability to apply the concept or principle instructor’s experience in conducting to a situation that has not previously such a demonstration. When confronted been memorized.” Her answer, based on with the evidence, students assumed PEG research and a growing body of that the demonstration had not worked other supporting literature was: “Teach- as planned. “We know what we were ing by telling is an ineffective mode of supposed to see,” they said to the instruction for most students…. Stu- instructor. Heron indicated that in dents must be intellectually active to laboratory situations students would develop a functional understanding…. often ask for better equipment if the Sitting and listening to lectures…, results fail to match their (erroneous) reading the textbook, solving the expectations. traditional end of chapter problems does During her presentation, Heron made not lead to this type of intellectual the point that effective instruction could engagement.” best be designed through research to Heron pointed out that even instruc- identify and detail specific student tion considered to be “pedagogically difficulties and assess instructional correct,” such as small group work, strategies meant to address those hands-on activities, and demonstrations, difficulties. By examining student may not address persistent conceptual responses to the bat problem, as well as and reasoning difficulties. When stu- to several other written problems, and dents in the engineering statics course probing student ideas through in-depth participated in small group exercises interviews, Heron and her colleagues devoted to centroids and center of mass, identified several specific student there was no evidence of improved difficulties with the concept of static performance on the bat problem. The equilibrium. Of primary concern was percentage of students who claimed that the failure of students to consider both the halves of the bat on each side of the the mass and its distribution relative to balancing point must have equal mass the balance point. was the same as in the prerequisite Heron explained how tutorials incor- introductory physics course. In re- porated into introductory physics sponse to a question by Lawson about courses at the University of Washington whether students would be affected by are designed to address students’ faculty demonstrating the different conceptual and reasoning difficulties E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 35

Hang board from a hole to the right lump of clay of the center of mass. center of mass of board frictionless pivot Use clay to balance the board. Predict how the total mass to the right of the pivot compares to the total mass to the left. a. Move the clay toward the pivot. Does the board remain balanced? Does the total mass to either side of the pivot change? b. Balance board again using larger piece of clay. Does the total mass to either side of the pivot change? FIGURE 3-2 Excerpt from tutorial “Equilibrium of Rigid Bodies.” SOURCE: McDermott, Shaffer, and the Physics Education Group (2002, p. 65). Reprinted with permission of Prentice-Hall. and to provide a mechanism for secur- rated into the UW introductory physics ing intellectual engagement despite course. A comparable improvement was large classes. Heron described an observed at Purdue University, where excerpt from the tutorial entitled “Equi- the same tutorial was introduced. librium of Rigid Bodies,” in which (Results at Purdue after traditional students explore the effects of reposi- instruction alone are very similar to tioning a piece of clay on an odd-shaped those obtained at UW.) board balanced on a nail (see Figure 3- To emphasize further the importance 2). Benefiting from the research on of instruction that specifically addresses student difficulties, this tutorial is persistent difficulties, Heron described designed to lead students to recognize another effort of PEG: a self-contained that both mass and its distribution laboratory-based curriculum to prepare relative to the pivot point influence K–12 teachers to teach science as a balancing. Correct responses by stu- process of inquiry. Physics by Inquiry is a dents on the bat problem increased set of instructional modules threefold after this tutorial was incorpo- (McDermott, Shaffer, and Rosenquist, 1996) that are designed to achieve four 36 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

primary objectives: (1) develop concep- and hands-on activities are appropriate tual understanding and reasoning ability techniques to promote student learning in the context of a coherent subject but are not sufficient in and of them- matter; (2) develop specific scientific selves to engage students at a suffi- reasoning skills (control of variables, ciently deep intellectual level. Unless proportional reasoning, etc.); (3) de- serious conceptual difficulties are velop skills in using and interpreting addressed explicitly, they are likely to scientific representations (graphs, persist and preclude meaningful learn- diagrams, equations, etc.); and (4) ing not only of the topic under consider- provide direct experience with the ation, but of others for which it is a process of science. Upon completion of prerequisite. the unit that deals with balancing, all of Both Heron and the subsequent the participating K–5 teachers answered presenter, Brian Reiser, detailed instruc- the bat problem correctly, whereas none tional strategies that would address the had done so prior to the unit. challenges in achieving desired learning outcomes. Heron’s presentation high- Tutorials That Address Students’ lighted the importance of science Difficulties education research2 to identify persis- The tutorial system described in tent difficulties that hinder students’ Heron’s presentation has been shown to conceptual understanding. Reiser be effective in improving student learn- described a “scaffolded” learning ing. (For a description of the tutorials environment that gradually shifted and their implementation at UW, see, for students from learning in lecture-based example, McDermott, 2001.) Weekly classrooms to learning in interactive, pretests are given in the tutorials to multiresource classrooms. During the elicit student difficulties about the topic. plenary discussion that followed The pretests are administered before Heron’s and Reiser’s presentations, the tutorial but usually after lecture other participants identified additional instruction on the topic. In the tutorials, effective instructional strategies. students work in small groups (3–4) using carefully structured worksheets; instructors are prepared to teach by questioning; and questions related to the tutorials are included on every 2 course exam. Heron emphasized that See distinction between science research and science education research made in footnote #4, interactive lectures, small group work, Chapter 2. E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 37

Scaffolding Tools engage students in the problem under Brian Reiser, Northwestern University consideration, provide assistance with tools, and structure classroom dis- In his presentation Supporting Investi- course around appropriate ideas. The gation and Argumentation in Science computer tools of the BGuILE program Classrooms, Reiser described a learning encourage and teach students to ma- environment, known as Biology Guided nipulate only one variable at a time, to Inquiry Learning Environments write better hypotheses, to look for (BGuILE) (http://www.letus.org/ disconfirming evidence, and to write bguile/overview.html), supported by down their observations, findings, the Center for Learning Technologies in interpretations, and questions continu- Urban Schools (LeTUS) (http:// ously. In this light, Reiser defined the www.letus.org). Scaffolded experiences scaffold as “a teacher or more knowl- are intended to work like an apprentice- edgeable peer [who] assists a learner, ship, whereby an expert (the instructor, so he or she can solve problems that teaching assistant, or a peer learning would otherwise be out of reach.” coach) models a problem-solving In BGuILE, four computer-based activity, then provides the learner with scenarios and associated classroom advice and examples, guides the learner activities are designed to help students in practice, and finally tapers off support learn biology by exploring significant and guidance until the student can do it research problems. The scenarios alone (Collins, 1990). Scaffolding can be selected are rich in data so that students provided by a computer-based resource do not all arrive at the same conclusion (White and Frederickson, 2000), as well but rather must defend their answers. as by an instructor, as in the case of Reiser illustrated the teaching strategies BGuILE. through one of the scenarios, a problem- Building scaffolding into a software based unit on ecology and evolution tool in addition to in a person has called The Galápagos Finches. Once the motivated Reiser’s work for the last ten teacher has “sold” students on the years. The teacher and software both significance of the Galápagos Finches have essential roles in the curriculum, problem the students will learn skills for which Reiser described and demon- graph interpretations and scientific strated through video clips and re- argumentation, as well as something corded classroom conversations. The about natural selection, intraspecies teacher’s role in these classrooms is to interactions, and balance in ecosystems, help students care about the material, through the scaffolding software tools. 38 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

Though much of Reiser’s work is at Students need to interact with each the K–12 level, he identified his purpose other and the teacher in new ways. They for presenting at this workshop as have to be willing to explain their exploring whether his work to help thinking, to ask questions of each other, middle and high school students go and to offer advice to other students. beyond the formalisms (memorized Teachers have to continually engage equations and terms) and develop students in conversations and questions functional understandings can be about the problem under consideration. replicated at the undergraduate level. Both the teacher and the software tool Reiser provided an example of a group aim to uncover students’ confusions and working with the scaffolding tool; the then guide the students into productive participants believed they had reached discussions about these confusions. consensus, but when they were forced to write in the tool they discovered Small Group and Pair disagreements and the need to further Cooperative Learning clarify definitions. Extending the idea that students To employ the strategies essential for should be more engaged in conversa- effective inquiry teaching, Reiser tions and questions with instructors and explained, instructors needed to change peers, several workshop participants the climate of the classrooms. He pointed to the benefits of student-to- pointed out examples of such climate student interactions. For example, changes in his video presentations. The Michael Zeilik, University of New teacher would often engage students in Mexico, asserted that students’ concep- questions about the problem and would tions can be changed in properly man- frequently profess that the answers are aged cooperative teams and that small not known. At the start of the school group or pair cooperative learning can year, the students did not believe her be facilitated even in large lecture and felt she was holding back answers. situations (Mazur, 1997; Schwartz and Through continued discussions, they Bransford, 1998). Priscilla Laws, allud- began to accept her acknowledgement ing to Heron’s message about targeting and to articulate explanations on their persistent difficulties, added that stu- own or through discussions with their dents must engage with the right issues peers. if cooperative teams are to be success- Reiser highlighted a student in a full ful. She pointed out that lecture demon- class discussion who asked a question strations can be effective, referring to of his classmates instead of the teacher. an article by Sokoloff and Thornton E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 39

(1997), if students are engaged in approaches to teaching and learning, discussions about what they saw and case studies promote interaction and what they conclude. provide relevance for the students Confirming that cooperative groups can (Herreid, 1999; Honan and Rule, 2002). be effective, Elaine Seymour, University of Colorado, added that students often Problem-Based Learning complain about this approach, but if the Both Herreid and Zeilik promoted instructor continues to form and work PBL as the “greatest method of all.” with such groups the students become Herreid pointed out that the PBL more comfortable with collaborative method, as described in The Power of learning. She also reported that men get Problem-Based Learning (Duch, Gron, the most out of structured groups be- and Allen, 2001), has thrived success- cause such work tends to be novel to fully at many medical schools for over them; women, in general, need little thirty years. However, it has also failed prompting to work in groups (e.g., at a few medical schools because of lack Stabiner, 2003). Richard McCray added of administrative support. that in situations where students are forced to explain their answers and Instructors’ Roles and Physical reasoning, they are able to identify what it Environment is that they don’t understand. In his presentation later in the work- shop, Jack Wilson, UMassOnline, Case Studies identified some teaching methods that In addition to instructional methods were relevant to include with this that encourage collaboration, another section. In his talk, he listed some of the strategy that many participants thought strategies that made Studio Physics to be effective was the use of case successful at Rensselaer Polytechnic studies. Katayoun Chamany, Eugene Institute. The major instructional Lang College, reminded the group that innovation was in the interaction be- subject matter could be made relevant tween the instructors (faculty and and useful to diverse populations of teaching assistants) and the students. students by teaching through case study Instructors in the Studio define their modules. In support of Chamany’s point role as guides, leading students to about incorporating case studies into information and helping them with the curriculum, Clyde Herreid, State difficult concepts, while students are University of New York at Buffalo, encouraged to take responsibility for explained that, like other inquiry-based their own learning. 40 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

Anton Lawson added that instructors suggested instructional methods— should be aware of the reasoning skills including Studio Physics, problem- and abilities students have when they based learning, case studies, in-class enter their classrooms and the need to conversations and small group work, focus on developing those to a higher scaffolding tools, and the tutorials at level throughout the course. Although UW—Ronald Henry, Georgia State assessing generalized cognitive skills is University, remarked that faculty should more difficult than measuring discipline- model in their teaching the ways in specific knowledge, there is a substan- which their own students should teach if tial literature on how to do it (Shavelson those students go on to become gradu- and Huang, 2003). ate teaching assistants, K–12 teachers, In Studio Physics, teaching assistants or science faculty. work collaboratively with faculty; one faculty member, one graduate TA, and one undergraduate TA interact with ASSESSING INSTRUCTIONAL students together in each studio section. IMPACT ON LEARNING The team approach reduced the error rate on transmitted information, since if During their deliberations, each of the one of the three misunderstood the workshop breakout groups developed problem the others could make correc- lists of student learning outcomes tions. The physical setting for the appropriate for introductory science course was completely reconstructed courses (see Chapter 2). But according from two theatre-style lecture sessions to Herb Levitan, National Science serving 500 students each to 12–15 Foundation, many faculty members do studios or labs where 50–60 students not know how to evaluate either student worked in small collaborative groups. learning achievements or their own Instruction was extended through the instructional efforts. Moreover, added mobile computing initiative, requiring a Lawson, although many current evalua- laptop computer for each student. tion practices measure performance on Laptop purchases were built into the tests or other tasks, they fail to indicate financial aid structure such that a the degree of learning. Learning, in this student receiving 100 percent financial context, is interpreted as conceptual aid was given a laptop. Most of the understanding measured by the ability didactic material for the course was of a student to apply knowledge and made available online. skills in new situations. This discussion Recognizing the importance of all the served as background for the two E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 41

speakers at the workshop who devoted defined only vaguely, Rogers stressed their presentations to assessment the importance of agreeing on defini- practices and tools, one to provide tions at the start of an evaluation pro- general background information, the cess to avoid future disagreements. other offering an example of an instru- Rogers distinguished “assessment,” the ment for evaluating an instructor’s collection and analysis of evidence, from performance in the college classroom. “evaluation,” which she defined as interpretation of evidence. She empha- Fundamentals of Evaluation sized that these were separate activities. She also defined inputs, processes, Gloria Rogers, Rose-Hulman Institute of outputs, and outcomes. See Table 3-2 for Technology examples of each of these terms. Each In her presentation Evaluating student term includes student, faculty, and outcomes: E=MC2, Rogers emphasized campus components. the importance of establishing a com- Inputs were defined as what the plete institutional assessment process, constituents bring into the system. covering classroom assessment, pro- Processes include the programs, ser- gram assessment, and “mapping strate- vices, loads, policies, and procedures gies” (i.e., methods for using assess- that are established to take advantage of ment data to chart instructional what is known about the inputs. Outputs improvements across a program). are easily measured indicators and Recognizing the wide range of expertise statistics. Outcomes refer to the effects, of the workshop participants, she aimed which are particularly important in to include information designed to link terms of accreditation. classroom and program assessments. Goals of Assessment The Purpose of Assessment Rogers focused next on the impor- The most difficult part of assessment tance of determining what is being for faculty, Rogers noted, is to define assessed. Is the target of the assess- explicitly what is meant by educational ment individuals or groups? Will the outcomes, and to articulate outcomes in assessment be used to evaluate achieve- terms of measurable criteria. ment, placement, gatekeeping, program enhancement, or program accountabil- Definition of Terms ity? She indicated that the assessment Since many of the terms commonly strategy would be dependent on assess- used in the field of assessment are ment goals. 42 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

TABLE 3-2 Examples for Assessment Terms Inputs Processes Outputs Outcomes Student credentials: Programs and services Student grades, What have students Test scores offered, populations graduation rates, learned; what skills served employment statistics have they gained; what attitudes have they developed? Faculty credentials Faculty teaching loads/ Faculty publication Faculty publication class size numbers citations data, faculty development Campus resources Policies, procedures, Statistics on resource Student learning governance availability, participation and growth rates SOURCE: Adapted by Rogers (2002b) from work cited in Middaugh, Trusheim, and Bauer (1994, p. 4). With input from workshop partici- suggested, should build educational pants, Rogers identified stakeholders in practices and strategies around desired the educational process as students, learning outcomes, incorporating their parents, other departments, measurable performance criteria into employers, scholarship agencies, and the curriculum to determine whether graduate programs. She noted that the those outcomes are achieved. constituents have different expectations Each aspect (objectives, criteria, and the makeup of constituents can vary practices/strategies, assessment, widely from institution to institution. outcomes, evaluation, and constituent She pointed out that educational objec- responses) is part of an interconnected tives should be tied to the institutional system of feedback for continuous mission and should be reevaluated improvement of the course or program. every several years. To accomplish the Rogers stressed the importance of educational objectives, students must defining and reevaluating: “If objectives achieve learning outcomes, which and outcomes are difficult to define, Rogers defined as what students should they will be difficult to measure.” know and be able to do by the end of a Ramon Lopez, University of Texas at El course or program. Instructors, she Paso, asked about the effectiveness of E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 43

the Accreditation Board for Engineering learning and “we can’t assess it if we and Technology criteria (2002). These don’t know what to expect.” standards, which emphasize outcomes and understanding for undergraduate Classroom versus Program Assessment engineering students, were intended to Rogers outlined the similarities and drive real change in engineering pro- differences between classroom and grams. Rogers responded that while the program assessment. Both can be ABET 2002 criteria themselves are formative and/or summative; both excellent, many schools have responded measure knowledge, skills, behaviors, with “surface” changes and most engi- attitudes, and values; and both focus on neering programs have not taken a hard individual students or groups of stu- look at defining outcomes, or teaching dents. The differences entail the degree in relation to these outcomes. Most of complexity, time span, cost, level of programs depend on exit surveys that specificity of the measure, degree of ask graduating seniors and employers if accountability for the assessment certain knowledge and skills have been process, and level of faculty commit- achieved. ment. Rogers identified constraints on Rogers pointed out, however, that assessment and practices that should be even these surface changes could have taken into consideration: time, facilities, some significance. Programs are begin- subject matter relevance, and student ning to recognize the importance of knowledge factors such as differences defining learning outcomes and to map in preexisting knowledge, out-of-class and identify gaps in their curriculum. experiences, and selected sequence of Alan Kay, Viewpoints Research Institute, courses. Inc., drew attention to the bigger picture She continued by describing what she of outcomes, or enlightenments, such as referred to as “mapping strategies”: those that enable individuals to invent procedures for identifying where in the entirely new technologies that could not curriculum students have the opportuni- be easily measured but would play ties to learn, apply, and demonstrate the powerful roles in future innovations. knowledge and skills proposed in the Rogers acknowledged the significance learning outcomes. The process of of his point, but reiterated that defining mapping informs the instructor or explicit learning outcomes is necessary, administration about existing opportuni- because instructors and institutions are ties as well as gaps in the course or accountable for assessing student program. Rogers shared her vision of a 44 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

syllabus for students that detailed their means of distinguishing between in- opportunities and responsibilities to structional strategies that are more gain specific knowledge and skills. effective and those that are less so in Faculty members at her institution reaching those goals. Yet, despite the receive a curriculum map every quarter importance of evaluating teaching, most and are asked to indicate learning colleges and universities continue to opportunities in their courses as they struggle with the question of how to do relate to the learning outcomes pro- it (Seldin, 1999). A common method of jected by the department or institution. evaluating faculty is through student Rogers listed common assessment ratings. However, more reliable and methods and made available a booklet productive assessments rely on multiple she authored, Evaluating Student sources of evidence. Faculty may also Learning: E=MC2 Assessment Methods be evaluated by examination of teaching (2002), which describes available tools portfolios that describe course organiza- in detail: standardized exams, local tion and teaching materials, determina- developed exams, oral exams, perfor- tion of level of student learning, evalua- mance appraisal, simulations, written tion by peers and administrators, and surveys and questionnaires, exit and classroom observations by an instruc- other surveys, focus groups, external tional consultant (Seldin, 1999; Fink, examiners, behavioral observations, 2002). Numerous teaching observation archival records, and portfolios. She and evaluation instruments are de- concluded with some words of motiva- scribed in the literature (reviewed in tion for the participants: start early with Wilkerson and Lewis, 2002; NRC, 2003). assessment plans, prioritize, pick appropriate battles, seek out resources An Instructional Assessment Tool and reference materials, recognize that Anton Lawson, Arizona State University any one assessment plan does not fit In his presentation Tools for Assessing every situation or institution, and adopt Quality USTEM Instruction: Reformed various strategies from different Teaching Observation Protocol (RTOP), sources to meet individual needs. Lawson described the context for the Anticipating the discussion in Chapter development of the RTOP instructional 4, Rogers noted that if an institution assessment tool, and also highlighted wishes to encourage and reward excel- many of the significant findings from the lence in instruction, it must have an application of RTOP by faculty of the established set of goals that any course Arizona Collaborative for Excellence in is expected to achieve and a reliable the Preparation of Teachers (ACEPT). E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 45

(See details in Lawson’s paper in Appen- A). On a Likert scale, an observer dix A, this volume.) assesses a lesson on the basis of items Following best practices in faculty such as: “3) In this lesson, student development (Wright, 2002), ACEPT exploration preceded formal presenta- introduced summer training institutes in tion”; “5) The focus and direction of the which college faculty could experience lesson was often determined by ideas teaching methods based on the prin- originating with students”; “6) The ciples of effective teaching introduced lesson involved fundamental concepts of by the American Association for the the subject”; “8) The instructor had a Advancement of Science (AAAS) in solid grasp of the subject matter content Science for All Americans (1990). These inherent in the lesson.”; “11) Students principles emphasize that teaching used a variety of means (models, draw- should be consistent with the nature of ings, graphs, concrete materials, scientific inquiry and recommend many manipulatives, etc.) to represent phe- of the teaching approaches discussed nomena”; “12) Students made predic- earlier in this workshop by Heron, tions, estimations and/or hypotheses Reiser, and others (see Lawson’s paper, and derived means for testing them.”; Appendix A). Lawson’s group then and “17) The instructors questions employed the RTOP instrument to triggered divergent modes of thinking.” evaluate whether the institutes had an In an ongoing investigation of the effect on faculty’s use of ACEPT teach- ACEPT program, instructors were ing methods in their courses and evaluated in five courses. To make whether these teaching methods in turn meaningful comparisons, several in- had an effect on student achievement. structors in each course were rated with (Details are available at http:// the instrument. Various instructors who purcell.phy.nau.edu/AZTEC/RTOP/ exhibited considerable variation in the RTOP_full/index.htm, and in Lawson’s extent to which they had embraced the paper in Appendix A.) reformed methods during the summer The 25-item RTOP observation institute were selected for rating, as well instrument is organized into the follow- as instructors who did not participate in ing evaluation categories: lesson design the institute. The examined courses and implementation, propositional and included an introductory physics and pedagogical content knowledge, class- mathematics course (each designed room culture (interstudent and teacher/ especially for preservice elementary student interactions), and problem- school teachers), a large introductory solving orientation (Box A-2, Appendix biology course, an introductory physics 46 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

course for physics majors, and a biology State from instructors who had partici- course for preservice teachers taken pated in ACEPT courses were signifi- near completion of their undergraduate cantly higher than those who had not. biology majors. For each course, data In a continuing investigation, Lawson included instructors’ RTOP scores and reported that ACEPT is building evi- students’ scores and/or normalized dence for extended positive effects of gains on tests appropriate to each ACEPT instruction for preservice course. teachers on the achievement of high Lawson outlined the significant school students that they teach. Lawson findings from the ACEPT investigation. concluded by pointing out that those The reliability of RTOP was demon- RTOP scores, which indicate the strated to be quite high. Trained inde- degree to which ACEPT instructional pendent observers were found to rate methods are implemented, are strongly individual instructors with similar correlated with improvements in stu- scores. The important result of the dent achievement not only in terms of investigation was that mean instructor conceptual understanding but also in RTOP scores correlated strongly (r = reasoning skills. The critical aspect of 0.88–0.97, p < 0.05, range of the five these instructional methods, according courses) with student achievement to Lawson, is that they include a broad gains, supporting the hypothesis that array of research-based teaching ACEPT teaching methods promote strategies. “Our project was based…on higher student achievement. Additional the teaching principles that are found in evidence for this conclusion was the the AAAS document called Science for finding of improved student content all Americans, which basically states knowledge and reasoning skills, as that teaching should be consistent with measured by an independent reasoning the nature of scientific inquiry. That skills assessment. Furthermore, a means [one should] start with ques- significant correlation (r = 0.70, p < 0.05) tions about nature. Engage students was noted between the RTOP scores of actively. Concentrate on the collection the TAs responsible for the introductory and use of evidence. Provide historical biology course labs and students’ perspective. Insist on clear expression. reasoning gains as measured in those Use a cooperative team approach and labs. In follow-up observations, the do not separate knowing from finding RTOP scores of in-service teachers who out and the memorization of textbook had received instruction at Arizona vocabulary.” E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 47

In response to participants’ concerns than other instructional strategies in about the accuracy of a one-time “snap- promoting understanding of complex shot” evaluation of a faculty member, concepts or the ability to apply such Lawson clarified that observations for concepts in new situations. Instructional these evaluations were conducted on programs known to the workshop three separate occasions for each participants to be effective in eliciting instructor at a time when the instructors such learning start by defining impor- were introducing new topics in the tant, measurable learning outcomes for classrooms. He concurred that an students; recognize that students have instructor’s RTOP score could increase diverse learning styles; provide varied if observed over extended periods or in experiences for students to develop different circumstances, such as labora- functional understanding of a subject; tory sessions. Richard McCray added promote students’ ability to work coop- that instructors must plan long-term eratively and to communicate orally and methods to encourage students to in writing; invest in training and become reflective about their learning; mentoring of instructors; and promote this does not occur in just one day. research on teaching and learning. Lawson responded that the summer Classroom observation assessment institutes did encourage developing instruments exist for evaluating an such procedures and that their single instructor’s degree of success in achiev- classroom observations are assumed to ing these goals. be appropriate snapshots of the results. An important element of effective instruction involves engaging students’ preconceptions and prior beliefs in ways SUMMARY that help them achieve a more mature understanding. Effective instructional The major points from the presenta- strategies for correcting misconceptions tions and discussions concerning the and producing conceptual understand- characteristics of effective instruction ing for most students require situations and how such instruction may be that demand active intellectual engage- assessed are summarized below. ment, such as tutorials, small group Accumulating research shows that learning, hands-on activities, case the traditional didactic lecture format studies, and problem-solving exercises can support memorization of factual with appropriate scaffolding. Scaffolding information but may be less effective (i.e., support and guidance in learning 48 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

specific concepts or tasks) can be described, they model in their teaching provided by an expert (instructor, the ways in which their own students teaching assistant, or peer learning should teach if those students go on to coach) or by a computer program. become graduate teaching assistants, When instructors employ effective K–12 teachers, or science faculty. instructional strategies of the types E VA L U AT I N G E F F E C T I V E I N S T R U C T I O N 49

Next: 4 Promoting Effective Instruction at Departmental and Institutional Levels »
Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop Get This Book
×
Buy Paperback | $54.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Participants in this workshop were asked to explore three related questions: (1) how to create measures of undergraduate learning in STEM courses; (2) how such measures might be organized into a framework of criteria and benchmarks to assess instruction; and (3) how such a framework might be used at the institutional level to assess STEM courses and curricula to promote ongoing improvements. The following issues were highlighted:

  • Effective science instruction identifies explicit, measurable learning objectives.
  • Effective teaching assists students in reconciling their incomplete or erroneous preconceptions with new knowledge.
  • Instruction that is limited to passive delivery of information requiring memorization of lecture and text contents is likely to be unsuccessful in eliciting desired learning outcomes.
  • Models of effective instruction that promote conceptual understanding in students and the ability of the learner to apply knowledge in new situations are available.
  • Institutions need better assessment tools for evaluating course design and effective instruction.
  • Deans and department chairs often fail to recognize measures they have at their disposal to enhance incentives for improving education.

Much is still to be learned from research into how to improve instruction in ways that enhance student learning.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!