National Academies Press: OpenBook
« Previous: References
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 87
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 88
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 89
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 90
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 91
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 92
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 93
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 94
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 95
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 96
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 97
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 98
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 99
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 100
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 101
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 102
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 103
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 104
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 105
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 106
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 107
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 108
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 109
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 110
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 111
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 112
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 113
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 114
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 115
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 116
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 117
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 118
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 119
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 120
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 121
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 122
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 123
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 124
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 125
Suggested Citation:"A Commissioned Papers." National Research Council. 2003. Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/10711.
×
Page 126

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix A Commissioned Papers 87

Using the RTOP to Evaluate Reformed Science and Mathematics Instruction1 Anton E. Lawson Department of Biology, Arizona State University INTRODUCTION Americans (1989). In turn, the AAAS teaching principles (see Box A-1) are The Arizona Collaborative for Excel- based on learning theory derived from lence in the Preparation of Teachers years of cognitive research. That theory (ACEPT) Program is a National Science posits that learning results from active, Foundation (NSF)-sponsored program learner-centered inquiry in which aimed at improving undergraduate students construct new concepts and science and mathematics instruction at conceptual systems by connecting new Arizona State University (ASU) and in information and concepts to what they the surrounding community colleges. already believe. Further, effective The primary reform mechanism has learning often requires restructuring, or been summer workshops in which even discarding, previous concepts and college faculty experience reformed beliefs when they prove incompatible teaching methods and then attempt to with, or contradictory to, new evidence implement those methods in their and new concepts (e.g., Alexander and courses. The reformed methods are Murphy, 1999). based on the principles of effective The ACEPT program has attempted teaching introduced by the American to incorporate reformed teaching Association for the Advancement of methods into several courses for Science (AAAS) in Science for All nonmajors and majors. These include 1 Based in part on Lawson et al. (2002). 89

BOX A-1 Principles of Effective Teaching • Teaching Should Be Consistent with the Nature of Scientific Inquiry: Start with questions about nature; Engage students actively; Concentrate on the collection and use of evidence; Provide historical perspectives; Insist on clear expression; Use a team approach; Do not separate knowing from finding out; Deemphasize the memorization of technical vocabulary. • Teaching Should Reflect Scientific Values: Welcome curiosity; Reward creativity; Encourage a spirit of healthy questioning; Avoid dogmatism; Promote aesthetic responses. • Teaching Should Aim to Counteract Learning Anxieties: Build on success; Provide abundant experience in using tools; Support the role of girls and minorities in science; Emphasize group learning. • Science Teaching Should Extend Beyond the School. • Teaching Should Take Its Time. SOURCE: AAAS (1989, pp. 200–207). Reprinted with permission of Oxford University Press. COMPARING REFORMED Introduction to Physical Geology, Funda- INSTRUCTION WITH STUDENT mentals of Physical Science, Theory of ACHIEVEMENT Elementary Mathematics, Patterns in Nature, The Living World, University Fundamentals of Physical Science Physics, and Methods of Teaching Biology. (PHS 110) is an introductory course Evaluation has focused on two central designed specifically for preservice questions: What effect, if any, have the elementary school teachers. A test of summer workshops had on participant physics concepts, developed by course faculty’s use of reformed teaching instructors and the ACEPT evaluation methods? And what effect, if any, does team, was administered to four experi- the use of reformed methods have on mental and two control PHS 110 sec- student achievement? The following tions at the beginning and again at the sections describe evaluation efforts in end of a recent semester. A member of five courses and a brief evaluation of the the ACEPT Program at ASU (the teaching methods used by some recent principal investigator) taught one graduates as they begin their elemen- experimental section. Community tary, middle, or high school teaching college instructors who had participated careers. 90 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

in an ACEPT summer workshop taught tive coefficients (16 pairs, r = 0.94; the other three experimental sections. 4 pairs, r = 0.99; 7 pairs, r = 0.97; 6 pairs, Importantly, these instructors were not r = 0.94; 5 pairs, r = 0.93; 9 pairs, selected at random. Rather, they were r = 0.90). selected because they exhibited consid- Mean RTOP scores for each PHS 110 erable variation in the extent to which instructor and the respective normal- they appeared to be embracing the ized pre- to posttest achievement gains reformed methods during the summer (i.e., percent gain/percent gain pos- workshop. Community college instruc- sible) for each instructor’s students tors who had not participated in a (n = number of students in each section) summer workshop taught the two were calculated. Among the experimen- control sections. tal sections, mean RTOP scores varied Instructional methods were evaluated from 27 to 73. Mean RTOP scores for using an ACEPT-developed instrument the two control instructors were 28 and called the Reformed Teaching Observa- 37. Normalized achievement gains tion Protocol (RTOP). The RTOP varied from 0–57 percent across all consists of 25 statements about the sections. Importantly, mean instructor extent to which reforms are incorpo- RTOP scores correlated strongly with rated into instructional practice (see student achievement gains (r = 0.88, p < Box A-2; details available at http:// 0.05). This result supports the claim ecept.net/rtop/). Each statement is that reformed teaching methods pro- scored on a 0–4 “Never Occurred” to mote higher achievement. Figure A-1 “Very Descriptive” scale. Thus, the shows instructor RTOP scores and RTOP allows observers to rate instruc- normalized gains on the test of physics tion on a 0–100 scale. Details of RTOP concepts for ACEPT (experimental) and development and administration can be control sections. found in Sawada (1999), Sawada et al. Theory of Elementary Mathematics (2000a), and Sawada et al. (2000b). (MTE 180) is an introductory course Estimates of inter-rater reliability have designed specifically for preservice been obtained using seven trained elementary school teachers. Four MTE evaluators as they observed several 180 instructors participated in the initial math and science instructors and ACEPT summer workshop. Subse- independently scored several lessons. quently, one of those instructors (from Inter-rater reliabilities have been high as ASU) helped two additional ASU MTE evidenced by the following pairs of 180 instructors develop reformed independent observations and respec- teaching methods. During a recent APPENDIX A 91

BOX A-2 Reformed Teaching Observation Protocol (RTOP) Lesson Design and Implementation (1) The instructional strategies and activities respected students’ prior knowledge and the preconceptions inherent therein. (2) The lesson was designed to engage students as members of a learning commu- nity. (3) In this lesson, student exploration preceded formal presentation. (4) The lesson encouraged students to seek and value alternative modes of investiga- tion or problem solving. (5) The focus and direction of the lesson was often determined by ideas originating with students. Content Propositional Knowledge (6) The lesson involved fundamental concepts of the subject. (7) The lesson promoted strongly coherent conceptual understanding. (8) The instructor had a solid grasp of the subject matter content inherent in the lesson. (9) Elements of abstraction (i.e., symbolic representations, theory building) were encouraged when it was important to do so. (10) Connections with other content disciplines and/or real world phenomena were explored and valued. Procedural Knowledge (11) Students used a variety of means (models, drawings, graphs, concrete materi- als, manipulatives, etc.) to represent phenomena. (12) Students made predictions, estimations, and/or hypotheses and devised means for testing them. semester, six sections of MTE 180 was administered at the beginning and participated in a study. Three ACEPT- again at the end of the semester. During influenced instructors taught three the semester, each instructor was sections at ASU and control instructors evaluated at least twice using the RTOP. taught three sections (one at ASU and Instructor mean RTOP scores and two at a nearby community college). A student posttest scores on the concept- test measuring concept understanding, understanding test were calculated for number sense, and computational skills each section. Instructor mean RTOP 92 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

(13) Students were actively engaged in thought-provoking activity that often involved critical assessment of procedures. (14) Students were reflective about their learning. (15) Intellectual rigor, constructive criticism, and the challenging of ideas were valued. Classroom Culture Communicative Interactions (16) Students were involved in the communication of their ideas to others using a variety of means and media. (17) The instructor’s questions triggered divergent modes of thinking. (18) There was a high proportion of student talk and a significant amount of it occurred between and among students. (19) Student questions and comments often determined the focus and direction of classroom discourse. (20) There was a climate of respect for what others had to say. Student/Instructor Relationships (21) Active participation of students was encouraged and valued. (22) Students were encouraged to generate conjectures, alternative solution strate- gies, and ways of interpreting evidence. (23) In general, the instructor was patient with students. (24) The instructor acted as a resource person, working to support and enhance student investigations. (25) The metaphor “instructor as listener” was very characteristic of this classroom. NOTE: Each item is scored on a 0–4 “Never Occurred” to “Very Descriptive” scale. SOURCE: Lawson et al. (2002, p. 390). Reprinted with permission of National Science Teachers Association. scores and student posttest scores were sense scores (r = 0.92, p < 0.001). These found to correlate strongly (r = 0.94, results further support the claim that p < 0.001). Mean RTOP scores and reformed teaching methods improve normalized gains also correlated student achievement. As predicted, no strongly (r = 0.86, p < 0.001). A very relationship was found between instruc- strong positive correlation was also tors’ mean RTOP scores and student found between instructors’ mean RTOP posttest performance on the computa- scores and student posttest number tional skills section. This result was APPENDIX A 93

100 90 RTOP Score / Normalized Gain (%) RTOP Normalized Gain 80 (n=46) (n=20) 70 60 (n=11) 50 (n=11) 40 (n=16) (n=14) 30 20 10 0 ACEPT PI Workshop Workshop Workshop Control 1 Control 2 (ASU) Participant 1 Participant 2 Participant 3 Fundamentals of Physical Science Course Section FIGURE A-1 Instructor RTOP scores and normalized gains on the test of physics concepts for ACEPT and control sections of PHS 110. SOURCE: Lawson et al. (2002, p. 390). Reprinted with permission of National Science Teach- ers Association. predicted because items in this section reformed teaching methods during a required only routine algorithmic three-day summer workshop followed procedures. by two-hour TA meetings each Friday The Living World (BIO 100) is an during the fall semester. introductory biology course enrolling A primary goal of BIO 100 is to about 750 students per semester. A improve students’ reasoning skills. faculty member presents three 50- Consequently, during the past several minute lectures each week. Graduate semesters, a 25-item pre- and posttest of teaching assistants (TAs) teach the labs. reasoning skills has been administered Labs meet once each week for two (Lawson et al., 2000). Figure A-2 shows hours. Students must enroll for both the the frequency of students at each score common lectures (all delivered by the on both the pre- and posttest and reveals faculty member) and one of several lab substantial and statistically significant sections (each taught by one of the pre- to posttest gains from a recent several TAs). TAs are introduced to semester (dependent T = 14.9, p < 0.001). 94 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

70 Pretest 60 Posttest 50 Frequency 40 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 The Living World : Reasoning Skills Pretest and Posttest Score FIGURE A-2 Pre and posttest scientific reasoning scores for students enrolled in BIO 100. SOURCE: Lawson et al. (2002, p. 391). Reprinted with permission of National Science Teach- ers Association. During that semester, the nine TAs taught by ACEPT-influenced instruc- were independently evaluated using the tors). Two experimental sections were RTOP. Regardless of the fact that all taught at ASU and one was taught at a TAs were introduced to teaching re- community college. A non-ACEPT- forms in the same manner, and all the influenced instructor taught the control BIO 100 labs are inquiry (learning section at a community college. A cycle) based, TA mean RTOP scores diagnostic test of mechanics concepts varied from 42 to 90 (inter-rater reliabil- called the Force Concept Inventory ity of r = 0.90, p < 0.001). Importantly, (Halloun and Hestenes, 1985) was TA mean RTOP scores correlated administered to all sections to assess significantly with normalized gains in pre- to posttest gains. Instructors’ student reasoning (r = 0.70, p < 0.05). mean RTOP scores and normalized University Physics 1: Mechanics (PHY gains were compared and a strong 121) is an introductory course designed positive correlation was found (r = 0.97, for physics majors that focuses on p < 0.01). Once again, this indicates that mechanics. A course evaluation was reformed teaching methods promote conducted using three experimental student achievement. sections of PHY 121 (i.e., sections APPENDIX A 95

BOX A-3 The Nature of Science Survey Next to each item write the number that best reflects your current belief: 1 = strongly disagree 2 = disagree 3 = don’t know 4 = agree 5 = strongly agree 1. The primary goal of modern science is to describe and explain natural phenomena. 2. Hypotheses are derived from controlled observations of nature. 3. A hypothesis is an educated guess of what will be observed under certain conditions. 4. A conclusion is a statement of what was observed in an experiment. 5. Hypotheses/theories cannot be proved to be true beyond any doubt. 6. Hypotheses/theories can be disproved beyond any doubt. 7. To be scientific, hypotheses must be testable. 8. To test a hypothesis, you need a prediction. 9. A hypothesis that gains support becomes a theory. 10. A theory that gains support becomes a law. 11. Truth is attainable through repeated supporting observations. 12. The primary goal of modern science is to discover facts about nature. 13. Scientific statements that are “just a theory” are of little value. SOURCE: Lawson et al. (2002, p. 391). Reprinted with permission of National Science Teachers Association. Methods of Teaching Biology (BIO assessed using the previously men- 480) is taught at ASU each spring for tioned reasoning test (Lawson et al., preservice biology teachers after they 2000). Students were also pre- and have completed, or are about to com- posttested using a 13-item ACEPT- plete, an undergraduate biology major. developed survey of the nature of In addition to using reformed methods science (see Box A-3). The survey to teach the preservice teachers about includes items that focus on the mean- those reformed methods, the course ing of terms such as hypothesis, predic- attempts to help students develop their tion, theory, law, proof, truth, fact, and reasoning skills and improve their conclusion. These are terms that are not understanding of the nature of science only central to the business of doing (NOS). During a recent semester, science but are also terms that are used students’ reasoning skills (classified into inconsistently and sometimes even developmental stages 3, 4, and 5) were contradictorily by many, if not most, 96 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

100 Pretest (n=5) Nature of Science Score (%) 80 Posttest (n=11) 60 (n=3) 40 (n=4) 20 0 Stage 3 Low Stage 4 High Stage 4 Stage 5 Methods of Teaching Biology : Student Developmental Level FIGURE A-3 Pretest and posttest performance on BIO 480 students at each developmental level. SOURCE: Lawson et al. (2002, p. 392). Reprinted with permission of National Science Teach- ers Association. scientists. The assumption is made that instruction is effective at improving these inconsistencies and contradictions NOS understanding, but (3) substantial are confusing to students who are trying gains in NOS understanding depend, at to better understand the research least in part, on students’ developmental process. level. Although current research on this As shown in Figure A-3, pretest NOS last point is preliminary, a plausible scores were low and unrelated to prediction is that becoming a skilled developmental level. However, posttest inquiry teacher requires advanced NOS scores were considerably higher. reasoning skills and a good understand- Further, posttest NOS scores were ing of the nature of science. If this is strongly related to developmental level indeed the case, then additional (F3,22 = 7.38, p < 0.01). These results are changes in the undergraduate curricu- important because they suggest that: lum will need to be made to insure that (1) without explicit NOS instruction, all students, particularly those who will biology majors learn very little about become teachers, develop advanced the nature of science, (2) inquiry in- reasoning skills. struction that includes explicit NOS APPENDIX A 97

HOW EFFECTIVE ARE ACEPT- More recently, we have found that INFLUENCED BEGINNING ACEPT-influenced high school biology TEACHERS? teachers have significantly higher RTOP scores than a group of control An important component of the teachers. Further, their students demon- ACEPT evaluation has focused on the strated significantly higher achievement teaching effectiveness of recent gradu- in terms of scientific reasoning, NOS ates as they begin their public school understanding, and understanding of teaching careers. A preliminary look at biology concepts than students of first-year teacher performance reveals control teachers (teacher n = 28, student significant differences (p = 0.05) in favor n = 1,115). Results were most divergent of ACEPT-trained teachers (i.e., mean for scientific reasoning. Depending on RTOP score of 48 among 20 teachers the amount of ACEPT influence, reason- who had enrolled in an ACEPT-influ- ing skills were from 25–46 percent enced science or mathematics course as better among students of ACEPT- undergraduate students compared with influenced teachers (Adamson et al., a mean RTOP score of 40 among a 2002). sample [n = 8] of teachers who had not encountered one or more ACEPT- reformed courses during their teacher CONCLUSIONS, preparation program). Similar data for RECOMMENDATIONS, AND SOME second- and third-year teachers were FURTHER QUESTIONS found. Importantly, the ACEPT-influ- enced teachers continue to outperform The primary result of the present the non-ACEPT teachers (mean RTOP evaluation is that, when implemented, score of 62 versus 45, p < 0.05). Also the AAAS teaching principles lead to RTOP performance improved from the improved student achievement in a first year for both groups. This improve- variety of undergraduate science and ment is encouraging because it suggests mathematics courses. This result not that a statewide movement to reform only supports the usefulness of the science and mathematics instruction AAAS teaching principles, but also (Arizona Department of Education, supports the active, learner-centered, 1997) and complementary local reform theory upon which those principles are efforts are having a positive and general based. Another important aspect of the impact on instructional reform. present evaluation is the development 98 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

and validation of a teaching observa- ones in the present sample, initially tional protocol (the RTOP). The RTOP know very little about the nature of enables trained observers to reliably science. Importantly, acquiring such evaluate instruction in terms of the knowledge appears to be linked to extent to which it incorporates reformed reasoning skill. This suggests that many teaching methods. In addition to evalu- science majors may not only need help ating current teaching methods, the in acquiring understanding of the nature RTOP could become an important of science, but they may also need help instrument to help instructors improve in developing scientific reasoning skills their classroom instruction. Perhaps a (Anderson and Mitchener, 1994; Coble useful extension of the present results and Koballa, 1996; Haney, Czerniak, and would be a study of the sort envisioned Lumpe, 1996; Lawson, 1999; Lawson et by Feuer, Towne, and Shavelson (2002) al., 2000). Clearly, much work remains that explores the relationship between to be done for college faculty to become RTOP scores and student achievement more effective in the classroom. Per- over a much larger number and diver- haps the present results will contribute sity of courses. to that ongoing process by suggesting The present evaluation indicates that one way in which such improvements when preservice teachers encounter can be made. reformed instruction as undergraduates Further questions include: they are more likely to incorporate those reforms into their own teaching • Why are some faculty members practices after graduation. This result resistant to reform? supports the familiar adage that “teach- • What can be done to overcome that ers teach as they have been taught.” resistance? This is an important finding as it offers a • What, if any, important reformed possible solution to the well-docu- method/strategy does RTOP not mented need for K–12 curricular re- measure? form. Namely, reform the way in which • What is the best way to help faculty preservice teachers learn science and members become skilled teachers? mathematics as undergraduates and • What support system needs to be they will carry those reforms with them in place to encourage reform? to K–12 classrooms. • What misconceptions exist regard- Finally, the results indicate that ing reform? preservice biology teachers, at least the APPENDIX A 99

ACKNOWLEDGMENTS Arizona Department of Education. (1997). Arizona academic standards. Phoenix, AZ: Author. This material is based upon research Coble, C.R., and Koballa, T.R. (1996). Science education. In J. Sikula, T. Buttery, E. Guyton supported by NSF under grant No. DUE (Eds.), Handbook on teacher education (2nd 9453610. Any opinions, findings, and ed.) (pp. 459–484). New York: MacMillan. Feuer, M.J., Towne, L., and Shavelson, R.J. conclusions or recommendations (2002). Scientific culture and educational expressed in this publication are those reform. Educational Researcher, 31(8), 4–14. Halloun, I.A., and Hestenes, D. (1985). The initial of the author and do not necessarily knowledge state of college physics students. reflect the views of the NSF. We would American Journal of Physics, 53(11), 1043– like to express gratitude to Terry 1055. Haney, J.J., Czerniak, C.M., and Lumpe, A.T. Woodin, Harriet Taylor, and Joan Prival (1996). Teacher beliefs and intentions regard- of the NSF, and O. Roger Anderson, ing implementation of science education reform strands. Journal of Research in Science chair of the ACEPT National Advisory Teaching, 33(9), 971–993. Committee, for their continuing support Lawson, A.E. (1999). What should students know about the nature of science and how should we and guidance. teach it? Journal of College Science Teaching, 28(6), 401–411. Lawson, A.E., Alkhoury, S., Benford, R., Clark, B., and Falconer, K.A. (2000). What kinds of REFERENCES scientific concepts exist? Concept construction and intellectual development in college biology. Journal of Research in Science Teaching. 37(9), Adamson, S.L., Banks, D., Benford, R., Burtch, 996–1018. M., Cox, F., Judson, E., Turley, J.B., and Lawson A.E., Benford, R., Bloom, I., Carlson, M., Lawson, A.E. (2002). Reformed undergraduate Falconer, K., Hestenes, D., Judson, E., Piburn, instruction and its impact on secondary school M., Sawada, D., Turley, J., and Wyckoff, S. teaching practice and student achievement: Does (2002). Evaluating college science and systemic reform work? (ACEPT Technical mathematics instruction. Journal of College Report). Tempe, AZ: Arizona Collaborative for Science Teaching, 31(6), 388–393. Excellence in the Preparation of Teachers. Sawada, D. (1999). Psychometric properties of Alexander, P.A., and Murphy, P.K. (1999). The RTOP (ACEPT Technical Report No. IN99-2). research base for APA’s learner-centered Tempe, AZ: Arizona Collaborative for Excel- psychological principles. In N.M. Lambert and lence in the Preparation of Teachers. B.L. McCombs (Eds.), How students learn: Sawada, D., Piburn, M., Falconer, K., Turley, J., Reforming schools through learner-centered Benford, R., Bloom, I., and Judson, E. (2000a). education (pp. 25–60). Washington, DC: Reformed teaching observation protocol (RTOP) American Psychological Association. (ACEPT Technical Report No. IN00-1). Tempe, American Association for the Advancement of AZ: Arizona Collaborative for Excellence in the Science. (1989). Science for all Americans. Preparation of Teachers. Washington, DC: Author. Sawada, D., Piburn, M., Turley, J., Falconer, K., Anderson, R.D., and Mitchener, C.P. (1994). Benford, R., Bloom, I., and Judson, E. (2000b). Research on science teacher education. In D.L. Reformed teaching observation protocol (RTOP) Gabel (Ed.), Handbook of research on science training guide (ACEPT Technical Report No. teaching and learning (pp. 3–44). New York: IN00-2). Tempe, AZ: Arizona Collaborative for MacMillan. Excellence in the Preparation of Teachers. 100 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

Effecting Faculty Change by Starting with Effective Faculty: Characteristics of Successful STEM Education Innovators2 Susan B. Millar Wisconsin Center for Education Research, University of Wisconsin–Madison INTRODUCTION was studying3 share certain characteris- tics, but I had no opportunity to system- For some 10 years now, I have been atically consider and then articulate learning through my work as an evalua- what these characteristics might be. So tor about education faculty innovators it was that upon considering Bob who work in the fields of science, DeHaan’s request to write about how to technology, engineering, and mathemat- promote curricular and pedagogical ics (STEM). As evaluators do, I moved improvements, I reasoned that findings without pause from one interesting on characteristics shared by STEM evaluation project to the next. In fleeting faculty who are largely successful at moments, I have become increasingly effecting change might help others certain that the faculty whose courses I (faculty as well as professional develop- 2 3 I shared an early draft of this document with These faculty members were attempting to the people listed in Box A-4, plus three social implement substantial innovations in their scientists who also work with STEM faculty courses. In almost all cases the evaluations were education innovators. Many responded with designed to provide formative feedback to inform comments, some of which I quote directly in this decisions about midcourse improvements. The revised paper. In particular, I thank Steve findings often were also used for summative Ackerman, Josefina Arce, Jean-Pierre Bayard, reporting purposes. All projects involved Aaron Brower, Ann Burgess, Diane Ebert-May, interviews with the key faculty, teaching assis- Art Ellis, Fiona Goodchild, Curt Hieggelke, tants, and at least some of the students. In Gretchen Kalonji, Elaine Seymour, Jerry Uhl, and addition, students usually completed surveys John Wright for their insightful comments. I also about their learning processes. Classroom and thank Denice Denton and John Moore for their laboratory observations were made, but on a support for the ideas presented in this document. limited basis. APPENDIX A 101

ment staff and policy makers on cam- Noticing this, he may then realize that puses and in funding agencies) effect he believes that students will take faculty change.4 responsibility for their own learning but How would such findings be helpful? that his practice (based on teaching as Trained as an anthropologist, it is he was taught) of maintaining control in evident to me that people are more able the class at all times is at odds with his to recognize who they are, and who they beliefs about student responsibility. Or, are not, by comparing themselves to the process of reviewing this set of others. And this process is more effec- characteristics of successful STEM tive when the characteristics of the education innovators might help a “others” are articulated in some detail. faculty member realize that some of her My hope, then, is that individuals who teaching practices, while unusual in her assess themselves and key colleagues in department, are common among innova- light of a set of characteristics shared by tors across the country. I also reasoned STEM education faculty innovators that knowledge of these characteristics might better identify, for example, might enable faculty and other change habits or implicit assumptions that may agents to recognize others who have be thwarting their efforts to achieve these characteristics, and who need a their goals as educators. word of encouragement, or a new skill To illustrate, a faculty member who or contact in order to keep the faith, or, cares deeply about teaching and learn- better yet, to really flourish. In other ing and knows on some level that words, this kind of learning through students should be more actively reflection might help faculty become involved in classroom activities might more accomplished and productive as realize that he differs from the innova- reflective practitioners (Schön, 1983, tors described here, in that he is not 1995). willing to hand over some decision- I also chose this topic for two other making authority to the students. reasons. One is that I anticipated that two other participants in the workshop, Elaine Seymour and Robert Zemsky, would complement my focus on faculty 4 Aaron Brower observed that the characteris- as individuals with talks that focused on tics presented here may well also be common to the organizational parameters that faculty innovators in the social sciences and humanities. I suspect he is correct, but could not, promote and constrain faculty efforts to on the basis of my experience as an evaluator, improve how undergraduate students extend the generalizations presented here beyond STEM faculty. learn in STEM courses and programs. 102 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

The other reason was that I knew this The process I used to formulate these paper would benefit from input from the common characteristics was to conduct workshop participants—many of whom an informal thematic analysis based on I know have deep knowledge of STEM findings that appear in evaluation faculty innovators—about the adequacy reports and case studies, and on points of these characterizations and about that some of these faculty made in how we might use them to inform action conversation. I organized the emergent strategies that faculty and other change characteristics of STEM faculty innova- agents might use to good effect.5 tors into topic areas pertaining to general personality features, attitudes Methods and habits of interpersonal interaction, The group about whom I am general- learning and teaching practices, pro- izing includes essentially all the STEM cesses for changing one’s own teaching faculty innovators whose innovations I practices, processes for fostering have studied during the last decade, change in the teaching practices of plus many others with whom I have communities, and the characteristic of worked and held extended discussions “peripheral vision.” In some places, I about teaching and learning. (I list many provide references to work in the of these individuals in Box A-4.) Almost emerging “learning sciences” literature all of these people are successfully that presents many of these same promoting pedagogical improvements, characteristics as key to effective and some are successfully promoting learning situations, and include some of curricular improvements (the latter the responses of those listed in Box A-4 being more difficult in that curriculum to these themes and to earlier drafts of tends to be a “canon” for which an this paper. entire discipline, or at least a depart- ment, shares responsibility). Moreover, most are also effecting change among COMMON CHARACTERISTICS OF their colleagues. STEM EDUCATION FACULTY INNOVATORS 5 For their insightful contributions to this General Personality Features paper during the CUSE workshop, I thank Katayoun Chamany, Robert DeHaan, Paula Certain general personality features Heron, Alan Kay, Priscilla Laws, Richard McCray, Lillian McDermott, Elaine Seymour, Susan stand out as common to the successful Singer, Lillian Tong, Carl Wieman, Jack Wilson, STEM education faculty innovators Michael Zeilik, and Robert Zemsky. whose work informs this paper. In APPENDIX A 103

BOX A-4 Stem Faculty Innovators Who Informed This Analysis I drew on evaluation and case studies involving the following faculty. Unless other- wise specified, these individuals are or were members of the University of Wisconsin- Madison teaching staff: • Melinda Certain and Mike Bleicher: Wisconsin Emerging Scholars calculus • Denice Denton: electrical engineering • Art Ellis: materials enriched general chemistry • Pat Farrell and colleagues, College of Engineering: introductory engineering design • Eric Frost, San Diego State University: geology for majors • Curt Hieggelke, Joliet Junior College: introductory physics • Gretchen Kalonji, University of Washington: materials science • Tim Killeen, Ben Van der Pluim, and colleagues, University of Michigan: global change • John Moore and colleagues: introductory chemistry • Robin Pemantle and colleagues: mathematics for preservice teachers • Jerry Uhl and colleagues, University of Illinois at Urbana-Champagne: biocalculus; at University of Massachusetts-Dartmouth: freshman engineering • John Wright: introductory analytic chemistry I drew on additional information obtained during extended conversations with: • Steve Ackerman, Center for the Integration of Research on Teaching and Learn- ing (CIRTL): meteorology • Josefina Arce, University of Puerto Rico at Rio Piedras: chemistry • Jean-Pierre Bayard, California State University–Sacramento: electrical engineer- ing • Ann Burgess: biology • Judith Burstyn, CIRTL: chemistry • Diane Ebert-May, Michigan State University: biology • Francis Halzen, CIRTL: physics • Jim Haynes, State University of New York College, Brockport: environmental sciences • Michelle Hluchy, Alfred University: geology • John Jungck, Beloit College: biology • Jean MacGregor, The Evergreen State College: environmental studies • Bob Mathieu, CIRTL: astronomy (principal investigator) • Greg Moses, CIRTL: computer science for engineering students • Jim Taylor: chemistry SOURCE: Millar (2002, November). 104 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

short, they are risk takers and very listen respectfully to students (“there hard workers. They make commitments are no dumb questions”), strive to build and stick with them to the end. Many on students’ questions and ideas, and are inspired by a sense of mission. And quickly recognize and are delighted by they are savvy and persistent about the occasional startling insight that a obtaining resources, including moral student presents. Mike Bleicher re- and material support from proactive sponded to this point by reminding me administrators and external funding of the Biblical saying, “A wise man agencies. They take pride in doing a learns more from a fool than a fool good job for their students and often for learns from a wise man.” their departments, disciplines, and/or • These faculty not only are com- institutions as well. Most are not espe- fortable admitting to students when cially charismatic in their personal style. they did not know something or made a mistake, but also value these situations Attitudes and Habits of as opportunities to engage their stu- Interpersonal Interaction dents in the kind of problem solving that Many people who are not engaged in is central to the scientific process. Most STEM education innovation can, how- of these educators are at least as inter- ever, be described by the general ested in teaching the process by which characteristics listed above. Thus, while discoveries are made as the outcomes of perhaps necessary, these general those discoveries. In his response to a features certainly are not exclusive to draft of this paper, John Wright affirmed successful STEM education innovators. this point and illustrated how he makes That is, they are not defining character- good use of mistakes: istics. By contrast, I believe that unless a person has the characteristic attitudes One of the powerful tools that I find useful and habits of interpersonal interaction in a course is to make sure that students know discussed below, they will not be in this that making mistakes is part of the scientific group of successful STEM education innovators. For brevity, I list these process and that the key to profiting from features as follows: them is making sure that you learn from them. Praising the aspects of a student’s • Their identity as a scholar does not work that are good and putting the mistakes depend on placing themselves above other faculty members, academic staff, in a proper perspective can do wonders for a graduate students or undergraduates student’s self-esteem and confidence. (Wilshire, 1990). Accordingly, they APPENDIX A 105

• They view students not as “outsid- different way by writing that he, for ers” but as less experienced potential example, really cares about students, peers. Accordingly, they design their and is motivated to earn his students’ courses and interact with students with respect. a “we’re in this together” attitude. They • They view graduate teaching make the effort to walk in students’ assistants as full members of the team shoes by taking time to recall what it and are eager for their input and feed- was like to not have concepts and skills back. They are willing to discuss their that they, as experts, take for granted failures (and what they have learned (Leamnson, 1999). Viewing students as from them) as well as their successes novice potential members of their with colleagues who also are experi- communities, they include them in the menting with innovation.7 real talk and real work of their “commu- nities of practice” (Lave and Wenger, Learning and Teaching Practices 1991). They therefore do not view I turn now to learning and teaching maintaining constant control of the practices that are common to the suc- classroom as a virtue, but rather seek cessful STEM education faculty innova- out ways to give students at least some tors whom I know. I would venture that decision-making power. this set of characteristics also consti- • In contrast to faculty who consider tutes a basic requirement for the people teaching a burden (“teaching load”) to I describe here, but my hunch is that, be accomplished in the least amount of compared to the characteristics listed time possible, these individuals feel above pertaining to attitudes and habits genuinely excited about students and of interpersonal interaction, those listed teaching.6 They enjoy seeing their below can more easily be developed students learn, and take a certain pride with experience. in their students’ accomplishments. As Josefina Arce put it, “We find pleasure in • Successful STEM education faculty seeing our students learn—a pleasure innovators experience teaching as similar to the one we feel when an intellectually exciting—as another experiment works well.” Jean-Pierre opportunity to learn that is no less Bayard expressed this point in a slightly engaging than the scholarship they 6 I thank Ann Burgess and several people at 7 the CUSE workshop for reminding me of this Special thanks to Ann Burgess for reminding characteristic. me of this characteristic. 106 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

pursue in their STEM discipline. In (as educators and researchers), they are other words, they make learning about quick to question the status quo and learning a part of their scholarship.8 their own beliefs when they notice an Many have explained that the chal- inconsistency. lenges of teaching force them to put • They have very high expectations their research in a larger context, which of students. They want them to go often leads to new insights useful in beyond “knowing that” to “knowing their research. For example, John how” (Brown and Duguid, 2000) and Wright finds that his best ideas—in his “knowing why” (Hieggelke, personal research as well as teaching—come communication). For that matter, they from students. want students to get so engaged with • These innovators hold the convic- their learning that “they try on for size” tion that good teaching demands ongo- the identity of scientist, mathematician, ing creative effort, believe that it is or engineer (Seymour et al., 2002). important to “understand understand- • They hold the conviction that if ing” (Wiggins and McTighe, 1998), and faculty will demand it, students will take the time to learn about teaching. As accept the challenge of becoming one member of the CUSE workshop put independent thinkers. Accordingly, they it, they recognize that “self-reeducation expect their students to push them- takes years.” These individuals eschew selves to comprehend and use difficult recipes or quick fixes, and believe that ideas and acquire new skills. Persuaded everything one tries—whether success- that attempts to think for students or to ful or not—enhances their capacity to do control their thinking may actually better the next time (Stevens, 1988). interfere with their learning, they seek • They understand that learning to provide course materials and an depends on feeling puzzled, perturbed, environment that pushes the students to and curious, and on tolerating ambigu- do the thinking and, as Jerry Uhl put it, ity. They value cognitive dissonance as a to “learn to learn.” As Ann Burgess precursor to the process of changing a commented, “Students learn more when person’s understanding (Jonassen and you figure it out together than when you Land, 2000). Thus, in their own practice just tell them the answer!” Accordingly, they know that they and any other (graduate or undergraduate) course instructors must eschew the role of 8 authoritative provider of answers and I thank Priscilla Laws and Lillian McDermott for emphasizing this point. instead play the role of a guide—some- APPENDIX A 107

one who has traveled these paths and abstract concepts, they use “real stuff” remembers how it was the first time.9 in the curriculum, that is, open-ended • They believe that learning entails a problems, hard problems that they can constant moving back and forth be- relate to their everyday lives. They tween “practice” (trying things out, resist including material that not only making things happen) and “beliefs” the students will never use, but the (theories about the nature of things and faculty themselves would never use.10 why things happen) (Lave and Wenger, 1991; Wertsch, 1993). Thus, they design Aligned with their efforts to engage their courses to provide students with students in genuine dialogue is their “practice” by using hands-on problems tendency to use assessment not to and challenges. (Some of them refer to grade/judge (a process that closes this approach as “learning on demand.”) opportunities to learn), but rather to They design their courses to provide figure out what their students are learning processes that engage students assuming and concerned about (a in reflection through genuine dialogue process that opens up opportunities to with senior peers (e.g., fast and context- learn). They consciously use these sensitive feedback from teachers and “formative” assessment practices to help expert practitioners), other students keep themselves aware that most of (e.g., problem-solving in groups), and their students do not possess the mental self-reflection (e.g., individual writing models and habits that they had when and problem solving). they were students, let alone now that • They believe that they should only they are experts in their discipline. ask students to learn things that there is Evaluation findings on courses taught good reason to believe will “matter” to with this approach revealed that for the students. Thus, while fully expecting many of the students in their classes, them to eventually master difficult and the learning, not the grade, was para- mount (see, for example, Courter and Millar, 1995; Millar, Alexander, and 9 Several of the CUSE workshop participants Lewis, 1995; and Wright et al., 1998). pointed out that, in fact, most successful scien- tists approach the undergraduate and graduate students who work in their labs or on their projects in this way. We agreed, however, that STEM education innovators are distinguished from other scientists in that they believe that not just “their own,” but the vast majority of the 10 students in their undergraduate lectures and Special thanks to Steve Ackerman for this laboratories will rise to these challenges. point. 108 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

Processes for Changing Their sustained—approach to solving their Own Teaching Practices problems with teaching and learning. In The STEM education faculty innova- particular, they transform their con- tors with whom I have worked or with cerns into actionable “problems,” whom I have discussed teaching at develop plans and strategies that they length are similar in the ways they go hypothesize will solve their problems, about changing their own teaching and have in mind from the beginning practices. what outcomes they will accept as sufficient evidence of success. They • They are proactive and very constantly seek and reflectively use pragmatic problem solvers. By this I feedback information. That is, they mean, borrowing a concept from Covey gather information on how well their (1990), that they work in their “circle of strategies work, analyze and reflect on influence,” and while aware of problems this feedback—often mulling it over in their “circle of concern,” they spend with colleagues (Lave and Wenger, little if any time or emotional energy on 1991)—and adjust their strategies these concerns (pp. 82–83).11 Related to accordingly. this point, they tend to be people who • They use this act-feedback-reflect- do not waste time casting blame (on the adjust and act cycle on an ongoing and students, K–12 teachers or the K–12 cumulative basis, working step-by-step system, or the system in place at their and bringing their entire store of past college or university) when they realize feedback information to bear on each there are problems. Instead, they focus new adjustment (Stevens, 1988). I their energies on the business of doing illustrate and improve on this point by what they can to address these prob- quoting Steve Ackerman’s response lems. upon reading it in an earlier draft: • They take an experimental—that is, an intentional, systematic, and 11 With regard to this point, Diane Ebert-May noted that successful senior innovators nonethe- less devote substantial time to helping younger scholars learn how to redirect their energies from their sphere of concern into their sphere of influence. APPENDIX A 109

foundation (predominately the National I relate this point to my own experiences in Science Foundation, often the Howard teaching an introductory weather and climate Hughes Medical Institute, among course. I began by seeking no feedback from others). In some cases, these cross- institutional networks consist of faculty peers. A couple of years into teaching it, I who worked with the same professor or realized the need for and value of this, and research groups as graduate students.12 for a few years I sought out lots of peer I saw no case in which the group con- review. Now, after about ten years, I don’t sisted of a faculty developer and an individual STEM faculty member. solicit peer feedback. Rather I get it from the • Last, I would list an eventual turn teaching assistants and students. So I to the larger community of educators as wonder, am I getting lazy, or overconfident, a characteristic that these STEM faculty or am I fooling myself that student feedback share with regard to how they go about making change in their own courses. is the most appropriate for this stage in my Once they are quite certain that they career? Or does the cycle you mention have accomplished something valuable include different groups? as innovative teachers, most of them begin to notice that there is a body of • They purposely engage with peer research on learning, and that there is a learning communities (2–10 people) big network of people in diverse disci- and/or networks (up to 100) of people plines who are involved in this business. who are interacting about shared At this point, they begin to participate in problems and pursuing similar action larger networks through meetings, strategies (Hutchings, 1996; Shulman, email, and listservs, sharing citations 1993). That is, they develop new ideas and reading certain key pieces that and insights, and obtain new informa- make the rounds in their disciplinary tion, about teaching by interacting with communities (Shulman, 1993). In “near peers” (Rogers, 1995) in local response to this point, Steve Ackerman communities and in professional societ- wrote, “I would add that the large ies, or at least the education branches of networks, in turn, play a role in seeking those societies. In a few cases these out the innovators.” And Diane Ebert- peer groups are department based. Most often they are cross-departmental or cross-institutional. The latter usually 12 are or were externally funded by a Special thanks to Alan Kay for this point. 110 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

May’s response to this same point ments, specific disciplines, and/or in beautifully illustrates Steve’s addition: STEM education overall. They are similar not only in their willingness to play these roles, but also with respect to their basic reason for doing so: they are I think more and more faculty are committed to helping others benefit becoming aware of this network and body of from the innovations in teaching and research before or while they are embarking learning about which they have learned. Whether operating at very local or on their pathway of change. There are national levels, each of these leaders has multiple points along the continuum for left the lab and the classroom for at least people to begin, and they don’t necessarily some of their time in order to marshal wait until they have accomplished something and then manage the resources that leaders need to be successful. Each is valuable. Often, it depends on who they meet guided by homegrown models of or hear from initially. For example, for [her change (such as the “dipping the toe” current STEM education improvement model that guided Art Ellis for many project], one team traveled by car 12 hours years) that they may or may not have articulated explicitly.13 Each knows the to a field station. Two of the three were going importance of building networks with only because their friend asked them to. Their other innovator colleagues and with arms were crossed in front of their chests all campus administrators, and many are the way, teeth clenched, and attitude, well, very skilled at building communities focused on finding ways to make not good. After five days there, they were change. They know that unless they ready to rock and roll, and have continued collaborate and build on one another’s doing so ever since. efforts, it is not likely that their innova- tions will become the new status quo, that is, be institutionalized.14 Processes for Fostering Change in the Teaching Practices of Communities 13 For a useful discussion of the theories of Essentially all of the STEM education change held by STEM education faculty innova- tors, see Seymour (2002a). faculty innovators about whom I gener- 14 Special thanks to Lillian McDermott and alize here are taking leadership roles in Priscilla Laws for their insights about these points. order to foster change in their depart- APPENDIX A 111

With the help of participants at the Washington), Curt Hieggelke (physical CUSE workshop, I realized that these sciences, Joliet Junior College), innovator-leaders might be organized Deborah Allen (biology, University of into different groups, depending on Delaware), and Dave Gosser (chemis- their motivation: try, City College of New York). • Some are primarily motivated by • Some of these leaders’ primary the belief that improving the way we motivation is to support and encourage teach science is an excellent way to other science education innovators, improve conditions in the world at large. many of whom feel, and indeed are, Gretchen Kalonji (materials science, marginalized and are vulnerable. University of Washington) exemplifies Among these leaders are some who find individuals with this motivation. As she that while their efforts are not appreci- explained, in response to a draft of this ated in their own departments, this paper: price is worth paying because of the influence they have on colleagues If we believe that education is indeed a elsewhere across the nation. To be sure, well-known researchers, such as Eric path for democratizing society, for providing Mazur (physics, Harvard University) economic opportunity for our youth, etc., and and John Wright (chemistry, University if we know that we are doing a poor job at of Wisconsin-Madison), are more likely it, and/or living with methodologies that are to have this type of national influence, due to their visibility within their disci- exclusionary, it is a moral issue. I know that plines. many of the colleagues I admire the most • Others are primarily motivated to share these motivations at the core. help their colleagues by developing reliable new knowledge about how students in their discipline learn, and by • Yet other innovator-leaders believe providing tested innovative curriculum they can most effectively help others by resources that can make others’ efforts influencing policy at the national level. to adapt these new methods much People in these positions generally need easier. Faculty at all types of institutions excellent reputations as researchers and are taking this leadership path with administrators, in addition to their noted success. Examples include John credentials as education innovators. Jungck (biology, Beloit College), Lillian Examples of these leaders include McDermott (physics, University of Judith Ramaley (assistant director, 112 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

Education and Human Resources As a sociologist (I suppose), I notice that Directorate, National Science Founda- tion), and Robert DeHaan (director of the innovators, which I also see as the Committee on Undergraduate participating in the loose, cross-institutional Science Education, Center for Educa- national networks that you reference, often tion, National Research Council). comprise people who are in some ways • Last, there is a group of STEM education innovator-leaders who, as marginal in their relationships to their members of the National Academy of departments. I note that women are Sciences or as Nobel Laureates, wield especially overrepresented, given their enormous influence because their smaller numbers in most science disciplines credibility as researchers and leaders within their disciplines and departments except biology. There is also an important is beyond question. In this group are, group of “radical seniors” whose for example, in mathematics—Hyman invulnerability as distinguished researchers Bass (University of Michigan) and gives them protection, high visibility, and a Richard Tapia (Rice University); in physics—Leon Lederman (University of role as spokespersons. There are also more Chicago) and Carl Wieman (University people with less traditional career paths— of Colorado, Boulder); in astronomy— including some who have walked away from Richard McCray (University of Colo- the university tenure process into community rado, Boulder); in chemistry—Bradley Moore (Ohio State University); and in and liberal arts colleges, and into research biology—Bruce Alberts (president, the and educational scholarship roles.15 In this National Academy of Sciences). group of people there are also more young A “Positional” Characteristic— faculty (despite the tenure risk), and perhaps Peripheral Vision more scholars of color. Before concluding, I add a character- My theory is that people who stand slightly istic about which Elaine Seymour off-center “see” the need for change in ways reminded me. Upon reading a draft of that people who are trying to compete with this document, she wrote: each other for mainstream recognition by traditional means can’t always see, or 15 Parker Palmer makes this point at some (perhaps more importantly) can’t afford to length in his piece on “a movement approach to educational reform” (1992). see. APPENDIX A 113

I have noted this same characteristic need for and possibilities for change, repeatedly, and especially when listen- and then return to the mainstream to ing to these educators (those in Box A-4 work on accomplishing those changes. and many others) respond to my re- The capacity to use peripheral vision quest that they recount why and how depends in part on one’s choice to do they got so involved in this education so. However, as Seymour suggested, reform business. Each told a story of people who are different, for some being, or at least observing the tradi- reason or other, are more likely to take tional classroom as, an “outsider.” Of this participant observer stance. Pursu- note, almost to a person, when telling ing this point, Lillian McDermott noted their stories, these individuals appeared that using peripheral vision is a great to enjoy their “off-center” position, even source of intellectual excitement, a though in some cases it placed them at fascinating way to learn—and in particu- some risk. lar, an excellent way to learn about Upon reflection, I am coming to teaching. believe that just as their ability to discover patterns in the unfamiliar is a key to these faculty members’ success DISCUSSION AND CONCLUSIONS as scientists, a key to their success as education innovators is their ability to This set of characteristics shared by discover patterns in the all-too-familiar STEM faculty who are largely success- world of traditional classrooms and ful at effecting change (at least in their other higher education settings. This own courses) begs the question of how ability to see the familiar anew depends these people fare within their depart- on the capacity to see, as Bateson (1994) ments and institutions. As noted above, puts it, with “peripheral vision,” to they are wise enough to work in their notice out of the corner of the eye sphere of influence and avoid wasting something important about what is in their energies on things that they front of you. It entails taking what cannot affect. To be sure, there are anthropologists call the “participant cases where these “things they cannot observer” stance toward situations in affect” affect them altogether too much, our everyday world, and not taking as when assistant professors are denied these situations for granted. Standing tenure because their departments did outside the taken-for-granted main- not recognize or sufficiently value their stream, a person is better able to see accomplishments in the scholarship of things in a new light, to perceive the teaching. But, as a growing body of 114 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

evaluation and research findings indi- tional practices, in which faculty and cate, there are many other cases where students tacitly agree to dispense with one can see that these innovators have the formal educational tasks required of become leaders whose spheres of them by following paths of least resis- influence have grown into and positively tance. They also pointed out a number transformed features of their depart- of key opportunities that current cir- ments and institutions. cumstances afford STEM education I venture that the extent to which innovators, such as market pressures STEM education innovators thrive and for more effective and efficient learning achieve their goals as educators de- that are experienced by, for example, pends not only on how well they man- medical and business schools (Zemsky); age those “things they can affect,” but and a climate of trust among faculty, also on the constraints posed, and graduate students, and undergraduates opportunities afforded, by the institu- in STEM classrooms (Seymour). tional and cultural circumstances in I would ask you to consider yet which they are embedded. Several of another factor—one that can just as the innovators who responded to the powerfully “afford” as it can “constrain” initial draft of this paper wanted to the kind of change sought by these pursue questions about what circum- faculty innovators. This factor is the stances pose constraints that are too faculty themselves. Here is my argu- risky and for whom and, more generally, ment. If universities are enduring about what lessons we can learn from institutions, it is not because they resist, innovators who successfully maneuver but rather because they selectively around the constraints and play into the embrace, change—following the best opportunities. I do not attempt to lessons learned and principles held by address those questions here. their typically “ungovernable” faculty. However, in their presentations at the Innovation in STEM education is under- CUSE workshop, Robert Zemsky and way in the myriad decisions made and Elaine Seymour highlighted some of the actions taken by ungovernable faculty constraints that innovators face. For who are learning from their students example, Zemsky (2002) called our and one another, and who are encourag- attention to the enduring resistance to ing one another in loose, cross-national, change that is characteristic of universi- and inexorably expanding networks ties, while Seymour (2002b) brought to comprised of people like those featured light the daunting power of a cultural here. system, supported by myriad organiza- Moreover, it is important to note that APPENDIX A 115

the successful STEM education innova- Brown, J.S., and Duguid, P. (2000). The social life of information. Boston: Harvard Business tors featured here include a number of School Press. our “radicalized seniors.”16 These Courter, S.S., and Millar, S.B. (1995). Final evaluation report of first-year design course people are important because faculty, 1994–95: Introduction to engineering. Madison, however ungovernable, are inclined to WI: University of Wisconsin-Madison, Learn- ing through Evaluation, Adaptation, and learn from respected peers and to notice Dissemination Center. Available: http:// the values and actions of the most www.cae.wisc.edu/~lead/pages/internal.html [March 26, 2003]. esteemed and altogether credible Covey, S. (1990). The seven habits of highly members of their disciplines. successful people. New York: Simon & Schuster. In conclusion, I venture that, as Hutchings, P. (1996). Building a new culture of teaching and learning. About Campus, 1, 4–8. STEM faculty innovators—“radicalized Jonassen, D.H., and Land, S.M. (2000). Theoreti- seniors” and many, many others— cal foundations of learning environments. Hillside, NJ: Erlbaum. expand their spheres of influence, they Lave, J., and Wenger, E. (1991). Situated learning: are reshaping and redefining what it is Legitimate peripheral participation. Cam- bridge: Cambridge University Press. that “the faculty” takes as acceptable Leamnson, R. (1999). Thinking about teaching norms for teaching STEM courses. And and learning: Developing habits of learning with first year college and university students. (coming full circle), to the degree that Sterling, VA: Stylus. this paper helps restructure how we Millar, S.B. (2002, November). Effecting faculty change by starting with effective faculty: perceive these innovators among us, Characteristics of successful STEM education helps make them visible in new ways, it innovators. Paper commissioned for National participates in this process of reshaping Research Council’s workshop Criteria and Benchmarks for Increased Learning from what we take for granted in STEM Undergraduate STEM Instruction, Washing- education. ton, DC. Millar, S.B., Alexander, B.B., and Lewis, H.A. (1995). Final evaluation report on the pilot Wisconsin Emerging Scholars Program, 1993– 1994 (vol. 1). Madison, WI: University of REFERENCES Wisconsin-Madison, LEAD Center. Palmer, P.J. (1992). Divided no more: A move- Bateson, M.C. (1994). Peripheral visions: ment approach to educational reform. Change, Learning along the way. New York: 24(2), 10–17. HarperCollins. Palmer, P. J. (1993). Good talk about good teaching: Improving teaching through conversation and community. Change, 25(6), 8–13. Palmer, P. J. (1998). The courage to teach: Exploring the inner landscape of a teacher’s life. 16 At Zemsky’s suggestion, I have added a San Francisco: Jossey-Bass. suffix to Seymour’s term (“radical seniors”) to Rogers, E.M. (1995). Diffusion of innovations (4th indicate that these people either actually became ed.). Westport, CT: Free Press. more radical, or allowed themselves to develop Schön, D.A. (1983). The reflective practitioner: and act on their peripheral vision, once they How professionals think in action. New York: became “seniors.” Basic Books. 116 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

Schön, D.A. (1995). The new scholarship Wertsch, J. (1993). Voices of the mind: A sociocul- requires a new epistemology: Knowing-in- tural approach to mediated action. Boston: action. Change, 27(6), 27–34. Harvard University Press. Seymour, E. (2002a). Tracking the process of Wiggins, G., and McTighe, J. (1998). Understand- change in U.S. undergraduate education in ing by design. Alexandria, VA: Association for science, mathematics, engineering, and Supervision and Curriculum Development. technology. Science Education, 86, 79–105. Wilshire, B.W. (1990). The moral collapse of the Seymour, E. (2002b, November). Barriers to university: Professionalism, purity, and alien- change: Resistance is the normative mode. Talk ation (SUNY Series in Philosophy of Educa- presented at Criteria and Benchmarks for tion). Albany, NY: State University of New York Increased Learning from Undergraduate Press. STEM Instruction Workshop, Committee on Wright, J.C., Millar, S.B., Kosciuk, S.A., Undergraduate Science Education, National Penberthy, D.L., Williams, P.H., and Wampold, Research Council, Washington, DC. B.E. (1998). A novel strategy for assessing the Seymour, E., Hunter, A-B., Laursen, S., and effects of curriculum reform on student DeAntoni, T. (2002). Establishing the benefits competence. Journal of Chemical Education, of research experiences for undergraduates: 75(8), 986–992. First findings from a three-year study. Manu- Zemsky, R. (2002, November). On encouraging script submitted for publication. faculty to pursue instructional reform. Paper Shulman, L.S. (1993). Teaching as community presented at Criteria and Benchmarks for property: Putting an end to pedagogical Increased Learning from Undergraduate solitude. Change, 25(6), 6–7. STEM Instruction Workshop, Committee on Stevens, E. (1988). Tinkering with teaching. Undergraduate Science Education, National Review of Higher Education, 12, 63–78. Research Council, Washington, DC. APPENDIX A 117

On Encouraging Faculty to Pursue Instructional Reform Robert Zemsky Graduate School of Education, University of Pennsylvania When challenged to defend the nothing more is needed; however, for staying power of their institutions, the naysayers among us, the image university presidents often invoke Clark suggests something more than Kerr Kerr’s (1987) observation: intended. What many see as enduring resilience, others perceive to be the About 85 institutions in the Western world established by 1520 still exist in academy’s early resistance to alteration recognisable forms, with similar functions and later its resistance to change. and with unbroken histories, including the Catholic church, the Parliaments of the Isle of Man, of Iceland, and of Great DEFINING THE CHALLENGE Britain, several Swiss cantons, the Bank of Siena and 70 universities. Kings that Kerr’s observation also suggests the rule, feudal lords with vassals, and guilds with monopolies are all gone. These 70 near impossibility of the assignment I universities, however, are still in the same have accepted: to explore “some of the locations with some of the same build- options that university administrators— ings, with professors and students doing presidents, deans, department chairs— much the same things, and with gover- have at their disposal to encourage and nance carried on much the same ways support their faculty in instructional (p. 184). reform.” What Kerr and his appeal to the historic university make clear is that Kerr was testifying to the enduring change in the academy is slow, probably nature of the university—its ability to imperceptible, and not likely to be the survive when challenged, to adopt when result of the strategies of individual necessary. For defenders of the faith, presidents, deans, or department chairs. 118 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

For purposes of discussion, let me less of the size of their budgets, are suggest three propositions that lend a spent before they can make a single practical perspective to this traditional decision or investment. Beyond their tension between resilience and resis- immediate staffs, they, like the pope, tance. The first is simply that most command no troops. Even the very universities—and almost all research words that frame this session reflect the universities—are presided over by problems nearly every president, dean, faculty guilds. Membership is for life. and department chair face: they cannot Independence and autonomy are guar- enforce change but merely explore anteed, as long as the guild members “options…to encourage and support respect the privileges and honor the their faculty in instructional reform.” obligations that membership confers, At the University of Pennsylvania, including the obligation not to meddle Nichole Rowles is completing a disserta- too deeply in the practices of one an- tion (2003) that will update Cohen and other. We teach largely as we were March’s application of the garbage can taught. When we experiment with new model (Cohen, March, and Olsen, 1972) modes of instruction, we tend to do so to describe decision making in the quietly, not wanting to draw too much modern university. Rowles is document- attention to ourselves. We tend to work ing the extent to which presidents and alone, largely eschewing group projects. their staffs, in particular, are attempting As in most guilds, acceptable practice is to adapt corporate models of decision what everybody does—a kind of implicit making while their faculties cleave to regression to the mean—so that the older, more established norms changes in curricula and instructional representative of guilds and garbage format require broad agreement that cans. The most striking differences something, in fact, is broken and re- involve the roles of strategy and data. In quires fixing. the corporate model, a strategy is what My second proposition concerns the sport enthusiasts will recognize as a nature of the offices that presidents, game plan: an envisioning of the job at deans, and department chairs occupy. hand, an enumeration of the resources We know they are administrators; we available to achieve the desired goal, can hope they are—or eventually and a focusing on the tactics necessary become—leaders. What we cannot to make one’s strategy operational. In expect them to be, however, are manag- all three modes, data play a critical role ers. They seldom command significant in defining possibilities, calculating resources. Most of their funds, regard- risks, and measuring results. On the APPENDIX A 119

other hand, most faculty think in differ- principle than of strategy—a matter of ent terms—not of strategies, but of what is intrinsically right as broadly strategic plans that, for the most part, understood by those vested with respon- are lists of things other people should sibility for determining what is to be be doing. taught and how. It is a perspective that What is most striking, however, is the is too easily caricatured, as when mem- relative absence of calls for data from bers of the faculty are quoted as saying, the faculty’s perspective. They enforce “It’s not a matter of what students want no culture of evidence for institutional but what they need.” As faculty, we decision making, despite the fact that have spent our lives learning what most scholars spend their lives in students need; we are collectively pursuit of data and empirical observa- responsible for the knowledge base they tion. Instead, there are experiences and must master, as well as exemplars of the lessons learned—and, above all, prin- role free and unfettered inquiry needs ciples derived from firmly held beliefs. to play within every educational institu- In one institution in Rowles’ study, the tion. When a president or dean speaks faculty came to believe that athletes of the need to update the curriculum, were being given preferential treatment incorporate more technology in the and were being credited with higher classroom, or recruit different kinds of grade point averages (GPAs) than they teachers, the faculty not surprisingly deserved. Despite the presence of a ask: “Why? Who says what we must study conducted by that university’s do?” And if the president or dean says, office of institutional research, which “Because we need to pay attention to the documented that athletes’ GPAs were market in order to enroll the kinds of not being inflated by the suspect prac- students we want to teach,” the natural tice, the faculty overwhelmingly voted response is: “But markets do not know to outlaw the practice. When Rowles what we know.” asked the head of the faculty senate why Actually there is a better rejoinder they had ignored the study, he re- which faculty are not likely to deliver, sponded simply, “You have to under- largely because, as a matter of principle, stand, it was not a matter of data but of they seldom pay attention to the work- principle.” ings of the market for undergraduate Hence, the problem faced by presi- education. What those of us who study dents, deans, and department chairs. those markets know is that there is no Curricular reform, like all academic market for good teaching—and that is decisions, becomes more a matter of my third proposition. There is precious 120 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

little evidence that students choose might exploit in pursuit of instructional where they enroll based on how faculty reform. teach. Alverno College has learned that The first is medical education leading lesson all too well. Universally ac- to the M.D. Schools of medicine were claimed for its pioneering curriculum among the first to experiment with and and innovative ways of teaching, then broadly adopt self-paced and Alverno remains an institution that has computer-assisted instruction. They proven far more successful at attracting have adapted a host of strategies to cope academic visitors and foundation grants with an exploding knowledge base that than students. Not surprisingly, the can no longer be mastered, in the sense U.S. News and World Report rankings that basic anatomy can be mastered. hardly bother to talk about teaching or And they have welcomed—some would curricula, choosing instead to focus on say shamelessly embraced— resources and reputations. Most nonphysician and non-Ph.D. instructors. presidents and deans know that the Why has medical education been able building of their institutions—and not so to achieve what most reformers of incidentally the building of their per- undergraduate education have only sonal portfolios—depends fundamen- flirted with? There are several answers. tally on increasing revenue and building In the first place, medical educators reputations, neither of which rest on teach very smart, highly disciplined instructional reform. students for whom efficient learning is of enormous benefit. If self-paced, computer-assisted instruction promises MAKING THE CASE that one can learn more and faster, then earnest students will believe it is worth Having defined the challenge, let me a try. It is also the case that, in medical hasten to add that achieving instruc- schools, teaching loads do not deter- tional reform is not impossible, just very mine the size of the faculty group. In difficult. To understand what it might undergraduate education, learning take to overcome the inertia of the efficiency all too often means the need guild, on the one hand, and the disinter- for fewer faculty slots. And not to be est of the market in good teaching on overlooked is the fact that most medical the other, I want to focus on a few schools have had and continue to have examples of success. They suggest the ample resources with which to experi- necessary conditions that an innovative ment with new instructional technolo- president, dean, or department chair gies. Finally, there is a measurable APPENDIX A 121

premium attached to good or at least began telling them that their traditional successful teaching: better perfor- bread-and-butter business programs mance on board exams and better were in danger of precipitating out of placements for the class as a whole in the market. As one executive was the competition for residencies. Out- reported to have said, in the past we did come measures spur reform, particu- not so much care what you taught your larly when those both within and be- undergraduates and M.B.A. students. yond the academy sense the value What we expected from you was screen- and appropriateness of the measures ing and certification, and figured that themselves. what happened in the classroom could My second example derives from the do no harm. Now we are not so sure. growth of executive education programs Maybe what you are teaching really is at most of the nation’s leading business standing in the way of the kinds of schools and their subsequent impact on companies we are trying to build. The the general business curriculum at both result across this set of select business the graduate and undergraduate levels. schools was a rush to introduce educa- In the early 1990s, when most of these tional experimentation and reform—a programs were being launched, I asked development that eventually came to the dean of one business school to energize business faculty across a wide account for the popularity of this par- spectrum of schools. ticular form of education. Poised to My last three examples are drawn build a hotel for his own new executive from the world of undergraduate sci- education program, he gave an answer ence and math instruction. In the that has stuck with me ever since. 1980s, Bill Massy and I conducted a The trend began as a kind of copycat study of how departments make deci- phenomena, after Northwestern’s sions about who teaches what (Zemsky, Kellogg School and then Penn’s Massy, and Oedel, 1993). It was funda- Wharton School had launched their big, mentally an interview study, in which expensive initiatives. Soon, more and Bill and I spent upwards of an hour with more schools followed suit; as they every chair from a department that began to attract seasoned executives taught undergraduates at ten selective and managers to their “exec-ed” class- colleges and universities. What struck rooms, the deans and faculty of these us was the degree to which physics schools made a crucial discovery. departments seemed to be different; Enrolled executives and managers their chairs evidenced a passion for 122 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

teaching and a willingness to be judged improving mathematics instruction. His by the quality of both their curricula and answer, as I best remember, went their teaching efforts. something like this: “We are an endan- Several years later I came across Jack gered species, and we know it. We are Wilson’s experiments with Studio not educating enough young people to Physics (http://www.rpi.edu/dept/ sustain ourselves. We are in a down- phys/education.html) at Rensselaer ward spiral: fewer young people inter- Polytechnic Institute (RPI) and was ested in mathematics translates into less again reminded of the unique commit- demand for mathematics instruction, ment to teaching evidenced across this which then increases the probability discipline. What helped to make Studio that among the next student cohort Physics work at RPI was the presence of there will be even less interest in math- an established means of verifying the ematics—and so the cycle repeats itself. quality of this alternate form of instruc- To break the cycle we need to be in the tion. All of the roughly 900 freshmen business of actively seeking converts.” each year who take the basic introduc- My last example derives from the tory physics course sit for the same set experiences of undergraduate geology of examinations, regardless of the programs, particularly those offered at section to which they were assigned. liberal arts colleges, over the last three Studio Physics was able to win adher- decades. The oil and related energy ents because it could prove not only that crises of the 1970s resulted in a boom in it was more efficient in terms of the geology majors, which in turn resulted resources it consumed, but also that it in rapid increases in the sizes of geology produced as good or better results than departments. By the 1990s, however, teaching physics the old-fashioned way. the boom had gone bust, and the de- Collegiate mathematics instruction partments that had enjoyed rapid provides the same pair of lessons: that a expansion suddenly found themselves disciplinary commitment is required, teaching fewer students and warding off paired with a way to ensure the disci- aggressive deans who wanted to shift pline that alternate ways of teaching their faculty billets elsewhere. At the produce measurable improvement. In time, I was engaged in a major study of the 1980s, the mathematician I knew the coherence of the collegiate curricu- best was Mort Lowengrub, then dean of lum, which examined the transcripts of arts and sciences at Indiana University. graduates from more than 200 colleges I asked him one night over dinner what and universities. Overall, we found what accounted for his discipline’s interest in most observers expected: there was APPENDIX A 123

little coherence, little course sequenc- LESSONS ing, little sense of an ordered progres- sion through an established body of There are four basic lessons I would knowledge. The principal exceptions extract from these stories and observa- were the sciences, primarily physics, tions, as a means of promoting the kind chemistry, and engineering. of discussion we need to have: Using the computer printouts of the statistical models that produced these 1. The first is that the guild itself results, the research team developed an must feel threatened before it is ready elaborate parlor game in which we to change. No amount of talking or would look at the structure of courses trying to explain that instructional and prerequisites and then try to guess reform is “good for you” is likely to the department and kind of institution to substitute for the cumulative experience which the particular printout belonged. of witnessing the marginalization of Although the output had been stripped what you consider to be important. of all departmental identifiers, we 2. Curricular and instructional became very good at noticing the subtle change, when it comes, is more likely to differences among disciplines and extend from the top of the institutional between institutional type. But one hierarchy down rather than bubble up profile stumped us nearly every time: from the bottom. What makes change those of departments of geology at so unlikely is the fact that it must come liberal arts colleges, which for the most from those most advantaged by current part we mistook for departments of arrangements and practices. English. When we followed up the 3. Curricular and instructional statistical analysis with a set of inter- change is easier to promote when the views, I gingerly asked the first geology students to be taught differently are not chair I encountered if he was surprised only smart and disciplined but also have that the structure of his curriculum was a vested interest in the outcomes of the indistinguishable from that of the experiment. English department. He replied, “Not at 4. Curricular change is inherently all. Actually we face the same challenge expensive, since the old ways of teach- of convincing undergraduates that what ing will not be abandoned until the new we know and teach is intrinsically means have fully demonstrated their interesting—that it can be fun!” staying power. (Zemsky, 1989). 124 I M P R O V I N G U N D E R G R A D U AT E I N S T R U C T I O N

And, finally, some advice: * The teaching of science in primary and secondary schools. • To deans, in particular, don’t tilt at * Making corporate groups windmills—rhetoric is nice, but the scientifically literate. frustration of unfulfilled promises in the * Building a public policy under- end overwhelms. standing of science. • While it sounds good, the recruit- ment of star teachers is likely to have little impact. REFERENCES • Changing the tenure rules only serves as a long-term strategy when the Cohen, M.D., March, J.G., and Olsen, J.P. (1972). A garbage can model of organizational choice. goal is curricular and instructional Administrative Science Quarterly, 17(1). reform. Kerr, C. (1987). A critical age in the university world. European Journal of Education, 22(2), • Pick your targets, spend your 183–193. money. Rowles, N.S. (2003). Halfway in the can: An examination of data use in college and univer- • Invest in strong programs. sity decision making. Doctoral dissertation in • Experiment with breaking the preparation, School of Education, University of Pennsylvania. rules—particularly those governing the Zemsky, R. (1989). Structure and choice. Curricu- time and mode of delivery. lum Data Base Series. Washington, DC: • Look for external markets to Association of American Colleges and Univer- sities. develop and then harvest those which Zemsky, R., Massy, W.F., and Oedel, P. (1993). On provide visibility plus funds for experi- reversing the ratchet. Change, 25(3), 56–62. mentation. Three markets are readily available: APPENDIX A 125

Next: B Reference Paper »
Improving Undergraduate Instruction in Science, Technology, Engineering, and Mathematics: Report of a Workshop Get This Book
×
Buy Paperback | $54.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Participants in this workshop were asked to explore three related questions: (1) how to create measures of undergraduate learning in STEM courses; (2) how such measures might be organized into a framework of criteria and benchmarks to assess instruction; and (3) how such a framework might be used at the institutional level to assess STEM courses and curricula to promote ongoing improvements. The following issues were highlighted:

  • Effective science instruction identifies explicit, measurable learning objectives.
  • Effective teaching assists students in reconciling their incomplete or erroneous preconceptions with new knowledge.
  • Instruction that is limited to passive delivery of information requiring memorization of lecture and text contents is likely to be unsuccessful in eliciting desired learning outcomes.
  • Models of effective instruction that promote conceptual understanding in students and the ability of the learner to apply knowledge in new situations are available.
  • Institutions need better assessment tools for evaluating course design and effective instruction.
  • Deans and department chairs often fail to recognize measures they have at their disposal to enhance incentives for improving education.

Much is still to be learned from research into how to improve instruction in ways that enhance student learning.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!