1
Innovations in Pedagogy and Technology

I hear and I forget; I see and I remember; I do and I understand.

—ancient Chinese Proverb

THE ROLE OF TECHNOLOGY

Marshall Smith, chair of the workshop planning group (and an officer of the William and Flora Hewlett Foundation), opened the workshop by inviting participants “to look deeply at the issues of improving teaching and learning.” Smith’s emphasis on pedagogy was reflected in the workshop presentations and discussions. Although workshop participants believed that information technology (IT) has great potential to support improved science, mathematics, engineering, and technology (SME&T) education, most agreed that there was nothing inherent in new technologies, by themselves, that would determine improvement. The participants noted that when IT is used for administrative purposes (for example, an instructor posts the course syllabus on the Internet), it is unlikely to help students understand and master scientific and technical subjects. Presenters and discussants focused instead on using IT to enable innovations in pedagogy that can increase learning. Although such innovations are possible without technology, the capabilities of IT make them easier and more practical. The new approaches to teaching and learning discussed at the workshop reflect developments in SME&T education, cognitive science, and educational research.

Traditionally, SME&T courses at U.S. colleges and universities have been comprised of lectures and laboratory sessions. However, a growing body of research indicates that this traditional approach is not effective for all undergraduates. Cognitive scientists have found that students have different ways of learning and benefit from different educational approaches (National Research Council [NRC], 1999a). In 1984,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary 1 Innovations in Pedagogy and Technology I hear and I forget; I see and I remember; I do and I understand. —ancient Chinese Proverb THE ROLE OF TECHNOLOGY Marshall Smith, chair of the workshop planning group (and an officer of the William and Flora Hewlett Foundation), opened the workshop by inviting participants “to look deeply at the issues of improving teaching and learning.” Smith’s emphasis on pedagogy was reflected in the workshop presentations and discussions. Although workshop participants believed that information technology (IT) has great potential to support improved science, mathematics, engineering, and technology (SME&T) education, most agreed that there was nothing inherent in new technologies, by themselves, that would determine improvement. The participants noted that when IT is used for administrative purposes (for example, an instructor posts the course syllabus on the Internet), it is unlikely to help students understand and master scientific and technical subjects. Presenters and discussants focused instead on using IT to enable innovations in pedagogy that can increase learning. Although such innovations are possible without technology, the capabilities of IT make them easier and more practical. The new approaches to teaching and learning discussed at the workshop reflect developments in SME&T education, cognitive science, and educational research. Traditionally, SME&T courses at U.S. colleges and universities have been comprised of lectures and laboratory sessions. However, a growing body of research indicates that this traditional approach is not effective for all undergraduates. Cognitive scientists have found that students have different ways of learning and benefit from different educational approaches (National Research Council [NRC], 1999a). In 1984,

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Godleski argued that lectures best serve students who are intuitive rather than sensory learners, and a decade later, McDermott, Shaffer, and Somers (1994) found that standard physics lectures do not help most students grasp fundamental concepts. Researchers have also found that when SME&T instructors recognize that student learning strategies vary and modify instruction accordingly, more students are able to learn and master these complex disciplines (Felder, 1993, 1996; Tobias, 1992). Laboratory sessions, too, do not help all students develop a deep understanding of SME&T concepts. Students can perform the experiments carefully, achieve the predetermined outcome, answer questions, and complete the lab report, yet still leave with very little understanding of the concepts they were supposed to learn (Poole and Kidder, 1996). Even in some SME&T courses based on newer curricula, laboratory experiences may emphasize verifying established knowledge and may not correlate with material presented in subsequent lectures (Hilosky, Sutman, and Schmuckler, 1998). Data on course completion also indicate that many students cannot master SME&T subjects as they are currently taught (Seymour and Hewitt, 1997). Among U.S. students who declared science and engineering majors as freshmen in 1989/1990, fewer than half had completed such a degree 5 years later, and about 22 percent had dropped out altogether (National Science Foundation [NSF], 2000). Among non-Asian minority students who planned to major in SME&T disciplines as freshmen in 1989/1990, only 25 percent had completed a science or engineering degree after 5 years (NSF, 2000). In response to these problems, some SME&T educators are experimenting with innovative pedagogical methods. Often, these new approaches are based on research into human cognition, which has identified four elements that are key to enhancing learning (NRC, 1999b, pp. 19-22): Schools and classrooms must be learner centered. Attention must be given to what is taught (information, subject matter), why it is taught (understanding), and what competence or mastery looks like. Formative assessments—ongoing assessments designed to make students’ thinking visible to both teachers and students—are essential. Learning is influenced in fundamental ways by the context in which it takes place. A community-centered approach requires that students, teachers, and others share norms that value learning and high standards. Workshop participants identified several key elements of this new pedagogy. First, instructors who use technology to implement these new approaches to teaching and learning typically move from lecturing, as a “sage on the stage,” to becoming a “guide on the side.” Although the instructor still plays a critically important role, deciding how

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary and when to intervene to enable and support learning, the focus of the educational process shifts from the instructor towards the learner. Second, in the new paradigm, education can become less abstract, as both students and faculty can create projects or perform research that is useful to employers, community groups, or others outside the classroom. Third, competition to succeed as an individual student or professor at the expense of others is reduced, and collaboration becomes an increasingly powerful learning tool. Harvard University professor Christopher Dede emphasized the importance of this new approach to pedagogy in his presentation on the educational potential of multiple interactive media. Dede noted that, until recently, many educators viewed IT primarily as a way to increase student access and provide economies of scale for traditional modes of education.1 However, according to Dede, there is a “new rationale” for using IT in higher education, beyond simply increasing access. Now, more college and university faculty recognize that IT has the power to transform education by supporting shared creation, collaboration, and mastery of knowledge (Dede, 2000; see also Hanna et al., 2000). Like the growing numbers of scientists who collaborate simultaneously with local and distant colleagues in virtual “knowledge networks,” students can use IT to collaborate and to learn. Information Technology can allow teaching and learning to be transformed in these ways, even when students do not interact with each other face-to-face, according to Dede. He noted that some students who are silent and passive in face-to-face settings “find their voice” and become active participants in technology-mediated communication. Using both synchronous and asynchronous media is important in producing this effect across the full range of learners. However, Dede argued that effective design of learning environments will include careful attention to the role of faculty mentors, as well as to technology. Summarizing his belief that IT is powerful “only if the medium is used well,” Dede stated “in the pedagogy lies the power.” To illustrate the importance of using IT to support this new pedagogy, rather than to simply expand delivery of current educational approaches, Ben Shneiderman of the University of Maryland (UMD) suggested using new words. He proposed replacing “Information” in “Information Technology” with “Communication,” reflecting the power of IT to support collaborative learning. Several recent comparative studies support Shneiderman’s suggestion that IT be used to enhance communication and collaboration among SME&T students. One recent study (Johnson, Johnson, and Smith, 1998) found that cooperative learn- 1   This point was reinforced during the workshop by Professor Richard Larson of MIT. Larson described the growth of private, Web-based education providers, and of partnerships between colleges and universities and these private vendors. Larson said that students anywhere around the world can now access upper-level MIT mathematics course materials at any time of the day or night.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary ing promotes higher individual achievement than either competitive or individualistic approaches to learning. Springer, Stanne, and Donovan (1997) analyzed comparative studies of small-group and individual education, concluding that collaborative groups promoted “greater academic achievement, more favorable attitudes toward learning, and increased persistence through SME&T courses and programs.” Another meta-analysis (Johnson et al., 1998) reached similar conclusions. Studies of students in biology (Watson and Marshall, 1995), chemistry (Wright, 1996), earth science (Macdonald and Korinek, 1995), and physics (Hake, 1998) all indicated that students collaborating in small groups learned more than those working independently. Shneiderman also suggested replacing the “Technology” in IT with “Philosophy,” in keeping with his view that teachers need a guiding philosophy in order to meet the many goals of undergraduate education (Shneiderman, 1998b). Thus, rather than a technology to transmit information, IT becomes a “philosophy” or pedagogy to enhance human communication and learning. Taken together, the views of those attending the workshop suggest that IT may make it possible to transform undergraduate SME&T education along the lines shown in Table 1-1. This new pedagogy reflects current thinking (although not yet current practice) in SME&T education. It is similar to the new educational approaches in primary and secondary schools called for in national standards for science (NRC, 1996), mathematics (National Council of Teachers of Mathematics, 2000), and technology (International Technology Education Association, 2000) education. When discussing current technological tools for education, presenters and partici- TABLE 1-1 How IT Might Transform Undergraduate SME&T Education Traditional SME&T Paradigm New Pedagogical Paradigm Teacher centered Learner centered Instructor delivers information (information may be transmitted via IT) Students and faculty engage in active learning/problem-solving together Individualistic (students work on their own, faculty work as individuals within a single discipline) Collaborative (students work in groups; faculty collaborate with other faculty within and across disciplines) Abstract Practical Academia is an “ivory tower,” remote from the outside world Students create service projects, useful to businesses, the university, or the community Information Technology transmits information Information Technology enables communication and learning

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary pants expressed varying views about their permanence and usefulness. Some agreed with Andries van Dam’s assertion that today’s Web-based educational offerings are “strictly transitional,” while others argued that in some current applications, IT is indeed effectively enhancing SME&T education. However, these disagreements were quickly put aside when participants recalled their focus on innovations in pedagogy, rather than in technology. Rensselaer Polytechnic Institute professor Jack Wilson, who was a member of the earlier Committee on Information Technology, emphasized the importance of saying “loud and clear” that it is time “to focus on the learner and to build our education systems based on the best research.” Whatever their views on current educational technologies for undergraduate learning, participants agreed that such technologies were likely to be very different, and possibly much better in the future. They noted that, based on the predictions and previous application of Moore’s Law,2 IT promises to continue its rapid evolution. However, many felt that changes in colleges and universities, as well as increased research and development, would be required to capture the true potential of IT to reform pedagogy and enhance SME&T learning (see Chapter 2). CASE STUDIES OF INNOVATIVE COURSES Professors Jean-Pierre Bayard, California State University, Sacramento (CSUS), and Ben Shneiderman, UMD, described four innovative, technology-intensive SME&T courses. They indicated that in each of these courses, currently available educational technology supported new teaching methods and enhanced undergraduate learning. The presenters’ descriptions of these courses also shed light on factors that can motivate faculty and administrators to try new approaches to technology and education. Among the four courses, two came about in response to an ongoing problem of high dropouts and failure rates among students taking introductory SME&T courses. Faculty members developed the other two courses after observing that students who appeared successful in upper-level classes did not perform well afterwards, in the workplace and in subsequent classes. Both types of problems proved to be powerful motivators, encouraging individuals and groups of faculty to develop and deliver the innovative courses. At the workshop, Bayard explained that 2   According to the Webopedia (www.webopedia.com [10/2/01]): “The observation made in 1965 by Gordon Moore, co-founder of Intel, that the number of transistors per square inch on integrated circuits had doubled every year since the integrated circuit was invented. Moore predicted that this trend would continue for the foreseeable future. In subsequent years, the pace has slowed, but data density has doubled approximately every 18 months, and this is the current working definition of Moore’s Law. Most experts, including Moore himself, expect Moore’s Law to hold for at least another two decades.”

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary he had learned about innovations in pedagogy and technology while working as a fellow at the University of Wisconsin’s National Institute for Science Education (NISE). NSF provided funding to Bayard and other members of the NISE “College Level One Team” to study and recommend improvements in undergraduate SME&T education, with the goal of retaining more students in these disciplines. Bayard noted that the College Level One Team initially focused on introductory courses, “what we call ‘pressure points’ and courses that…turn off students from science, engineering, math and technology.” Bayard said that the team used several criteria to select exemplary applications of IT that appeared to help engage and retain students in SME&T education. They sought applications of currently available technology (that could easily be adopted by other colleges and universities) across a variety of different types of undergraduate institutions. They also hoped to demonstrate that IT could help individual faculty members, departments, or institutions solve local problems. Finally, Bayard noted that they selected only those courses and programs in which IT was “transformative,” helping to achieve at least one of seven principles for good practice in undergraduate education outlined by Chickering and Gamson (1987):3 Encouraging student-faculty contact Encouraging cooperation among students Encouraging active learning Giving prompt feedback Emphasizing time on task Communicating high expectations Respecting diverse talents and ways of learning Based on these criteria, the College Level One team identified nine courses, including not only introductory “pressure points,” but also upper-level classes. They conducted detailed case studies of these applications of IT to SME&T education. Three of these case studies are summarized here, and all nine can be viewed at the College Level One “Learning through Technology” Web site (Millar et al., 2001). Bayard presented two examples of innovative SME&T courses designed to increase retention and learning among first-year students. The first was an introductory college algebra course at the University of Houston-Downtown (UHD). This urban university has an open admission policy and a diverse student body. About one-third of the students are African American, one-third are Hispanic, and one-third are of other races. Many students have had negative experiences with mathematics in high school and lack basic skills. Although the university has offered a variety of remedial programs, seniors graduating in the early 1990s were overwhelmingly white. The lack of minority students successfully completing courses and moving on to graduate 3   A more recent study identified a similar list of factors affecting college success: student-student interaction, student-faculty interaction, and time on task (Astin et al., 1993).

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary suggests that greater efforts are needed to help all students succeed (Millar et al., 2001). The catalyst for change at UHD was the high failure rate from introductory college algebra, averaging about 70 percent across all sections and instructors. Among those students who did pass and go on to take calculus, instructors found that few understood the core concept of a function. As a result of these problems, university faculty did not like being assigned to teach the class. At that time, most mathematics faculty followed a standard curriculum, which outlined specific mathematical skills to be taught in the introductory linear algebra class. For example, students were expected to learn how to solve a three-by-three system of equations. Instructors presented students with detailed step-by-step approaches to solving such equations, as well as other types of problems included in the curriculum. Faculty developed and presented example problems, theorems, and proofs. A group of mathematics instructors decided to try a new approach, emphasizing real-world problems, teamwork, and technology. These instructors introduced graphing calculators, both to help students visualize the concept of a mathematical function and to eliminate some of the tedious calculations. Today, students in some sections of introductory college algebra use this new approach, working with graphing calculators in small groups, where they discuss and solve problems. Bayard presented data comparing the passing rate (a grade of C or higher) in these “reformed” sections of college algebra with the rate in traditional sections, and the relative performance of the two groups in their next course (see Table 1-2). Bayard emphasized that, although these quantitative data reveal only a modest increase in student success as a result of the TABLE 1-2 Outcomes of Reforms in College Algebra at University of Houston-Downtown Semester Sections/Enrollment Grade C or Better Grade C or Better in Next Course   Traditional Reformed Traditional Reformed Fall 1996 Traditional: 3/75 Reformed: 6/202 38% 46% 35% 36% Spring 1997 Traditional: 3/91 Reformed: 3/75 38% 39% 32% 26% Fall 1997 Traditional: 3/74 Reformed: 3/51 45% 48% 20% 29% Spring 1998 Traditional: 3/87 Reformed: 3/69 35% 51% 19% 28%   SOURCE: Bayard, 2000.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary changes enabled by IT, more qualitative indicators do indicate a large change. For example, he said that students “feel a huge difference” in terms of their interest in algebra, instructors are more enthusiastic about teaching, and attendance is up. Bill Waller, one of the instructors who developed and teaches the innovative course, puts it this way (Millar et al., 2001): I can only give you anecdotal evidence, but the ambiance in the classroom is totally different compared to before. The biggest thing that I’ve enjoyed about it is when you go into the classroom, you’re not dreading going. You’re not thinking, How many are going to show up today? How many are going to be paying attention? Are they going to ask any questions? Am I going to get people to ask any questions? It’s just a much different classroom atmosphere than before (University of Houston-Downtown, “Evidence of Success,” p. 2). Bayard’s second example of innovative introductory SME&T education for first-year students was a course called “IMPULSE” (Integrated Math, Physics Undergraduate Laboratory Science, and Engineering) at the University of Massachusetts, Dartmouth (UMASSD). Because this university was formerly a technical institute, the mathematics and physics departments were happy to work with engineering faculty to create an integrated approach to teaching these subjects. As at UHD, the catalyst for change was high failure rates, but in this case, the student population was quite different. Engineering students who were accepted on the basis of their high scores on the mathematics portion of the SAT frequently failed courses or dropped out altogether during their first year. According to Bayard, first-year students saw their required mathematics, physics, and chemistry courses as irrelevant—an obstacle they had to get through in order to reach the engineering courses they were interested in. Among those who succeeded, some found their second-year engineering courses were not what they had expected. To overcome this problem, engage students, and retain them, an interdisciplinary team of faculty developed the IMPULSE curriculum. Students enrolled in IMPULSE attend studio classrooms, where they spend most of their time working collaboratively in teams of three to four on a variety of laboratory projects. For example, each team may be assigned to conduct and videotape a physics experiment demonstrating the motion of a projectile across the room. Using technology, students are able to capture data (such as data on the position of the projectile at various times) and record it in personal computers. Then, each team uses a computer to plot and analyze the data, answering questions about physical concepts such as velocity and acceleration. These projects help students learn across disciplines. Conducting physics experiments helps them understand concepts in this field. At the same time, to successfully analyze the data from the physics experiments, students must apply concepts of basic calculus, such as derivatives. And, because students often must troubleshoot the equipment, they develop engineering skills, including the ability to localize problems and develop alternative solutions.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary To support the understanding of these concepts developed in the studio classrooms, IMPULSE also includes mini-lectures in physics and calculus. The overall approach is borrowed from the successful “Workshop Physics” program at Dickinson College (Laws, 1997; Laws et al., 2001) and “Studio Physics” at Rensselaer Polytechnic Institute (Large, 2001). Collaboration is central to the IMPULSE program. Engineering students work together in project teams, live together in assigned areas of the dormitory, and take many classes together. According to Bayard, students objected strongly to being forced to live with one another, and also said they found the work very hard, but nevertheless felt that the group approach “helps them a lot.” The IMPULSE program appears to have met its goal of reducing attrition. During the fall 1999 semester, students enrolled in IMPULSE earned an average of nearly 16 credit hours, or 5 credits more than the average among first-year engineering students enrolled in traditional mathematics, physics, and science classes in the fall of 1998. In addition, higher percentages of IMPULSE students successfully completed calculus and physics classes on schedule than had students in previous years, before the integrated approach was adopted (see Figure 1-1). Overall, attrition among first-year engineering students has dropped from 40 percent to 17.3 percent since the implementation of the new program (Millar et al., 2001). Bayard’s third example illuminates how FIGURE 1-1 Percentage of first-year engineering students passing courses on schedule at UMASSD. SOURCE: Bayard, 2000. IT can help upper-level faculty increase student learning. This example was an upper-level, elective geology course at San Diego State University (SDSU). Bayard said that Eric Frost, an associate professor at this comprehensive research and teaching university, became increasingly unhappy with his lecture-based approach in the early 1990s. Frost observed that students who performed well in the class did not really understand the underlying concepts, based on their subsequent performance in other geology classes and in the workplace. This problem led Frost to dramatically change his approach. To match this course more closely to job requirements and obtain IT resources, he worked with an oil company and the San Diego Supercomputer Center of the University of California at San Diego.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Today, instead of lecturing, Frost teaches students to find oil and gas by learning how to identify fault lines. He assigns groups of three to four students to semester-long projects in which they use data from the supercomputer to create maps. The governments of oil-rich nations sponsor some of these projects. After creating the maps, students search for fault lines, and they present their results to sponsoring partners and at scientific conferences. Frost acts as a “cheerleader and guide,” spending many hours with individuals and teams as needed. Although Bayard noted that there has been no formal evaluation of this new approach, he said that the oil company now provides scholarships to Frost’s students, hires them following their graduation, and moves them up the corporate ladder. Bayard joined Frost’s students for one of their regular weekly lunches to discuss their projects, and saw that they were enthusiastic and shared a spirit of community. Following Bayard’s presentation of these case studies, UMD computer science professor Ben Shneiderman described his collaborative, technology-based approach to teaching upper-level undergraduate and graduate classes. Shneiderman explained that he was motivated to take this new approach during a visit to a large corporation that hires many graduates of the UMD computer science program. He reported that managers at this corporation said, “your students are great, but they don’t know how to work in teams.” This problem led Shneiderman to develop a more collaborative approach to computer science education over the following two decades. He explained that this collaborative approach resulted from years of experimenting with various elements of technology and pedagogy, observing which elements were most effective, and incorporating those elements in subsequent courses. As a result of this scientific approach, Shneiderman said, his current teaching is guided by a philosophy that includes three components: Relate-create-donate (Shneiderman, 1998b). Students relate to each other in collaborative groups. These groups create ambitious projects and then donate them to people outside the classroom (Shneiderman, 2000). Shneiderman said that students in his current upper-level course, Human Factors in Computer and Information Systems, have donated a variety of projects. Some students in this class have helped nursing home directors by analyzing and reporting on different strategies for teaching elderly residents how to use computers and the Internet. Another team set up a database allowing a regional charity to track donors and volunteers, and another helped a local high school develop a plan for computer usage. As a way to encourage students in this course to relate to each other, Shneiderman has introduced “open” projects. He requires each student team to post its term-length project on the Internet and has found that this process encourages students to polish the projects and to learn more about each other’s work. Near the end of the semester, each team posts its project report, and students outside the project team are assigned

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary to review these draft reports. Students send both positive and negative comments, with a copy to Shneiderman. Each project team has three days to revise its posted report, based on comments received (Shneiderman, 1998a). The final projects are posted on the class Web site each year and are maintained on the Internet as a resource for future students (Students of Dr. Ben Shneiderman, 2000). Acknowledging that “I was never taught this stuff,” Shneiderman shared what he has learned about collaborative approaches to education. He allows students to place themselves on teams, based on short biographies that they prepare at the beginning of each class. Students sign contracts agreeing to accept responsibility as team members, and each team is supported both by specially trained teaching assistants and by Shneiderman. Shneiderman argued that the open, collaborative approach is more successful in helping students master computer science than his past methods. He described two indications of this increased success. First, he noted that, three or four of the papers posted on the Web by student teams enrolled in his Human Factors class in 1998 included publishable results. One of the projects was accepted at an international conference, and a student presented it in Italy. By comparison, student teams enrolled in earlier Human Factors classes, who were not required to openly post and critique each others’ projects, produced only one or two projects that included some publishable results (Shneiderman, 1998a). Shneiderman said that he continues to find that three to four in every ten undergraduate student projects can be published or presented in peer-reviewed venues. Second, Shneiderman noted that one in ten student projects developed currently results in an operational computer system that is either used directly or serves as the basis for immediate development of software and hardware. Bayard and Shneiderman provided data indicating that in these four selected undergraduate SME&T courses, technology has supported new teaching approaches and yielded positive impacts. The anecdotal and quantitative data they presented indicates that these transformed courses have had positive effects on students, faculty, employers, and the community. A summary of the positive impacts described by these two presenters is found in Table 1-3. EVALUATION AND ASSESSMENT CHALLENGES Workshop participants agreed that it is very difficult to evaluate the full impact of IT-enabled education reforms on students, faculty, higher education institutions, employers, and communities. Their views were similar to those participating in an earlier workshop on the uses of IT in undergraduate education, who observed that “assessment and evaluation of technology-enriched courses are in their early stages of

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary TABLE 1-3 Indicators of Positive Impacts in Case Study Courses Program/Course Institution Indicators of Positive Impacts Geological Sciences 505 San Diego State University Increased student enthusiasm Students routinely present at conferences Students hired and promoted Graduating students offered higher salaries College Algebra University of Houston Improved passing rates Increased student motivation to study math Increased attendance Increased faculty motivation, enjoyment IMPULSE (Integrated mathematics, physics, engineering) University of Massachusetts Dartmouth Increased retention of first-year students Improved grades in first semester Students perceive they are working harder Students report they are learning a lot Human-Computer Interaction University of Maryland Increased student motivation More student papers accepted at conferences and by journals Provide IT services to university and community Increased teacher motivation   SOURCE: Bayard, 2000, June; Shneiderman, 2000, June. development” (Ellis, Seiter, and Yulke, 1999, p. 6). Workshop presenters pointed to several reasons for this difficulty. First, evaluations of educational programs are usually based on analysis and interpretation of data gathered from assessments of student learning. Yet, most current assessment methods have been designed based on a more traditional view of education and evaluation. Often, undergraduate SME&T students are assessed using methods that reflect only the ability to remember facts over a short time period. In a recent review of such methods, an NRC committee (NRC, 2001a) concluded that: “most widely used assessments of academic achievement are based on highly restrictive beliefs about learning and competence not fully in keeping with current knowledge about human cognition and learning” (p. 2). As a result, current assessment methods may be ill-suited to measuring student learning gains when faculty innovate with pedagogy and technology. For example, assessments that have been designed to measure an individual student’s achievement may be difficult to apply when learning is enhanced through small group collaboration. And,

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary although a few researchers are using computers to link ongoing assessment with instruction, many current assessments rely on paper-and-pencil methods, making them poorly aligned with learning enabled by IT (NRC, 2001a). A key question in evaluating the impact of IT in education is whether technology-intensive approaches provide any advantage in student learning, when compared with more traditional approaches. In theory, computers are capable of providing individualized, interactive instruction, like that provided by a human tutor. Because one-on-one tutoring has been shown to increase student achievement dramatically, when compared with one instructor teaching 20 to 30 students, some experts argue that IT can yield similar gains in learning (Fletcher, 2001). The reality, however, is that most current applications of IT in U.S. classrooms do not take advantage of these capabilities. Researchers have conducted hundreds of studies that compare test scores and satisfaction of students attending classes where there is a live instructor with those of students receiving the same material via video, audio, or other technology-based delivery mechanisms. Most of these comparative studies find no significant difference in student learning with the alternative modes of education (Russell, 2001). Two recent reviews of such comparative studies (Wisher et al., 1999; Phipps, 1999) found that most have been poorly designed. Often, investigators compared the scores of students who volunteered for technology-rich classes (and hence may be highly motivated to succeed with this delivery system) with students enrolled in traditional classes. Few studies used an experimental research design, with students randomly assigned to one of the two modes of education. Another problem is that many education and training courses involve a mix of on-line and face-to-face interaction, making it difficult to separate the effects of the two modes of education. As a result of such problems, the two reviews reached similar conclusions—that most comparative studies to date yield little concrete information about the effectiveness of IT-enabled education. Based on the assumption that education takes place when an expert teacher instructs students, most evaluations of the effectiveness of technology-rich courses have focused on delivery methods, rather than on content and learning. Ben Shneiderman, noting that his computer science students like to conduct studies focusing on delivery methods, said, “I sometimes joke that it is like we have a running experiment to see whether the strawberries delivered by trucks or strawberries delivered by cars are fresher or better in some way.” The second, more fundamental difficulty in evaluating innovative applications of IT to transform SME&T education relates to goals. Workshop participants noted that until there is a greater consensus about the goals of higher SME&T education, it may be difficult to conduct evaluations that would be widely accepted. The first step in evaluating any program is to define the program’s goals. However, administrators, faculty, students, and par-

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary ents have varying short-term and long-term goals. For example, some SME&T faculty may feel that, currently, the most important goal is to retain students within their chosen discipline. However, others may believe that the most important goal of undergraduate education is to create a group—possibly quite small—of highly qualified graduates, with complete mastery of specific content areas. This goal is reinforced by the strict prerequisite structures in many SME&T undergraduate programs. At the workshop, Jean-Pierre Bayard highlighted the tension between this goal and the goal of retaining more students. He contrasted the success of the IMPULSE program in retaining first-year engineering students with his perception that many engineering schools view high dropout rates among first-year students as a mark of success in eliminating people who would not become successful engineers. In addition to faculty and administrators, others involved in undergraduate SME&T education have varying goals. Some students and their parents may care little about formal evaluations, but place high value on the future career success that might result from innovative educational programs. Employers may be most concerned about the ability of colleges and universities to prepare students to perform in the workplace, caring less about student grades or program evaluations. Given the problem of varying goals, Christopher Dede suggested applying a variety of evaluation methods to influence different audiences to adopt, develop, or demand innovative, technology-rich education. Many SME&T faculty, as well as outside funding agencies that support development of educational technology, will require rigorous quantitative evaluations. On the other hand, informal evaluations, including personal testimonials from faculty and examples of student work, might help influence some faculty to consider using new, technology-rich approaches. Bayard and others involved in the Learning through Technology project have gathered these types of informal evaluations, including “hallway conversations” with faculty who are using IT, and have made them available on the Internet (Millar et al., 2001). Workshop participants acknowledged that IT not only supports new educational approaches that may be difficult to evaluate, but also provides new opportunities for assessment and evaluation (NRC, 2001a). When students interact with educational software, that software can be designed to collect detailed records of student activities that provide information about student learning. Medical professor Ron Stevens (University of California at Los Angeles [UCLA]) presented one such approach. Stevens originally developed the IMMEX (Interactive Multimedia Exercises) program for medical students, and it has been modified for use in Los Angeles-area elementary and secondary schools. Although IMMEX is not currently used in undergraduate SME&T education, the program illustrates the potential benefits of embedding ongoing assessment into instruction.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Stevens was originally motivated to develop IMMEX because he saw that multiple-choice examinations failed to fully assess the skills required of future medical doctors. Today, IMMEX is being used in middle schools and high schools to help students learn to solve problems and measure changes in students’ problem-solving abilities. Extensive professional development is provided to teachers, who play a central role in helping students use the interactive software. Meeting in teams under the guidance of UCLA researchers, teachers create conceptual problems for students. Each problem has 50 to 60 variants, allowing students multiple opportunities to practice solving problems that use a variety of types of resources and data. The software tracks the steps the students take in solving the problems and uses pattern recognition to categorize each student’s strategy. Teachers can use the tracks of individual student performance to assess progress and to compare a student’s individual progress with that of his or her peers. These tracks can also help teachers to refine instruction, feedback, and coaching. The information can also be provided directly to students, allowing them to think about how to improve their own performance. Stevens noted that a full evaluation of IMMEX, or any other program, should be multidimensional, including assessment of student learning, curriculum, and teaching practice. Chapter 2 outlines possibilities for such multifaceted approaches to evaluation. CULTURAL AND INSTITUTIONAL CONSTRAINTS Over the past decade, some policy makers, educators and members of the public, have questioned the quality of U.S. higher education. In response, individual colleges and universities, as well as educational associations have launched efforts to improve undergraduate teaching and learning (American Association of Universities, 2001; Project Kaleidoscope, 1991, 1998). Experts have called on colleges and universities to tap the power of IT as they undertake such efforts (Boyer Commission, 1998; NRC, 1999d). However, despite these reports and activities, workshop participants observed that transforming undergraduate SME&T education with IT is a slow and difficult process. This is because faculty, administrators, students, parents, and employers are required to adopt new goals and assume unfamiliar roles (see Table 1-1). Workshop presenters observed that widespread diffusion of innovative undergraduate courses, such as those they described, might be constrained by the current culture and institutions of undergraduate SME&T education. For example, Shneiderman noted that he is the only one among 38 professors in his computer science department at the UMD who uses collaborative, learner-centered approaches. Bayard said that other faculty at SDSU see Eric

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Frost as unusual, and not necessarily as a model to be followed. Commenting on Frost’s new, project-based approach to learning, one participant noted: “I think we have all heard of these kinds of gems…. Usually they get sealed off by the rest of the institution, because they say, ‘that is [effective] because of that particular person.’ ” Three types of constraints—institutional expectations, cultural factors, and technological constraints—could potentially prevent innovative, IT-enabled approaches from being implemented on a larger scale. First, many colleges and universities reward and recognize faculty based on more “traditional” conceptions of research and teaching. Workshop participants, most of whom were mid-career or senior-level faculty, noted that they enjoyed secure, tenured positions, allowing them the freedom to use IT in new ways. However, given the competitive reality of academic science, mathematics, and engineering today, younger faculty members are motivated by, and preoccupied with, the goal of obtaining tenure. To achieve this goal, younger faculty must devote so much time to research and publishing in peer-reviewed journals that they may be unable to devote attention, time, and energy to fundamentally restructuring their teaching. Christopher Dede asserted that some elite universities view publishing papers in refereed journals as the primary measure of success for younger faculty. If these views are correct, then current institutional reward systems, emphasizing publications,4 may well constrain young faculty who not only want to improve their teaching, but also want to do so by using IT to transform their educational approach. In addition, as illustrated by the IMPULSE example, these new approaches to teaching often involve collaboration among faculty as well as students, including collaboration across academic disciplines. Christopher Tucker noted that it is extremely difficult to try to place articles in peer-reviewed journals, and be awarded tenure, based on truly interdisciplinary collaboration. Time availability may constrain faculty members from adopting new approaches to pedagogy and technology. SME&T faculty who use IT to create active learning environments typically find themselves spending more time with students. In describing Eric Frost’s activities at SDSU, Bayard noted that Frost works with these students “a lot more than just the three hours a week that some of us are familiar with.” Elias Deeba, one of those who developed and now teaches the algebra course at UHD, has said, “Using technology, my own workload, of course, has increased tremendously” (Millar et al., 2001). Some SME&T educators may be unwilling to move toward these IT-enabled approaches if the increased workload leaves less time for research or other professional or leisure activities. 4   As discussed further below, some administrators may be reluctant to base tenure or other rewards on teaching quality because of a lack of good measures of teaching quality.

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Institutional expectations about faculty time could potentially constrain development of technology-rich SME&T courses, as well as their delivery. At UHD, faculty members are not expected to spend extensive time doing research, because the university sees its main mission as teaching. Reflecting this focus, faculty members are required to teach four courses per semester. This large teaching load adds to the office hours during which faculty are expected to be available for consultations with students. These teaching demands make it difficult to find time for faculty collaboration and design of new curricula. Without the grant from UHD’s Teaching with Technology Learning Center, which offered release time, the professors who incorporated IT into the algebra course would not have been able to develop their new approach (Millar et al., 2001). Technological constraints could also slow the use of IT to transform traditional undergraduate SME&T courses into more active learning environments. At the most basic level, not all colleges and universities have access to current technology. As illustrated in the UHD example, outside funding is often essential to provide the hardware and software that students and faculty need. These inequities in access to technology at the undergraduate level are somewhat similar to the broader problem in homes and elementary and secondary schools. Educational consultant Martha Darling expressed her concern about the possibility of a growing “digital divide,” between the rich, who have ready access to advanced technology at home and school, and the poor, who often lack such access. Funding to purchase and install hardware and software, by itself, will not overcome this problem. Student learning can be enabled by IT systems only when those systems work. For example, at the time Jean-Pierre Bayard visited the IMPULSE studio classrooms at UMASSD, technological problems prevented students from videotaping the physics experiments. This problem illustrates the importance of having adequate technical support staff when implementing IT-enabled approaches. The teams of computer science students enrolled in Shneiderman’s classes at the UMD are supported not only by specially trained teaching assistants, but also by dedicated technical support staff. Many higher education institutions may be unable to provide this level of support. In addition, some SME&T faculty outside computer science departments may lack the expertise needed to integrate IT into design and delivery of new forms of education. Cultural factors influence the willingness of SME&T students and faculty to embrace new technology and new forms of pedagogy. On the positive side, undergraduates who have grown up with video games and home Internet access may welcome technology-intensive approaches to education. University of Arizona mathematics professor Deborah Hughes Hallett pointed out that “today’s students are powerful and fluent and agile in technology in ways that their elders are often not. This...is a powerful boost to education.” On the other hand,

OCR for page 1
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary those students who have not had access to advanced tools may be at a decided disadvantage in a technology-rich classroom. The changing approaches to education outlined in Table 1-1 engage students in active learning, often in collaborative groups. Although one might think that students would be eager to respond to these “learner-centered” forms of education, not all students are. Shneiderman noted that every semester, he encounters one or two students who are reluctant to participate in group projects. At UHD, mathematics instructors have found that some students, who either dropped or failed the more traditional sections of introductory algebra, complain that “I’ve had this course before, and this is not the way it’s supposed to be done” (Millar et al., 2001). Although workshop participants noted that all of these cultural and institutional constraints currently slow widespread transformation of undergraduate education, these constraints could possibly be reversed in the future. For example, high-quality, interactive software that supports individualized instruction and learning of SME&T disciplines might be widely available. In this case, a single professor could support learning among a large, lecture-sized group of students, without spending hours with each student. In such a future, faculty and administrators would likely welcome the use of IT and the time savings that would result. The next chapter summarizes some participants’ views about the future of undergraduate education, including the role of IT. The chapter also describes participants’ suggestions about ways to use IT more effectively and capture its full potential to enhance learning among SME&T students.