2
Planning for Uncertainty

A FRAMEWORK OF GIVENS AND UNKNOWNS

Information technology (IT) is changing at a breathtaking pace, making it virtually impossible to accurately predict its future impact on teaching and learning in undergraduate science, mathematics, engineering, and technology (SME&T) education. As Philip Agre of the University of California, Los Angeles Department of Information Studies put it, “everyone knows that IT is going to change the world, but nobody knows how.” Agre went on to say that, although no one can predict the future, it is possible to prepare. After discussing many examples of current uses of IT in undergraduate SME&T education on the first day of the workshop, during the second day, participants looked toward the future. To facilitate this process, Alan Schwartz from PolicyFutures1 led workshop participants through the first phase of a multiple scenario-planning exercise, designed to help prepare them for future possibilities.

Schwartz launched the exercise with three examples of how different organizations ended up planning for the “wrong” future. His first example was the U.S. Army’s decision to build a large fleet of Abrams M1A1 tanks in the late 1970s. Designed to fight the Soviet army on the plains of Europe, the tanks are heavily shielded for protection against Soviet shells. They are so heavy that a military plane can carry only one at a time. This design, based on outdated assumptions about where battles would be fought and what the enemies’ capabilities would be, has proven ill-suited for more recent battles. During Desert

1  

A consulting firm that provides strategic planning, scenario planning, and policy forecasting services to public and private clients.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary 2 Planning for Uncertainty A FRAMEWORK OF GIVENS AND UNKNOWNS Information technology (IT) is changing at a breathtaking pace, making it virtually impossible to accurately predict its future impact on teaching and learning in undergraduate science, mathematics, engineering, and technology (SME&T) education. As Philip Agre of the University of California, Los Angeles Department of Information Studies put it, “everyone knows that IT is going to change the world, but nobody knows how.” Agre went on to say that, although no one can predict the future, it is possible to prepare. After discussing many examples of current uses of IT in undergraduate SME&T education on the first day of the workshop, during the second day, participants looked toward the future. To facilitate this process, Alan Schwartz from PolicyFutures1 led workshop participants through the first phase of a multiple scenario-planning exercise, designed to help prepare them for future possibilities. Schwartz launched the exercise with three examples of how different organizations ended up planning for the “wrong” future. His first example was the U.S. Army’s decision to build a large fleet of Abrams M1A1 tanks in the late 1970s. Designed to fight the Soviet army on the plains of Europe, the tanks are heavily shielded for protection against Soviet shells. They are so heavy that a military plane can carry only one at a time. This design, based on outdated assumptions about where battles would be fought and what the enemies’ capabilities would be, has proven ill-suited for more recent battles. During Desert 1   A consulting firm that provides strategic planning, scenario planning, and policy forecasting services to public and private clients.

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Storm in the early 1990s, when rapid deployment was important, it took fully 6 months to deploy the Abrams tanks. In addition, another aspect of the tanks’ design—night vision that was superior to that of the enemy—allowed the tanks to avoid enemy attack during Desert Storm, making the heavy armor superfluous. A second example of misdirected planning, based on the assumption that the future would resemble the past, was the creation of the Alaska Natural Gas Transportation Institute in 1977, to tap the natural gas that is found in the oil fields on Alaska’s North Slope. At that time, when experts were predicting oil prices would reach $200 per barrel, parts of the pipeline were built, but after oil prices dropped, it was never completed. (With oil prices now rising, it is possible that the pipeline will be built someday; nevertheless, it is clear that the plans made in 1977 were not appropriate to the future that developed over the following two decades.) Schwartz’s final example was the U.S. automobile industry in the 1960s, which believed that Americans would always love big cars, that gasoline would continue to be plentiful, and that reliability wasn’t an issue because Americans traded in their cars every two years. Because U.S. auto makers acted based on this prevailing wisdom, Japanese firms were able to capture a large share of the U.S. market with smaller, more reliable, and more fuel-efficient cars in the 1980s. To further illustrate his point that one cannot reliably plan for an uncertain future TABLE 2-1 Changing Benchmarks Benchmark 1990 2000 Number of Web Pages 0 Millions Dow Jones Industrial Average on January 3 2,810 11,358 Soviet Union Exists Does not exist   SOURCE: Schwartz, 2000. by simply extrapolating from current trends, Schwartz presented data on dramatic technological, economic, and political changes that have taken place over the past decade (see Table 2-1). Schwartz suggested that an important step in planning for an uncertain future is to make a list of relevant events and to categorize the events as certain or uncertain to continue into the future. (For example, the Abrams tank was designed because the army felt that the existence of the Soviet Union was a certainty.) Schwartz maintained that this process of categorizing events can allow one to identify signposts that might be monitored to indicate how the future is unfolding. He led the workshop participants in brainstorming to identify factors influencing development of technologies that: (1) might be useful in science and mathematics education, and (2) might influence implementation of those technologies over the next 10 years. Using the resulting list, participants then assembled in small groups to discuss which factors they felt were certain and which were uncertain.

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Next, the small groups assembled in a plenary session, compared their lists, and created a framework of certain and uncertain factors that are likely to influence the course of IT in SME&T education over the next decade (see Table 2-2). Had this session been a full multiple-scenario planning exercise, the participants would have been led to develop a framework for looking at the future based on the list of issues, and to identify signposts that might be used to monitor how the future unfolds. The framework and signposts could then be used by decision makers to build strategies for dealing with an uncertain future. Despite the lack of such a full framework, workshop participants suggested several strategies. Presenters and discussants suggested that the following steps could help to overcome the cultural and institutional constraints to change discussed in chapter 1. STEPS TOWARD DIFFUSION OF INNOVATIONS IN PEDAGOGY AND TECHNOLOGY Steps for Educators and Researchers Several workshop presenters suggested that the first, and most important, step that SME&T faculty might take would be to adopt a more “scientific” approach to improving SME&T education. For example, Shneiderman emphasized the importance of establishing the overriding goal of the course or program at the outset. Based on this goal, a faculty member could then formulate a hypothesis about what educational approaches and/or technologies are most likely to achieve this goal, and develop a protocol for evaluating progress toward the goal. Shneiderman suggested testing the hypothesis by experimenting with alternative educational delivery methods and learning processes. Experts in the wider community, as well as those attending the workshop, are calling for a more rigorous, scientific approach to development of new educational approaches (Anderson, Greeno, Reder, and Simon, 2000; National Research Council [NRC], 2001b). Educators and researchers who adopted this approach would continually evaluate and refine their approach to teaching and learning. They would base their evaluations on assessments of student learning gains and might also assess learning among faculty and instructors involved in innovative courses. This process would help educators and researchers to identify those educational approaches that have been shown to enhance student learning. Policy makers might focus educational funding on these approaches for maximum effectiveness (Lyon, 2001). At the workshop, Dede called for a new “culture of assessment and evaluation, where professors are willing to try new things, and view their own work critically.” This requires putting time and energy into multilayered evaluation of teaching and learning, including the following steps:

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary TABLE 2-2 Factors Influencing SME&T Education and IT over the Coming Decade Certainties Uncertainties Information flood Entertainment Retirement of faculty Demographics Need for learner outcomes Structure of universities will change Multiple technologies to reach millions Scientific developments outside the United States Price of educational commodities will drop Universities will lose monopolies Learning is social Student awareness of own skills Changed role of public funding Globalization Human pride Moore’s Law Lifelong learning Bandwidth cost Whether the IT tools needed to improve SME&T education will exist, and, if so, what they will cost Universal access Student preparation for undergraduate SME&T Economic disparities between rich and poor students and institutions Business shifts high technology abroad State funding of institutes of higher education Whether quality mentoring will be available to all students The roles of traditional higher education institutions and new, for-profit education providers What does certification look like? Foreign reaction to U.S. dominance Whether technology can foster deep communities Role of assessment Will there be believable evidence of impact on outcomes? Economy Tenure policy Who will bear the costs of learners? One possibility is development of new cost-sharing mechanisms, such as “educational maintenance organizations,” modeled on HMO’s Direction of change in new institutional arrangements Cultural attitudes towards innovations in SME&T education Professional values

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Gather multiple indicators of student learning. Although evidence of the benefits of IT-enabled innovations in SME&T education is growing, Dede said that “tough” assessments are needed to demonstrate that the new educational paradigm really enhances student learning. These multiple indicators may include data gathered by the instructors of courses as well as data obtained by outside evaluators. Assess student motivation by measuring the amount of time that students spend on projects, courses that they take over time, attendance, and retention. IT can help to do this, by providing logs of student interactions with a course or Web site. Include both quantitative and qualitative measures (which may require using more than one external evaluator). Focus on formative assessment2 of student learning and, if possible, aggregate to summative. Formative assessment will provide the information on how the project worked, which is essential for improvement and replication of the project. The National Institute for Science Education (NISE) provides resources for assessment through its Field-tested Learning Assessment Guide (FLAG) Web site (NISE, 2001). Match evaluation methods to the intended audience. For example, Martha Darling noted the need to demonstrate that the innovations “work” to end-users of the education system, including employers. Focus assessment on students’ intellectual products and learning processes, rather than on technology. As one part of this scientific approach to SME&T education, faculty might collaborate across disciplines to develop new courses as well as to evaluate and refine courses they have already created. Ongoing assessment, evaluation, and revision of courses would lead to new findings about effective approaches. Several workshop participants suggested that SME&T educators collaborate to create new interdisciplinary journals focusing on pedagogy and technology. The journals would provide a place where the results of this ongoing research could be published and widely shared. Steps for Higher Education Institutions Commenting on the innovative courses described by Bayard, one participant likened the situation in higher education to that of consumers and businesses faced with the possibility of purchasing innovative technology. He recommended that faculty and 2   Assessments are typically used for a variety of purposes. Nevertheless, many experts distinguish between formative assessments, used primarily to guide instruction and learning, and summative assessments, used primarily to determine whether a student has attained a certain level of competency after completing a particular phase of education (NRC, 2001a).

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary administrators read Crossing the Chasm (Moore and McKenna, 1999; see also Rogers, 1982), which describes why high technology products typically are never widely adopted by the general public. He said the book argues that individuals fall onto a normal curve, with “early adopters” (who are technically literate) on one end and “laggards” at the opposite end. The problem, he said, is that individuals in the middle of the bell curve are unlikely to adopt high technology without incentives and rewards. As a first step toward bridging the “chasm” between early adopters of technology and the majority of other SME&T faculty, workshop participants suggested that colleges and universities try to increase the majority’s knowledge about both IT and new approaches to education. Dede and other presenters identified several ways in which institutions might do this: Help faculty learn about the potential of collaborative, learner-centered educational approaches. Colleges and universities might consider involving groups of faculty (both in person and in virtual learning communities) in shared learning about the new approaches. Support cross-disciplinary collaboration to develop and implement innovative, IT-enabled approaches. Provide financial and other support for student and/or faculty projects that provide service to the university and/or to the community. However, as illustrated in Moore’s book, knowledge alone is unlikely to lead to widespread innovation, unless it is supported by recognition and concrete rewards for faculty who take advantage of professional development opportunities to restructure their teaching. Colleges and universities could publicly recognize excellent teaching that incorporates IT. Shneiderman noted that the University of Maryland recognizes innovators by inviting them to give public presentations on successful uses of IT. Outside organizations, too, might grant recognition. One workshop participant suggested that the U.S. Department of Education provide awards and recognition for faculty who are exemplary in their uses of IT to enhance student learning. Several presenters and participants called on higher education institutions to evaluate and reward SME&T faculty, based not only on their research, but also on the quality of their teaching. Some suggested basing tenure, salaries, sabbaticals, and other employment decisions on indicators of teaching quality, as well as research. These indicators might include publications in new journals focusing on pedagogy, peer review by master teachers, the number of publishable papers produced by students, and employers’ views of graduates. However, these suggestions may be difficult to implement, because many faculty and administrators question the validity, fairness, and reliability of currently available methods for assessing teaching quality.

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary Steps for Public and Private Research and Development At the workshop, Christopher Dede presented a videotape made by a private firm to market its concept for a future form of human-computer interface. He also showed another vendor’s videotape, in which people wore wireless computing devices and interacted with “smart objects” in an office where computers are embedded in the walls and furniture. Dede noted that these products, envisioned by a few innovators in private firms, have the potential to enhance human communication and learning. However, he said that there is a wide gap between these early prototypes and what private firms actually develop. Dede argued that, because most private research currently focuses on using IT for information exchange rather than for communication, public funding is essential in order to develop uses of IT that enhance learning. Workshop participants emphasized that the National Science Foundation (NSF) has played an essential role in supporting and financing the kinds of innovations in SME&T education that were discussed at the workshop. The NSF’s role has included analyzing, and disseminating information about, current innovations. For example, Bayard’s case studies of applying IT to undergraduate education were part of a yearlong NSF project on “Improving Learning through Technology.” As one outcome of that project, these case studies are now easily accessible via the project’s Web site (Millar et al., 2001). In 1999, NSF organized a workshop to share experiences and define future priorities for the use of IT in undergraduate SME&T education (Ellis et al., 1999), and NSF supported the National Academies’ workshop summarized here. Research and development funding from the NSF also was critical to developing the innovative undergraduate courses discussed at this workshop. To cite just one example, in 1992, University of Houston-Downtown (UHD) was selected to host summer workshops on the writing and use of interactive texts as part of a project funded by NSF and IBM. Through these workshops, mathematics faculty gained access to hardware and software that could be used to teach algebra, and also learned about new approaches to mathematics. At about the same time, the NSF-supported Harvard Calculus Consortium began publishing a series of textbooks (e.g., Hughes Hallett and Gleason, 1999), which gave legitimacy to the innovative approaches being developed by UHD mathematics faculty. Finally, in addition to this new knowledge and technology, NSF provided the critical grant that supported the three instructors’ release time while they developed the new curriculum (Millar et al., 2001). Despite such successes, several workshop participants wondered whether innovations in technology and pedagogy could spread more widely, across departments and institutions, without additional, and larger-scale, funding. Andries van Dam asserted that the NSF typically funds educational technology research in $100,000 grants over one or two years, and argued that these amounts were

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary insufficient.3 He noted that some workshop participants had mentioned costs of about $1 million to produce a single course “using today’s technology.” Van Dam called for larger amounts than this, over a sustained period, to support the long-term research needed to develop and implement tomorrow’s technology. Van Dam suggested that SEMATECH, a consortium of semiconductor manufacturers supported by both member companies and federal funds, might provide a useful model for long-term research and development. SEMATECH was created as a non-profit organization by 14 U.S. firms in 1987, and the following year, Congress appropriated $100 million annually for 5 years to the new group, matching the companies’ contributions. Originally organized to develop procedures for manufacturing next-generation computer chips, SEMATECH soon began to emphasize strengthening the smaller firms that supply the chip manufacturers with equipment. Acting as a communication channel, SEMATECH helped the chip makers define their needs and supplier companies develop the capabilities to meet those needs. These efforts succeeded in helping U.S. firms gain a larger share of the global computer chip market in the early 1990s, and the federal government continued funding SEMATECH beyond the original 5 years. However, since the mid-1990s, the organization has chosen to eliminate all federal support in order to open its membership to foreign firms and increase its flexibility (NRC, 1999c). Jack Wilson described a current initiative that provides smaller investments designed to help diffuse innovation based on currently available technology. At Rensselaer Polytechnic Institute (RPI), he and other faculty and administrators have begun to make institutional reforms, focusing on three key elements of undergraduate education: computing, communication, and cognition. As described by Wilson, RPI chose to follow a “scientific” approach, first establishing goals, then testing alternative ways to reach those goals “in a way that would allow you to continuously refresh and change.” RPI’s innovations—particularly replacing most large lectures, recitations, and laboratory sections with studio classrooms—have been so successful that RPI received an $8.8 million grant from the Pew Charitable Trusts. The Pew Learning and Technology Program, housed at RPI, is currently using these funds to provide financial and technical support to other institutions that wish to “undertake substantial systemic change.” Wilson emphasized that the new program does not simply provide funding, but also works actively with institutions to ensure that IT is used in ways that reduce costs and enhance learning (Bartscherer, 2001). Whether through new partnerships or through existing methods, public and private funding agencies could support indi- 3   The NSF has awarded much larger grants for educational technology in the past, ranging up to nearly $1 million over two years in one case.

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary vidual faculty and higher education institutions as they take a scientific approach to deploying IT in undergraduate SME&T education. In this approach, researchers start by establishing the goals of the educational program. They build assessment into the instructional program from the start, and revise curriculum and delivery methods on an ongoing basis, based on data about student learning and achievement of other program goals. (Examples can be found in NRC, 2001a.) Workshop participants generally agreed that the focus of research, development, and demonstration should be on student learning and quality of instruction, rather than on any particular technological tools. Nevertheless, some participants called for increased investments in research and development of particular technologies. First, Christopher Tucker noted that private firms are rapidly deploying broadband networks for business and residential customers, yet little is known about the educational potential of these networks. He said that the growing availability of broadband raises fundamental questions about how best to tap its potential to enhance learning. For example, in traditional approaches to education, an instructor (whether in person or online) initiates the process by presenting some information to the learners. Tucker raised the question of whether, in a broadband education system, the learner would receive a “story line” or lecture to begin the learning process. He noted that another possibility would be to create learning systems that would be entirely self-selected or self-guided, leaving open the question of which approach might be more effective at enhancing learning. Second, Tucker noted that new generations of intelligent browsers can allow faculty and students to transcend current digital databases which replicate academic disciplines. For example, the American Economic Association produces one such database, EconLit. With further research and development, these new browsers might support interdisciplinary collaboration to enhance faculty and student learning. Third, Dede noted that virtual reality technology is moving very quickly. His research has established that for certain types of SME&T subject matter, modeling and visualization in virtual reality can be very powerful in enhancing student learning (Dede, Salzman, Loftin, and Sprague, 1999). Dede stressed the need for funding to support further research that would examine how best to use this technology for educational purposes. CONTINUING THE DIALOGUE Many workshop presenters and participants expressed a desire for ongoing collaboration with others interested in developing innovative, effective approaches to undergraduate SME&T education. They noted the importance of continuing to learn about, and from, innovations in education and technology in both the public and private sectors. Some workshop participants

OCR for page 19
Enhancing Undergraduate Learning with Information Technology: A Workshop Summary expressed a desire to learn more about the ways in which particular educational tools (such as those listed in Appendix A) might support the innovative approaches to pedagogy discussed at the workshop. Their concerns suggested a need to create an ongoing forum for exchange of information. With support from the U.S. Department of Education, the NRC Center for Education initiated such a forum in late 2000, the Improving Learning with Information Technology (ILIT) Project. Although this project is focused on elementary and secondary education, its activities and findings are likely to guide innovations in pedagogy and technology in higher education as well. The ILIT project is designed to create an ongoing community of experts in technology, cognitive science, and education who are devoted to improving education through creative applications of information technology.4 These experts will work to develop a path toward improving teaching and learning in elementary and secondary schools through the use of IT. The overall project is designed to enable educational decision makers to make rational and strategic decisions about how they purchase and use education technology. Just as SEMATECH helped supply firms better understand the needs of the chip makers to whom they sold, the ILIT project seeks to empower the education community to drive the development of hardware and software tools that meet the needs of children and educators alike. A final project goal is to help guide the federal research agenda in the relevant areas. 4   Further details on the project, including contact information, are available at: http://www4.nationalacademies.org/cfe/cfe.nsf. Click on “Organizational Structure and Projects,” and on “Improving Learning with Information Technology.” [July 9, 2001].