Enhancing Undergraduate Learning with Information Technology: A Workshop Summary (2002)

The Committee on Information Technology in Undergraduate Science Education was organized in December, 1995 and completed its work in November of 1999. The committee’s charge was to improve undergraduate science education through the use of information technology. Nicholas Turro of Columbia University chaired the committee. Committee members included: Stephen Ehrmann, American Association for Higher Education; Bernard Gifford, Academic Systems Corporation; Steven Gilbert, American Association for Higher Education; B. James Hood, University of Central Arkansas; Deborah Hughes Hallett, University of Arizona; John Jungck, Beloit College; Stephen Lerman, Massachusetts Institute of Technology; Ronald Stevens, University of California at Los Angeles; and Jack Wilson, Rensselaer Polytechnic Institute. Committee members who served limited terms included Linda Chaput, Interactive Sciences, (December, 1995— October, 1996); Stephen Hurst, University of Illinois at Urbana-Champaign (February, 1996—November, 1999); C. Bradley Moore, Ohio State University (December, 1995—April, 1997); Dorothy Stout (February, 1996-November, 1999); and James Whitesell, University of Texas at Austin (May, 1997— November, 1999).

Nancy Devino served as Staff Officer to the committee from December, 1995 through February, 1999. Jay Labov served as Staff Officer from March, 1999 until the committee completed its work in November, 1999. Other staff included: Gail Pritchard, Research Associate; Terry Holmer, Project Assistant; Stacy Lucas, Consultant; Steven Olson, Editor; and James Lawson, Editor.

Information technology (IT) has dramatically transformed scientific practice in the last 50 years. The ability to handle massive amounts of data, collect data continuously in extraordinarily fine detail, perform millions of computations per second, visualize information in three and four dimensions with rotations and translations, and simulate immensely complex phenomena with multivariate, multidimensional models has become commonplace. Computers are ubiquitous in scientific workplaces, whether in offices, laboratories, or field settings. Electronic preparation of manuscripts has allowed professionals greater control over layout, graphics, and symbols, such that many now use publishers as distributors rather than as editors, layout specialists, or typesetters. The World Wide Web, faxes, and e-mail allow scientists, mathematicians, and engineers to collaborate on a daily basis over long distances and across many time zones. New technologies allow scientists to control and collect real-time data from distant locations with expensive, highly specialized research instruments.

This information technology revolution also has changed the undergraduate learning environment. Students are as likely to “surf the Web” from their dormitory rooms or teaching laboratories as they are to visit a library. They know that a portable computer or handheld calculator can perform almost all of the calculations expected in exams or problem sets. Having used computers extensively in high school, some students now arrive at college already knowing how to enter and analyze scientific data in spreadsheets and statistical graphing packages. Many more students who enter college are comfortable using these tools for a variety of other purposes.

Reflecting these changes, many faculty members have begun to post lectures and reading lists on the Internet, create on-line discussion groups, and to use IT in other ways to deliver courses. However, to be truly effective, information technology needs to be embedded in instruction, not just provided as an additional activity to a standard course or program. For example, in several science, mathematics, engineering, and technology (SME&T) disciplines, professors and instructors are using more than one technological innovation to help students learn. Earth scientists employ geographic information systems (GIS), global positioning systems (GPS), and remote sensing. When teaching calculus classes, some faculty use information technology to emphasize multiple representations of mathematical ideas (symbolic, verbal, numerical, and graphical) and realistic problem solving. Graphing calculators and spreadsheets have made numerical and graphical representa-

tions easily accessible to undergraduate and precollege students. This means that most students—though not yet all faculty—expect technology to be part of a mathematics classroom (see Box A-1).

However, the challenge to rethink the mathematics curriculum has just begun. The emergence of calculators that can do symbolic manipulation (computer algebra systems, or CAS) will push faculty and administrators to consider which topics in algebra and calculus they need to teach with pencil and paper—even though machines can do them—because pencil and paper develop understanding. In addition, faculty need to learn how to use computer algebra systems to promote understanding. Can they be used to illuminate topics that students have in the past learned by rote, without much understanding? Mathematicians, scientists, engineers, teachers, and educational researchers will need to reach a consensus on such questions.

Both the power and widespread availability of these new technological capabilities, and recent scientific findings about how people learn are forcing science, mathematics, and engineering faculty members to rethink what and how they teach. Many have begun to use new pedagogical practices. The committee found that IT can play a particularly effective role in supporting six such practices:

• visualization

• simulation

• real-world problem solving

• collaboration

• inquiry

• design.

The following section describes current technological tools that enable and support each of these pedagogical practices.

Computers are powerful tools to help

learners visualize, simulate, and construct objects in the created or natural worlds. Improvements in technology and significant decreases in the price of high-end computers mean that molecular modeling workstations now can be bought for as little as $2,000 compared to ten or more times that much only several years ago. Molecular modeling offers science educators a vehicle for helping students understand difficult abstract concepts. For example, students often have difficulty understanding the relationship between the macroscopic appearance of a substance, its symbolic representation as a chemical formula, and its molecular shape and structure as predicted by quantum mechanics (Gabel, 1990). Talley (1973) documented student difficulties in understanding the particulate nature of matter and proposed three-dimensional visualization to overcome the problem. Technology offers a way to move beyond static two-dimensional representations of three-dimensional phenomena into a format that more closely simulates reality. Current molecular modeling software can alternate images of a given object at rates approaching thousands of times per second, creating a realistic three-dimensional effect. The rapid increase in computer processing speeds means that software developers now are producing pedagogical software that was mere fantasy not long ago. While the full impact and influence of this technology on science education are not yet known, preliminary results are encouraging. Williamson and Abraham (1995), for example, developed computer animations using molecular modeling images. They found that students who used these tools developed better conceptual understanding of phenomena such as diffusion of intermolecular forces than did students who received conventional instruction.

Molecular modeling also can improve the linkage between lecture and laboratory. Crouch, Holden, and Samet (1996) described an innovative instructional sequence in which students learn about nucleophiles and electrophiles in a lecture, construct electron density maps of assigned compounds using molecular modeling software, and then use these maps to predict the outcome of the laboratory experiment they are about to undertake.

In contrast to visualization tools that exploit the computational capabilities of up-to-date computers, a GIS uses powerful information databases to construct detailed, precise computer maps. According to the U.S. Geological Survey,¹ “a GIS is a computer system capable of assembling, storing, manipulating, and displaying geographically referenced information (i.e., data identified according to their locations).” A GIS works by relating information from different sources for the same location. Different kinds of data in map form can be entered into a GIS, and a GIS can also convert existing digital information into maps. There are numerous GIS applications in wide-

spread use in many different fields, including:

• Agriculture (crop monitoring; land use management; commodity forecasting; soil studies; irrigation planning; and water resource assessment);

• Environment (pollution, weather, and climate monitoring; cause-effect studies; landscape assessment; conservation planning; and biodiversity libraries);

• Health care (asset management; ambulance routing and mobilization; epidemiological studies; and road traffic accident analysis); and

• Local government (planning and building control; land searches; boundary change modeling; property and highways maintenance; crime analysis; natural disaster management; and police and fire service command and control).

This list, while not exhaustive, illustrates the range of disciplines that can be expanded and deepened with GIS technology. According to the U.S. Geological Survey, an active GIS market has resulted in lower costs and continual improvements in hardware and software components. Thus, GIS tools that only recently were restricted to use by professional scientists and engineers, are now becoming part of undergraduate science and engineering programs and courses.²

These developments are likely to result

in much wider GIS application in both the public and private sectors, which in turn will pressure higher education institutions to provide instruction in this kind of learning to science and engineering students. This example also indicates how information technology costs can be shared among the public and private sectors to everyone’s benefit.

Many other visualization tools are finding a place in the postsecondary SME&T curriculum (see Box A-2). Spreadsheets and graphing calculators are now widely used for mathematical modeling. Large databases of all types of images are accessible to students and faculty from all over

the world. The National Library of Medicine’s Visible Human Project contains thousands of digitized cryosection photographs, magnetic resonance images (MRI), and transverse computerized tomography (CT) images, which together comprise anatomically correct, three-dimensional representations of the male and female human body. Although used heavily in clinical medicine and biomedical research, these images also are used widely in undergraduate courses. The Visible Human World Wide Web site³ lists dozens of applications of the image data sets, most of which were developed by and are used at colleges and universities around the country. Other databases contain images taken by the Hubble Space Telescope of planets, stars, asteroids, and comets,⁴ which can be downloaded for use in astronomy courses. Digitized video can help students understand the physics involved in sporting events (e.g., pole-vaulting), transportation (a plane take-off), and nature (river flow).

Technologies that rely on interactivity with the World Wide Web are still developing and hold promise as yet another avenue for pedagogical change. Small Java applets⁵ make it possible for a Web page user to perform interactive animations, immediate calculations, and other simple tasks without having to send a user request back to the server. Wolfgang Christian at Davidson College has created a Web site with numerous Java applets for physics, which he calls Physlets.⁶

The array of computer-based visualization tools available can dramatically affect not just how postsecondary SME&T educators teach their courses, but how they select the topics to cover.

Simulations let students view multiple aspects of complex systems simultaneously or sequentially. For example, an ecosystem or an architectural development can be constructed from many components with various connections of material and energy flow. Output data can be displayed in scatter plots, histograms, pie charts, and on maps concurrently and can be linked so that all representations change simultaneously when variables are manipulated. Similarly,

a transformation of the data (e.g., to log-log scales) can be reflected in all displays in a linked manner.⁷ Picture a situation where students are collaborating with each other or with research scientists from remote sites and are able to illustrate alternative graphical and iconic representations of the same data to highlight patterns in that data. The integration of simulations with real-world data allows the learner to incorporate both theoretical and empirical data to understand a complex phenomenon.

Many simulations have been constructed to help students learn long-term research strategies. These micro worlds engage students in the exploration of open-ended problems and complex data sets, giving them numerous modes and opportunities for data analysis, visualization, and the ability to explore multiple variables using tools from many disciplines. Simulations allow students to investigate contemporary research problems, explore “what-if” questions, pose new problems, and examine fundamental concepts and classical experiments. These learning environments are content- and process-rich, since these types of simulations are open to such diverse manipulation for long periods of investigation. These elaborate simulations help students learn which tools are appropriate in which contexts and why one tool is more powerful than another in a given context.

Several features of currently available simulation tools offer advantages to faculty interested in enhancing student mastery of scientific and technical disciplines. These features and advantages are listed in Box A-3.

Because of these features and advan-

tages, simulations probably will play an increasingly important role in undergraduate SME&T classrooms and laboratories. For example, in some fields, certain experiments may be too risky for novices to perform, they might be too advanced for a student’s laboratory skills, or the necessary equipment might be too costly. Another possibility is that conducting a real-world experiment would require more time than is available in a course (e.g., multiple crosses or crosses of organisms with long breeding cycles in a genetics experiment). Simulations are a reasonable alternative (see Box A-4). However, wet labs offer an indispensable experience to students that often cannot be replaced by the use of information technology.

Simulations can provide critical support for education based around realistic problem solving, such as the programs described earlier. By linking to research databases, simulations may enhance data mining⁸ and, in some cases, allow students to do original, open-ended research. Using simulations, students can replicate aspects of historically important models and classical experiments. They also can learn long-term strategies of scientific research using strategic simulations. Finally, simulations are a means of enhancing conceptual integration and conceptual change.

Our society faces challenges that were unknown to our parents and grandparents. Increases in global atmospheric carbon dioxide may be linked to a gradual warming of the planet. Run-off of fertilizers used to grow crops and fecal waste from livestock can contaminate drinking water supplies hundreds of miles away. The safety of the food supply, the advantages and disadvantages of genetic testing, and the development of reproductive technologies to benefit infertile couples all pose difficult

challenges to society. The solutions to these problems will be as complex and multifaceted as the problems themselves and will draw on a wide range of expertise.

Providing opportunities for students to develop their problem-solving skills is becoming a high priority for postsecondary science and engineering educators. Faculty members, departments, and institutions are recognizing the value of constructing contemporary science curricula around real-world problems. Institutions that require science credits for graduation have led the development of many courses for non-specialists organized around modern scientific problems. In some cases a semester or yearlong course is composed of interchangeable modules of different topics, such as water quality (e.g., the Modular Chemistry Consortium based at University of California-Berkeley⁹); global warming (e.g., Workshop Physical Science¹⁰), and environmental decision-making (e.g., The BioQUEST Curriculum Consortium¹¹). Publishers have responded with textbooks and other educational resources that emphasize problem solving, including those with a multidisciplinary approach (Trefil and Hazen, 1995) and those with a disciplinary emphasis (Schwartz et al., 1997).

Focusing SME&T courses on real-world problems will not only encourage faculty members to develop sophisticated and realistic assignments for students, but also will introduce students to a variety of information technology tools. Information technology can play a critical role in helping faculty develop real-world problems, and in helping students search among many possible solutions. The Internet provides access to a wide range of contemporary scientific problems and potential solutions. Students can tap into vast amounts of unfiltered information, including large databases, images of objects of all types and sizes, and textual information (some of which is available in print, and which may or may not be peer reviewed). As they seek solutions to real-world problems, SME&T students can connect with one another, with their instructors, and with practicing professionals. Learning from scientists, mathematicians, and engineers in industry and in government (e.g., National Aeronautics and Space Agency, Environmental Protection Agency) will be crucial, since they tend to be at the cutting edge of the field and know about the most promising solutions. These experts in the field can help a student understand a complicated, interdisciplinary scientific problem, in ways that were impossible only a few years ago (see Box A-5).

Scientific research often involves extensive collaboration among colleagues. Single-author papers are increasingly unusual in scientific and mathematical journals. Similarly, manufacturing relies on

teams of designers to create new products, as do software companies to write code for a software application. Postsecondary institutions also need to provide students the opportunity to participate in collaborative projects in which they can develop and nurture their teamwork skills using modern information technology as a tool.

Information technology can offer all students the opportunity to learn teamwork skills, including students who live off campus. These students often have full-time jobs in addition to family responsibilities. Without access to information technology and telecommunications, they would be at a distinct disadvantage to their on-campus

peers. But students need not be limited to virtual interactions on the World Wide Web. Collaborative projects also can involve more discrete team activities such as the development of a CD-ROM. The first of the three examples in Box A-6 is a project designed to develop both information technology and teamwork skills.

Recent changes in science, mathematics, and technology education at the K-12 level promise to have a profound impact on postsecondary education. National standards in mathematics (National Council of Teachers of Mathematics, 1989), science (American Association for the Advancement of Science, 1993; National Research Council [NRC], 1996b), and technology (International Technology Education Association, 2000) have been developed and are being adopted and translated by state departments of education. In the not-too-distant future, high school students will begin arriving at college with very different learning experiences and expectations (NRC, 1999b).

For example, the National Science Education Standards (NRC, 1996b) emphasize inquiry as a means of learning fundamental scientific concepts. Inquiry is defined in the Standards as “a set of interrelated processes by which scientists and students pose questions about the natural world and investigate phenomena” (NRC, 1996b, p. 214). When engaging in inquiry, students construct explanations, test those explanations against current scientific knowledge, and communicate their ideas to others. They

identify their assumptions, use critical and logical thinking, and consider alternative explanations. In this way, students actively develop an understanding of science by combining factual scientific knowledge with reasoning and thinking skills.

College courses historically have consisted of lecture, recitation, and laboratory. However, this configuration, and particularly the lecture, may contribute little to some students’ learning. Standard physics lectures do not help most students grasp fundamental concepts (McDermott, Shaffer, and Somers, 1994). Birk and Foster (1993) compared student grades in large multi-section lecture courses and found that the experience and lecturing skills of the instructor did not correlate with student achievement in the course. Although Birk and Foster conceded that results could be attributed to a mismatch between the instructor’s goals and the assessments used to determine student grades, they maintained that the lecture method is not the best learning tool for most students. For these reasons, students who have been exposed to inquiry-based learning in their K-12 education could be frustrated by many of the prevailing postsecondary SME&T pedagogical practices, particularly at the introductory level.

Unfortunately, the laboratory component of many undergraduate SME&T courses also can be less than optimal in developing students’ conceptual understanding. The experiments and activities in these courses are often structured in such a way that students can perform the actions step-by-step and achieve the predetermined outcome to within a small margin of error (if they are careful). Students may answer questions that are drawn from the preparatory information at the beginning of the activity and fill in the lab report at the end—yet still leave the laboratory with very little understanding of what was supposed to be learned (Poole and Kidder, 1996). Even in “newer” reform-minded curricula, laboratory experiences often are self-contained, emphasize verification of established conclusions, and do not correlate with material presented in subsequent instructional sessions (Hilosky, Sutman, and Schmuckler, 1998).

Inquiry as defined by the above standards, and “extended inquiry” in which students undertake real scientific research projects extending over several days or weeks, present major pedagogical challenges to their instructors. In the process of investigating a scientific question, both students and educators are required to use such diverse skills as the following (Tinker, 1997):

• modes of thinking, including posing good questions and developing experimental strategies;

• procedural skills, such as data collection and instrument calibration;

• familiarity with design and construction (safety procedures or machine shop skills); and

• analytical techniques, such as graphing, mathematical modeling, and statistics.

Although most scientists with doctoral degrees acquire the skills listed above during completion of their dissertation research, they may not feel qualified to teach them. They also may feel constrained by department-wide curricula and the need to cover a specified range of topics for students to enroll in subsequent courses. Faculty time also can be a factor when considering the effort necessary to develop or modify laboratory activities (for which professional recognition or rewards are often minimal). Another barrier to the kinds of reform advocated by the NRC (1996a, 1999b) and the National Science Foundation (NSF) (1996) is the presence of faculty members with master’s degrees who have not conducted an extended research project. An additional problem is that beginning graduate students, who have little experience with extended inquiry projects, are often employed as teaching assistants and lead laboratory classes.

An increasing number of academic institutions are beginning to integrate computers into introductory science and engineering courses. In the traditional lecture/recitation/laboratory system, computers have been used as aids for demonstrations in lectures, to run simulations, in microcomputer-based laboratories, and for out-of-class problem solving. This initial practice often encourages even more ambitious technology-based projects such as the use of Calculus & Mathematica,¹² Workshop Physics (Laws, 1991, 1999), and Studio Physics (Wilson, 1994).

Technology now provides exciting new opportunities for learners to conduct their own investigations of real problems (see Box A-7). In addition to working with data generated or collected by others, information technology enables students to collect and analyze their own scientific data without regard to time or place. Commercially available computer-interfaced sensors can measure almost any phenomenon, and they can be used with handheld calculators, battery-operated computer-based laboratory interfaces, and personal data assistants, making them opportune for fieldwork.¹³ In experiments not suited for computer-based data collection, technology still can allow learners to manipulate and analyze data, fit data to known mathematical equations, rapidly analyze large data sets with sophisticated statistical tests, simulate with models, and draw conclusions based on the outcomes of these processes. Students may even have access to “real” sensors used by scientists, such as devices for measuring the temperature of a lava flow or obtaining water samples as a function of depth. With modern networking, these instruments that collect data may be a world away from the students.

Computer-aided design (CAD), once exclusively the domain of professional engineers, is now used in undergraduate engineering courses as early as the freshman year. Although CAD bears a superficial resemblance to simulations—they both attempt to reproduce real phenomena graphically—CAD is only one step on the pathway to designing and manufacturing a real product. Growing numbers of engineering schools are offering CAD instruction, because it provides important workplace skills. For example, many professional engineers now use AutoCAD, a high-quality design tool. Even some high school students are now learning to use these tools (Ercolano, 1998).

As distance education and collaboration between individuals at different sites become more common, the use of design tools that can be transmitted over the Internet will likely increase. Designers can build a sequence of visual images into Web settings, and a user can view, move, and rotate those images, as in the example found in Box A-8.¹⁴

Many of the six pedagogical practices described above can be incorporated into courses that traditionally have not had a strong information technology component. In addition to the examples presented above, information technology is central to many other current efforts to improve teaching and learning in undergraduate SME&T fields. Table A-1 provides additional examples of innovative pedagogical practices, organized by discipline.

Currently, the dearth of effective instruments for measuring the effectiveness of new teaching and learning approaches that incorporate IT poses a tremendous barrier to the diffusion of these new approaches. Evaluation instruments must allow for metrics that the science and engineering teaching community are willing to accept. Some educators who resist curricular change argue that, until standardized tests such as the Graduate Record Examinations and Medical College Admission Test change their orientation to emphasize analysis of information, the solution of open-ended problems, and design of experiments to test hypotheses, using information technology to transform undergraduate education cannot succeed.

To make progress, pioneering faculty members will need to become actively involved with changing these tests. Although developing student learning assessments to correspond with new pedagogical methods can be extremely challenging, it can be done. For example, a group of chemistry professors at the University of Wisconsin-Madison felt strongly that their “structured active learning” (SAL) approach in introductory chemistry resulted in increased student learning. However, their colleagues were reluctant to embrace it. Consequently, faculty members from the science, engineering, and mathematics disciplines and on-campus experts in assessment and statistics

were recruited to assess learning by students emerging from either the traditional introductory chemistry course or the SAL course. The faculty members used their own criteria and instruments to see if they could distinguish the two groups of students. All statistical measures showed that the assessors ranked the SAL students as more competent (Millar et al., 1996; Springer et al., 1997a). As a result of this assessment, educators who had previously expressed skepticism about the effectiveness of the new approach became more open to new teaching methods.

As Garnett et al. (1995) have observed, when passive transmission of information is replaced by active learning activities, “the student is required to think carefully about the information, analyze it, and then apply it to new situations that may not have a clear solution or at first glance seem to be unrelated to the original presentation context.” As a result, these researchers find that student understanding improves— provided the tools used to measure it are consistent with what one defines as “understanding.”

As the SAL example illustrates, faculty members have considerable control over the assessment strategies they use in their own classes. These faculty members will need to consider how to improve the alignment between instructional goals and assessment tools. Tests often drive student behavior, especially at the classroom level. Researchers, teachers, and testing specialists therefore are exploring and experimenting with new assessment approaches, including discovering those that map more directly onto integrative and constructed learning. Whatever approach is chosen, assessment must at least provide feedback to teachers and students to make a fair judgment of progress and sensible next steps.

Before describing good assessment practices in current use and suggesting ways to use information technology to improve the process, it is important to define the various forms of assessment and distinguish between assessment and evaluation. The American Association for Higher Education (AAHE) defines assessment as “an ongoing process aimed at understanding and improving student learning.”¹⁵ Most definitions of evaluation, on the other hand, put little emphasis on feedback and instead concentrate on discrete measures of the overall success or failure of a process or project. Assessment often has been categorized as formative or summative. Formative assessments provide feedback to the student and instructor in a regular and systematic way, but often are not part of the student’s final grade. Summative assessments measure student knowledge at the conclusion of instruction (whether a chapter, a part of a semester, or at the end of the semester), but those results are rarely used to improve teaching and learning. Although tests of various sorts are by far the most common form of assessment, other assessments that measure student ability to carry out a task (performance assessments) or the ability to apply learning to

the solution of real-world problems (authentic assessments) are becoming more popular.

Several national efforts are aimed at helping postsecondary faculty members reconsider assessment issues. One of these, the Assessment Forum of the AAHE, builds on the foundation laid more than a decade ago by Chickering and Gamson (1987). Although not aimed specifically at SME&T educators, the following recommendations by AAHE (1992) are flexible and relevant enough to encompass a wide range of educational innovations and can frame design assessments of information technology-based innovations in the SME&T curriculum:¹⁶

1. The assessment of student learning begins with educational values.

2. Assessment is most effective when it reflects an understanding of learning as multidimensional and integrated, and is revealed in performance over time.

3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.

4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.

5. Assessment works best when it is ongoing and not episodic.

6. Assessment fosters wider improvement when representatives from across the educational community are involved.

7. Assessment makes a difference when it begins with issues of use and illuminates questions people really care about.

8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.

9. Through assessment, educators meet responsibilities to students and to the public.

In the current climate of accountability in all types of higher education institutions, assessment must be included as a regular feature of all information technology activities. Student performance and progress will be key indicators for the next decade (Baker, 1998). Some of these assessment metrics can be as simple as usage and time on task. Others can be more probing by delving into the subtler aspects of individual or group learning. The acquisition and dissemination of data from these assessments may begin to bridge the gap between educational theory and practice. Assessment data can also help researchers and educators explore varying hypotheses about effective teaching and learning outcomes.

While information technology is increasingly used to improve undergraduate teaching and learning, its use in assessment

is lagging. Helgeson and Kumar (1993) reported that the most common use of computer technology in assessment was to administer multiple-choice tests. More recently, of the 42 university presentations at an undergraduate program directors meeting of the Howard Hughes Medical Institute (HHMI), only one mentioned using information technology in student assessment, and that involved delivering a series of multiple-choice items (HHMI, 1996). This lack of use in assessment reflects the imma-turity of information technology’s integration into the wider educational arena. It also underscores the fact that evaluation strategies must parallel new classroom practice and learning goals (Linn, Baker, and Dunbar, 1991) and not be neglected as information technology enables a paradigmatic shift in postsecondary teaching and learning.

Information technology offers cost and scale benefits, decreased reporting time, and increased validity in assessment of postsecondary student learning. Information technology also has the potential to:

• provide continual rather than discrete measures of student learning;

• guide assessments of student learning in complex, real-world activities;

• increase the quality of assessment data while reducing costs; and

• provide measures of learning and skills among students with special needs.

Electronic classrooms can now make formative assessment feasible even in large lecture courses where the instructor may not know students by name. Classrooms equipped with communications systems such as Classtalk¹⁷ (see Box A-9) allow instructors to assess student understanding at crucial points during the class period, thus encouraging learning that is consistent with the research presented in this section. Traditional lectures are best suited for those students who make sense of ideas while listening and taking notes. Relying on a single instructional strategy that helps only a portion of students to learn creates inequities for all students, particularly those who assimilate materials better by discussing it, writing about it, and using it to solve problems (Dufresne, Gerace, Leonard, Mestre, and Werk, 1996).

When students discuss their answers in groups before responding to a query, their collective understanding is an example of distributed cognition. By using information technology to promote active engagement, e.g., by allowing students to run a simulation or visualize some phenomenon prior to engaging in discussions, SME&T faculty members improve the alignment between instruction and learning research.

Like a fine arts portfolio representing an artist’s range and quality of work, a portfolio used for assessment includes samples of work that represent a student’s best efforts. The portfolio could include assignments, project reports, reports on significant events, results from experiments, and reflective notes in journal format. In short, a portfolio is a collection of evidence that learning has taken place. Because portfolio assessment is designed for integration with instruction, it is becoming one of the most appealing forms of assessment, and is one area where increased use of technology in the classroom can translate directly into modified assessment activities.

Some of the benefits of portfolio assessment include:

1. By showing what they can do through their portfolios, students demonstrate skills and competencies for teachers, parents, potential employers, and even policy makers.

2. Portfolios provide useful information to evaluate comprehensively the quality of education and the quality of student achievement.

3. Because portfolios offer students a variety of ways to demonstrate what they know and can do, students are steered towards becoming reflective learners responsible for their own growth, thus encouraging and supporting multiple learning styles.

4. Portfolios offer educators multiple avenues to understand what and how their students are learning, thus contributing to

faculty efforts to design appropriate instruction that can improve student achievement.

Digital portfolios are an example of an incremental enhancement to an existing assessment based on information technology. The types of materials that might be included in a digital portfolio range from student-teacher e-mails to more complex materials such as graphics or presentations created with multimedia software tools. (See, for example, Box A-10.) Opportunities for improving assessment using digital tools depend primarily on the creativity and ingenuity of faculty and the institutional support and rewards that administrators put in place to encourage using these kinds of assessment tools.

Portfolios can require substantial effort on the part of students to produce and on the part of teachers to evaluate. Because of the new spectrum of skills displayed by students in portfolios, new rubrics to evaluate them also are needed (Gearhart, Novak, and Herman, 1996).

Computer-based tools can be cost-effective in assessing students’ problem-solving skills. For example, the Interactive Multimedia Exercises (IMMEX) Project at the University of California at Los Angeles School of Medicine,¹⁸ which was created to

assess medical students’ ability to diagnose a wide variety of illnesses, has evolved into a problem-solving platform for students from elementary school onwards. IMMEX is a software package with three components:

1. An authoring tool that instructors use to design customized problems in their particular discipline;

2. A user interface for students, who are given a problem and a menu of choices for accessing information needed to solve the problem; and

3. An analysis tool that records each student’s problem-solving actions in sequence.

As an assessment instrument, the most important component of IMMEX is the analysis tool, which contains embedded artificial neural networks that are “trained” to recognize problem-solving patterns. Stevens, Lopo, and Wang (1996) used this component of IMMEX to assess the ability of hundreds of UCLA medical students to correctly diagnose infectious diseases. As the first step in this process, infectious disease specialists completed six computer-based clinical diagnosis simulations. Next, the IMMEX analysis tools analyzed the diagnostic strategies used by these experts, and identified common features of their solutions to a problem. Then, these expert-trained neural networks were used to assess the diagnostic abilities of medical students. Only 17 percent of the medical students were classified as demonstrating the same strategies as the experts.

This IMMEX assessment has informed medical school faculty about areas for improvement in their efforts to train students to become capable diagnosticians. Information technology has become central to the medical school’s ability to perform learning assessments of individual students that otherwise would be time- and cost-prohibitive.

The examples of curricular innovations based on information technology described in this section are by no means exhaustive. Nevertheless, they illustrate the range of ways that individuals, departments, institutions, and consortia have used information technology to give their students new, more diverse, and more authentic learning experiences. These innovations also are consistent with recommendations made in SME&T reform documents that encourage the following pedagogical and learning strategies:

• visualization

• simulation

• real-world problem solving

• collaboration

• inquiry

• design.

The projects also are consistent with what research has shown about how students learn. Finally, many of these information technology-based innovations include appropriate ongoing assessment and evaluation. They are the result of careful, deliberate planning and adequate support, features that characterize successful, sustainable reform efforts.

The next section provides an overview

of current research on teaching and learning, using examples from this section to illustrate how educational research findings can be applied in the classroom.

The committee’s fundamental premise is that information technology (IT) can become a tool to enhance undergraduate science, mathematics, engineering, and technology (SME&T) learning for all students. To achieve this objective, pedagogy that includes information technology should draw on models of cognition that recognize the learning needs of today’s student population.

This section of the report describes the committee’s survey of recent educational research and theories. It also examines possibilities for using IT to apply these theories in undergraduate SME&T classrooms. Based on the examination discussed below, the committee found that IT can be an important vehicle to engage students and help them develop a deeper understanding of complex scientific and technical concepts.

Traditional postsecondary instruction has relied primarily on oral presentation of fundamental science and engineering concepts, sometimes accompanied by a laboratory component for students to verify the basic principles presented during lectures. These mostly passive learning environments can help some postsecondary students learn effectively. However, the growing need for a more scientifically literate (National Research Council [NRC], 1999b) and more technologically fluent (NRC, 1999a) citizenry, combined with the increasingly multicultural backgrounds of the postsecondary student population, call those traditional methods into question. Current students, including those interested in becoming future scientists, mathematicians, and engineers, are reporting their dissatisfaction with teaching methods and course structure as well as obvious mismatches between their goals and those of the instructor (e.g., Tobias, 1990; Seymour and Hewitt, 1997).

To face this issue directly, and fueled in part by advances in neuroscience and learning theory, educational researchers have developed a deeper understanding of how people learn at different ages and have documented the importance of active engagement in the learning process (Millar et al., 1996; Springer et al., 1997a; NRC, 1999b). Fischer (1996) summarizes five

fundamental assumptions about learning (p. 125):

• Learning is a process of knowledge construction, not of knowledge recording or absorption.

• Learning is knowledge-dependent; people use their existing knowledge to construct new knowledge.

• Learning is highly tuned to the situation in which it takes place.

• Learning needs to account for distributed cognition, requiring knowledge in the head of the individual learner to be combined with knowledge of the world.

• Learning is affected as much by motivational issues as by cognitive issues.

The implications of these educational research findings for postsecondary SME&T educators are becoming more widely understood and, consequently, are fueling reform initiatives. Many of the curriculum initiatives stemming from the reform movement, including those that make extensive use of information technology, have drawn on research in cognitive and social psychology, science and mathematics education, and ethnography and anthropology. This literature has stressed that students come into courses with definite beliefs and preconceptions about how the world works and that their conceptual development may involve a process of resistance, compromise, and then assimilation of new or competing knowledge.

Instructional activities also build on existing knowledge about the:

• context of learning (situated cognition);

• acquisition of knowledge across a group of individuals (distributed cognition); and

• influences of gender, ethnic and cultural influences; learning style; and motivation on student learning.

There have been significant advances in understanding the process of knowledge acquisition in the last two decades. Learning improves when instruction is contextualized (Brown, Collins, and Duguid, 1989); when it is driven by student interests and prior experiences (Shymansky et al., 1997); and when a variety of pedagogical methods are used to reach students with a diversity of learning styles (Felder, 1993).

As faculty members shift their pedagogy to the kind of instruction that is grounded in cognitive theories of learning, information technology can be a powerful instrument to help achieve these ends. As information technology plays a larger role in and out of the classroom, it will provide both structured and unstructured learning experiences. Instructors can incorporate more student-student collaboration and problem-solving activities into their courses to meet the educational needs of a diverse student population. Educators also can use information technology to assess student learning and use this information to improve teaching and learning.

A variety of learning theories have been proposed based upon a large body of educational research. However, three theories of learning are particularly relevant to learning in the natural sciences: constructivism, situated cognition, and distributed cognition.

Constructivism is a model of learning that asserts that knowledge is not passively received but is actively created inside the mind of every learner.

Situated cognition contends that knowledge and skills are best acquired in contexts from daily life (Brown et al., 1989). For example, Wilson, Teslow, and Osman-Jouchoux (1995) argue that because knowledge and learning are dependent on context, the content of knowledge within a population of learners is uneven.

The central theory of distributed cognition contends that learning is distributed in an environment and among learners and is not the product of isolated individual cognitive activity (Jonassen, Campbell, and Davidson, 1994). Therefore, learners construct understanding by using a variety of resources, including other people.

Other factors not accounted for in these theories also can affect the way individuals learn. Gender, for example, can affect learning by shaping preferences for information presentation and processing. The research literature contains numerous well-documented studies of differences in learning styles of male and female students. Similarly, individual differences in motivation to learn and achieve can profoundly impact student learning and need to be considered in developing technology-enhanced curricula. These factors, as well as the three educational theories of constructivism, situated cognition, and distributed cognition, are discussed below.

According to the theory of constructivism, learning is a dynamic process in which an individual actively integrates new information with existing knowledge. This idea has its roots in Jean Piaget’s groundbreaking research on cognitive development in children (Herron, 1978). Piaget was the first to propose that learning has two essential components. First, learners must take in information and compare it with the information already stored in their brain, a process known as assimilation. Second, when the new information is foreign or when it is at odds with what is already known, the learner must create a new mental representation in a process called accommodation. Cognitive growth occurs when new mental representations are created or modified to fit the demands of reality. Extensions of Piaget’s work have provided more precise definitions of these mental representations, or schema. Anderson (1980) describes schemas as “large, complex units of knowledge that organize much of

what we know about general categories of objects, classes of events, and types of people.”

According to proponents of constructivism, meaningful learning requires deliberate, personal restructuring of one’s conceptual framework. Cognitive scientists have elaborated on Piaget’s theories by developing several learning models. One model, proposed by Shymansky et al. (1997), proposes that learning consists of six phases (Box A-11). Three essential phases precede what Piaget referred to as the assimilation phase. During conceptual equilibrium, learners are comfortable with their present understanding. They then have an experience with a new body of information that challenges their conceptual understanding and leads to disequilibrium. Assimilation then occurs when learners adopt the new information to reestablish equilibrium. Development of new schema in accommodation is then followed by re-equilibration, a stage that persists until the next

challenge to the learners’ existing knowledge structures.

Through these six phases, learners must integrate new information with prior knowledge to build more elaborate knowledge structures or schema. The role of the learner’s prior experiences in developing conceptual understanding has been well documented (Bodner, 1991). Furthermore, research indicates that highly persistent misconceptions can arise from erroneous original learning, which forms a portion of the foundation of subsequent information that the student perceives as related to the existing knowledge (Zoller, 1996). Learners often retain erroneous models and explanations because they seem more reasonable and more useful to the learner (Mayer, 1987). In many cases, if not challenged, these beliefs can persist, potentially hindering further learning (McDermott, 1991; Nakhleh, 1992).

Once learners discard their misconceptions, they must then integrate their new knowledge with what Wittrock (1974) calls “generative processing.” Although faculty members often “tell” students things, such as factual information in a large lecture setting, the students must become personally engaged with the material and perform mental operations on this new knowledge before the knowledge becomes their own. Robust learning best occurs when the student can examine and test the information received (Bellamy and McNeill, 1994; Wang, 1996). SME&T faculty members can select information technology-based instructional strategies to help their stu-

dents interpret new information, reason from what is known, and solve complex problems.

Models for instructional design, teaching, and learning need to reflect the ways scientists, mathematicians, and engineers create new knowledge. Likewise, these models must be grounded in current knowledge about how students learn as well as the context in which that learning takes place. In the decades since Atkin and Karplus (1962) first proposed the concept of a learning cycle, a variety of instructional models have become available to link teaching decisions with learning research (National Center for Improving Science Education, 1991). Bybee (1997) proposed one such model, known as the “5 Es” model, which contains the following elements (pp. 178-179):

1. Engage students in a learning task.

2. Enable students to explore their ideas.

3. Guide students to devise an Explanation for the observations and results of their exploration.

4. Elaborate on students’ experiences by extending or applying the learning to new situations.

5. Evaluate students’ explanations and understanding.

The 5 Es model can guide faculty in selecting instructional strategies that help learners overcome their misconceptions. When the learning task includes discrepant events that yield unexpected results, learners will try to understand their observations. Their exploration will naturally lead to proposals for explanations, some of which are now no longer plausible or feasible in light of the unexpected results. By letting students apply what they have learned to new situations, the instructor can help the students reinforce new conceptual understandings and permanently discard old, incorrect ones.

Many of the examples of undergraduate SME&T reform initiatives described in the previous section are grounded in educational research about student learning. For example, the use of the computer as a data-collection device is consistent with constructivist learning theory. Three programs (Studio Physics, Workshop Physics, and the Center for Science and Mathematics Teaching) extensively use Microcomputer-Based Laboratory (MBL) sensors and software. Students in these programs use the MBL tools to measure and graph such physical quantities as position, velocity, acceleration, force, temperature, light intensity, pH, pressure, sound pressure, radiation, current, and voltage. They use the results to construct their own understanding of physical phenomena in a process that mirrors the 5 Es instructional model. Students receive a scenario, discuss the scenario with other group members, suggest predictions about the outcome, and then use real-time measurement tools to collect preliminary data.

Unexpected results occur frequently, which lead the group to more rigorous experimental procedures as members struggle to make sense of an outcome that seems counterintuitive. Extensive investigation of this type of student learning demonstrates that instruction employing information technology can provide students with a conceptual understanding that is superior to other modes of instruction (Laws, 1991).

Construction of dynamic computer models is another means of implementing the 5 Es instructional paradigm. Such models encourage students to analyze, synthesize, reason, and explain complex systems (Spitulnik and Krajcik, 1998). Using information technology this way also introduces students to scientists’ uses of models, as advocated by national science education reform efforts (e.g., American Association for the Advancement of Science, 1993). Casti (1997) explains why scientists use models:

• to predict a system’s behavior in the future, based on the system’s properties and current behavior (a predictive model);

• to provide a framework for understanding past observations as part of an overall process (an explanatory model); and

• to offer a picture of the real world with additional features built in, to let users bend or shape that reality to their own liking (a prescriptive model).

Each type of model enables learners to understand the connections between real-world concepts by constructing an environment to formulate and test a phenomenon (Spitulnik and Krajcik, 1998).

According to constructivism, learners use prior experiences and existing schema to understand new information and create new knowledge. The theory of situated cognition expands that premise by examining the context and culture in which knowledge is constructed. A central tenet of situated cognition is that learning is affected by the environment (Brown et al., 1989; Carr, Jonassen, and Litzinger, 1998; Jonassen et al., 1994). For example, some Brazilian school-age children assist their street-vendor parents by performing mathematical calculations to determine the total of a customer’s purchase, despite their limited formal education (Carraher, Carraher, and Schliemann, 1985). The context of street vending contributes to developing mathematical skills.

Researchers once viewed the process of acquiring knowledge and the context in which it is applied as two separate entities. Proponents of situated cognition challenge this premise and assert that the environment should reflect how the knowledge is used (Brown et al., 1989), so that acquiring and applying knowledge becomes a cohesive, integrated process (Brown et al., 1989; Jonassen et al., 1994). Information acquired apart from its context is unusable or “inert” (Griffin and Griffin, 1996; Jonassen et al.,

1994; Young, 1993). In the aforementioned example, the marketplace gives meaning to the mathematical calculations performed by Brazilian children.

Proponents of situated cognition often use the term “cognitive apprenticeships” to describe experiences in which learners engage in “authentic practices” through activities or social interactions. The term is reminiscent of a craft apprentice who learns from a more experienced tradesperson (Brown et al., 1989; Wilson, 1993). Ultimately, the goal of cognitive apprenticeships is to promote learning in the context of an activity, culture, or tool—to allow learners to become “enculturated in a community of practice” by observing and experiencing the values and practices of a professional working community (Carr et al., 1998, p. 6). The concept of cognitive apprenticeships supports the idea that individuals must experience the culture and context for meaningful learning to occur (Wilson, 1993), which in fact happens in most graduate research in SME&T disciplines. Unlike traditional lecture instruction, cognitive apprenticeships encourage learners to develop knowledge through the modeling, coaching, and guidance of mentors (Wilson, 1993). Moreover, cognitive apprenticeships enable learners to collaborate with practitioners and gain access to the wisdom of mentors. In joining a community of practice, learners also develop their own identity as members of the community (Lave and Wenger, 1991).

Not all educators fully subscribe to situated cognition as a theory of learning. For instance, Tripp (1993) acknowledges that teaching concepts without context is an exercise in futility, but he questions the emphasis placed on immersing students in a “community of practice” without the support of classroom experience. Without a balance of authentic and classroom experiences, Tripp argues, students may gain only partially developed skills for a setting or activity. Anderson, Reder, and Simon, (1996) advocate balance when applying the principles of situated cognition, asserting that learning is both context dependent and context independent. They maintain that, while some skills develop best within the appropriate social context, other skills may be developed better via abstract instruction in classroom settings. Overall, researchers who are cautious about situated learning are not opponents of the theory per se but are interested in a feasible balance between practical and instructional experiences.

The examples of successful information technology-based innovations in postsecondary SME&T education described in the previous section of this appendix are consistent with research into effective, efficient learning, especially for students who historically have been underrepresented in the SME&T disciplines. Many of the examples cited are grounded in realistic, practical problems in science and engineering. These kinds of problems not only can help

students integrate new information into their existing knowledge base but also can enhance their motivation to learn by providing a relevant context. The committee’s examples of information technology-based courses include several instances of students working with real data on real problems of interest to professional scientists.

Psychology has identified cognition as occurring “inside the head” of individuals, with only secondary influences from social, cultural, and historical factors. In effect, thought processing takes place in isolation from the context of the thought. According to Salomon (1993), such a perspective is appropriate for examining information-processing models but does not necessarily represent daily situations involving collaboration with others and the use of available tools and resources.

In contrast, the theory of distributed cognition holds that cognition is distributed across people, environments, situations, and objects, all of which are referred to as artifacts. Artifacts serve as vehicles that represent or act on information (Hunt, 1992; Pea, 1993). In essence, artifacts alter how a person performs a task and may compel the individual to employ entirely different cognitive skills to complete the task (Norman, 1991).

While many researchers use the term “distributed cognition,” Pea refers to the same concept as “distributed intelligence” to make the distinction that “people, not designed objects, ‘do’ cognition” (Pea, 1993, p. 50). For the discussion that follows, “distributed cognition” and “distributed intelligence” are used interchangeably.

In his study of distributed cognition among airline crew members, Norman (1992) identified numerous instances where crew members used common objects as artifacts to communicate information. This study was motivated by a tragic circumstance. A plane crashed—reportedly, because the captain had a heart attack that went undetected by the first officer, who dismissed the captain’s unresponsive behavior as typical and, therefore, not alarming. This example illustrates several principles of distributed cognition, including the danger of having all the knowledge reside in one individual (Norman, 1992). As with so many activities, a commercial plane cannot be flown by a single individual but requires collaboration among all crew members. Distributed cognition also discourages cognitive overload, in which individuals either fail to notice critical pieces of information or concentrate on a single explanation to the exclusion of other possibilities.

Group learning, in which students draw on and learn from the experiences of others, is one manifestation of distributed cognition. Collaborative learning and cooperative learning are two forms of group learn-

ing. Although similar, the concepts are not synonymous. Collaborative learning is “…an umbrella term for a variety of educational approaches involving joint intellectual effort by students, or by students and teachers together” (Goodsell et al., 1990). Cooperative learning (Box A-12), a form of collaborative learning, is an instructional technique that allows students to work in groups to achieve a common goal, to which they each contribute in individually accountable ways (Slavin, 1995). Learning increases when students learn interactively in groups (Brufee, 1993). For example, in a meta-analysis of published research on undergraduate SME&T education, Springer et al. (1997b) found that small-group learning results in greater academic achievement, better attitudes toward learning, and increased persistence in SME&T courses. Johnson, Johnson, and Smith (1998) conducted a similar meta-analysis and came to the same conclusion—namely, that cooperative learning promotes higher individual achievement than either competitive or individualistic approaches to learning. These findings seem to hold regardless of discipline or class size. Group learning in biology (Watson and Marshall, 1995), chemistry (Wright, 1996), earth science (Macdonald and Korinek, 1995), and physics (Hake, 1998) all resulted in greater student achievement than did courses where students worked independently.

The committee reviewed several examples of curricula that incorporate collaborative learning techniques. The Workshop

Physics curriculum¹⁹ uses cooperative learning strategies for guided inquiry activities and student projects. Likewise, students using BioQuest curriculum materials²⁰ work in groups. The University of California at Los Angeles’ Science Challenge²¹ offers lower-division students hands-on experience using real scientific problems, and provides support structures that foster teamwork rather than pitting students against

their peers. These and other information technology-based curriculum innovations use a distributed-cognition learning model. Group members are responsible for gathering data on a facet of a problem, sharing that data and their knowledge with other group members, and integrating that knowledge into the overall solution to the problem.

The examples identified also enable students to connect to rich resources on a network and communicate with their peers outside the classroom, consistent with the precepts of distributed cognition and, to some degree, situated cognition. Class activities that incorporate distributed intelligence encourage students to assume greater ownership and participation in learning. Students actually become “inventors of distributed-intelligence-as-tool, rather than receivers of intelligence-as-substance” (Pea, 1993, p. 82). Information technology affords students the resources to tackle their assignments more fully by accessing databases, the Internet, and other materials. And the mark of a successful student is generally the one who accesses a variety of diverse resources and then uses the information inventively to solve problems and complete tasks (Fischer, 1996; Pea, 1993). In the BioQuest instructional modules, complex simulations of real phenomena are used to generate data for student problem solving. The processes students use to solve a problem, including the resources they access, also provide the instructor with a great deal of insight about student understanding of the problem. In Project Interactive Multimedia Exercises, SME&T educators can use information technology to track student analytical skills by documenting each step in problem solving, which is virtually impossible to do with any other method. Harley (1993) points out that in situated learning settings such as this, the instructor’s role becomes supportive rather than directive. Such a role can be equally demanding, since the instructor must determine which problems to explore, provide the necessary background learning or “scaffolding,” and assess and reassess the situated learning that occurs (Young, 1993).

Distance learning courses also illustrate the application of distributed cognition. For example, Professor Kawagley’s “Native Ways of Knowing” at the University of Alaska enable students who are separated both physically and culturally to engage in cooperative learning (see Box A-13). The emphasis in the Alaska example is not on the thought process and work of individuals per se but more on how group members exchange thoughts and ideas to complete the assignment. This project exemplifies the essence of distributed cognition, in which thought processing occurs between group members and by means of a diverse set of resources.

An article in The Washington Post, headlined “Gender Gap in Fairfax Computer

Classes,” reported on the disproportionate number of male students who take high school computer classes in the Fairfax County, Virginia, school system when compared with the number of female students enrolled.²² The 3-to-1 ratio of males to females in these classes is part of a national trend, according to a report by the American Association of University Women (AAUW, 1998). AAUW convened the AAUW Educational Foundation Commission to Examine Gender, Technology, and Teacher Education in order to study the underlying causes of gender difference in computer usage.²³ Do males and females use computers differently? Do teachers contribute to gender-related differences in students’ attitudes toward computers and learning with or about computers? What can be attributed to the fact that computers and user interfaces were mainly developed by males (NRC, 1997a)? Are males more predisposed than females to exhibit strength in visual-spatial skills, which computer-based learning emphasizes? Regardless of the eventual outcome, studying these differences will contribute to understanding and possibly correcting any inequities in access to high-technology.

Research to understand gender differences in learning has increased during the

last 20 years and has included studies of a variety of age groups. These studies have ranged from examinations of verbal abilities to studies of visual-spatial skills in females and males. The finding that males consistently score higher than females on science tests (Howe and Dowdy, 1989) suggests the need for further research on gender differences. Hamilton (1995) suggests several explanations for different SME&T achievement between males and females. Although environmental factors and personal experiences can influence a student’s ability to learn and apply information, biological and physiological factors also can affect that learning, although to what extent remains under debate (NRC, 1999b).

Neuroscientists who study gender differences in learning have focused their investigations on the anatomy and physiology of the brain, particularly the lateralization (specialization) of the brain’s two hemispheres to perform specific tasks. Language, for example, is thought to reside primarily in the left hemisphere, while visual-spatial processing occurs predominantly in the right hemisphere. However, some researchers have hypothesized that females have better verbal abilities because language resides in both hemispheres of the brain in women but is primarily lateralized in the left hemisphere in men. By using magnetic resonance imaging to track brain activity during language tasks, Shaywitz et al. (1995) demonstrated that these tasks primarily involved the left brain hemisphere among males, while, among women, the same region of both hemispheres was active. These results are consistent with the current hypothesis about hemispheric specialization, but Shaywitz et al. (1995) also acknowledge that other regions in the brain may contribute to phonological and language tasks. If, in fact, language is bilateralized across both hemispheres of the brain in females, then the theory of “cognitive crowding” proposed by Levy (1976) becomes more plausible. Levy argued that visual-spatial cognition, which is typically attributed to the right hemisphere, must share neural space with language functioning in the right hemisphere in females. In comparison, language functioning for males is lateralized to the left hemisphere, allowing more neural space in the right hemisphere for functioning related to visual-spatial activities.

Although there is an extensive body of research from many areas of science that attempts to identify underlying biological explanations for differences in cognition between males and females, many studies have simply documented these differences without attempting to attribute them to either physiological or social factors. These studies are useful because they help to identify fertile areas for research on the underlying causes of gender differences in cognition and, equally important, because they often have immediate, practical applications.

For example, Hall and Hickman (1997) conducted a study in which 27 undergradu-

ate psychology students ranging from 19 to 45 years old evaluated five Web pages that described a neuron. Three of the pages consisted of text, one page consisted of a picture of the neuron, and one page consisted of both the text and the neuron. The background of three of the pages varied in complexity, which allowed investigators to investigate the effects of both the form of information and the context in which it was delivered. After reviewing the pages, students answered a questionnaire regarding their age, gender, experience with computers, and experience with the World Wide Web. Results indicated that both sexes gave higher ratings to the Web pages that contained both the text and the picture of the neuron; however, results also revealed a significant difference between male and female ratings regarding the visual complexity of the Web pages. The men gave a higher rating to the more visually complex pages, while women rated those pages lower on the scale. While Hall and Hickman (1997) stated that females reported less experience with computers and the World Wide Web than did men, this factor did not seem to account for the significant difference in the complexity ratings.

In an era of increasing student diversity, educators can enhance learning opportunities for all students by being more aware of learning differences. It has been well established that individuals develop a preferred, consistent set of behaviors or approaches to learning (Litzinger and Osif, 1993). This set of behaviors, otherwise known as a learning style, can be described in terms of four layers (Curry, 1983), with different learning styles tending to concentrate on one of the layers:

1. The personality layer describes an individual’s basic personality, often in a continuum from introvert to extrovert.

2. The information processing layer refers to the way a learner prefers to take in and process information.

3. The social interactions layer centers on how students behave and interact in the classroom (e.g., focused solely on learning or on grades).

4. The instructional preference layer describes the mode in which learning occurs most easily (e.g., listening, reading, and direct experience).

These learning style layers are not independent of each another, since the traits of one level will influence the next. Nonetheless, cognitive psychologists have developed a number of models for each of the layers. For a comprehensive review of these models, see Claxton and Murrell (1987), who focused on how these layers overlap; options for helping students determine and understand their own learning style; trends in gender-related differences in learning style; and the implications of this information for SME&T faculty members.

Two models of learning styles have be-

come widely accepted, in part because of simple-to-use instruments that help determine a student’s learning style. Kolb’s model of experiential learning (1984) describes learning as a continuum from concrete to abstract and from active to reflective. The resulting 2×2 matrix yields four learning styles, as shown in Table A-2.

Felder (1996) proposed another model of learning styles. In his model, five key questions are posed to probe a student’s learning preferences:

• Does the learner prefer sensory information (sights, sounds, and physical sensations) or intuitive information (memories, ideas, and insights)?

• Does the learner perceive sensory information most effectively through visual or verbal modes?

• Does the learner prefer to process information actively or reflectively?

• How does the learner progress toward understanding, sequentially or globally?

• Is the learner more comfortable with information that is obtained inductively (principles inferred from facts and observa

• tions) or deductively (consequences and applications inferred from principles)?

Felder and Soloman²⁴ have developed a “Learning Styles Inventory” to help students answer the first four of Felder’s questions about their own learning. Like Kolb (1984) and Curry (1983), Felder (1993, 1996) sees these traits as a continuum and describes students according to their preference for one mode or another. He further argues that, although students may be uncomfortable with instruction in their less-preferred modes of learning, faculty members should teach in multiple modes to accommodate the range of individual differences in a class (Felder, 1996). In the end, much of the responsibility for learning lies with the individual student. Knowledge about one’s own learning style is only useful if students know what to do with that information. After students complete the Learning Styles Inventory, Felder provides them with a handout of tips for maximizing their learning in ways that are consistent with their own learning style.²⁵

Information technology-based instruction has particular implications for the in-

formation processing and instructional preference layers in Curry’s model of learning styles (Curry, 1983). Information technology can help SME&T faculty members address the learning needs of all of their students by offering students instruction in a more preferred mode. Active learners, for instance, like to discuss ideas with their classmates or solve problems in a group setting. Electronic communications technologies make this possible even when students are in different locations. Information technology is especially helpful for visual learners, who gain more from pictures and diagrams than they do from the written and spoken words that fill most lecture classes. Sensory learners prefer facts and observations to concepts and interpretations and can be frustrated by courses devoid of real-world context.

Traditional lecture courses best serve students who are intuitive rather than sensory learners (Godleski, 1984). Faculty members who use information technology to provide science and mathematics instruction are likely to enhance student retention in their courses by maintaining the interest and motivation of students with other learning styles. The examples discussed in this document showcase how information technology could present information in a variety of ways. Computers offer unique information processing capabilities, such as multiple representations, interactive real-time assessment, and document revision, all of which can help individual students better construct their own understanding (Spitulnik and Krajcik, 1998). In addition, the use of multiple representations (diagrams, drawings, graphs, animations, and video) more closely approximates the way that scientists understand and describe phenomena, and can help students understand scientific concepts (Kozma and Quellmalz, 1995). Multiple representations of information also allow faculty members greater flexibility and creativity in developing effective pedagogical practices.

Finally, the use of multiple representations can make SME&T courses more inclusive for all students, not just those who learn the same way as their instructors. Halpern (1992) emphasized that knowledge about learning styles can be used to facilitate and improve curricula and instruction for both sexes. Material in SME&T courses can be presented in multiple ways, and various aspects of problem solving can incorporate both visual-spatial and verbal skills. Information technology-based initiatives that allow students to visualize complex phenomena, such as mathematical functions or molecular structures in three dimensions, can broaden the range of learning styles the instructor can accommodate.

For example, visualization of mathematical functions is central to the use of information technology in the calculus initiatives described in the previous section. Project CLEA²⁶ is another program that makes use of visualization of complex phenomena to teach. It allows students to con-

struct and manipulate images of stars and other astronomical bodies constructed from digital data. The Molecular Science Initiative at UCLA²⁷ uses molecular modeling hardware and software to provide students with images of phenomena that are otherwise impossible to visualize without information technology tools. Projects that feature GIS usage or multimedia tools similarly enable instructors to reach students with a variety of preferred learning styles.

The role of science education is to help learners grow in all three educational domains: cognitive, psychomotor, and affective. The development of problem-solving and reasoning skills and the acquisition of facts and concepts occur in the cognitive domain, while the development of physical and dexterity skills takes place in the psychomotor domain. The affective domain incorporates the learner’s motivation, attitudes, and beliefs.

The keys to success in education often are rooted in how a student feels about home, self, and school. These factors can confound efforts to transform postsecondary SME&T education because much of the affective domain is well established before students complete high school. Motivating students to learn science is perhaps the greatest challenge for SME&T educators. Unfortunately, current teaching practices often dampen student motivation and interest in science. The information technology-based initiatives highlighted in this document have been successful in part because they are sensitive to the affective dimensions of learning. The examples showcase a variety of instructional strategies and indicate how they accommodate a wide range of learning styles.

Motivation affects the likelihood that students will persist in a course. If success is defined partly in terms of a student’s course completion, student motivation becomes an important consideration.

McMillan and Forsyth (1991) define motivation as “purposeful engagement in classroom tasks and study to master concepts and skills.” Although this definition may need to be broadened to encompass non-campus based learning (vis-à-vis “classroom” tasks), it nonetheless suggests important research questions about the catalysts for engagement and student persistence mastering subject matter.

As discussed above in the section on situated cognition, some motivation to learn derives from the link between the information being learned and its application. If the information seems irrelevant, students may not pay close attention to the details, and this can lead to misconceptions (Cognition and Technology Group at Vanderbilt University, 1993). While there are several ways to define “relevance,” ultimately, it is in the mind of the learner. When courses and assignments incorporate the kinds of

experiences described in the previous section, increased motivation and “time on task” may help students focus not only on the “big picture” but also on those very details that will help them build a solid foundation for further learning of SME&T.

Student interest and satisfaction with instructional materials also can enhance motivation. For example, multimedia approaches can increase student motivation and interest (Fifield and Peifer, 1994; Powers, 1998). Multimedia can be fun, and “having fun with learning may enhance motivation and achievement” (Druger, 1997). Information Technology can also be used to tailor instruction to the needs and learning styles of individual students. Instruction that recognizes how much each student has already mastered and what significant questions remain can enhance motivation and learning.

For example, physicists at the U.S. Air Force Academy, Indiana University-Purdue University Indianapolis, and Davidson College have collaborated to develop Just in Time Teaching (Novak, Patterson, and Gavrin, 1999). In this approach, instructors develop web-based assignments, and students respond electronically. The instructor reads the student submissions “just in time” to adjust the lesson content and activities to suit the students’ needs. This creates a feedback loop, in which student preparation outside of class fundamentally affects what happens during the classroom session. As a result, most students come to class already prepared and engaged with the material. And, the faculty member knows at the beginning of class exactly how much each student has learned and how to best spend the classroom time. Originally developed to support physics instruction, more than 60 colleges and universities are now using this approach to provide instruction in a variety of academic disciplines (Novak, Patterson, and Gavrin, 2001). It appears that Just in Time Teaching motivates students to increase both the amount and quality of their interactions with other students and with faculty, as well as increasing their time on educational tasks. One extensive study found that these three factors—student-student interaction, student-faculty interaction, and time on task are critical to success in the undergraduate years (Astin, 1993).

Motivation, attitudes, and beliefs are equally important for “traditional” and “nontraditional” postsecondary SME&T learners and must be considered when developing courses and curricula for nontraditional populations of students. Many researchers contend that the motivators for adult learners are quite different from those of other populations. For example, Cennamo and Dawley (1995) found that internal motivators such as self-esteem, quality of life, and increased job satisfaction are far more important to adult learners than external motivators such as grades.

There is now considerable research on how students learn science and mathematics and how best to teach these disciplines (e.g., Gabel, 1990; NRC, 1999b). This research has contributed to the growing diversity of instructional strategies in postsecondary SME&T education (McNeal and D’Avanzo, 1997; Glassick et al., 1997). This trend must continue so that greatest potential learning gains from the use of information technology for teaching and learning SME&T can be achieved.

The applications of information technology that are described in this section are firmly grounded in the educational research literature. They are based on what is known about how students learn, both individually and in groups, and about how to establish inclusive learning opportunities that student find relevant to their personal experience and goals. These applications also share another important feature: considerable effort was expended on designing, evaluating, and using the outcomes of those evaluations to improve each application. This process of continual improvement also has catalyzed further research. Each application has contributed to the growing body of literature about educational assessment methods.

Accreditation Board for Engineering and Technology. (1997). Criteria for accrediting programs in engineering in the United States. Baltimore, MD: Author. Available: http://www.abet.org. [7/27/ 01].

Alexander, P.A. and Judy, J.E. (1988). Interaction of Domain-Specific and Strategic Knowledge in Academic Performance. Review of Educational Research, 58(4):375-404.

American Association for Higher Education. (1992). Principles of good practice for assessing student learning. Washington, DC: Author.

American Association for the Advancement of Science. (1993). Benchmarks for science literacy. New York: Oxford University Press.

American Association of University Women. (1998). Gender gap: Where schools still fail our children. Washington, DC: Author.

Anderson, J R. (1980). Cognitive psychology and its implications. San Francisco: W.H. Freeman & Co.

Anderson, J.R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.

Anderson, J.R. (1995). Cognitive psychology and its implications. New York: W.H. Freeman & Co.

Anderson, J.R., Reder, L.M., and Simon, H.A. (1996). Situated Learning and Education. Education Researcher, 4(5):5-11.

Astin, A.W. (1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass.

Atkin, M.J. and Karplus, R.H. (1962). Discovery or Invention? The Science Teacher, 29(2):121-143.

Baker, E.L. (1998). Model-based performance assessment. CSE Technical Report 465. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.

Baker, W., Hale, T., and Gifford, B.R. (1997). From Theory to Implementation: The Mediated Learning Approach to Computer-Mediated Instruction, Learning, and Assessment. EDUCOM Review, 32(5):42-50.

Balestri, D., Ehrmann, S.C., and Ferguson, D.L. (1992). Preface (pp. xi-xiv), Learning to design, designing to learn: Using technology to transform the curriculum. NY: Taylor & Francis.

Basili, P.A. and P.J. Sanford. (1991). Conceptual change strategies and cooperative group work in chemistry. Journal of Research in Science Teaching, 28(4):293-304.

Baxter, G.P., Elder, A.D, and Glaser, R. (1997). Assessment and Instruction in the Science Classroom. CSE Technical Report 432. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.

Bellamy, L. and McNeill, B. (1994). Active learning in the engineering classroom. Instructional Materials for Classroom Use, College of Engineering and Applied Sciences. Tempe, AZ: Arizona State University.

Benning, V. (1998, July 14). Gender Gap in Fairfax Computer Classes: Report Says Boys Outnumber Girls 3 to 1; Some Minorities Also Underrepresented. Washington Post, pp. B1, B5.

Benoit, R. (1998). Mr. Chips Meets Computer Chip: the Consequences of the Microbial Literacy Project of 21st Century Teaching of General Microbiology. Focus on Microbiology Education, 4:1.

Berger, C. (1998). Ann Jackson and the Four Myths of Integrating Technology into Teaching. Syllabus, March:18-20.

Berselli, B. (1997, September 30). Read It and Weep: Online Publishing Actually Boosts Sales. Washington Post, pp. C1-C2.

Birk, J.P. and Foster, J. (1993). The importance of lecture in general chemistry course performance. Journal of Chemistry Education, 70:180-182.

Bloom B.S. and Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: Longmans, Green.

Blumer, H. (1969). Symbolic interactionism: Perspective and method. Berkeley, CA: University of California Press.

Bodner, G.M. (1991). I Have Found You an Argument: The Conceptual Knowledge of Beginning Chemistry Graduate Students. Journal of Chemistry Education, 68:385-388.

Borgman, C. (Ed). (1990). Scholarly communications and biblitometrics. Newbury Park: Sage Publications.

Bourque, D.R. and Carlson, G.R. (1987). Hands-on Versus Computer Simulation Methods in Chemistry. Journal of Chemistry Education, 64:232-234.

Boverie, P., Nagel, L., and Garcia, S. (1997). Learning Styles, Emotional Intelligence, and Social Presence as Predictors of Distance Education Student Satisfaction. Selected papers from the eighth national conference on college teaching and learning. Jacksonville, FL: Florida Community College at Jacksonville.

Boyer Commission on Educating Undergraduates in the Research University. (1998). Reinventing undergraduate education: A blueprint for America’s research universities. Stony Brook, NY: Author.

Brown, A.L. (1978). Knowing when, where and how to remember: A problem of metacognition. Pp. 367-406 in R. Glaser (Ed.), Advances in instructional psychology. Hillsdale, NJ: Erlbaum.

Brown, A.L., Bransford, J.D., Ferrara, R.A., and Campione, J.C. (1983). Learning Remembering and Understanding. In J.H. Flavell and E.M. Markman (Eds.) Handbook of child psychology: Vol. 3, Cognitive Development (4^th ed.).

Brown, J.S., Collins, A., and Duguid, P. (1989). Situated Cognition and the Culture of Learning. Education Researcher, 18:32-42.

Brufee, K.A. (1993). Collaborative learning: Higher education, interdependence, and the authority of

knowledge. Baltimore: The Johns Hopkins University Press.

Bybee, R.W. (1997). Achieving science literacy: From purposes to practice. Portsmouth, NH: Heinemann.

Carr, A.A., Jonassen, D.H., and Litzinger, M.E. (1998). Good Ideas to Foment Educational Revolution: The Role of Systemic Change in Advancing Situated Learning, Constructivism, and Feminist Pedagogy. Educational Technology, 38(1):5-15.

Carraher, T.N., Carraher, D.W., and Schliemann, A.D. (1985). Mathematics in the Streets and in Schools. British Journal of Developmental Psychology, 3:21-29.

Casti, J.L. (1997). Would-be worlds: How simulation is changing the frontiers of science. New York: J. Wiley.

Cech, T.R. (1999). Science at Liberal Arts Colleges: A Better Education? Daedalus, 128 (1), Winter:195-216.

Cennamo, K.S. and Dawley, G.W. (1995). Designing Interactive Video Materials for Adult Learners. Performance & Improvement, 34(1):14-19.

Chase, W.G. and Simon, H.A. (1973). Perception in chess. Cognitive Psychology, 4:55-81.

Chi, M.T.H., Feltovich, P.J., and Glaser, R. (1981). Categorization and Representation of Physics Problems by Experts and Novices. Cognitive Science, 5:121-152.

Chickering, A.W. and Gamson, Z.E. (1987). Seven Principles for Good Practice in Undergraduate Education. American Association for Higher Education Bulletin, 39(7):3-7.

Choi, J.I. and Hannafin, M. (1995). Situated cognition and learning environments: Roles, Structures, and implications for design. Educational Technology, Research, and Development, 43(2):53-69.

Chonacky, N. and Myers, J. (1997). Exploring Collaboratory Partnerships for Interdisciplinary Undergraduate Science Education Reform. Council on Undergraduate Research Quarterly, 18(1):18-23.

Claxton, C.S. and Murrell, P.H. (1987). Learning Styles: Implications for Improving Educational Practices. ASHE-ERIC Higher Education Report No. 4. Washington, DC: George Washington University.

Clinton, W.J. (1997). Opening College Doors to All Americans: Excerpts from Remarks at San Jacinto Community College. Journal of Chemistry Education, 74(12):1392-1393.

Cognition and Technology Group at Vanderbilt University. (1993). Anchored Instruction and Situated Cognition Revisited. Educational Technology, 33(3):52-70.

Cole, M. and Engestrom, Y. (1993). A cultural-historical approach to distributed cognition. Pp. 1-46 in G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations. Cambridge, UK: Cambridge University Press.

Coley, R.J., Cradler, J., and Engel, P.K. (1997). Computers and classrooms: The status of technology in U.S. schools. Princeton, NJ: Educational Testing Service.

Collins, A. (1988). Cognitive apprenticeship and instructional technology (Report No. BBN-R-6899). Cambridge, MA: BBN Laboratories. (ERIC Document Reproduction Service No. ED 331 465).

Collis, B. (1997). Cooperative Learning on the World Wide Web. Selected Papers from the Eighth National Conference on College Teaching and Learning. Jacksonville, FL: Florida Community College at Jacksonville.

Consumer Electronics Association. (1996). Senior market potential. Arlington, VA: Author. Available: http://www.ebrain/org/ers/crs_all.asp. [October 24, 2001].

Crouch, R.D., Holden, M.S., and Samet, C. (1996). CAChe Molecular Modeling: A Visualization Tool Early in the Undergraduate Curriculum. Journal of Chemistry Education, 73(10):916-917.

Curry, L. (1983, April). An Organization of Learning Styles Theory and Constructs. Paper presented at the American Educational Research Association annual meeting, Montreal.

Daniel, J.S. (1996). Mega-universities and knowledge media: technology strategies for higher education. London: Kogan Page.

Davis, P. (1997). What Computer Skills do Employers Expect from Recent College Graduates? Technological Horizons in Education, Sept:74-78.

Department for Education and Employment (UK). (1997). Connecting the learning society. London: Author.

Derry, S.J., DuRussel, L.A., and O’Donnell, A.M. (1997). Individual and distributed cognitions in interdisciplinary teamwork: A developing case study and emerging theory. Madison, WI: National Institute for Science Education.

DiAmico, M., Baron, L.J., and Sissons, M.E. (1995). Gender Differences in Attributions about Microcomputer Learning. Sex Roles, 33:353-385.

Doerr, John. (1998, March 16). Business Week, p 29.

Druger, M. (1997). Motivating the Unmotivated. In E. Siebert, M. Caprio, and C. Lyda (Eds.), Effective teaching and course management. Dubuque, IA: Kendall-Hunt.

Duderstadt, J.J. (1997). The Future of the University in an Age of Knowledge. Journal of Asynchronous Learning Networks, 1(2):78-88.

Dufresne, R.J., Gerace, W.J., Leonard, W.J., Mestre, J.P., and Wenk, L. (1996). Classtalk: A Classroom Communication System for Active Learning in the College Lecture Hall. Journal of Computing and Higher Education, 7:3-47.

Dyson, E. (1997) Release 2.0. New York: Broadway Books, Bantam Doubleday Dell Publishing Group.

Ehrmann, S.C. (1988). Improving a Distributed Learning Environment with Computers and Telecommunications. Pp. 255-259 in R. Mason and A.Kaye (Eds.), Mindweave: Communication, computers and distance education. New York: Pergamon Press.

Ehrmann, S.C. (1990). Reaching Students, Reaching Resources: Using Technologies to Open the College. Academic Computing, IV(7):10-14, 32-34.

Ehrmann, S.C. and Balestri, D. (1992). Learning to Design, Designing to Learn: A More Creative Role for Technology. pp. 1-20. In D. Balestri, S. Ehrmann, and D. Ferguson (Eds.), Learning to design, designing to learn: Using technology to transform the curriculum. New York: Taylor & Francis.

Ellis, J.D. (1990). Preparing Science Teachers for the Information Age. Journal of Computers in Mathematics and Science Teaching, 9(4):55-70.

Ercolano, V. (1998). Students, Start Your Designs! American Society for Engineering Education Prism, 7(5):16.

Esiobu, G.O. and Soyibo, K. (1995). Effects of concept and vee mapping under three learning modes on students’ cognitive achievement in ecology and genetics. Journal of Research in Science Teaching, 32(9): 971-95.

Faison, C.L. (1996). Modeling Instructional Technology Use in Teacher Preparation: Why We Can’t Wait. Educational Technology, XXXVI(5):47-59.

Felder, R.M. (1993). Reaching the Second Tier: Learning and Teaching Styles in College Science Education. Journal of College Science Teaching, 22(5):286-290.

Felder, R.M. (1996). Matters of Style. American Society for Engineering Education Prism, 6(4):18-23.

Fifield, S.J. and Peifer, R.W. (1994). Enhancing Lecture Presentations in Introductory Biology with Computer-Based Multimedia . Journal of College Science Teaching, 23:235-239.

Fischer, G. (1995). Distributed Cognition, Learning Webs, and Domain-Oriented Design Environments. Pp. 125-129 in Proceedings of the conference on computer supported for collaborative learn-

ing. Available: http://www-cscl95.indiana.edu/cscl95/fischer.html. [7/27/01].

Fischer, G. (1996, June). Making Learning a Part of Life: Beyond the Gift Wrapping Approach to Technology. Paper presented at the National Science Foundation Symposium, Learning and Intelligent Systems. Available: http://www.cs.colorado.edu/~l3d/presentations/gf-wlf/. [7/27/01].

Flavell, J.H. (1979). Metacognition and Cognitive Monitoring: A New Era of Cognitive-Development Inquiry. American Psychologist, 34:906-911.

Floyd, B.P. (1998, September 11). Professor’s Web Site Makes Her “Dear Abby of Math.” Chronicle of Higher Education, p. A31.

Gabel, D.L. (1990). Students’ Understanding of the Particle Nature of Matter and its Relationship to Problem-Solving. Pp. 92-105 in Empirical research in mathematics and science education. Proceedings of the International Seminar, University of Dortmund, Germany.

Garmer, A.K. and Firestone, C.M. (1996). Creating a learning society: Initiatives for technology and education. Washington, DC: The Aspen Institute.

Garnett, P.J., Hacking, M., and Oliver, R. (1995). Refocusing the Chemistry Lab: A Case for Laboratory-Based Investigations. Australian Science Teachers Journal, 41(2):26-32.

The Gartner Group, Inc. (1993). Management strategies: PC cost/benefit and payback analysis. Stamford, CT: Author.

Gearhart, M., Novak, J.R., and Herman, J.L. (1996). Issues in Portfolio Assessment: The Scorability of Narrative Collections. CSE Tech. Report No. 410. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.

Gick, M.L. and McGarry, S.J. (1992). Learning From Mistakes: Inducing Analogous Solution Failures to a Source Problem Produces Later Successes in Analogical Transfer. Journal of Experimental Psychology: Learning, Memory and Cognition, 18:623-639.

Gilbert, S.W. (1996). Posted to the American Association for Higher Education information technology listserve (aahesgit@list.cren.net), 1/ 23/96. Subject: AAHESGIT: 10th of 14; Support Service Crisis.

Glanz, J. (1998). Cosmos in a Computer. Science, 280:1522-1523.

Glaser, R. (1991). Expertise and assessment. Pp. 17-30 in M.C. Wittrock and E.L. Baker (Eds.), Testing and cognition. Englewood Cliffs, NJ: Prentice Hall.

Glassick, C.E., Huber, M.T., and Maeroff, G. I. (1997). Scholarship assessed: Evaluation of the professoriate. San Francisco: Jossey-Bass.

Glennan, T.K. and Melmed, A. (1996). Fostering the use of educational technology: Elements of a national strategy. Santa Monica, CA: Rand.

Gobbo, C., and Chi, M.T.H. (1986). How knowledge is structured and used by expert and novice children. Cognitive Development, 1:221-237.

Godleski, E. (1984). Learning Style Compatibility of Engineering Students and Faculty. Proceedings, Annual Frontiers in Education Conference. American Society for Engineering Education/ Institute of Electronic and Electrical Engineers, Philadelphia, PA.

Goodsell, A., Maher, M., Tinto, V., Smith, B.L., and MacGregor, J., (Eds.) (1990). Collaborative learning: A sourcebook for higher education. University Park, PA: National Center on Postsecondary Teaching, Learning, and Assessment.

Graves, W.H. (1997). A Framework for Universal Intranet Access. CAUSE/EFFECT, 20(2):48-52. Available: http://cause-www.colorado.edu/. [7/27/01].

Green, K.C. (1996a). Campus Computing Survey. Pomona, CA: Claremont Graduate School.

Griffin, M.M., and Griffin, B.W. (1996). Situated cognition and cognitive style: Effects on students’ learning as measured by conventional

tests and performance assessments. Journal of Experimental Education, 64(4):293-308.

Hake, R.R. (1998). Interactive Engagement Versus Traditional Methods: A Six-Thousand Student Survey of Mechanics Test Data for Introductory Physics Courses. American Journal of Physics, 66:64-75.

Hall, R H. and Hickman, L.L. (1997, March). The Effect of Contiguity and Complexity of Web Page Displays on Subjective Ratings: The Role of Gender. Paper presented at the annual meeting of the American Educational Research Association, Chicago IL. Available: http://www.umr.edu/~rhall/research/aera/aera97.html. [7/27/01].

Halpern, D. (1992). Sex differences in cognitive abilities. 2nd Ed. Hillsdale, NJ: Erlbaum.

Hamilton, C.J. (1995). Beyond Sex Differences in Visuo-Spatial Processing: The Impact of Gender Trait Possession. British Journal of Psychology, 86:1-20.

Harley, S. (1993). Situated Learning and Classroom Instruction. Educational Technology, 33(3):46-51.

Helgeson, S.L. and Kumar, D.D. (1993). A Review of Educational Technology in Science Assessment. Journal of Computers in Mathematics and Science Teaching, 12(3/4):227-243.

Hellige, J. (1993). Hemispheric asymmetry: What’s right and what’s left. Cambridge, MA: Harvard University Press.

Herreid, C.F. (1997/1998). What Makes a Good Case? Journal of College Science Teaching, 27(3):163-165.

Herron, J.D. (1978). Piaget in the Classroom: Guidelines for Applications. Journal of Chemistry Education, 55(3):165-170.

Hilosky, A., Sutman, F., and Schmuckler, J. (1998). Is Laboratory-Based Instruction in Beginning College-Level Chemistry Worth the Effort and Expense? Journal of Chemistry Education, 75(1):100-104.

Holyoak, K.J. (1991). Symbolic Connectionism: Toward Third-Generation Theories of Expertise. Pp. 301-336 in K.A. Ericsson and J. Smith (Eds.), Toward a general theory of expertise. Cambridge, UK: Cambridge University Press.

Howard Hughes Medical Institute. (1996). New tools for science education, Undergraduate Program Directors’ Meeting, October 25-27, 1995. Chevy Chase, MD: Author.

Howe, A.C. and Dowdy, W. (1989). Spatial Visualization and Sex-Related Differences in Science Achievement. Science Education, 73(6):703-709.

Hunt, W.T. (1992). Shared Understanding: Implications for Computer-Supported Cooperative Work. Qualifying exam paper, Department of Computer Science, University of Toronto. Available: http://www.dgp.utoronto.ca/people/WilliamHunt/qualifier.html. [7/27/01].

Iaccino, J.F. (1993). Left brain-right brain differences: Inquiries, evidence, and new approaches. Hillsdale, NJ: Lawrence Erlbaum.

International Technology Education Association. (2000). Standards for technology education. Reston, VA: Author. Available: http//www.itea.www.org. [7/27/01].

Johnson, D.W., Johnson, R.T., and Smith, K.A. (1998). Cooperative Learning Returns to College: What Evidence is there that it Works? Change, 30(4):27-35.

Johnson, J. (1997). It Takes a (Global) Village to Prepare Teachers: Teaching/Technology Reflection. Selected Papers from the Eighth National Conference on College Teaching and Learning. Jacksonville, FL: Florida Community College at Jacksonville.

Johnson, S.D. and Thomas, R.G. (1994). Implications of Cognitive Science for Instructional Design in Technology Education. Journal of Technology Studies, 20(1):33-45.

Jonassen, D.H., Campbell, J.P., and Davidson, M.E. (1994). Learning with Media: Restructuring the

Debate. Educational Technology Research and Development, 42(2):31-39.

Jones, B.F., Valdez, G., Nowakowski, J., and Rasmussen, C. (1996). Plugging in: Choosing and using educational technology. Oak Brook, IL: North Central Regional Educational Laboratory.

Jones, J.B. (1998). The Non-Use of Computers in Undergraduate Engineering Courses. Journal of Engineering Education, 87(1):11-14.

Jones, L.M. and Kane, D.J. (1994). Student Evaluation of Computer-Based Instruction in a Large University Mechanics Course. American Journal of Physics , 62(9):832-836.

Keefe, J. (1987). Learning style: Theory and practice. Reston, VA: National Association of Secondary School Principals.

Keller, J.M. (1987). Development and use of the ARCS model of motivational design. Journal of Instructional Development, 10(3):2-10.

Kershaw, A. (1996). People, Planning, and Process: The Acceptance of Technological Innovation in Post-Secondary Organizations. Educational Technology, XXXVI(5):44-48.

Khan, B.H. (Ed.) (1997). Web-based instruction. Englewood Cliffs, NJ: Educational Technology Publications Inc.

Kimura, D. (1987). Are Men and Women’s Brains Really Different? Canadian Psychology, 28:133-147.

Kimura, D. (1992). Sex Differences in the Brain. Scientific American, 267(3):118-125.

King, J.E. (1998, May 1). Too Many Students Are Holding Jobs for Too Many Hours. Chronicle of Higher Education, p. A72.

Kolb, D. (1984). Experiential learning: Experience as the source of learning and development . Englewood Cliffs, NJ: Prentice Hall.

Kotovsky, K., Hayes, J.R., and Simon, H.A. (1985). Why Are Some Problems Hard? Evidence from the Tower of Hanoi. Cognitive Psychology, 17:248-294.

Kouzes, M., and Wulf, W.A. (2001). EMSL Collaborative. Available: http://www.emsl.pnl.gov:2080/docs/collab/CollabHome.html. [7/27/01].

Kozma, R.B., and Quellmalz, E.S. (1995). Issues and needs in evaluating the educational impact of the national information infrastructure. Menlo Park, CA: Center for Technology in Learning, SRI International. Paper presented at workshop sponsored by the Office of Educational Technology, U.S. Department of Education. Available: http://www.ed.gov/Technology/Futures/kozma.html [7/27/01].

Lanctot, R.C. (1998, March 11). U.S. Home PC Penetration Tops 45 Percent. Computer Retail Week. Available: http://www.techweb.com/wire/story/TWB19980311S0025 [7/27/01].

Larkin, J.H., McDermott, J., Simon, D., and Simon, H. (1980). Expert and Novice Performance in Solving Physics Problems. Science, 208:1335-1342.

Laurillard, D. (1993). Rethinking university teaching: A framework for the effective use of technology. London: Routledge.

Lave, J. and Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.

Laws, P.W. (1991). Calculus-Based Physics Without Lectures. Physics Today, Dec:24-31.

Laws, P.W. (1999). New Approaches to Science and Mathematics Teaching at Liberal Arts Colleges. Daedalus, 128(1):217-240.

Lehman, J.R. (1986). Microcomputer Offerings in Science Teacher Training. School Science and Mathematics, 86(2):119-125.

Levy, J. (1976). Cerebral Lateralization and Spatial Ability. Behavior Genetics, 6:171-188.

Linn, R.L., Baker, E.L., and Dunbar, S.B. (1991). Complex, Performance-Based Assessment: Expectation and Validation Criteria. Educational Researcher, 20:15-21.

Litzinger, M.E., and Osif, B. (1993). Accommodating diverse learning styles: Designing instruction for electronic information sources. In L.

Shirato, (Ed.) What is good instruction now? Library instruction for the 90s. Ann Arbor, MI: Pierian Press.

Macdonald, R.H. and Korinek, L. (1995). Cooperative Learning Activities in Large Entry-Level Geology Courses. Journal of Geology Education, 43:341-345.

MacLeod, C.M., Hunt, E.B., and Mathews, N.N. (1978). Individual Differences in the Verification of Sentence-Picture Relationships. Journal of Verbal Learning and Verbal Behavior, 17:493-508.

Massy, W.F., and Zemsky, R. (1996). Using Information Technology to Enhance Academic Productivity. Educom Review, 31(1):12-14. Available: http://www.educause.edu/nlii/keydocs/massy.html. [7/27/01].

Mayer, M. (1987). Common Sense Knowledge Versus Scientific Knowledge: The Case of Pressure, Weight and Gravity. Pp. 299-310 in Proceedings of the second international seminar: Misconceptions and educational strategies in science and mathematics, Vol. 1. Ithaca, NY: Cornell University Press.

Mazur, E. (1997). Peer instruction. Upper Saddle River, NJ : Prentice-Hall.

McCandless, G. (1998). Creating a Level Playing Field for Campus Computing: Universal Access. Syllabus, 11(6):12-14, 29. Available: http://www.syllabus.com [7/27/01].

McDermott, L.C. (1991). What we teach and what is learned-closing the gap. American Journal of Physics, 59:301-315.

McDermott, L.C., Shaffer, P., and Somers, M. (1994). Research as a guide for curriculum development: an illustration in the context of the Atwood’s machine. American Journal of Physics, 62:46-55.

McLellan, H. (1993). Evaluation in a Situated Learning Environment. Educational Technology, 33(3):39-45.

McLellan, H. (1994). Situated Learning: Continuing the conversation. Educational Technology, 34(8):7-8.

McMillan, J.H., and Forsyth, D.R. (1991). What Theories of Motivation Say about How Learners Learn. Pp. 39-52 in R.J. Menges and M.D. Svinicki, (Eds.) College teaching: From theory to practice . New Directions in Teaching and Learning, No. 45. San Francisco: Jossey-Bass.

McNeal, A.P. and D’Avanzo, C. (Eds.) (1997). Student-active science: Models of innovation in college science teaching. Fort Worth, TX: Saunders College Publishing.

Means, B., and Olson, K. (1995). Technology’s role in education reform: Findings from a national study of innovating schools. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

Millar, S.B., Kosciuk, S., Penberthy, D. and Wright, J.C. (1996). Faculty Assessment of a Freshman Chemistry Course. Proceedings of the American Society for Engineering Education Annual Conference. Washington, DC: American Society for Engineering Education, Session Number 2530.

Mislevy, R.J., and Gitomer, D. (1996). The Role of Probability-based Inference in an Intelligent Tutoring System. User modeling and user-adapted interaction, 5: 253-282.

Mitchell, W.J and Dertouzos, M.L, (Eds.) (1997). MIT Educational Technology Council Report. Cambridge, MA: MIT.

Morris, P.M., Ehrmann, S.C., Goldsmith, R.B., Howat, K.J., and Kumar, M.S.V. (Eds.) (1994). Valuable, viable software in education: Case studies and analysis. New York: PRIMIS-McGraw-Hill.

Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. (1998). Mathematics achievement in the final year of secondary school: IEA’s third international mathematics and science study. Chestnut Hill, MA: Boston College, Center for the Study of Testing, Evaluation, and Educational Policy.

Munro, B.C. (1984). B. F. Skinner. British Columbia Journal of Special Education, 8(1):45-60.

Nakhleh, M.B. (1992). Why Some Students Don’t Learn Chemistry. Journal of Chemistry Education, 69(3):191-196.

National Academy of Sciences. (1996). Careers in science and engineering: A student planning guide to grad school and beyond. Washington, DC: National Academy Press.

National Center for Education Statistics. (1998). The condition of education, 1998. Washington, DC: Author.

National Center for Improving Science Education. (1991). The high stakes of high school science. Washington, DC: Author

National Council for the Accreditation of Teacher Education. (1997). Technology and the new professional teacher: Preparing for the 21st century classroom. Washington, DC: Author. Available: http://www.ncate.org/pubs/m_pubs.htm#tech_prof_teach. [7/27/01].

National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.

National Council of Teachers of Mathematics. (1991). Professional standards for teaching mathematics. Reston, VA: Author.

National Institute for Science Education. (1998). Proceedings of the NISE forum on assessment. Madison, WI: Author.

National Research Council. (1993). National collaboratories: Applying information technology for scientific research. Committee Toward a National Collaboration: Establishing the User Developer Partnership. Washington, DC: National Academy Press.

National Research Council. (1994a). Information technology in the service society: A twenty-first century lever. Committee to Study the Impact of Information Technology on the Performance of Service Activities. Washington, DC: National Academy Press.

National Research Council. (1994b). Realizing the information future: The internet and beyond. NRENAISSANCE Committee. Washington, DC: National Academy Press.

National Research Council. (1995). Reshaping the graduate education of scientists and engineers. Committee on Science, Engineering, and Public Policy. Washington, DC: National Academy Press.

National Research Council. (1996a). From analysis to action: Undergraduate education in science, mathematics, engineering, and technology. Center for Science, Mathematics, and Engineering Education. Washington, DC: National Academy Press.

National Research Council. (1996b). National science education standards. National Committee on Science Education Standards and Assessment. Washington, DC: National Academy Press.

National Research Council. (1996c). The unpredictable certainty: Information infrastructure through 2000. NII 2000 Steering Committee. Washington, DC: National Academy Press.

National Research Council. (1997a). More than screen deep: Toward every-citizen interfaces for the nation’s information infrastructure. Toward an Every-Citizen Interface to the NII Steering Committee. Washington, DC: National Academy Press.

National Research Council. (1997b). Adviser, teacher, role model, friend: On being a mentor to students in science and engineering. Committee on Science, Engineering, and Public Policy. Washington, DC: National Academy Press.

National Research Council. (1998). Developing a digital national library for undergraduate science, mathematics, engineering, and technology education . Center for Science, Mathematics, and Engineering Education. Washington, DC: National Academy Press.

National Research Council. (1999a). Being fluent with information technology. Committee on

Information Technology Literacy. Washington, DC: National Academy Press.

National Research Council. (1999b). How people learn: Brain, mind, experience and school. Committee on Developments in the Science of Learning, J.D. Bransford, A.L. Brown, and R.R. Cocking (Eds.). Washington, DC: National Academy Press

National Research Council. (1999c). Transforming education in science, mathematics, engineering, and technology. Center for Science, Mathematics, and Engineering Education. Washington, DC: National Academy Press.

National Science Foundation. (1993). Indicators of science and mathematics education. Arlington, VA: Author.

National Science Foundation. (1996). Shaping the future: New expectations for undergraduate education in science, mathematics, engineering, and technology. Arlington, VA: Author.

National Science Foundation. (1998). Information technology: Its impact on undergraduate education, science, mathematics, engineering and technology. Available: http://www.nsf.gov/pubs/1998/nsf9882/nsf9882.pdf. [7/27/01].

National Science Teachers Association. (1998). NSTA standards for science teacher education. Arlington, VA: Author.

Naour, P. (1985). Brain/Behavior Relationships, Gender Differences, and the Learning Disabled. Theory into Practice, 24(2):100-105.

Neal, E. (1998, June 19). Using Technology in Teaching: We Need to Exercise Healthy Skepticism. Chronicle of Higher Education, p. B4.

Newell, A. and Simon, H.A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.

Niaz, M. (1991). Reasoning strategies of students in solving chemistry problems as a function of developmental level, functional M-capacity and disembedding ability. (ERIC Document Reproduction Service No. ED360180).

Norman, D.A. (1991). Cognitive Artifacts. Pp. 17-38 in John M. Carroll (Ed.), Designing interaction: Psychology at the human-computer interface. Cambridge, UK: Cambridge University Press.

Norman, D.A. (1992). Turn signals are the facial expressions of automobiles. Reading, MA: Addison-Wesley.

Norman, D.A. (1993). Things that make us smart. Reading, MA: Addison-Wesley.

Novak, G.M., Patterson, E.T., and Gavrin, A.D. (1999). Just-in time teaching: Blending active learning with Web technology. Upper Saddle River, NJ: Prentice Hall.

Novak, G.M., Patterson, E.T., and Gavrin, A.D. (2001). A Brief Overview: What Is JiTT? Available: http://webphyics.iupui.edu/jitt/whatOVR.html. [9/25/01].

Novak, J.D. and Gowin, D.B. (1984). Learning how to learn. New York: Cambridge University Press.

Oberlin, J.L. (1996). The Financial Mythology of Information Technology: The New Economics. Cause/Effect, Spring: 21-29.

Okebukola, P.A. and Jegede, O. J. (1988). Cognitive preference and learning mode as determinants of meaningful learning through concept mapping. Scientific Educator, 74:489-500.

O’Loughlin, M. (1992). Rethinking Science Education: Beyond Piagetian Constructivism toward a Sociocultural Model of Teaching and Learning. Journal of Research in Science Teaching, 29(8):791-820.

Orlansky, J. and String, J. (1979). Cost-effectiveness of computer-based instruction in military training. Washington, DC: U.S. Department of Defense.

Osborne, J.F (1996). Beyond Constructivism. Science Education, 80(1):53-82.

Panitz, B. (1998). Learning on Demand. American Society for Engineering Education Prism, 7(8):18-24.

Pea, R. (1993). Practices of distributed intelligence and designs for education. Pp. 47-87 in G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations. Cambridge, UK: Cambridge University Press.

Pence, H.E. (1997). Are Simulations Just a Substitute for Reality? Paper #9, Summer On-Line Conference on Chemical Education, June 1-August 1.

Peters, P.C. (1982). Even Honors Students Have Conceptual Difficulties with Physics. American Journal of Physics, 50(6):501-508.

Pew Research Center for The People and The Press. (1995). Technology in the American Household. Available: http://www.people-press.org/tech.htm. [7/27/01].

Phillips, D.C. (1995). The Good, the Bad, and the Ugly: The Many Faces of Constructivism. Educational Researcher, 24(7):5-12.

Poole, B.J. and Kidder, S.Q. (1996). Making Connections in the Undergraduate Laboratory. Journal of College Science Teaching, 26(1):34-36.

Powers, P. (1998). One Path to Using Multimedia in Chemistry Courses: Enlivening Students’ Learning Through Visual Presentations. Journal of College Science Teaching, 27:317-318.

President’s Committee of Advisors on Science and Technology. Panel on Educational Technology. (1997). Report to the President on the use of technology to strengthen K-12 education in the United States. Washington, DC: U.S. Government Printing Office.

Project Kaleidoscope. (1991). What works: Building natural science communities: A plan for strengthening undergraduate science and mathematics. Volume I. Washington, DC: Author.

Project Kaleidoscope. (1994). What works. Leadership: Challenges for the future. Volume II. Washington, DC: Author.

Project Kaleidoscope. (1997). The question of reform: Report on Project Kaleidoscope 1996-1997. Washington, DC: Author.

Project Kaleidoscope. (1998). Shaping the future of undergraduate science, mathematics, engineering and technology education: Proceedings and recommendations from the PKAL day of dialogue. Washington, DC: Author.

Project Kaleidoscope. (1999). Steps toward reform: Report on Project Kaleidoscope, 1997-1998. Washington, DC: Author.

Rogoff, B. and Gardner, W. (1984). Adult Guidance of Cognitive Development. Pp. 95-116 in B. Rogoff and J. Lave (Eds.), Everyday cognition. Cambridge, MA: Harvard University Press.

Ruiz-Primo, M.A., Schultz, S.E., and Shavelson, R.J.. (1997). Concept Map-based Assessment in Science: Two Exploratory Studies. CSE Technical Report 436. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing.

Salomon, G. (1993). Editor’s introduction. Pp. xi-xxi in G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations. Cambridge, UK: Cambridge University Press.

Schoenfeld, A.H. (1990). On mathematics as sense-making: An informal attack on the unfortunate divorce of formal and informal mathematics. In D.N. Perkins, J. Segal. and J. Voss (Eds.), Informal reasoning and education. Hillsdale, NJ: Erlbaum.

Schwartz, A.T., Bunce, D.M., Silberman, R.G., Stanitski, C.L., Stratton, W.J., and Zipp, A.P. (1997). Chemistry in context. 2nd ed. New York: McGraw-Hill.

Seymour, E. and Hewitt, N.M. (1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview Press.

Seymour, E. and Hunter, A-B. (1998). Talking about disability: The education and work experience of graduates and undergraduates with disabilities in science, mathematics, and engineering majors. Washington, DC: American Association for the Advancement of Science.

Shaywitz, B.A., Shaywitz, S.E., Pugh, K.R., Constable, R.T., Skudlarskl, P., Fulbright, R.K., Bronen, R.A., Fletcher, J.M., Shankweller, D.P., Katz, L., and Gore, J.C. (1995). Sex Differences in the Functional Organization of the Brain for Language. Nature, 33:607-609.

Sherald, M., and Ward, S. (1994). Market Predictions with Backprop Neural Nets. AI in Finance, Fall 1994:25-30.

Shymansky, J.A., Yore, L.D., Treagust, D.F., Thiele, R.B., Harrison, A., Waldrip, B.G., Stocklmayer, S.M., and Venville, G. (1997). Examining the Construction Process: A Study of Changes in Level 10 Students’ Understanding of Classical Mechanics. Journal of Research in Science Teaching, 34(6):571-593.

Simpson, R.D. and Frost, S.H. (1993). Inside college: Undergraduate education for the future. New York: Insight Books.

Sinnott, J. and Johnson, L. (1996). Reinventing the university: A radical proposal for a problem-based university. Norwood, NJ: Ablex Publishing Company.

Slavin, R.E. (1995). Cooperative learning: Theory, research, and practice (Second ed.). Boston: Allyn and Bacon.

Smith, S.G., Jones, L.L., and Waugh, M.L. (1986). Production and Evaluation of Interactive Videodisc Lessons in Laboratory Instruction. Journal of Computer-Based Instruction, 13:117-121.

Society for Industrial and Applied Mathematics. (1995). The SIAM report on mathematics in industry. Philadelphia, PA: Author. Available: http://www.siam.org/mii/miihome.htm. [7/27/ 01].

Software Publishers Association. (1997). The effectiveness of technology in schools, ’90-’97. Washington, DC: Author.

Spitulnik, M.W and Krajcik, J. (1998). Technological Tools To Support Inquiry in a Science Methods Course. Journal of Computers in Mathematics and Science Teaching, 17(1):63-74.

Springer, L., Millar, S.B., Kosciuk, S. and Penberthy, D. (1997a). Relating Concepts and Applications through Structured Active Learning. Presented at the American Educational Research Association meeting, Chicago, IL.

Springer, L., Stanne, M.E., and Donovan, S. (1997b). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Madison, WI: National Institute for Science Education (pre-publication manuscript).

SRI Consulting. (1997). Digital literacy: survival skills for the information age. London, England: BIT3M Futurescript.

State of California. (1997). Interim County Population Projections. Sacramento, CA: Department of Finance. Available: http://www.dof.ca.gov/html/demograp/post2nd.htm. [7/27/01].

Sternberg, R.J. and Rifkin, B. (1979). The Development of Analogical Reasoning Processes. Journal of Experimental Child Psychology, 27:195-232.

Stevens, R.S., Lopo, A.C., and Wang, P. (1996). Artificial Neural Networks Can Distinguish Novice and Expert Strategies During Complex Problem Solving. Journal of the American Medical Informatics Association, 3:131-138.

Strassman, P. (1996, April 15). Spending without results? Computerworld, p. 88.

Sugrue, B. (1994). Specifications for the design of problem-solving assessments in science. CSE Technical Report No. 387. Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing.

Talley, L.H. (1973). The Use of Three-Dimensional Visualization as a Moderator in the Higher Cognitive Learning of Concepts in College Level Chemistry. Journal of Research in Science Teaching, 10:263-269.

Thomes, K. and Clay, K. (1998). University Libraries in Transition. American Society of Engineering Education Prism, 7(8):26-29.

Tinker, R. (1997). The problem of extended inquiry in science teaching: Technology-rich curricula to the rescue. Concord, MA: The Concord Consortium. Available: http://concord.org/pubs/extinq.html. [7/27/01].

Tissue, B.M. (1997). The Costs of Incorporating Information Technology in Education. Paper presented in the Summer On-Line Conference on Chemical Education, June 1-August 1.

Available: http://www.chem.vt.edu/archive/chemconf97/paper04.html. [7/27/01].

Tobias, S. (1990). They’re not dumb, they’re different: Stalking the second tier. Tucson, AZ: Research Corporation.

Tobias, S. and Raphael, J. (1997a). The hidden curriculum: Faculty-made tests in science. Part 1: Lower-division courses. New York: Plenum.

Tobias, S. and Raphael, J. (1997b). The hidden curriculum: Faculty-made tests in science. Part 2: Upper-division courses. New York: Plenum.

Trefil, J. and Hazen, R.M. (1995). The sciences: An integrated approach. New York: Wiley.

Tripp, S.D. (1993). Theories, Traditions, and Situated learning. Educational Technology, 33(3):71-77.

Uecker, A. and Obrzut, J.E. (1993). Hemisphere and Gender Differences in Mental Rotation. Brain and Cognition, 22:42-50.

U.S. Department of Education. National Center for Education Statistics. (1996). The condition of education 1996. Washington, DC: U.S. Government Printing Office. Available: http://nces.ed.gov/pubs/ce/c9609a01.html. [7/27/01].

Van Dusen, G.C. (1997). The Virtual University: Technology and Reform in Higher Education. ASHE-ERIC Higher Education Report, Volume 25, No. 5. Washington, DC: George Washington University.

von Glasersfeld, E. (1996). Footnotes to The Many Faces of Constructivism. Educational Researcher, 25(6):19.

Wang, R. (1996). Learning chemistry in laboratory settings: A hands-on curriculum for non-science majors. (ERIC Document Reproduction Service No ED399189.)

Watson, S.B. and Marshall, J.E. (1995). Effects of Cooperative Incentives and Heterogeneous Arrangement on Achievement and Interaction of Cooperative Learning Groups in a College Life Science Course. Journal of Research in Science Teaching, 32(3):291-299.

Weiner, B. (1990). History of Motivational Research in Education. Journal of Educational Psychology, 82:616-622.

Williamson, V.M. and Abraham, M.R. (1995). The Effect of Computer Animation on the Particulate Mental Models of College Chemistry Students. Journal of Research in Science Teaching, 32(5):521-534.

Wilson, A.L. (1993). The Promise of Situated Cognition. New Directions for Adult and Continuing Education, 57(3):71-80.

Wilson, B.J., Teslow, J., and Osman-Jouchoux, R. (1995). The Impact of Constructivism on Instructional Design Fundamentals. Pp. 137-157 in B.B. Seels (Ed.), Instructional design fundamentals: A review and reconsideration. Englewood Cliffs, NJ: Educational Technology Publications.

Wilson, J.M. (1994). The CUPLE Physics Studio. The Physics Teacher, 32:518.

Wilson, J.M. (1997). Distance Learning for Continuous Education. EDUCOM Review, 32(2):12-16. Available: http://educause.edu. [7/ 27/01].

Wittrock, M.C. (1974). Learning as a Generative Process. Educational Psychologist, 11:87-95.

Wright, J.C. (1996). Authentic Learning Environment in Analytical Chemistry Using Cooperative Methods and Open-Ended Laboratories in Large Lecture Courses. Journal of Chemistry Education, 73(9):827-832.

Wulf, W.A. (1989). The National Collaboratory: A White Paper. Appendix A in Towards a National Collaboratory, the unpublished report of an invitational workshop held at the Rockefeller University, March 17-18, 1989.

Young, J.R. (1998, March 13). For Students with Disabilities, the Web Can Be Like a Classroom without a Ramp. The Chronicle of Higher Education, p. A31.

Young, M.F. (1993). Instructional Design for Situated Learning. Educational Technology, Research and Development, 41(1):43-58.

Zoller, U. (1996). The Use of Examinations for Revealing and Distinguishing between Students’ Misconceptions, Misunderstandings and “No Conceptions” in College Chemistry. Research in Science Education, 26(3):317-326.