Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering (2012)

7

Some Emerging Areas of Discipline-Based Education Research

Discipline-based education researchers study several other important aspects of teaching and learning beyond those described in previous chapters. For many of these topics, the research base in discipline-based education research (DBER) is not yet robust. This chapter highlights a few of these topics that are vital to learning science and engineering and warrant further study:

•   The role of science and engineering practices in undergraduate education, including in undergraduate research experiences

•   Students’ ability to apply knowledge in different settings (transfer)

•   Students’ ability to monitor their own learning processes (metacognition)

•   Students’ dispositions and motivations to study science and engineering (affective domain)

Some of these topics have been studied by cognitive science researchers or educational psychologists, but they are understudied in DBER for a variety of reasons. They may be addressed implicitly or as a secondary focus in studies on other topics; they may involve basic research (rather than the applied research that dominated the early stages of DBER and remains a strong emphasis today); or they may simply not yet be a priority for DBER scholars. In addition, DBER scholars are just beginning to deploy some of the measurement tools used by scholars in other disciplines that are necessary to research these topics. Despite the relatively sparse DBER literature on these topics thus far, they are of central importance.

This chapter discusses each of the above topics in turn, recognizing the conceptual overlap among some of them and with the topics discussed in Chapters 5 and 6. In contrast to earlier chapters that included a discipline-by-discipline summary of each topic, the evidence base for these four topics does not support such treatment. Instead, we briefly discuss the cross-disciplinary findings—all of which we have characterized as limited because few studies exist, much of the existing research consists of small-scale investigations, and no reviews have been published—and discuss in more detail the findings from the particular DBER field(s) with the most research to date on these emerging topics. Because these topics warrant further study in the context of DBER, each section ends with an identification of directions for future research. However, unlike previous chapters, this chapter does not conclude with a summary of key findings because DBER on these topics is too limited to support conclusions.

SCIENCE AND ENGINEERING PRACTICES

In part, science may be thought of as a vast and powerful compendium of factual information, concepts, principles, and laws that describe the nature of the universe and its inhabitants. But science also comprises a set of investigative processes, or ways of empirically and systematically studying the natural world, to advance the collective understanding of its order. These investigative processes—which we refer to as practices—and the knowledge gained from their application are critical components of scientific disciplines. Without those investigative practices, there would be no new scientific and engineering knowledge. Thus, an understanding of the attributes of science and engineering practices is vital, as is imparting them to new generations of learners.

In contrast to the clear delineation of content knowledge presented in introductory textbooks, no consensus exists on core disciplinary practices at the undergraduate level. Professional societies emphasize science and engineering practices in different ways. In physics, the American Association of Physics Teachers (1997) provides a set of goals for instructional laboratories that emphasize the central role of practices. In engineering, the ABET accreditation criteria F, G, and H focus on the needed skills of teamwork, communication, and ethics (see Chapter 3). In chemistry, the American Chemical Society Committee on Professional Training revised its guidelines for the training of chemists to include the same skills as engineering.¹

__________________

¹The guidelines are available at http://portal.acs.org/portal/PublicWebSite/about/governance/committees/training/acsapproved/degreeprogram/WPCP_008491 [accessed March 10, 2012].

At the K-12 level, the nature of science has historically received greater attention (Collins and Pinch, 1993; DeBoer, 1991; Petroski, 1996). Indeed, “The idea of science as a set of practices has emerged from the work of historians, philosophers, psychologists and sociologists over the past 60 years” (National Research Council, 2012, p. 43). More recently, A Framework for K-12 Science Education identifies core disciplinary ideas, practices, and cross-cutting concepts in the physical, life, and Earth sciences and engineering. That report’s conceptualization of practices is useful to consider here (National Research Council, 2012, pp. 44-45):

One helpful way of understanding the practices of scientists and engineers is to frame them as work that is done in three spheres of activity, as shown in Figure [7-1]. In one sphere, the dominant activity is investigation and empirical inquiry. In the second, the essence of work is the construction of explanations or designs using reasoning, creative thinking, and models. And in the third sphere, the ideas, such as the fit of models and explanations to evidence or the appropriateness of product designs, are analyzed, debated, and evaluated…. In all three spheres of activity, scientists and engineers try to use the best available tools to support the task at hand.

The framework goes on to identify eight specific science and engineering practices that advance an understanding of science among students.

FIGURE 7-1 The three spheres of activity for scientists and engineers.
SOURCE: National Research Council (2012, p. 45).

These practices also apply to undergraduate education. They include the following (National Research Council, 2012, p. 49):

1.   Asking questions and defining problems

2.   Developing and using models

3.   Planning and carrying out investigations

4.   Analyzing and interpreting data

5.   Using mathematics and computational thinking

6.   Constructing explanations and designing solutions

7.   Engaging in argument from evidence

8.   Obtaining, evaluating, and communicating information

Learning and becoming adept at science and engineering practices should not be separated from content learning. Rather, research at the K-12 level has shown that well-designed curricula and instructional practices can support deeper learning of content at the same time that students are engaging with these practices (National Research Council, 2007).

Overview of Discipline-Based Education
Research on Science and Engineering Practices

In contrast to K-12 education (see Flick and Lederman, 2004), science and engineering practices at the undergraduate level are largely understudied across the disciplines in this report. Most of the available evidence comes from physics and chemistry, with fewer studies in biology the geosciences, engineering, and astronomy. The studies that are available reflect the considerable range of accepted methods in science, as well as the lack of clear consensus among scientists and engineers about which practices are most important at the undergraduate level.

Research Focus and Methods

DBER studies of science practices typically adopt one of two perspectives: (1) examining practices as an outcome (i.e., how proficient students are in a specific practice or practices of science and engineering), or (2) engaging in practices as a means of leveraging content learning or other outcomes. Much of the engineering education research on professional skill-related process falls outside of these categories. That research largely describes “how to”—for example, how to build and use teams—rather than studying what works for developing students’ professional skills and how those strategies work. However, research on teamwork in particular is shifting toward the latter focus (Svinicki, 2011). Similarly, some notable curriculum development efforts promote practices in biology (e.g., BioQuest,

which focuses on problem posing, problem solving, and peer persuasion),² although with few efforts to evaluate their efficacy.

Most DBER studies that the committee reviewed on science practices do not explicitly situate themselves in broader theories of learning. An implicit framework for some biology education studies on this topic harkens back to Karplus’ (1977) learning cycle as elaborated by Lawson (1988). The learning cycle is fundamentally a constructivist approach, which argues that people generate their own understandings and form meaning as a result of their experiences and ideas (Piaget, 1978; Vygotsky, 1978).

DBER scholars use a range of methods to study science and engineering practices. Evidence is typically derived from self-developed assessments, surveys, interviews, and observations of students in class or laboratory sessions. Five of the 11 studies that the committee reviewed in biology are quasi-experimental studies of students in different courses (Dirks, 2011). The majority of these studies involve only students majoring in the biological sciences, and it is much more common for these studies to take place in lower division courses than upper division courses (Dirks, 2011).

Students’ Proficiency with Practices

Regarding practices as an outcome, findings from DBER suggest that undergraduate students have little experience or expertise in aspects of designing or conducting scientific investigations that are important to practicing scientists and engineers. Specifically, students struggle to

•   distinguish between data and evidence (Lyons, 2011);

•   classify matter (Stains and Talanquer, 2008);

•   ask scientifically fruitful questions (Karelina and Etkina, 2007; Slater, Slater, and Lyons; 2010; Slater, Slater, and Shaner, 2008); and

•   make predictions, observations, and explanations (Kastens, Agrawal, and Liben, 2009; Mattox, Reisner, and Rickey, 2006; Tien, Teichert, and Rickey, 2007).

Some research suggests that students also lack an understanding of experimental uncertainty unless they are explicitly taught about it (see Sere, Journeaux, and Larcher, 1993). These results are consistent with the prevalence of traditional laboratory exercises for introductory students, which are not effective in teaching higher-order skills (Redish, Steinberg, and Saul, 1997).

__________________

²See http://www.bioquest.org for a description of this curriculum [accessed March 31, 2012].

In addition to these studies of general practices, DBER is beginning to generate evidence about specific practices of individual disciplines. In physics, the research on problem solving is extensive, as is research about how students do and do not use discipline-specific models and graphical representations (see Chapter 5 for a discussion of research on problem solving and the use of representations in various disciplines). This research shows that students who use representations outperform those who do not but that students rarely use representations on their own (De Leone and Gire, 2006; Kohl and Finkelstein, 2005; Rosengrant, Etkina, and Van Heuvelen, 2006; Van Heuvelen and Zou, 2001). Research also has shown that some visualization skills that are important to the geosciences can be improved through a targeted set of practice exercises (Titus and Horsman, 2009; see Chapter 5). Also discussed in Chapter 5, engineering education courses that use case analyses, model-eliciting activities, worked out problem examples, and heuristics have been shown to help students develop the practices of problem analysis and design (Svinicki, 2011).

The laboratory is an important setting for students to engage in science and engineering practices, either in the context of regular course work or through research experiences with faculty. In astronomy education research, a limited number of studies address the role of traditional laboratories in improving proficiency with practices, and the results so far are mixed. Although some research suggests that these laboratories do not help students understand that scientists use a wide variety of methods to conduct investigations, they have been shown to help students improve their ability to develop appropriate scientific questions (Slater, Slater, and Lyons, 2010; Slater, Slater, and Shaner, 2008). As discussed in Chapter 6, research on students’ experiences in general chemistry (Miller et al., 2004) and analytical chemistry (Malina and Nakhleh, 2003) has found that the goals of the faculty determine the features of the laboratory that students identify as important. Depending on how faculty structure the laboratory experiment and assess student learning, students can perceive of instruments (e.g., spectrophotometers) simply as objects, without any knowledge of their internal workings, or as useful tools for collecting evidence about the behavior of molecules and their properties.

Using Practices to Enhance Conceptual Understanding

Considerably less research exists on using science and engineering practices to leverage learning. In physics, some studies demonstrate that engaging in scientific practices improves conceptual understanding (Cox and Junkin, 2002; Etkina et al., 2010) and that making predictions can enhance the educational impact of professor-led demonstrations (Crouch et al., 2004; Cummings et al., 1999). In biology, explicit instruction in

science practices—either through supplemental instruction or during a regular course—also appears to help students learn content (Coil et al., 2010; Dirks and Cunningham, 2006; Kitchen et al., 2003). For example, at one university, a preparatory short course for first-year students who were not prepared for rigorous coursework in the sciences explicitly taught graphing, experimental design, and science communication. Students who took the short course before taking a biology course earned, on average, higher overall grades in their introductory biology courses than students who did not participate (Coil et al., 2010; Dirks and Cunningham, 2006). Likewise, engineering students identified as having low-spatial ability who were selected to participate in a one-semester preparatory course on spatial visualization skills had better grades in subsequent courses and better retention in the major than similar students who did not participate in the program (Sorby and Baartmans, 2000).

Research Experiences for Undergraduates

Some colleges and universities use undergraduate research experiences and internships to supplement traditional learning experiences and offer students additional opportunities to engage in the practices of science and engineering outside the course setting. Many undergraduate research experiences are built on the same apprenticeship model as graduate education. These experiences might include opportunities for discovery of new knowledge; rediscovery of knowledge already acquired from mentors or from the larger disciplinary literature, including through replication or simulation of earlier results; and acquiring skills—perhaps doing bench science, learning to use analytical instruments, mastering modeling programs, or doing field work. Research experiences can give the student a sense of whether advanced study and a career in the particular field is a good personal fit (Hunter, Laursen, and Seymour, 2007; Seymour et al., 2004); allow them to experience the social dimensions of the work of science and engineering (Dunbar, 1995); and begin the long process of induction into science and engineering by involving them in a community of practice that shares goals, values, assumptions, and methods (Mogk and Goodwin, 2012; Zuckerman, 1992).

Studies on undergraduate research experiences generally cut across science disciplinary boundaries (e.g., Hunter, Laursen, and Seymour, 2007; Kardash, 2000; Lopatto, 2010; Reuckert, 2002; Sadler et al., 2010). These studies show that students who participate in undergraduate research believe that they have enhanced their research skills and report being more motivated to pursue a career in science after the experience. Other benefits include improved attitudes or dispositions toward science and a better understanding of the nature of science (Jarrett and Burnley, 2010).

A widespread assumption is that extended research experiences will promote more robust knowledge of science content and understandings of scientific ideas and principles, but this assumption has not been adequately tested or borne out (Sadler et al., 2010). The typical methodology is students’ or mentors’ self-reports via survey or interview; direct assessment of students’ pre- and post-apprenticeship knowledge/understanding is rare.

The few studies that have examined group differences show that research experiences enhanced retention in science for students from underrepresented groups (Gregerman, 2008; Locks and Gregerman, 2008; Nagda et al., 1998; Russell, Hancock, and McCullough, 2007), and in some cases, improved academic performance (Gilligan et al., 2007). The most promising findings come from a longitudinal study of the Undergraduate Research Opportunity Program (UROP), which includes many science disciplines and has been in existence at the University of Michigan since 1988. In that research, 75 percent of the African American men who participated in UROP completed their degrees, compared with 56 percent of a control group who applied but were not accepted into the program (Gregerman, 2008; Locks and Gregerman, 2008).

Although research experiences appear to improve student retention, they do not appear to affect students’ decisions about their future courses of study. In a multi-institutional survey study of the benefits of research for undergraduates and the progression rate of those students to advanced degrees,³ 83 percent of the 1,135 respondents earned, or intended to go on to earn, advanced degrees (Hunter, Laursen, and Seymour, 2007; Lopatto, 2007). The percentages of underrepresented minorities and whites who intended to earn advanced degrees were similar. However, the research experiences did not change students’ minds about pursuing future study. Along similar lines, a separate study of 51 students found that an undergraduate research program made little difference in the intent of females to pursue a graduate degree in astronomy (Slater, 2010).

It is important to note that most undergraduate research experiences are voluntary. The self-selection of students into these programs potentially confounds the research findings because the students who opt to participate might be more motivated or inclined to pursue further study or a career in science or engineering. Notable efforts to counter self-selection bias in the research include the evaluation of UROP (Gregerman, 2008), which compared students in the program to students who applied but were not

__________________

³Lopatto (2007) validated an instrument to measure how students experience undergraduate research (Survey of the Undergraduate Research Experience, SURE). The quality of the instrument adds weight to the evidence that undergraduate research enhances student perceptions of their science skills and interest in science.

accepted, and a review of four undergraduate programs that compared students and faculty who were involved in the programs and students and faculty who were not involved (Hunter, Laursen, and Seymour, 2007).

Directions for Future Research on
Science and Engineering Practices

For most of the disciplines examined in this report, the research on how students learn in laboratories or in the field—where they are likely to engage in science and engineering practices—is scarce. Additional research is needed to better understand how to measure and promote proficiency with these practices, and to explore relationships among practices and other outcomes such as overall understanding of concepts, practices, and ways of thinking of science and engineering.

In addition to the general practices described in this chapter, which span all of the science and engineering disciplines, individual disciplines may emphasize other practices or nuances of the general practices. Future research on practices at the undergraduate level might involve scientists, engineers, and/or scholars of the nature of science in reflecting on such discipline-specific variations and taking them into account when studying student learning.

More specifically, given the increase in undergraduate research programs and the expense of these programs in both time and money, it is important to understand the short- and long-term impacts of undergraduate research experiences and other research apprenticeships. Most reports on research apprenticeships document general trends (e.g., experiences have a positive effect on a specified outcome), but do not investigate the processes or mechanisms by which the outcome is achieved (Sadler et al., 2010). Ideally, future research examining these mechanisms would be conducted in ways that minimize or account for the effects of the self-selection bias of undergraduate research experiences. Additional studies are also needed on research experiences that occur during the regular school year, as most of the published research on the impact of undergraduate research experiences has been conducted on 10-week summer research apprenticeships rather than ongoing, independent, or mentored research in faculty laboratories.

It would be useful to study a wider variety of opportunities that engage students in science and engineering practices (e.g., internships in government or industry settings, service learning experiences, or museum and planetarium programs) and that emphasize different dimensions of practice. As one example of the latter, although knowledge of professional ethics is mandated through the ABET engineering accreditation criteria, engineering education research on enhancing student knowledge of professional ethics is scant, and represents a promising area for future study.

APPLYING KNOWLEDGE IN
DIFFERENT SETTINGS (TRANSFER)

Instructors expect that students will be able to apply what they learn in the classroom to new situations encountered inside and outside the classroom. Indeed, a prime goal and fundamental assumption of education is the transfer of knowledge from one context to another. Elementary school students are expected to transfer the subtraction skills learned in the classroom to the problem of making change at a neighborhood lemonade stand. At the college level, if students learn how to apply Newton’s second law of motion to a problem involving a block on an inclined plane, they are expected to recognize the applicability of that law in understanding the data collected in a physics laboratory or in designing a device for a mechanical engineering class. Indeed, the idea of knowledge transfer is inherent in the discipline of physics because it is assumed that a very small set of fundamental ideas can be used to explain the diversity of the universe. As a result, physics curricula are driven by the assumption that students will be able to apply what they learn across the physics curriculum and to other science and engineering courses. Geoscience curricula, on the other hand, are rich in opportunities for the transfer and application of concepts and insights from physics, chemistry, and biology to the Earth system—such as when concepts from chemistry are applied in mineralogy or concepts from physics are applied in structural geology.

Regardless of the discipline, if students were only able to use what they have learned in exactly the conditions under which the learning occurred, that learning would have little practical value. However, the application of knowledge in different contexts is limited by students’ understanding of the conditions under which knowledge applies. That understanding, in turn, typically stays fixed in the domain in which students initially learn the content (Bassok, 2003). For example, a student who learns an equation in physics may have difficulty seeing the applicability of the same equation in algebra. Thus, one of the enduring problems in education—no less an issue at the undergraduate level than in K-12 education—is that transfer of learned material to new situations is much more difficult than educators expect.

Discipline-Based Education Research on Transfer

Research Focus and Methods

Transfer is a two-part process: transfer during learning (effect of past learning on new knowledge acquisition) and transfer of learning (the degree to which the new learning is applied in future situations) (Sousa, 2011). DBER scholars have partially addressed the first dimension through their

extensive exploration of students’ conceptual understanding (see Chapter 4). Indeed, much of that research is predicated on the assumption that instructors need to know what their students already know, because prior knowledge can either interfere with new learning (negative transfer) or facilitate it (positive transfer) (Ausubel, 1968; Ausubel, Novak, and Hanesian, 1978; Bretz, 2001; Novak, Gowin, and Kahle, 1984). However, only a small amount of DBER has explicitly focused on either transfer during learning or transfer of learning. In contrast, a considerable number of cognitive science studies on physics problem solving (Bassok, 1990; Bassok and Holyoak, 1989) and mathematical problem solving (Bassok, 2003; Catrambone, 1998; Kaminsky and Sloutsky, 2012; Novick and Holyoak, 1991; Reed, 1987) have investigated transfer.

More DBER studies might be identified post hoc as exploring one or both of these dimensions of transfer. Studies on analogical reasoning (Jee et al., 2010; Sibley, 2009), concept mapping (Clark and James, 2004), the application of skills to novel situations such as the use of the petrographic microscope to identify minerals and then to interpret textures (Gunter, 2004; Milliken et al., 2003), and the relationships between learning in the classroom, laboratory, and field (Mogk and Goodwin, 2012) could provide early insights into transfer.

Much of the physics research on transfer in the context of problem solving is based on the theoretical underpinnings of information processing (Simon, 1978). However, the meaning and even the utility of the idea of knowledge transfer as a theoretical construct is controversial within physics education research (Mestre, 2005). Some scholars attempt to find the basic building blocks of physics knowledge and the mechanism for their interaction. This type of theoretical framework is expressed, for example, in terms of mental resources that are activated by situational perception (Hammer et al., 2005). Scholars working within this approach study building blocks of different scales. Within this theoretical framework, transfer is not a process but the outcome of interactions among building blocks. Knowledge transfer, in this approach, is a derived construct.

Other physics education researchers take a more phenomenological approach, driven by the assumption that learning is so complex that a linear set of mechanisms and interactions, even if they exist, are not adequate to describe knowing. Some of these theoretical frameworks are concerned with specifying meaningful characterizations of transfer (Schwartz, Bransford, and Sears, 2005). Others are concerned with characterizing teaching systems that achieve transfer without ascribing the underlying mental mechanisms (Brown, Collins, and Duguid, 1989).

In chemistry, researchers have analyzed students’ transfer of knowledge related to the characteristics and behavior of NaCl and NaCl (aq) (Kelly,

2007; Kelly and Jones, 2008; Teichert et al., 2008), and the transfer of knowledge gained from computer animations designed to help students learn concepts related to the particulate nature of matter (Kelly and Jones, 2008). This research is guided by Ausubel, Novak, and Hanesian’s (1978) theory of meaningful learning, which proposes that students store new information according to what they identify as similarities between what they already know and what they need to know. Meaningful learning stands in stark contrast to the strategy of rote memorization, explaining why the latter leaves chemistry students with fragmented pieces of knowledge not useful for building connections to new information (transfer during learning) or in future courses (transfer of learning).

Methods of studying transfer range from individual interviews and experiments in controlled research environments to the analysis of student behavior and written work in classes. Participants typically include students in general introductory courses.

Students’ Difficulties Transferring Knowledge

In DBER, most a priori studies of transfer come from chemistry education research. These studies generally suggest that students have trouble applying knowledge in a new context. A series of seminal studies in chemistry (Nakhleh, 1993; Nakhleh and Mitchell, 1993; Nurrenbern and Pickering, 1987; Pickering, 1990; Sawrey, 1990) have demonstrated that students can memorize how to solve problems that require mathematical manipulations of symbols and chemical formulas, but cannot transfer these skills forward to a similar problem involving particulate images (i.e., drawings of the atoms and molecules). In these studies, students could successfully calculate the mass of products when given information about the reactants and a balanced equation of the reaction. However, when presented with a particulate image of the reactants, they could not correctly identify the particulate image of what remained after the reaction.

A few studies in chemistry have documented students’ difficulties identifying the critical attributes of a problem (Kelley and Jones, 2008; Tien, Teichert, and Rickey, 2007), which is an important component of transfer (Sousa, 2011). Those difficulties, in turn, appear to preclude students’ abilities to transfer their existing knowledge to a new concept. For example, in one study (Teichert et al., 2008), 19 students were interviewed after completing a laboratory experiment that involved measuring the conductivity of sodium chloride solutions in water and explaining the results in terms of particulate images. They were asked to predict the conductivities of NaCl(aq), AgNO₃(aq), and the resulting solution upon their mixing, and to draw particulate images to support their reasoning. Eighteen of 19

students correctly drew NaCl(aq) as solvated ions, and 15 of 19 did so for AgNO₃(aq).⁴ Students were then asked to read a paragraph about boiling point elevation—a property they had not yet learned—which explained that this property depended solely upon the number of particles in solution and not their chemical identity. In this context, only 10 of the 19 students drew NaCl(aq) as solvated ions. When the 9 who responded incorrectly were asked to look back at their drawing from the first interview, only 6 of them altered their drawing to reflect solvated ions. Consistent with findings discussed in Chapter 6, these findings suggest that students had difficulty identifying the critical attributes of the problems at hand.

It is not surprising that if novices focus on superficial rather than structural features of problems, their ability to apply learned solution features to new situations will be limited (see Novick, 1988, for relevant experimental evidence from studies of mathematical problem solving). Consistent with research from cognitive science (National Research Council, 1999), the findings of Teichert et al. (2008) suggest that students need help to understand which aspects of problems are critical for determining the appropriate solution method and which are not.

Directions for Future Research on Transfer

In science and engineering, educators and researchers need a greater understanding of how to widely assess and promote the transfer of knowledge and skills within courses; across courses in the major; and among majors and nonmajors as they take different science courses. This understanding is vital because the same students often take chemistry, physics, and biology courses, and each instructor assumes a certain level of prior knowledge. Longitudinal studies would offer the opportunity to develop measures and use them to carefully document and track instances of negative and positive transfer for multiple cohorts of students. Ideally, these studies would be interdisciplinary to understand how students’ knowledge transfers across the suite of science and engineering courses they take in college.

One of the greatest research challenges will be framing transfer in a way that is both measureable and acceptable to instructors and students. The course structure and the usual assessment of students in those courses tend to channel DBER into the narrowest form of transfer, namely assessing the direct application of something learned to a new context as measured by the number of correct answers on a task. Bransford and Schwartz (1999) have developed a broader theory of transfer that emphasizes preparation for future learning, which could be a useful framework for further research

__________________

⁴When a substance dissolves in a solvent, its ions spread out and become surrounded by the molecules in the solvent. These ions are then called “solvated ions.”

efforts in this area. For example, college students were more successful than precollege students when presented with the unfamiliar challenge of designing a recovery program for baby eagles, although neither group was completely successful. Bransford and Schwartz argue that the college students had greater general education experience and that they were able to transfer this experience to the new learning situation—evidence of positive transfer that might be missed using more traditional definitions.

Cognitive science research has illuminated some factors that influence transfer, including the quality and context of original learning; the similarity of problems across settings; and, as discussed, students’ recognition of the critical attributes of problems (Bassok, 2003; Sousa, 2011). Although this research has been conducted primarily in the context of investigating analogical problem solving in mathematics and physics, DBER scholars in other disciplines might be able to use this research to design studies that advance the understanding of transfer in different disciplines and learning environments.

Cognitive science research also provides insight into specific conditions under which successful transfer occurs, such as when instructors teach students to monitor their own thinking and learning processes (Bransford and Schwartz, 1999; see discussion of metacognition below) or, in the context of problem solving, when source and target problems are superficially and structurally similar (e.g., Chi and Bassok, 1989; Holyoak and Koh, 1987; Ross, 1984). These insights also might inform future DBER studies.

METACOGNITION

No one doubts that cognition is important to the work of the scientist and engineer and to learning in those disciplines. Cognition, a synonym for “thinking,” is necessary for understanding science concepts, for applying the methods of science to discover new aspects of the natural world, and for using scientific ideas to solve important problems. What may be less obvious is that cognition must be supplemented with metacognition. Metacognition is the mind’s ability to monitor and control its own activities (Brown, 1978; Flavell, 1979). Metacognition is “thinking about thinking.” It is a form of higher-order cognition that allows individuals to monitor the quality of their thoughts and to redirect their thinking processes as needed. Metacognition also comes into play during problem solving when the individual judges whether the chosen strategy is effective.

Metacognition is a potent intellectual competency. Students need to be metacognitive so that in every learning context—during lecture, independent reading, laboratory work, research, or discussions—they monitor their comprehension and, if comprehension fails, take corrective steps. A metacognitive learner asks: Do I truly understand? Is my strategy working?

When a student is preparing for a midterm or final exam, a crucial question must be posed: Am I ready to take the test?

An extensive research base in psychology indicates that the ability to make an honest and accurate appraisal of one’s own knowledge state is crucial to academic success. Differences in metacognitive ability translate to differences in students’ learning outcomes (Tobias and Everson, 1996). In general, students who are more metacognitive are better students overall, which suggests that one goal of education should be to help students become more metacognitive (Lin, Schwartz, and Hatano, 1995).

Metacognition is more than the binary distinction of to know or not. Understanding is always incremental. Indeed, the entire scientific enterprise is predicated on a sense of which aspects of the natural world are well understood, which are partially understood, and which are unknown. These metacognitive sensibilities steer the research agenda, alert investigators to promising questions, and give insights into whether investigative probes are answering the driving questions. Thus, metacognition is an essential competency for both learning and knowledge creation.

Metacognitive approaches are embedded in instructional practices such as problem-based learning, knowledge surveys, and reflective exercises during classes, and in activities designed to support critical thinking. Unfortunately, many instructors assume either that undergraduate students already have the requisite metacognitive skills, or that these skills are too advanced to teach in introductory courses (Trigwell et al., 2001; Yerushalmi et al., 2007). However, research has shown the benefits of teaching metacognitive skills as part of learning content (Collins, Brown, and Newman, 1989). Such practices include being explicit with students about the rationale for learner-centered pedagogy, including defining learning objectives, demanding more student responsibility in mastering content, and using class time for problem solving (Heller et al., 1992). The concomitant need is for students to become more aware of their own study habits and how to improve them (e.g., Silverthorn, 2006). (See also the discussion of cognitive apprenticeship in Chapter 5.)

Discipline-Based Education Research on Metacognition

Research Focus and Methods

Students’ metacognition is an implicit focus of some research on problem solving and other kinds of decision making, and is increasingly an explicit focus of some DBER. Most of the research that the committee reviewed investigates or assesses the role of metacognition in specific learning environments, typically in the context of problem solving (see Chapter 5), and some research focuses on the development of tools to

measure metacognition (e.g., Cooper and Sandi-Urena, 2009). A few studies document efforts to develop students’ metacognitive skills (McCrindle and Christensen, 1995). DBER scholars study metacognition through interviews, case studies, quantitative studies, mixed methods, and phenomenology.

Physics studies on metacognition typically take place in a controlled environment outside of the classroom with a small number of students. Participants in these studies are typically students who know some physics, often paid volunteers who have taken one or two introductory courses in physics. This sort of study often uses interviews or students’ self-explanations to analyze students’ reasoning processes as they engage in a task. In other disciplines, researchers have assessed metacognitive activity in the context of a learning environment such as an inquiry laboratory, or a specific problem solving activity. In biology, for example, McCrindle and Christensen (1995) conducted an experiment in which they randomly assigned students in their introductory class to either a treatment group or a control group to test a strategy for developing metacognitive skills.

Assessing metacognition during learning can be challenging because it is largely a hidden skill, although there are techniques to infer its existence. Moreover, asking students to document metacognitive activities might artificially prompt metacognitive behavior. Such behavior is desirable, but the false positive results could confound the research findings. In chemistry, the Metacognitive Activities Instrument (MCAI), a validated self-report instrument, probes students’ thinking about problem solving (Cooper and Sandi-Urena, 2009). As described in Box 7-1, students’ MCAI scores have been shown to correlate with their problem-solving strategies and abilities as measured by the online Interactive Multi-Media Exercises system (Cooper, Sandi-Urena, and Stevens, 2008). Besides the MCAI, evidence of metacognition in chemistry has been gathered from self-reports and observations of student work samples and behavior.

Promoting Metacognition to Enhance Learning

The effectiveness of deeper, more meaningful processing for retention of information was first documented in cognitive psychology by Craik and Lockhart (1972). Consistent with findings from cognitive science research (National Research Council, 1999), DBER suggests that students can develop metacognitive skills over time when metacognitive strategies are built into instruction (McCrindle and Christensen, 1995; Weinstein, Husman, and Dierking, 2000), but that relatively few students report using metacognitive strategies such as self-testing when studying on their own (Karpicke, Butler, and Roediger, 2009).

One focus of DBER on metacognition concerns the self-explanation effect, which is the benefit to learning and problem solving that accrues

from attempts by learners to explain to themselves or another person concepts that are unclear. For example, studies of how college students use example solutions provided in physics textbooks to learn topics in mechanics found that successful students explained and justified solution steps to a greater extent than did unsuccessful students (Chi et al., 1989). The quality of the explanations also differed; good students referred to general principles, concepts, or procedures they had read about in an earlier part of the text and examined how they were instantiated in the current example (Chi and VanLehn, 1991).

Replication of these studies in biology (Ainsworth and Loizou, 2003) has yielded similar results. This research examined whether the way information about the human circulatory system is presented affects learners’ self-explanations and subsequent learning. The learners were college students who had not taken a biology class beyond high school. They took a pre-test, learned about the circulatory system via either text or diagrams, and then took a post-test. Students were directed to generate explanations to themselves during the study phase, and the students who generated more

self-explanations did better on the post-test. In addition, students who received diagrams provided more self-explanations, performed better on the post-test, and spent less time studying the material than students who received text. Similarly, research in engineering education has found that incorporating metacognitive reflection steps and self-explanation prompts into instruction can improve students’ problem solving (Svinicki, 2011).

Together, this research suggests that generating and articulating explanations can be an effective pedagogical tool to help students process more deeply the underlying structure of unfamiliar concepts and problems in all disciplines. This deeper processing can lead to enhanced learning compared with what is achieved by simply reading textbook paragraphs and examples or examining textbook diagrams.

Directions for Future Research on Metacognition

Metacognition is a necessary skill for meaningful learning and thus merits continued study in the context of DBER. Further research could clarify which metacognitive skills are useful to science and engineering. Because the skills may not be the same for each discipline, additional DBER could examine these similarities and differences. Other efforts might be directed at developing easy-to-use assessments that align to the appropriate metacognitive skills and that measure the effects on metacognition of instructional interventions and learning environments. Finally, the kinds of conditions and strategies that limit or promote metacognition (e.g., writing across the curriculum, structured problem solving, and assessments that promote metacognition) also merit examination. In this regard, the MORE model (model, observe, reflect, explain) (Tien, Rickey, and Stacy, 1999), which promotes metacognition in the chemistry laboratory, might be adapted to other settings and disciplines. The spread of cooperative group problem solving from physics to other disciplines such as chemistry, engineering, and health sciences also might be instructive (Heller, Keith, and Anderson, 1992). This approach emphasizes the practice of metacognitive skills by making them evident through the social interaction of co-constructing a problem solution with the use of specific scaffolding.

DISPOSITIONS AND MOTIVATION TO STUDY SCIENCE
AND ENGINEERING (THE AFFECTIVE DOMAIN)

Successful science and engineering education cannot be defined solely in terms of how many concepts and practices students learn. Students have attitudes, beliefs, and expectations about learning that can influence their behavior and performance in courses (Halloun, 1997; Hammer, 1994, 1995; May and Etkina, 2002; Perkins et al., 2005). As an example, a

common student belief is that physics consists of many unrelated pieces of information. This belief often leads students to approach physics by memorizing formulas without connecting them to broader, underlying concepts and principles (Hammer, 1994, 1995). More generally, helping students become members of a scientific or engineering community requires attention to a wider range of outcomes and how to achieve them.

Educational outcomes beyond the mastery of discipline-specific content are considered part of the affective realm (Snow, Corno, and Jackson, 1996). The affective domain is very broad. Psychologists use the term “affect” to refer to an observable expression of emotion, but the term can extend to motivation, attribution, and willful action (i.e., volition). Affect can also refer broadly to belief systems, including the important concept of self-efficacy (Bandura, 1986), or the belief that one is capable of accomplishing a goal such as learning within a particular discipline. The affective domain also can be defined to include a range of external and internal factors that influence a student’s ability to learn, including values, social pressures, stereotypes, perceptions, feelings, anxiety, or fear (Krathwohl, Bloom, and Masia, 1964). Another important concept is conation, which refers to volition, or more generally, to the connection between knowledge and affect (personal will, motivation) and intentional, goal-oriented personal actions or behaviors (the desire to learn more, or to engage in proactive activities). Conation provides critical evidence of self-direction and regulation (Hilgard, 2006; Snow, 1989). All such feelings, motives, and beliefs fall under the banner of affect or the affective domain as we use those terms here.

Scholars are increasingly aware that affective aspects of learning are directly linked to cognitive and memory functions (Gray, 2004; Pessoa, 2008; Storbeck and Clore, 2007). Indeed, sociocultural perspectives on learning recognize that a changing sense of one’s own identity and competence in a domain occurs at the same time that one is becoming increasingly adept at disciplinary practices and knowledge application and that these concurrent processes are mutually supportive (Lave and Wenger, 1991; Resnick and Klopfer, 1989; Rogoff and Wertsch, 1984). These findings suggest that researchers and instructors should not consider cognitive and affective development apart from each other.

Discipline-Based Education Research on the Affective Domain

The Carnegie Preparation for the Professions Program (Sullivan, 2005) describes three apprenticeships: the apprenticeship of the head (intellectual development), the hand (skill development), and the heart (development of habits of mind, values and attitudes). Many science disciplines and engineering stress the first apprenticeship (the head), place less emphasis on the

second (the hand), and are silent or implicit about the third apprenticeship (the heart) (Sullivan, 2005). Similarly, in DBER, the affective domain has received less attention than the cognitive domain.

Research Focus and Methods

The research that has been done in DBER is as broad as the affective domain itself, and ranges from students’ views about the discipline, to their motivations for pursuing science and engineering, to the social dimensions of fieldwork, to the role of student beliefs in conceptual change.

Scholarly research on the affective domain rigorously probes students’ attitudes and beliefs about content, pedagogy, the discipline as a whole, and/or learning in general. DBER scholars use a range of instruments to measure aspects of the affective domain. Some widely used, validated instruments include the following:

•   Epistemological Beliefs Assessment for Physical Sciences (Elby, 2001), which assesses students’ views about the nature of knowledge and learning in physics.

•   Science Motivation Questionnaire, which assesses six components of motivations: (1) intrinsically motivated science learning, (2) extrinsically motivated science learning, (3) relevance of learning science to personal goals, (4) responsibility (self-determination) for learning science, (5) confidence (self-efficacy) in learning science, and (6) anxiety about science assessment (Glynn and Koballa, 2006).

•   Colorado Learning Attitudes about Science Survey (CLASS), which is designed to compare novice and expert perceptions about the content and structure of a specific discipline, the source of knowledge about that discipline, and the connection of a discipline to the real world (Adams et al., 2006). Discipline-specific versions of the CLASS exist for physics, (Adams et al., 2006), biology (Semsar et al., 2011), and chemistry (Barbera et al., 2008).

Insight into student and faculty attitudes and beliefs has also come from in-depth interviews and case studies of a few individuals.

Students’ Attitudes, Beliefs, and Motivations

Although the evidence is limited, DBER on the affective domain suggests that student attitudes about physics, chemistry, biology—and, indeed, science in general—differ markedly from the views of practicing scientists. Student attitudes in physics after instruction diverge further from “expert-like” norms than before instruction, even when instruction

is more student-centered (Adams et al., 2006; Kost-Smith, Pollock, and Finkelstein, 2010; Redish, Steinberg, and Saul, 1998). In biology as well, students in one study became less expert-like following the introductory course, but subsequently became more expert-like in upper division courses (Semsar et al., 2011). In chemistry, nonmajors’ attitudes, as measured by an instrument known as CHEMX, move toward faculty norms during the first year of chemistry instruction. Chemistry majors, on the other hand, move away from the faculty’s norms during the first year, and then begin moving closer toward faculty views when they take organic chemistry (Grove and Bretz, 2007).

Although motivation is largely understudied in DBER, a longitudinal (7-year) study in engineering found that several factors influence students’ motivation to study engineering, including “psychological/personal reasons, a desire to contribute to the social good, financial security, or, in some cases, seeing engineering as a stepping stone to another profession” (Atman et al., 2010, pp. 3-4). That study also demonstrates that motivation is related to important outcomes such as the intention to pursue engineering as a major and a career.

The Geosciences. The nature of some subject matter covered in the geosciences makes consideration of the affective domain (e.g., sensory input from a natural setting) particularly important to scholars in that discipline. In addition, many applications of the geosciences also are directed toward controversial issues such as evolution, the age of the Earth, or climate change. Studying these issues may require students to confront prior beliefs and values. (Student beliefs are also of interest to biologists because of the debates surrounding evolution among some nonscientists—see Chapter 4 for a discussion.)

The Geoscience Affective Research Network has conducted research on the affective domain, with an emphasis on the attitudes and motivations of introductory students (McConnell and Kraft, 2011; van der Hoeven Kraft et al., 2011). In a composite study of introductory classes (7 colleges, 14 instructors, 800 students), student performance was most strongly correlated with scores on the self-efficacy section of the Motivated Strategies for Learning Questionnaire (Pintrich and DeGroot, 1990). In addition, students with low Geoscience Concept Inventory (GCI) scores or low incoming grade point average (GPA) but high self-efficacy earned the same grade as students with high GCI scores or high GPA and low self-efficacy (McConnell et al., 2009, 2010).

Geoscience education research also has documented differences in attitudes and self-efficacy among males and females. One study of 539 males and 607 females from 14 introductory classes at 7 institutions used the Motivated Strategies for Learning Questionnaire to measure pre- and

post-course attitudes (Vislova et al., 2010). On the pre-test, females reported lower self-efficacy and higher test anxiety than males. On the post-test, females reported lower likelihood of engaging in future geoscience courses, despite earning similar course grades as their male peers. In a different study, Liben, Kastens, and Christensen (2011) found that female undergraduates’ self-reported confidence in the quality of their performance on strike-and-dip and direction tasks was lower than their actual performance or their general spatial ability.

Learning in the field setting, which is an integral part of geoscience education, also has a strong affective component. Fieldwork can engage a wide spectrum of students in learning, in part because of the social interaction it entails (Boyle et al., 2007; Fuller et al., 2006; Maguire, 1998; Marques, Praia, and Kempa, 2003; Stokes and Boyle, 2009). Social aspects of learning in the field include heightened interpersonal interactions, building friendships, and reducing social barriers (Crompton and Sellar, 1981; Fuller et al., 2006; Fuller, Gaskin, and Scott, 2003; Kempa and Orion, 1996; Kern and Carpenter, 1984; Tal, 2001). Well-designed field experiences are seen as an effective means to recruit students to Earth science majors (Karabinos, Stoll, and Fox, 1992; Kern and Carpenter, 1984, 1986; Manner, 1995; McKenzie, Utgard, and Lisowski, 1986; Salter, 2001) and to introduce nontraditional students to the geosciences (Elkins, Elkins, and Hemmings, 2008; Gawel and Greengrove, 2005; Semken, 2005). In addition, some studies have shown that student attitudes toward the geosciences—and indeed, science in general—become increasingly positive as a result of fieldwork (Huntoon, Bluth, and Kennedy, 2001; Stokes and Boyle, 2009), perhaps because students view learning in the field as more interesting than learning in other contexts (Maguire, 1998; Stokes and Boyle, 2009).

Directions for Future Research on the Affective Domain

To date, much DBER has treated cognitive and affective outcomes as distinct “variables.” Future DBER on the affective domain should avoid this dichotomy and recognize the interdependence of affect and cognitive outcomes.

Instructors and researchers would benefit from a greater understanding of the attitudes and beliefs that are the most salient to learning science and engineering, including the role of cultural and social factors and potential differences among different groups of students (e.g., Brandriet et al., 2011). Cognitive science can help DBER scholars to clarify distinctions in theories of affect as they apply to student learning. Such distinctions are useful because they offer new ways to think about undergraduate science education. To that end, research on the affective dimensions of K-12 science learning (e.g., Simpson et al., 1994) also might be applied to DBER.

Systematic research on student motivation in the sciences and engineering is lacking. Future DBER studies might build on the extensive literature on motivation, especially expectancy and value orientation in cognitive science and in the broader higher education literature (e.g., Ambrose et al., 2010; Svinicki, 2004).

Research on a range of teaching strategies that engage the affective domain (e.g., collaborative study; teaching controversial issues; human impacts of course content) and that have the potential to change student attitudes and beliefs also would be useful (e.g., Middlecamp, 2008). In this realm, the interplay between faculty behavior and students’ affect merits further exploration: Do faculty responses to student reactions influence teaching strategies and, as a result, student learning? Instructors also have attitudes, beliefs, and values about students and how they learn. The complex interaction of these elements influences how and what instructors teach. Thus, the attitudes and beliefs of instructors themselves should be studied to understand their expectations for student learning in science and engineering—perhaps building on work that has already been conducted in physics (Geortzen, Sherr, and Elby, 2009, 2010; Henderson and Dancy, 2007; Henderson et al., 2004, 2007; Yerushalmi et al., 2007).

On a broader level, research on multiple dimensions of the affective domain would enhance the understanding of “what works” in the recruitment and retention of students into science and engineering majors, with longitudinal studies to determine which career paths students ultimately choose (e.g., Connor, 2009). As one example, in light of the larger percentage of undergraduate females majoring in biology compared to the physical sciences, studies that focus on the persistence of females in undergraduate majors and careers in the life sciences would be illuminating.