This report comes at a time when our nation and our species face profound challenges. Ensuring adequate food, water, energy, and mineral resources to support a growing human population competes with the need to control negative impacts such as pollution, global climate change, and loss of biodiversity. Understanding and addressing these challenges will require all the wisdom, ingenuity, and knowledge that humans can muster. These efforts will necessarily involve people with a diverse array of educational backgrounds and expertise, including scientists and engineers. Undergraduate education in science and engineering plays a crucial role in providing future generations with the knowledge and skills to address these challenges.
Undergraduate education in science and engineering in the United States serves multiple purposes, including providing all students with foundational knowledge and skills, motivating some students to complete degrees in science or engineering, and providing students who wish to pursue careers in science or engineering with the knowledge and skills required to be successful. Students who go on to have successful careers in science or engineering must be problem solvers, skilled in quantitative reasoning and modeling, effective at communication and cross-disciplinary collaboration, and cognizant of relationships between science and society (Brewer and Smith, 2011). Students who do not pursue these careers need to understand science and engineering to serve in their roles as citizens, consumers, and leaders of business and government who need to make wise science-informed
decisions in their personal and professional lives. Choosing healthcare for one’s children, buying a car, voting about land-use regulations, or retrofitting one’s house or business to be more earthquake-resistant are but a few of the decisions that today’s undergraduates may face. Their decisions on these and other issues will be based, in part, on their confidence in the methods of science and engineering and their understanding of the findings of science and engineering.
The importance of science and engineering in preparing the technical workforce and a science-literate citizenry has drawn increased attention to the quality of undergraduate science and engineering education and how it can be improved. There are persistent concerns that undergraduate science and engineering courses are not providing students with high-quality learning experiences, nor are they attracting and retaining students in science and engineering fields (President’s Council of Advisors on Science and Technology, 2012). Colleges and universities also face the challenge of serving an increasingly socially, economically, and ethnically diverse undergraduate population entering college classrooms directly from high school, after a military career or other life experiences, or from postsecondary educational experiences at another institution. Sustained attention to motivating, engaging and supporting the learning of all students who enter college science and engineering classrooms is an imperative.
Completion rates for all undergraduate students, including whites and Asians, are significantly lower in science, technology, engineering, and mathematics than in other disciplines. For example, Hispanic and African American students are as likely as white and Asian students to start college with an interest in science and engineering, but less likely to persist (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011). Specifically, underrepresented racial and ethnic groups comprised roughly 30 percent of the national population in 2006, but only 9 percent of the college-educated science and engineering workforce (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011).
Recognizing these challenges and the need for improvements in undergraduate science and engineering instruction, many institutions are working to identify effective approaches (Association of American Universities, 2011). Faculty members—alone or in collaboration with others—also are engaged in efforts to improve instruction, measure the efficacy of these teaching practices, and understand how students learn the concepts and practices that are fundamental to their disciplines (National Research Council, 2012; Project Kaleidoscope, 2011a, 2011b). Discipline-based education research (DBER)—by systematically investigating learning and teaching in science and engineering and providing a robust evidence base on which to base practice—is playing a critical role in these efforts.
DBER is grounded in the science and engineering disciplines and addresses questions of teaching and learning within those disciplines. The roots of this type of research can be traced to the early 1900s, but DBER emerged more prominently in the 1980s and 1990s (see Chapter 2 for a detailed discussion of the history). DBER can be defined both by the focus of the research and by the researchers who conduct it. In the following sections, we define DBER and who conducts it. This definition guided the committee in identifying the relevant bodies of research, and examining how to advance DBER and strengthen its impact.
DBER investigates learning and teaching in a discipline using a range of methods with deep grounding in the discipline’s priorities, worldview, knowledge, and practices. It is informed by and complementary to more general research on human learning and cognition. Although the focus of this report is learning and teaching in undergraduate institutions, DBER scholars have also examined learning and teaching in the K-12 context, particularly at the high school level.
The long-term goals of DBER are to
• understand how people learn the concepts, practices, and ways of thinking of science and engineering;
• understand the nature and development of expertise in a discipline;
• help to identify and measure appropriate learning objectives and instructional approaches that advance students toward those objectives;
• contribute to the knowledge base in a way that can guide the translation of DBER findings to classroom practice; and
• identify approaches to make science and engineering education broad and inclusive.
Thus the research has the practical goal of improving science and engineering education for all students.
Achieving these goals requires that DBER studies be grounded in expert knowledge of the discipline and the challenges for learning, teaching, and professional thinking within that discipline. All fields of DBER share a common focus on issues that are important for understanding and fostering student learning of the most crucial topics, techniques, procedures, and ways of knowing that define the particular discipline. This focus includes investigating student learning within that discipline per se, along with issues affecting enrollment and retention of students in classes and the adoption of best practices by instructors.
To progress toward these goals, DBER relies on several types of knowledge from outside the science or engineering disciplines: (1) the nature of human thinking and learning as they relate to the discipline of interest, (2) factors that affect student motivation to initially engage in and then to persist in the learning necessary to understand the discipline and apply findings of the discipline, and (3) research methods appropriate for investigating human thinking, motivation, and learning. By its very nature, DBER is an interdisciplinary field of study. This means that discipline-based education researchers must bridge the gaps in language, background, and ways of thinking between their home discipline and several areas of research on learning and teaching.
DBER embraces the full spectrum of research approaches for understanding human learning, cognition, and affect. Its research methods are drawn not only from the home discipline (e.g., chemistry or engineering) but also from a variety of other fields such as experimental and social psychology, education, and anthropology. Discipline-based education researchers use experimental, correlational, ethnographic, and exploratory designs, and to collect quantitative and qualitative evidence.
As with other areas of research, DBER includes a range of studies from fundamental to applied, and from theoretical to empirical. A useful framework for thinking about the range of questions that can be addressed in research was proposed by David Stokes in his book Pasteur’s Quadrant (Stokes, 1997; see Figure 1-1). Using this framework, some DBER studies might be categorized as pure basic research, driven by a quest for fundamental understanding that is connected to the practical goal of improved education, but with no immediate application. Basic research in DBER might include research on the cognitive underpinnings of groups of students’ misconceptions (see Chapter 4). Pure applied DBER, on the other hand, might include studies of the effectiveness of collaborative problem solving or the use of technology for improving classroom instruction (see Chapter 6). Many DBER studies fall in the “use-inspired basic research” category of Pasteur’s quadrant. For example, researchers have investigated students’ competence at authentic tasks within the discipline, such as translating between different representations of a molecule in chemistry (Cooper et al., 2010) or using diagrammatic representations to reason about evolutionary relationships among taxa in biology (Novick and Catley, in press).
DBER has sometimes been characterized by the training and professional positions of the contributing scholars rather than solely in terms of substantive focus. As discussed in Chapter 2, DBER scholars have a diverse array of backgrounds. A number of DBER scholars have a Ph.D. in a science or engineering discipline and additional training or experience in education research. Many of these scholars hold positions in natural science departments. Yet other scholars also contribute to DBER through collaborations
FIGURE 1-1 Pasteur’s quadrant showing basic and applied DBER.
that bring together individuals with expert knowledge in science or engineering and those with expertise in education research or research on learning and teaching. The discussions in Chapter 2 of this report focus more heavily on individual scholars who have dual training in the natural sciences or engineering and experience or training in education research.
Relation of DBER to Other Research Areas
Another way to define DBER is to consider it in relation to other fields that study learning and teaching (Bodner, 2011). In this section we consider DBER alongside three related fields: the scholarship of teaching and learning, educational psychology, and cognitive science. The category of DBER overlaps each of these other categories, but stands distinct from all of them. As noted, DBER is distinguished by an empirical approach to investigating learning and teaching that is informed by an expert understanding of disciplinary knowledge and practice. In making these distinctions we focus on the research perspective of scholars rather than on their organizational location within the institution.
Scholarship of Teaching and Learning
The activities that have come to be known as the scholarship of teaching and learning (SoTL) have developed in parallel with DBER. SoTL
emerged from Scholarship Reconsidered (Boyer, 1990), which emphasized the need for classroom research and sparked conversations about teaching among colleagues at the university level. In 1997, the Carnegie Foundation for the Advancement of Teaching established the Carnegie Academy for the Scholarship of Teaching and Learning (CASTL). CASTL supported faculty fellows in developing classroom research skills, and several reports have been published on the positive impact this work has had on the fellows and their institutions (Hatch, 2005; Huber and Hutchings, 2005; Hutchings, Huber, and Ciccone, 2011). SoTL has focused on engaging faculty across disciplinary boundaries, including the humanities, social sciences, and natural sciences with their wide-ranging epistemologies and standards of evidence. SoTL emphasizes developing reflective practice and using classroom-based evidence. Some faculty engage in SoTL to inform their own work in the classroom, and some have gone on to become deeply engaged in more general education research. Thus, the boundaries between SoTL and DBER are blurred and some researchers belong to both the SoTL and DBER communities.
While DBER scholars gravitate to discipline-specific journals, SoTL researchers mostly publish in broad journals on teaching and learning such as the Journal of College Student Development or through the International Journal for the Scholarship of Teaching and Learning (IJSoTL). IJSoTL states that “SoTL is a key way to improve teaching effectiveness, student learning outcomes, and the continuous transformation of academic cultures and communities. …[C]ollege and university teaching is seen as a serious intellectual activity that can be evidence and outcome based.”1
Educational Psychology Research
Educational psychologists investigate learning in students of all ages, including undergraduates (e.g., Mayer, 2011). In contrast to DBER, research in educational psychology typically focuses on general principles of learning, and the content domain under investigation is often secondary. For example, science content might be used in a study of how students learn from diagrams, but the particular science content per se is not necessarily the object of investigation. DBER, on the other hand, is concerned with undergraduate students’ learning of a particular aspect of the scientific discipline. As a consequence, the science content that is incorporated in research in educational psychology is often not equivalent in depth or breadth to that considered in DBER.
1This statement appears on the IJSoTL website: http://academics.georgiasouthern.edu/ijsotl/index.htm [accessed March 30, 2012].
Cognitive Science Research
Cognitive science is a multidisciplinary field dedicated to understanding the nature of the human mind and other intelligent systems, primarily from a basic research perspective. Theoretical investigations generally focus on issues of knowledge representation, cognitive processes, and (in humans) brain theory (Friedenberg and Silverman, 2006). Computational modeling of human thinking is a strong focus in the field. Cognitive science researchers value both novel experimental tasks (e.g., abstract puzzles) and those drawn from or modeled on real-world tasks (e.g., authentic science materials such as physics problems) (Friedenberg and Silverman, 2006).
The core disciplines of cognitive science are artificial intelligence, linguistics, anthropology, psychology, neuroscience, philosophy, and education. Cognitive Science, the flagship journal of the Cognitive Science Society, publishes research on intelligent systems that is multidisciplinary across two or more of these named disciplines. The journal historically has not published many articles related to education and would consider many DBER studies—such as solving problems from undergraduate physics and reasoning from diagrams in evolutionary biology—as belonging to the field of cognitive psychology, which includes the study of problem solving and diagrammatic reasoning in undergraduates. Although such studies are interdisciplinary between cognitive psychology and another science discipline, the second discipline is not considered to be part of cognitive science.
It can be difficult to determine whether studies of complex, high-level cognitive tasks such as problem solving and diagrammatic reasoning, which are related to DBER studies, belong to cognitive psychology or cognitive science. In many cases, both fields would reasonably lay claim to the research. In this report, we have opted to classify this research as cognitive science because (a) it generally is directed to and is cited by a multidisciplinary audience and (b) at the time of this report such research is much more likely to be presented at the annual meeting of the Cognitive Science Society than at the annual meeting of the Psychonomic Society (the home of cognitive psychology). In the minority of cases in which we refer to supporting research as coming from cognitive psychology, it is because that research is directed toward a cognitive psychology audience.
Finally, evaluation of educational interventions and programs is related to the work in DBER that measures the effectiveness of particular instructional strategies, course structures, or programs of study. Many educational evaluators use sophisticated methods for studying the implementation and impact of interventions in context, as well as efforts to take those
interventions to scale. These methods include large-scale, mixed methods designs and a wide variety of quasi-experimental designs (see, for example, the journal Educational Evaluation and Policy Analysis). In contrast to DBER, however, these studies do not typically closely examine the nature of the science or engineering discipline being learned.
Increased calls to improve instructional practices in the natural sciences intersect with growing interest in DBER as an important area of scholarship, generating new opportunities to apply this research. Recognizing this important juncture, the National Science Foundation (NSF) requested that the National Research Council (NRC) convene a committee to conduct a synthesis study on the status, contributions, and future directions of DBER across undergraduate physics, biology, the geosciences, and chemistry. In response to this request, the NRC convened the 15-member Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research to answer questions that are essential to advancing DBER and broadening its impact on science teaching and learning at the undergraduate level. Over a 13-month period in 2010-2011, the committee explored those questions. This report synthesizes the committee’s findings.
Charge to the Committee
The three broad elements of the committee’s charge were to
1. synthesize empirical research on undergraduate teaching and learning in the sciences,
2. examine the extent to which this research currently influences undergraduate science instruction, and
3. describe the intellectual and material resources that are required to further develop DBER.
More specifically, the committee was charged with addressing the following questions:
1. What is the state of DBER scholarship as a whole and what currently is being done across each of the natural sciences? Are there research synergies across disciplines?
2. What findings are robust across disciplines?
3. What discipline-specific instructional practices are most clearly linked to increased performance across student groups (especially low socioeconomic status, minority, and female students)?
4. To what extent and how has DBER informed teaching and learning in the various disciplines?
5. What factors are influencing differences in the state of research and its impact in the various disciplines?
6. What are the resources, incentives, and conditions needed to advance this research?
7. What resources and incentives are needed to ensure that teaching and learning in the various science disciplines is informed by DBER?
8. What questions should DBER scholars prioritize in the next generation of research?
Scope of the Study
The original charge to the committee specified that the committee consider undergraduate physics, biological sciences, geosciences, and chemistry. As work on the study began, two changes were made to the disciplines that were included. First, engineering was included because early discussions suggested engineering education research was robust and the engineering education research community was establishing an infrastructure for research. Second, a consideration of the literature in physics education research revealed that astronomy education research differed in terms of timeline and trajectory and merited inclusion as a discipline separate from physics. Thus, although astronomy typically is linked with physics at the undergraduate level, the committee decided to treat them separately for this study because astronomy education research and physics education research are at different points in their development, emphasize different methodological approaches, and involve distinct (though overlapping) student populations.
It is important to note that DBER can be a field of study within any academic discipline, in the sciences and beyond. However, because this study focuses on education research in a select set of science and engineering disciplines, throughout this report we use the term DBER to refer only to these disciplines.
In addressing the specific questions in the charge, the committee agreed on the following approaches. In question 1, we interpreted “scholarship” to encompass the community of DBER scholars in the sciences and engineering and the body of literature that those researchers generate. Determining the state of scholarship includes examining the types of questions that DBER scholars ask or the problems they study, how they study them, what counts as evidence, and key findings. It also includes examining degree programs, postdoctoral and faculty positions, conferences, professional societies, journals, and other indicators that reflect the development and
identity of DBER in a discipline and its practitioners within the academic culture. And finally, determining the state of scholarship also includes a historical sense of how the research community has developed over time.
In discussing question 2, we determined that it would be necessary to summarize the findings within each discipline before analyzing the findings across disciplines. Concerning question 3, we also agreed that, depending on how the research was disaggregated, it would be useful to consider student characteristics other than socioeconomic status, minority status, and gender. However, overall, our synthesis revealed that relatively little DBER has been designed to examine group differences.
Finally, noting that question 7 creates a sense of direction for the field, we focused on ensuring the widespread use of research-based practices. We also recognized the importance of identifying whom the resources should target, and of considering future science and engineering faculty and future DBER scholars—including graduate students—when answering this question.
Approach and Sources of Evidence
The committee carried out its charge through an iterative process of gathering information, deliberating, identifying gaps and questions, gathering further information to fill these gaps, and holding further discussions. In the search for relevant information we held four public fact-finding meetings, reviewed published reports and unpublished research, and commissioned experts to prepare and present papers. During a fifth, private meeting, we intensely analyzed and discussed our findings and conclusions.
Our approach began with an examination of the research on teaching and learning at the undergraduate level in each discipline of the study charge, with a focus on DBER. To this end, we commissioned literature reviews of DBER in astronomy, biology, chemistry, engineering, the geosciences, and physics. Equipped with this foundational understanding, we addressed issues that cut across disciplines by considering some general principles of teaching and learning from cognitive science and educational psychology. Finally, we examined a broader set of factors that influence faculty, departmental, and institutional change, and considered a set of strategies designed to promote research-based instructional and institutional change in undergraduate science instruction.
We found a limited number of studies that identify the extent to which DBER has informed instruction (study question 4). Thus, to address this question we also sought to identify the factors that influence faculty decisions about instruction, primarily through a commissioned paper that drew
on the broader research about individual and institutional transformation in postsecondary education.
When considering the issues of advancing DBER (study question 6), we found that the research base was similarly sparse. Therefore, to address this issue, we commissioned papers examining the history of DBER in each of the disciplines in the study charge. Those papers helped to define DBER, designate milestones associated with the development of emerging fields, and identify relevant journals for education research in each discipline. They also enabled a cross-cutting comparison of the development of DBER. Another aspect of advancing DBER relates to preparing and placing future faculty members. However, no systemically collected data existed on graduate or postdoctoral programs or career pathways for discipline-based education researchers, so we also commissioned a paper that would allow us to explore the role of postdoctoral programs in preparing DBER faculty.
Although the committee considered information from a variety of sources during the course of this study, the conclusions we have drawn about the research on teaching and learning within each discipline give the most weight to research published in peer reviewed journals and books. Following an earlier National Research Council report (2002), we adopted the view that “A wide variety of legitimate scientific designs are available for education research. They range from randomized experiments…to in-depth ethnographic case studies…to neurocognitive investigations…using emission tomography brain imaging” (p. 6). Reflecting this view, we developed a set of categories to characterize the strength of the conclusions we could draw from the available evidence (see Box 1-1).
The bulk of this report (Chapters 4 through 7) is dedicated to a synthesis of research on undergraduate teaching and learning in physics, chemistry, engineering, biology, the geosciences, and astronomy. With the synthesis we have attempted to strike a balance between preserving characteristics or challenges that are tied to just one or two disciplines (e.g., students’ difficulties understanding deep time or that matter is made of discrete particles), and identifying general themes in science and engineering learning that cut across most disciplines (e.g., students’ difficulties solving problems and interpreting visual and mathematical representations).
The report also discusses the emergence and current state of the individual fields of DBER (Chapter 2); analyzes the use of DBER findings among faculty members (Chapter 8); and provides a roadmap for the future of DBER by proposing a research agenda and identifying actions that
Characterizing the Strength of Conclusions
Supported by the Evidence Base
A Limited Level of Evidence
• Few peer-reviewed studies of limited scope with some convergence of findings or convergence with nonpeer-reviewed literature or with practitioner wisdom.
A Moderate Level of Evidence
• A well-designed study of appropriate scope that has been replicated by at least one other similar study. Often such evidence will include both quantitative and qualitative data OR
• A few large-scale studies (e.g., across multiple courses, departments, or institutions) with similar results OR
• A moderate number of smaller-scale studies (e.g., in a single course or section) with general convergence but possibly with contradictory results. If the results are contradictory, more weight might be given to studies that reflect methodological advances or a more current understanding of teaching and learning, or are conducted in more modern learning environments.
• Numerous well-designed qualitative and/or quantitative studies, with high convergence of findings.
postsecondary institutions, disciplinary departments, journal editors, professional societies, and funding agencies can take to support and advance DBER (Chapter 9). Because the fields of DBER have been and will continue to be an important way of improving science and engineering education, our hope is that the findings and recommendations in this report invite and assist postsecondary institutions to increase interest and research activity in DBER.