1

Introduction

I hear and I forget.
I see and I remember.
I do and I understand.
—Confucius

Undergraduate research experiences (UREs) are a meaningful opportunity for undergraduates to learn about the work and perspectives of science, technology, engineering, and mathematics (STEM) researchers. Many faculty members and other scientists recall that their own research experiences as an undergraduate were pivotal to their career success. Today, there are various forms of UREs available to students at a wide variety of institutions. However, while many students report that they enjoy the experiences and learn a lot from them (Harsh et al., 2011), there has been little analysis of which types of UREs might best serve students at different academic institutions and with diverse career aspirations.

Attention to UREs has grown significantly in the last few years as policy actions have promoted their expansion. In 1998, the Boyer Commission on Educating Undergraduates in the Research University considered a capstone research experience as an essential element in the reinvention of undergraduate education (Miller, 2013). However, the most prominent call was from a committee of the President’s Council of Advisors on Science and Technology (PCAST). The second recommendation of its 2012 report, Engage to Excel: Producing One Million Additional College Graduates with Degrees in STEM, is to “advocate and provide support for replacing standard laboratory courses with discovery-based research courses”

(President’s Council of Advisors on Science and Technology, 2012, p. 25). Specifically, the report discussed how undergraduates working on faculty projects can allow students to experience real discovery and innovation and to be inspired by STEM subjects. The PCAST report recommended that all relevant federal agencies examine their programs and make changes in an effort to decrease any policy or practice that creates barriers to early engagement of students in research. It also called on the agencies to “encourage projects that establish collaborations between research universities and community colleges or other institutions that do not have research programs” (President’s Council of Advisors on Science and Technology, 2012, p. v).

As efforts have been made to expand opportunities for UREs, many questions have arisen. This report provides a comprehensive overview of and insights about the current and rapidly evolving types of UREs, in an effort to improve understanding of the complexity of UREs in terms of their content, their surrounding context, the diversity of the student participants (including the educational pathways of those students), and the opportunities for learning provided by a research experience. The report discusses the various types of UREs and the crosscutting characteristics that most UREs exhibit. The type and level of evidence available on the efficacy of UREs and how the evidence base might be strengthened are examined. The way that UREs currently fit with the educational “ecosystem” in higher education is discussed, as well as the problems that designers and implementers of such programs often encounter within the current structures for governance and funding of higher education. Recommendations are presented on how such barriers might be overcome in the future by rethinking how academic departments, institutions, and funding agencies might support UREs and how UREs could be assessed and evaluated more effectively and comprehensively.

As noted in Chapter 2 and discussed throughout the report, the authoring committee has examined many varieties of UREs. Students engage in research during capstone experiences, co-ops,¹ and internships; as part of community engagement projects; and as part of bridge programs to assist with transitions between high school and college or between college and graduate school. Traditionally, two general categories of UREs are most often discussed and analyzed in the literature: the apprentice model and the course-based undergraduate research experience (CURE). Other variations of UREs exist and will also be discussed throughout this report. In an apprentice model experience, one or a small number of students work with

___________________

¹ Undergraduate co-ops are full-time paid educational experiences designed to provide an opportunity for students to apply knowledge and skills from their coursework while working in a professional setting such as in industry. They are particularly common for students in engineering programs.

an individual or small group of established scientists, technologists, engineers, or mathematicians on a research or design problem, typically outside of the classroom. Individual students may play different roles depending on the wishes and needs of the sponsoring individual and the background of the student. In a CURE, small to large groups of students enrolled in a formal course or sequence of courses participate in a discovery-based project designed to engage them in the use of STEM practices, discovery, collaboration, iteration, and pursuit of broadly relevant or important work (Auchincloss et al., 2014; Brownell and Kloser, 2015; Litzinger et al., 2011). These course experiences may be offered over part of an academic term/semester, for full semesters, or for multiple semesters in a sequence of courses. Some CUREs are developed by individuals, while others are part of large national consortia. CUREs appear to have increased in popularity in recent years and are the subject of several new studies (discussed in Chapter 4).

HISTORY OF URES

Undergraduate research is sometimes thought to be a relatively recent development in higher education. However, faculty-mentored, apprentice-based undergraduate research has a long and rich history, dating back more than 200 years to Wilhelm von Humboldt (Zupanc, 2012). Many U.S. institutions of higher education adopted the “Humboldtian Ideal” of an unceasing process of inquiry that unified teaching and research (Kinkead, 2012). In keeping with this ideal, the National Science Foundation launched a program supporting undergraduate research participation in 1958. The program was canceled in 1981 but relaunched in its current form as Research Experiences for Undergraduates in 1987 (Bennett, 2015).

The Council on Undergraduate Research (CUR), founded in 1978, is a national organization that helps connect faculty and college administrators across institutions engaged with undergraduate research. Many of the first participants involved with CUR were faculty from liberal arts colleges who saw undergraduate research as good pedagogy that could expand horizons for the students while furthering basic or applied research being undertaken by faculty members themselves.

Many different types of academic institutions have now explored and established strong undergraduate research programs; the types of offerings have varied depending on the academic environment, the research infrastructure available, and the culture of the particular institution or discipline. With new avenues of funding through public and private foundations to support these efforts, programs have been replicated, modified, and expanded. As discussed in Chapter 4, the reported benefits of UREs include increasing the number of students who choose to major in STEM

disciplines, continue throughout the program, and subsequently graduate with a STEM degree, in addition to helping students develop an interest and identity as a STEM researcher. They may also encourage more faculty members and other academic personnel to engage in some kind of basic or applied research.

The committee chose to take an inclusive view of the individual disciplines included in the definition of STEM by considering the social sciences, natural sciences, engineering, and mathematics. We attempted to find examples and identify literature from all of these fields that would provide evidence to inform the discussion of UREs. However, the relevant literature is limited for many of these specific disciplines, so examples and references cited in the report are not evenly distributed across all intended STEM disciplines. In particular, the discipline of engineering received attention in the committee’s discussions and review of the literature. Engineering has a long history of capstone courses and other opportunities for undergraduate students to engage in work done by engineers (Rowles et al., 2004). However, much of the work on UREs has focused on examining the ways that students learn science. Therefore, throughout the report where those studies are discussed, the language may appear to be ignoring the engineering perspective.

As research has become a more institutionalized component of undergraduate education, faculty members have created links with colleagues at other campuses. In addition to CUR, other national groups that have joined the conversation on UREs include the National Conferences on Undergraduate Research, which started in 1987 and merged with CUR in 2010, and Project Kaleidoscope (PKAL). PKAL began in 1969 under the auspices of the Independent Colleges Office, and is now operating as a component of the American Association of Colleges and Universities (AAC&U). Founded on the principle of “discovering what works,” PKAL has focused on catalyzing professional development for STEM faculty in ways that will enhance their success as scholar-educators who can then promote undergraduate student learning through hands-on approaches and through research in the classroom, laboratory, engineering design environment, or field.

Many professional and disciplinary societies have initiatives to fund student research or engage faculty in discussions about improving undergraduate research. For example, the Mathematical Association of America works to provide avenues for undergraduate students to engage in research and hosts a special interest group devoted to research in undergraduate mathematics education. Moreover, the American Society for Engineering provides access to programs that sponsor undergraduate research. Individually and collectively, these national organizations have played a significant catalytic role by bringing a strong and professional framework and culture

to the affirmation and expansion of undergraduate research in all types of institutions across the nation.

Although initial efforts emphasized the classic apprenticeship model, often based on a summer experience in the lab, recent years have seen an expansion of CUREs in which undergraduate students are engaged in research as part of a formal course. The creation of a CURE is often driven by a desire to provide research experiences for a larger group of students than can be accommodated in a faculty member’s research environment (the member’s laboratory or field site). Alternatively, it can be part of the fieldwork for a given course. Some institutions see CUREs as a way to further engage students and encourage them to pursue and continue their education in a particular major or to continue their studies at that institution (Rodenbusch et al., 2016). These courses often have been developed by individual faculty members and are sometimes based on the faculty member’s own research. Some CUREs have served as models and have been adapted or replicated at other institutions, including two-year colleges—for example, the Center for Authentic Science Practice in Education (Weaver et al., 2006). The Course-Based Undergraduate Research Experiences Network was initiated in 2012 to help faculty address challenges inherent to integrating research experiences into undergraduate courses in the biological sciences.²

CONTEXT OF THE STUDY

In the past decade, discussions of undergraduate research have intensified among various national and regional groups across campuses, in response to the need to better prepare an increasingly diverse student population to face 21st century challenges (Auchincloss et al., 2014; Bangera and Brownell, 2014; Brownell et al., 2015; Litzinger et al., 2011). Conversations have centered not only on student outcomes such as fostering students’ learning and other psychosocial factors (e.g., engagement, belonging, interest in research) and how these may influence retention, but also on the costs inherent in expanding the availability of UREs. Recent years have also seen an increase in the numbers of students from underrepresented groups who are enrolled in undergraduate courses and programs (Bangera and Brownell, 2014; National Academies of Sciences, Engineering, and Medicine, 2016), thereby increasing the need to ensure that students from all groups are considered in the design of UREs. More discussion of student demographics and inclusion follows later in this section.

Multiple national reports have both stimulated and captured these discussions and called for expanding UREs. AAC&U reports that address

___________________

² For more information, see http://www.lifescied.org/content/13/1/29.full [November 2016].

undergraduate research include College Learning for the New Global Century; High Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter; and The LEAP Vision for Learning: Outcomes, Practices, Impact, and Employers’ Views (respectively, Association of American Colleges and Universities, 2007, 2008, 2011). These reports present UREs as one of the high-impact practices that can dramatically influence undergraduate education. Another report, Vision and Change in Undergraduate Biology Education (American Association for the Advancement of Science, 2011), called for integrating UREs into curricula. Follow-on activities prompted by that report have included formation of the PULSE (Partnership for Undergraduate Life Science Education) online community, conferences, and sharing of resources.

As discussed earlier in this introduction, the PCAST report Engage to Excel: Producing One Million Additional College Graduates with Degrees in STEM (President’s Council of Advisors on Science and Technology, 2012) specifically highlighted the potential of UREs to improve the nation’s undergraduate STEM education during the first 2 years of college and recommended their expansion so that eventually all undergraduates are afforded this kind of learning opportunity. The report recommended that current and future STEM faculty learn about and incorporate effective teaching methods into their STEM courses, particularly including the opportunity for students to generate or apply knowledge through research. The National Science Foundation has taken the lead in many aspects of implementing the PCAST report because enhancing the quality of STEM education is a high priority for that agency. However, questions remain about how best to achieve this goal, and some of those questions motivated this study.

The emphasis on UREs is part of a larger effort to improve and broaden participation in undergraduate STEM education, which has been the focus of numerous efforts and projects by a range of groups. In addition to the organizations mentioned above (American Association for the Advancement of Science, AAC&U, PCAST), work has been done by the American Association of Universities through its Undergraduate STEM Initiative, including the Framework for Systemic Change. Funding from the National Science Foundation, the National Institutes of Health, Howard Hughes Medical Institute, and numerous other funding entities has also driven efforts in undergraduate STEM education that have increased undergraduate opportunities to participate in research.

Consensus studies and other activities of the National Academies of Sciences, Engineering, and Medicine (the National Academies) have addressed the topic in multiple ways over the past 6 years. The 2011 report Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads (National Research Council, 2011) examined the role of diversity in the STEM workforce and called for efforts

to increase demand for and access to postsecondary STEM education and technical training by historically underrepresented students. Community Colleges in the Evolving STEM Education Landscape (National Research Council, 2012a) reported on a summit that addressed the relationships between community colleges and four-year institutions, with a focus on partnerships and articulation processes that can facilitate student success in STEM. It also considered how to expand participation of students from historically underrepresented populations in undergraduate STEM, as well as how subjects such as mathematics can serve as gateways or barriers to college completion. The 2012 report Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering (National Research Council, 2012b) studied the new field that combines knowledge of teaching and learning with deep knowledge of discipline-specific science content. It analyzed empirical research on undergraduate teaching and learning and the extent to which the resulting evidence influences undergraduate instruction. Reaching Students: What Research Says About Effective Instruction in Undergraduate Science and Engineering (National Research Council, 2015) presented information from Discipline-Based Education Research and additional examples in a format designed to be more practical for faculty and instructors.

The recent report Barriers and Opportunities for 2-Year and 4-Year STEM Degrees (National Academies of Sciences, Engineering, and Medicine, 2016) addressed the changing demographics of college students and the multiple pathways they take in their education. It also described the challenges of needing to take developmental courses before being able to enroll in credit-earning courses, as well as the complications of transferring credits. This report includes recommendations about ways policy makers and institutions can learn more about students’ varied pathways to better support them in reaching their goals and completing their degrees.

The National Academies publication most closely related to this project is the report of a convocation on Integrating Discovery-based Research into the Undergraduate Curriculum, convened to explore aspects of recommendation #2 of the 2012 PCAST report. The convocation report presents efforts to improve instruction through engaging students in research, with a focus on the opportunities and challenges of CUREs (National Academies of Sciences, Engineering, and Medicine, 2015).

As mentioned above, another important aspect of the study context is the changing demographics of today’s undergraduate students, including more historically underrepresented students, first generation college students,³ and nontraditional (e.g., part-time, delayed start, financially independent, and caregiver) students. Many private and publicly funded

___________________

³ First generation students are those who are the first in their families to attend college.

programs have focused specifically on providing UREs to historically underrepresented minorities, women, and first generation students because members of these groups are less likely to persist in STEM fields once they enter their undergraduate education and CUREs are proposed as a mechanism for broadening access to research for members of underrepresented groups (Bangera and Brownell, 2014). Although students from historically underrepresented groups express greater interest in pursuing a STEM degree now compared with 30 years ago, there has not been a corresponding increase in the overall completion rates in STEM degrees, nor has there been a decrease in the notable disparities among historically underrepresented groups (Eagan et al., 2013, 2014; Estrada et al., 2016; Hurtado et al., 2012; National Science Foundation, 2014). Data on different participation rates of various groups are presented in Appendix A.

Overall, enrollment by women in STEM majors has increased in recent years; however, this change is not consistent across all disciplines. Of students pursuing STEM degrees in 1971, 62 percent were men and 38 percent were women. In 2012, 48 percent of those pursuing STEM degrees were men and 52 percent were women (Eagan et al., 2014). However, within specific STEM disciplines, there is still evidence of a gender gap. Men are more likely than women to pursue a degree in engineering (79 percent men versus 21 percent women) or in math or computer science (75 percent men versus 25 percent women). However, the pattern is reversed for the biological sciences (60 percent women versus 40 percent men) and the social sciences (70 percent women versus 30 percent men). In addition, even in fields with roughly equal numbers of women, concerns still remain with respect to discrimination in the selection of students to participate in UREs, discrimination during a URE, and the potential for sexual harassment (Clancy et al., 2014; Eddy and Brownell, 2016; Moss-Racusin et al., 2012).

Faculty and administrators at community colleges face additional or enhanced challenges compared to other types of institutions, particularly with respect to resources, including lower levels of funding for research and lack of appropriate facilities. This can make it difficult to implement and support UREs. In addition, the student populations at community colleges are generally quite diverse. Community colleges are more likely to serve first generation students and students who are slightly older, have families, or are working. Notably, students from Hispanic/Latino backgrounds were more likely to be enrolled in community college compared to students from other racial/ethnic backgrounds. Moreover, students in community colleges were on average less prepared than those in four-year institutions, often requiring some form of developmental education in their first year, especially in mathematics (Van Noy and Zeidenberg, 2014).

Although data exist on the demographics of students attending colleges and universities today, there is very limited information on how many stu-

dents from historically underrepresented groups or at particular types of institutions participate in UREs. That is, there is currently no standardized metric used at educational institutions to track the type and duration of undergraduate research engagement for students. Programs that receive federal funding collect and maintain some information regarding the students involved in such UREs, but this represents only a fraction of the undergraduate research programs available to students. Thus, these data provide an incomplete picture of the demographics of students participating in UREs.

CHARGE TO THE COMMITTEE

In response to a request from the National Science Foundation, the National Academies of Sciences, Engineering, and Medicine convened a 16-member expert committee to evaluate the state of knowledge on the current broad array of UREs. The committee charge was to recommend ways to design, evaluate, and study UREs, based on a review of the available research evidence and taking into account the needs and resources of colleges and universities (see complete Statement of Task in Box 1-1). Membership on the committee included faculty from various STEM disciplines who have been involved in research opportunities for undergraduates at two- and four-year institutions, experts in education research and policy, and those with experience in higher education leadership.

Interpreting the Charge

The committee met five times over a 10-month period in 2015 and 2016 to gather information and explore the range of issues associated with UREs. In addition to reviewing published materials pertaining to the committee’s charge, committee members heard from many experts and commissioned three papers during the information-gathering phase of the committee process (see below).

The committee spent a great deal of time discussing the charge and the best ways to respond to its call. We gathered evidence from literature reviews and presentations, by contacting faculty and administrators at numerous colleges and universities, and by sharing the members’ own experiences and expertise. The conversations with faculty and administrators provided information about the range of UREs offered, examples of how students and faculty are compensated, and various examples of institutional support mechanisms, among other topics. The literature reviews searched for information on UREs, undergraduate research opportunities, research experiences for undergraduates, CUREs, mentors, apprentices, advisors, identity, and persistence. We also searched for evaluations of URE programs. Although some evaluations were found in the literature, the committee

could not determine a way to systematically examine the program evaluations that have been prepared. The National Science Foundation and other funders require grant recipients to submit evaluation data in their annual reports, but that information is not currently aggregated and published.⁴

Over the course of this study, members of the committee benefited from discussion and presentations by the many individuals who participated in our three fact-finding meetings. At the first meeting, Jo Handelsman (Office of Science and Technology Policy), David Asai (Howard Hughes Medical Institute), and Beth Ambos (CUR) described different perspectives on UREs, existing work to build upon, sources to evaluate, and the changing land-

___________________

⁴ Personal knowledge of Janet Branchaw, member of the Committee on Strengthening Research Experiences for Undergraduate STEM Students.

scape. During the second meeting, the committee heard expert testimony on institutional-level data gathering and analysis from Bethany Usher (George Mason University), Stephany Hazel (George Mason University), and Marco Molinaro (University of California, Davis). Mitch Malachowski (University of San Diego) and Paul Hernandez (West Virginia University) provided information on institutional change. Erin Dolan (The University of Texas at Austin) and Tuajuanda Jordan (St. Mary’s College of Maryland) provided a commentary on the presentations and helped provide additional perspectives on the day’s topics.

The third meeting involved some panel discussions. The first panel included Michael Wolf (Rice University), Suzanne Weekes (Worcester Polytechnic University), and Michael Dorff (Brigham Young University), who discussed UREs in mathematical sciences. The second panel on faculty perspectives on undergraduate research was presented by Sandra Laursen (University of Colorado Boulder), Tracy Johnson (University of California, Los Angeles), and Ariel Anbar (Arizona State University). The third panel involved Lisa Benson (Clemson University) and Ann Saterbak (Rice University), who discussed engineering perspectives on undergraduate research.

The committee commissioned three papers to provide in-depth input on specific topics. Erin Dolan (The University of Texas at Austin) authored an analysis of CUREs. Christine Pfund (University of Wisconsin–Madison) wrote a summary of current thinking about mentorship and how it relates to UREs. Linda Blockus (University of Missouri) prepared a document on issues related to co-curricular research experiences. In addition, the committee built upon the information and experience of the Convocation on Integrating Discovery-based Research into the Undergraduate Curriculum, described above.

Discussions about the evidence engaged the full committee, and members shared their expertise in designing and running URE programs; training other faculty to run URE programs and to mentor students; evaluating URE programs; and designing and conducting research on learning, STEM education, higher education, and learning in UREs. The committee’s discussions frequently grappled with contrasts between the large body of positive descriptive evidence, the lack of extensive causal evidence, the impassioned calls for expansion of UREs, and the numerous creative UREs that have already been established. We worked to reconcile the perspectives in order to provide guidance to the field. This report synthesizes the committee’s findings based on the evidence reviewed and the expertise of its members.

Request to Define Authentic Research

The committee’s charge (see Box 1-1) includes providing a definition for authentic undergraduate research experiences. While discussing which

characteristics were appropriate and necessary to include in a definition of UREs, the committee grappled with the inclusion of the term “authentic” in their charge. The committee sought out examples in which others had used the term “authentic” to inform their discussion. The term “authentic” in the context of STEM education is used in the PCAST report Engage to Excel and in multiple other documents. “Authentic” has also been used by researchers, notably in previous work on education in STEM fields (e.g., Spell et al., 2014) and on education generally (Newmann, 1996). In addition, a framework is forthcoming from the federal government that defines and explains an authentic STEM experience. In this framework, a URE would be one example of an authentic STEM experience, but the federal approach encompasses more than undergraduates and more than research. Its definition states that “an Authentic STEM Experience is an experience inside or outside of school designed to engage learners directly or indirectly with practitioners and in developmentally-appropriate practices from the STEM disciplines that promote real world understanding.”⁵ It lists the characteristics of an “Authentic STEM Experience” as an active-doing, collaborative, meet learners where they are, appropriate learning approach/practice, leading to real-world understanding.

With these precedents in mind, the committee again discussed the wide variety of UREs and which features of UREs are essential to a definition. The committee found that an attempt to sort them into binary categories of “authentic” or “unauthentic” would not help to achieve a useful construct. The committee’s definition of UREs is detailed in Chapter 2, which provides a discussion of the characteristics that make up a URE. Many of these characteristics are similar to the activities identified by Auchincloss and colleagues (2014) and by Brownell and Kloser (2015) in their work on CUREs. In this report the committee considers a URE to mean that the student is doing the type of work that STEM researchers would typically do; that is, the student is engaging in discovery and innovation, iteration, and collaboration as the student learns STEM disciplinary knowledge and practices while working on a topic that has relevance beyond the course. A URE is structured and guided by a mentor; the students are intellectually engaged and assume increasing ownership of some aspects of the project over time. The extent and focus on each particular activity will vary across different types and examples of UREs. Students can engage in STEM-based undergraduate research in many different ways, across a variety of settings, and along a continuum that extends from and expands upon learning opportunities in other educational settings. UREs therefore include many

___________________

⁵ Personal communication from Susan Camarena, National Science Foundation, to the Committee on Strengthening Research Experiences for Undergraduate STEM Students, November 10, 2016.

different types of research. For example, undergraduates may participate in wet bench research (such as characterizing human genetic diversity in Pacific Island populations), non–wet bench research (such as exploration and analysis of a genome), hypothesis-driven research (such as hypothesizing that the depths of aftershocks from the 2011 Tohoku earthquake, which was a magnitude 9.0, will be deeper to the west due to the direction of plate subduction), or nonhypothesis-driven research (such as case comparisons done to analyze the geological record).

HOW LEARNING SCIENCE INFORMS THE DISCUSSION

One of the major claims about UREs is that they can motivate students to persist in STEM by providing a window into the creation of knowledge, by strengthening student identity as a member of the STEM community, and by showcasing career options. Claims are also made that research experiences promote the development of robust, integrated, conceptual knowledge by engaging participants in STEM practices (Brownell and Kloser, 2015; Litzinger et al., 2011). Though there is a lack of strong causal and mechanistic evidence to support these claims, research from the learning sciences provides some very strong principles that are relevant to UREs and from which URE designers and researchers can benefit in their efforts to create and study UREs. To develop hypotheses about how UREs might promote the outcomes described above, it is important to draw both on research in the learning sciences broadly (National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2005; National Research Council, 1999) and on research that specifically examines STEM learning (National Research Council, 2006, 2009, 2012a).

Research from the learning sciences provides a way of thinking about how students engage with their education (Johri and Olds, 2011). This research indicates that prior knowledge and experiences shape learning: in other words, the learners’ existing understanding, skills, and beliefs significantly influence how they remember, reason, solve problems, and acquire new knowledge. Therefore, providing students with the opportunity to engage in the work of a STEM professional—focusing on the requisite research and disciplinary skills—through a URE can encourage deeper learning (Auchincloss et al., 2014; Brownell and Kloser, 2015; Johri and Olds, 2011; Litzinger et al., 2011). It is important to remember that when students have misconceptions—ideas, beliefs, and understandings that differ from accepted STEM-specific explanations—they may have difficulty integrating new knowledge with their inaccurate notions. This is because learning is a process of actively constructing knowledge via the process of conceptual reorganization. Individuals actively seek to make sense of new knowledge by connecting it with prior knowledge and experience (diSessa,

1996; Linn, 1995; Linn and Eylon, 2011). The act of discovery can allow students to work through ambiguous results and use evidence-based reasoning (Auchincloss et al., 2014).

The role of metacognition—the mind’s ability to monitor and control its own activities—in this process is important. Students who are encouraged to reflect on their learning have a better chance of constructing deeper, more robust knowledge (Litzinger et al., 2011). They monitor their comprehension as they learn—for example, by asking themselves if they truly understand when they encounter a new concept or by pausing to consider whether their strategy is working when they tackle a problem (National Research Council, 2012b).

Students often have difficulty applying their knowledge in a new context. For students to be able to use what they have learned, they need to understand the core concepts and use them as a structure for organizing their knowledge. Spending a lot of time studying material and practicing in rote ways is not sufficient to promote transfer of knowledge; what matters is how this time is spent. The goal is to spend time on activities that promote deeper learning, such as engaging in the work of a STEM professional, as this can develop the necessary expertise to know how the research fits within the landscape of the discipline (Litzinger et al., 2011). Evidence suggests that collaborative activities can enhance the effectiveness of student-centered learning over traditional instruction and improve retention of content knowledge (see, for example, Cortright et al., 2003; Johnson et al., 1998, 2007; National Research Council, 2015). When students work together on well-designed learning activities, they sometimes establish a community of learners, which provides cognitive and social support.

As discussed in Chapter 4, researchers have examined many questions about UREs. For instance, does the opportunity to participate in a sustained research experience where the student takes on increasing ownership foster the development of a sense of agency and efficacy (a belief that one’s actions can lead to improved understanding)? In typical STEM courses, students often find that they are following a set procedure and have little choice or opportunity for creativity. Does following a procedure that involves STEM practices help students develop a personal belief that they can learn disciplinary content and use the knowledge to solve relevant problems? Do research experiences promote agency by giving students choices in managing their experiment, recognizing and addressing problems, refining the research design, and exploring alternative explanations? Do students develop a sense of belonging, acceptance, and identity as a STEM professional when they feel they are participating in a community that is solving novel problems, have choices to make, and have the opportunity to provide creative input?

A well-designed URE builds on the evidence generated by researchers

seeking to answer these types of questions. It builds on evidence-based principles and seeks to provide an inclusive culture in which students from diverse backgrounds feel welcome in the program and are able to generate deeper learning that is relevant to their interests and perhaps values (Johri and Olds, 2011). By its very nature, a research experience requires that students do more than “know” something; it requires that they use their knowledge to “do” something. At various stages, across the various forms of UREs, students may design and carry out experiments or build and test new products or applications. They may analyze and interpret data, using the evidence that they have generated to make arguments; they may design solutions to problems, and almost always, they will need to communicate their work to other audiences. Their knowledge is not generated solely for academic purposes but rather to use in a research setting, and the latter objective enables more robust, deeper learning and integration to occur, tied into current practice within the STEM profession. For some students, the URE also provides a place to explore how the goals of the relevant STEM discipline relate to their personal and perhaps cultural values, which may or may not be reflected in the dominant culture. Furthermore, the setting of most UREs provides an experience that offers the potential for collaboration—engaging others from diverse backgrounds—as well as opportunities for these undergraduate researchers to think about, reflect on, and consolidate what they are doing and learning, which can potentially connect to what is meaningful to the student. In short, the experience provides opportunities both for metacognitive reflection and for integration of their personal and budding professional identities.

STANDARDS OF EVIDENCE

This committee was charged in part with the task of reviewing “the empirical evidence of benefits across a range of outcomes associated with the multitude of educational, student, and institutional goals.” In approaching this task, we found it useful to build on an earlier report, Scientific Research in Education (National Research Council, 2002). The committee that authored that report distinguished among three types of research questions: descriptive, causal, and mechanistic. Descriptive questions simply ask what is happening without making claims as to why it is happening. In the present context, one might ask how students experience undergraduate research and the degree to which their understanding of key concepts or procedures, or their beliefs in their capacity as a scientist or researcher, changes over the course of their research. Note that this description makes no claims as to whether the research experience caused these changes, only that these changes occurred over the same period of time during which students were engaged in undergraduate research. Causal questions seek

to discover whether a specific intervention leads to a specific response; whether, for example, a summer URE reduced the chances that students would subsequently switch out of STEM fields to pursue degrees in other majors. Finally, questions of mechanism or of process seek to understand why a cause leads to an effect. Perhaps the URE enhances a student’s confidence in her ability to succeed in her chosen field or deepens her commitment to the field by exposing her to the joys of research, and through these pathways it enhances the likelihood that she will persist in STEM.

Approaches to answering descriptive, causal, and mechanistic questions require a combination of theory, method, and measurement. In plain terms, you need to know the question you want to test (theory), know how to look for the outcome of that test (method), and be able to measure that outcome. The committee views the question of URE benefits to be one of cause: did the URE support the student in the career path she was on? Did it provide insights into the nature of STEM and a STEM career that the student would not have gained absent the experience? Did the student acquire new knowledge regarding the STEM discipline to which she was exposed?

Implicit in the causal claim is what social science researchers call a counterfactual: an alternative outcome an individual would have experienced in the presence of a different cause, or absent the cause under investigation. Examining differences between comparable students allows for causal claims. For example, a claim that UREs increase persistence in STEM fields is equivalent to the counterfactual claim that persistence rates in STEM would be lower in the absence of UREs. What is the warrant for such claims? One can never know for sure what would happen to a given individual subject to two different treatments—say a course with a strong, classroom-based research component and one that consists of lectures only. The student takes one class or the other. One could, however, make claims about average differences across groups of students experiencing these different approaches to instruction if one believes the groups are, on average, more or less identical prior to enrolling in these disparate courses. The design of the study, and fidelity to that design, forms the foundation of the belief that the groups of students subject to these different experiences are truly comparable.

In evaluating the research on the benefits of UREs, the committee looked for designs that would support not only descriptive but also causal and mechanistic claims. The latter designs would have (1) a clearly identified treatment, (2) a treatment group and at least one comparison group, and (3) an approach to assignment to treatment, or retrospective matching, that would lead one to have some confidence that groups in the two (or more) conditions were likely the same on average, prior to treatment. We were able to find very few such studies. However, some studies used plausible strategies for supporting the claim that the groups on average were

equivalent prior to the URE. These studies offered evidence suggestive of various benefits of UREs, in particular in retention of students in STEM programs (e.g., Rodenbusch et al., 2016). For example, Lopatto (National Academies of Sciences, Engineering, and Medicine, 2015; Appendix B) showed the effectiveness of quasi-experimental designs to study UREs.

Finally, many of the studies we reviewed lacked a control or comparison group. The committee considers studies of this sort to be descriptive but not causal or mechanistic. They offer a good foundation for developing hypotheses about causes, and they may be informative regarding potential mechanisms. Descriptive studies may provide a warrant for looking for causal relationships (benefits), but individually they do not offer hard evidence about those benefits. Many of the studies in this category relied either on student self-reports of their increased knowledge of the research process, confidence in their ability to participate, development of their research identity, or some other attribute, with student responses collected either retrospectively or at the beginning and end of the URE being studied.

REPORT ORGANIZATION

This report examines the types of UREs available and considers the roles of students, faculty, administrators, funders, and others involved with UREs.

Variations in types of UREs are examined in Chapter 2, including their structure, location, and the ways they reward students. This chapter also provides examples of the many creative approaches to UREs that can be found at institutions around the country.

Chapter 3 provides a framework for looking at the interacting actors and the situational components influencing UREs. The forces operating on students, faculty, nonfaculty mentors, academic departments, and institutions are complex and multilayered. This chapter also discusses the claims that are made about the benefits of UREs within the context of what is known about learning and learning science.

Chapter 4 examines the evidence for impact of UREs on students by analyzing the available research literature. Many of the most robust studies focus on historically underrepresented groups of students. There are many unanswered questions and opportunities for further research into the role of UREs in student learning and the mechanisms through which UREs have an impact on retention.

Faculty and mentoring are the topics of Chapters 5 and 6. URE programs are not always run by faculty; undergraduates are frequently mentored by nonfaculty instructors, postdoctoral fellows, graduate students, and even fellow undergraduates. Mentoring is a key aspect of the research experience for undergraduates and is therefore discussed in detail here. Mentoring has been studied extensively in many different settings, and there

is much to be learned from the literature on this topic. The faculty role in UREs is much larger than the opportunity to serve as a mentor. Faculty incentives and rewards for engaging in UREs vary across departments and institutions. The opportunity for faculty or staff to engage in relevant professional development also varies. Little research has been done on these aspects of faculty roles.

Chapter 7 presents a research agenda that describes topic areas where further studies could greatly improve understanding of how UREs work. Potential questions to be answered, as well as potential methodologies for pursuing the answers, are included in the agenda. Although the chapter advocates for a broad range of research, it stresses the importance of conducting research on the causal effects of UREs. It also discusses the different kinds of evidence and the importance of designing good studies that can provide insight into cause and mechanism.

Chapter 8 presents considerations in designing and implementing UREs. Although the committee advocates for further studies to better understand UREs and identify optimal approaches, we also recognize that many campuses are currently expanding the UREs available to their students. Therefore, this chapter aims to provide guidance based on the currently available information for institutions, campus leaders, and URE designers and implementers. It looks at the current policy context and considers campus culture as well as the perspectives of students and faculty. There is a section on the importance of considering equity and access. In addition, big-picture issues and practical questions are presented, as well as topics to consider in the design, implementation, evaluation, and improvement of UREs. While the committee was not able to find the information that would be necessary to do a cost-benefit analysis of UREs, this chapter does address the topic of financial, human, information, space, and equipment resources.

The final chapter lays out the committee’s conclusions about UREs and the recommendations for future actions involving the implementation and analysis of UREs.

REFERENCES

Association of American Colleges and Universities. (2008). High Impact Educational Practices: What They Are, Who Has Access to Them and Why They Matter. Washington, DC: Association of American Colleges and Universities. Available: http://provost.tufts.edu/celt/files/High-Impact-Ed-Practices1.pdf [November 2016].

Association of American Colleges and Universities. (2011). The LEAP Vision for Learning: Outcomes, Practices, Impact, and Employers’ Views. Washington, DC: Association of American Colleges and Universities. Available: https://www.aacu.org/sites/default/files/files/LEAP/leap_vision_summary.pdf [November 2016].

Auchincloss, L.C., Laursen, S.L., Branchaw, J.L., Eagan, K., Graham, M., Hanauer, D.I., Lawrie, G., McLinn, C.M., Pelaez, N., Rowland, S., Towns, M., Trautmann, N.M., Varma-Nelson, P., Weston, T.J., and Dolan, E.L. (2014). Assessment of course-based undergraduate research experiences: A meeting report. CBE–Life Sciences Education, 13(1), 29-40.

Bangera, G., and Brownell, S.E. (2014). Course-based undergraduate research experiences can make scientific research more inclusive. CBE–Life Sciences Education, 13(4), 602-606.

Bennett, N. (2015). Overview of the NSF REU Program and Proposal Review. Presentation at the GRC Funding Competitiveness Conference [February 18-21, 2015], Arlington, VA: National Science Foundation.

Brownell, S.E., and Kloser, M.J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology. Studies in Higher Education, 40(3), 525-544.

Brownell, S.E., Hekmat-Scafe, D.S., Singla, V., Seawell, P.C., Conklin-Imam, J.F., Eddy, S.L., Stearns, T., and Cyert, M.S. (2015). A high enrollment course-based undergraduate research experience improves student conceptions of scientific thinking and ability to interpret data. CBE–Life Sciences Education, 14(2), ar21. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4477737/pdf/ar21.pdf [January 2017].

Clancy, K.B.H., Nelson, R.G., Rutherford, J.N., and Hinde, K. (2014). Survey of academic field experiences (SAFE): Trainees report harassment and assault. PLOS ONE, 9(7), e102172. Available: http://dx.Doi.org/10.1371/journal.pone.0102172 [January 2017].

Cortright, R.N., Collins, H.L., Rodenbaugh, D.W., and DiCarlo, S.E. (2003). Student retention of course content is improved by collaborative-group testing. Advances in Physiology Education, 27, 102-108.

diSessa, A.A. (1996). Faculty opponent review: On mole and amount of substance: A study of the dynamics of concept formation and concept attainment. Pedagogisk Forskning i Sverige, 1(4), 233-243.

Eagan, M.K., Hurtado, S., Chang, M.J., Garcia, G.A., Herrera, F.A., and Garibay, J.C. (2013). Making a difference in science education: The impact of undergraduate research programs. American Educational Research Journal, 50(4), 683-713.

Eagan, K., Hurtado, S., Figueroa, T., and Hughes, B. (2014). Examining STEM Pathways among Students Who Begin College at Four-Year Institutions. Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees. Washington, DC. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_088834.pdf [November 2016].

Eddy, S.L., and Brownell, S.E. (2016). Beneath the numbers: A review of gender disparities in undergraduate education across science, technology, engineering, and math disciplines. Physical Review Physics Education Research, 12(2). Available: http://journals.aps.org/prper/pdf/10.1103/PhysRevPhysEducRes.12.020106 [January 2017].

Estrada, M., Burnett, M., Campbell, A.G., Campbell, P.B., Denetclaw, W.F., Gutierrez, C.G., Hurtado, S., John, G.H., Matsui, J., McGee, R., Okpodu, C.M., Robinson, T.J., Summers, M.F., Werner-Washrune, M., and Zavala, M. (2016). Improving underrepresented minority student persistence in STEM. Cell Biology Education, 15(3), es5. Available: http://www.lifescied.org/content/15/3/es5.full [November 2016].

Harsh, J.A., Maltese, A.V., and Tai, R.H. (2011). Undergraduate research experiences from a longitudinal perspective. Journal of College Science Teaching, 41(1), 84-91.

Hurtado, S., Eagan, M.K., and Hughes, B. (2012). Priming the Pump or the Sieve: Institutional Contexts and URM STEM Degree Attainments. Paper presented at the annual forum of the Association for Institutional Research [June 2-6, 2012], New Orleans, LA. Available: http://www.heri.ucla.edu/nih/downloads/AIR2012HurtadoPrimingthePump.pdf [November 2016].

Johnson, D.W., Johnson, R.T., and Smith, K.A. (1998). Cooperative learning returns to college: What evidence is there that it works? Change, 30, 26-35.

Johnson, D.W., Johnson, R.T., and Smith, K.A. (2007). The state of cooperative learning in postsecondary and professional settings. Educational Psychology Review, 19(1), 15-29.

Johri, A., and Olds, B.M. (2011). Situated engineering learning: Bridging engineering education research and the learning sciences. Journal of Engineering Education, 100(1), 151-185.

Kinkead, J. (2012). What’s in a name? A brief history of undergraduate research. CUR on the Web, 33(1). Available: http://www.cur.org/assets/1/7/331Fall12KinkeadWeb.pdf [August 2016].

Linn, M.C. (1995). Designing computer learning environments for engineering and computer science: The Scaffolded Knowledge Integration framework. Journal of Science Education and Technology, 4(2), 103-126.

Linn, M.C., and Eylon, B.S. (2011). Science Learning and Instruction: Taking Advantage of Technology to Promote Knowledge Integration. New York: Routledge.

Litzinger, T.A., Lattuca, L.R., Hadgraft, R.G., and Newstetter, W.C. (2011). Engineering education and the development of expertise. Journal of Engineering Education, 100(1), 123-150.

Miller, R.E. (2013). The Almost Experts: Capstone Students and the Research Process. Paper presented at the Association of College & Research Libraries, Indianapolis, IN. Available: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2013/papers/Miller_Almost.pdf [January 2017].

Moss-Racusin, C.A., Dovidio, J.F., Brescoll, V.L., Graham, M.J., and Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109(41), 16474-16479.

National Academies of Sciences, Engineering, and Medicine. (2015). Integrating Discovery-Based Research into the Undergraduate Curriculum: Report of a Convocation. Committee for Convocation on Integrating Discovery-Based Research into the Undergraduate Curriculum. Division on Earth and Life Studies. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

National Academies of Sciences, Engineering, and Medicine. (2016). Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students’ Diverse Pathways. S. Malcom and M. Feder (Eds.). Committee on Barriers and Opportunities in Two- and Four-Year STEM Degrees. Board on Science Education, Division of Behavioral and Social Sciences and Education. Board on Higher Education and the Workforce. Policy and Global Affairs. Washington, DC: The National Academies Press.

National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. (2005). Facilitating Interdisciplinary Research. Committee on Facilitating Interdisciplinary Research. Committee on Science, Engineering, and Public Policy. Washington, DC: The National Academies Press.

National Research Council. (1999). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press.

National Research Council. (2002). Scientific Research in Education. R. Shavelson and L. Towne (Eds.). Committee on Scientific Principles for Education Research. Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

National Research Council. (2006). America’s Lab Report: Investigations in High School Science. S. Singer, M. Hilton, and H. Schweingruber (Eds.). Committee on High School Science Laboratories: Role and Vision. Board on Science Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

National Research Council. (2009). Learning Science in Informal Environments: People, Places, and Pursuits. P. Bell, B. Lewenstein, A. Shouse, and M. Feder (Eds.). Committee on Learning Science in Informal Environments. Board on Science Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

National Research Council. (2011). Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads. Committee on Underrepresented Groups and the Expansion of the Science and Engineering Workforce Pipeline. Committee on Science, Engineering, and Public Policy. Policy and Global Affairs. Washington, DC: The National Academies Press.

National Research Council. (2012a). Community Colleges in the Evolving STEM Educational Landscape: Summary of a Summit. S. Olon and J. Labov (Rapporteurs). Committee on Evolving Relationships and Dynamics Between Two- and Four-Year Colleges and Universities. Board on Higher Education and Workforce. Policy and Global Affairs. Board on Life Sciences. Division on Earth and Life Studies. Board on Science Education; Teacher Advisory Council. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

National Research Council. (2012b). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research. S. Singer, N. Nielsen, and H. Schweingruber (Eds). Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

National Research Council (2015). Reaching Students: What Research Says about Effective Instruction in Undergraduate Science and Engineering. N. Kober (author). Board on Science Education. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

National Science Foundation. (2014). Science and Engineering Indicators 2014. Arlington, VA: National Science Board.

Newmann, F. (1996). Authentic Achievement: Restructuring Schools for Intellectual Quality. San Francisco: Jossey-Bass.

President’s Council of Advisors on Science and Technology. (2012). Engage to Excel: Producing One Million Additional College Graduates with Degrees in STEM. Washington, DC: Executive Office of the President. Available: http://files.eric.ed.gov/fulltext/ED541511.pdf [February 2017].

Rodenbusch, S.E., Hernandez, P.R., Simmons, S.L., and Dolan, E.L. (2016). Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE–Life Sciences Education, 15(2), ar20. Available: http://www.lifescied.org/content/15/2/ar20 [February 2017].

Rowles, C.J., Koch, D.C., Hundley, S.P., and Hamilton, S.J. (2004). Toward a model for capstone experiences: Mountaintops, magnets, and mandates. Assessment Update, 16(1), 1-2, 13-15.

Spell, R.M., Guinan, J.A., Miller, K.R., and Beck, C.W. (2014). Redefining authentic research experiences in introductory biology laboratories and barriers to their implementation. CBE–Life Sciences Education, 13, 102-110.

Van Noy, M., and Zeidenberg, M. (2014). Hidden STEM Knowledge Producers: Community Colleges’ Multiple Contributions to STEM Education and Workforce Development. Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_088831.pdf [November 2016].

Weaver, G., Wink, D., Varma-Nelson, P., Lytle, F., Morris, R., Fornes, W., Russell, C., and Boone, W. (2006). Developing a new model to provide first and second-year undergraduates with chemistry research experience: Early findings of the center for authentic science practice in education (CASPIE). Chemical Educator, 11, 125-129.

Zupanc, G.K. (2012). Undergraduate research and inquiry-based learning: The revitalization of the Humboldtian ideals. Bioscience Education, 19(1), 1-11.