Undergraduate Research Experiences in the Larger System of Higher Education: A Conceptual Framework
National reform efforts have begun to look at undergraduate research experiences (UREs) as a potential mechanism to encourage interest and retention within science, technology, engineering, and mathematics (STEM) fields. This interest has resulted in an overall increase in the funding and implementation of URE-oriented programs over the past decade. From the committee’s review of the extant literature on UREs, it is clear that there is a substantial range in the type and design of URE programs. The committee has developed the conceptual framework presented here to help designers, researchers, and evaluators organize their thinking about UREs. The committee sought to create a framework that would take into account two different components that contribute to the design, implementation, and evaluation of UREs. The first part of the framework articulates the goals for students participating in UREs and how these goals are related to different features of UREs, giving rise to a set of design principles. The second part characterizes the multiple systemic factors of the higher education landscape and how UREs are situated within that context.
The chapter begins with a review of the goals for student outcomes that have been associated with UREs. This review is followed by a discussion on the design of UREs that reflect the varied goals for students. The chapter concludes with a review of the relevant systemic factors (institutional, departmental, disciplinary, and financial), as well as policy issues impacting undergraduate research.
Reviewing the extant literature on programs of undergraduate research, the committee found several different themes for the goals that have informed the design and evaluation of UREs. As described in Chapter 1, programs of undergraduate research arose from national calls that encouraged institutions to provide high-impact practices that would allow students to better face the challenges of the 21st century (Boyer Commission on Education of Undergraduates in the Research University, 1998). The focus of these reports was to find opportunities that helped to keep students in STEM programs to support workforce needs (Auchincloss et al., 2014; Brownell et al., 2015; Litzinger et al., 2011). Therefore, retention and persistence in STEM fields was a primary motivating factor.
As more programs developed, the emphasis of the research on UREs began to shift away from simply trying to determine whether UREs led to retention in STEM (majors and graduation) and toward understanding why these programs had an effect. Evaluation of the programs began to look at student outcomes, such as content learning, and affective outcomes, such as whether URE students like doing research more than non-URE peers. Framing the questions in this way has begun to set the stage for uncovering answers about the importance of UREs not only for the purposes of keeping students in STEM majors or developing the STEM workforce, but also for their potential to have broader impacts on the citizenry.
Synthesizing across the literature and based upon the committee’s experience, we identified the primary goals for UREs to include developing and supporting students’ identities as researchers, increasing student knowledge of STEM content, increasing feelings of belonging in STEM, improving the understanding of the research enterprise, promoting greater ability to engage with STEM issues they will face as citizens, developing academic skills and strategies, increasing student persistence in STEM fields, and guiding student decisions about STEM courses and careers (Blockus, 2016; Dolan, 2016; Pfund, 2016). The outcomes students gain from UREs are shaped by how the experiences are constructed by faculty and supported by the department; institution; professional organizations; and external policy, accreditation, and funding structures at the state and national levels (Blockus, 2016; Dolan, 2016). As discussed in later sections of this chapter, these external factors influence which of the potential goals are prioritized by URE designers and implementers and also the details of how, when, and where UREs are implemented.
We discuss the goals for students participating in UREs under three major categories: (1) increasing retention and persistence of students in STEM, (2) promoting STEM disciplinary knowledge and practices, and
(3) integrating students into STEM culture (see Figure 3-1). These categories were determined by organizing the literature into themes that captured the primary motivations discussed above, whether it be participation/retention (category 1), cognitive outcomes (category 2), or affective outcomes (category 3). As discussed in Chapter 2, URE designers make choices about which goals to emphasize depending on their situation (e.g., how the URE fits into the curriculum, background of the students) and the types of
students participating in the URE. They select methods for implementing these goals based on their beliefs about how students learn.
For example, an overarching goal of student participation in UREs, as part of an undergraduate’s overall STEM learning experience, could include increasing conceptual understanding of relevant disciplinary knowledge, to learn to conduct an investigation, and to develop “literacy” for STEM. That is, the goal might not always be to persist in a STEM discipline but to be an informed citizen and a savvy consumer of scientific information in order to know how to make reasonable conclusions and arguments based on the strength of evidence. Any single URE may be designed to emphasize some goals and not others. For example, for students making decisions about STEM courses and careers, STEM majors might be inspired by their URE to continue to graduate school or get a job in a STEM field because of a love of research, whereas others may decide against these paths because they do not enjoy research; nonmajors may make progress toward becoming more STEM literate.
Overall, the goals presented here for students participating in UREs could be viewed through the lens of research on learning and instruction as it provides a way of thinking about the mechanisms that lead to outcomes (National Academy of Sciences, National Academy of Engineering, Institute of Medicine, 2005; National Research Council, 2000a, 2000b, 2006, 2009, 2012). This research provides a context for considering how learners’ and designers’ existing understanding and beliefs influence how UREs impact remembering, reasoning, solving problems, and acquiring new knowledge. Each of the primary goals are described in more detail below.
Increase Retention and Persistence of Students in STEM
A primary goal of UREs, driven by national-level calls for reform, is to improve STEM education in an effort to strengthen the STEM workforce. Research has suggested that participation in UREs could improve student outcomes such as higher grade point averages and increased retention in STEM majors, as well as an associated increase in college completion. (Chapter 4 provides a more nuanced discussion of these outcomes.) In this context, UREs are seen as a potential way to increase retention of students in STEM majors through graduation. Many argue that since UREs allow students to engage in the work of a STEM researcher, this experience can provide confirmation/clarification of their intended career paths (Auchincloss et al., 2014; Corwin et al., 2015). These paths might include pursuing opportunities outside of STEM by becoming a more literate citizen. Alternatively, the student may matriculate into a graduate program and/or enter into the STEM workforce.
Promote STEM Disciplinary Knowledge and Practices
To persist in a field, one must acquire knowledge about it. UREs seek to help students to better understand what it means to do research and what the process entails. This understanding consists of (at least) three parts: (1) understanding the disciplinary knowledge related to the topic under investigation and how the research questions fit within the landscape of the discipline, (2) development of the requisite research skills, and (3) understanding of the research enterprise and how disciplinary knowledge is built. Understanding the research enterprise includes being able to use disciplinary research practices (see Figure 3-1 for a list of these practices), understanding the importance of interaction, and appreciating the value of teamwork.
For students to develop a conceptual understanding of how a research question fits within the landscape of the discipline, they must also develop the relevant content knowledge associated with the field. That is, students need to understand the nature of the research discipline, be it science, engineering, or math. As discussed in How People Learn (National Research Council, 2000b) and Discipline-Based Education Research (National Research Council, 2012), learning is not only the accrual of information but also a process of conceptual reorganization. This has been explained as a process in which individuals actively seek to make sense of new knowledge by connecting it with prior knowledge and experience (National Research Council, 2015).
To develop coherent and robust understanding of a URE’s research question and related content knowledge, students need to sort out their existing ideas along with the new ideas they encounter as part of that URE. Often, new ideas have a fleeting trajectory, and the pre-existing ideas students bring to research experiences (and STEM courses) have been used and refined over multiple experiences. Therefore, a goal for UREs is to encourage students to engage in a process of distinguishing among ideas so that their understanding of the research topic grows, based on the evidence and their experience (diSessa, 1996; Johri and Olds, 2011; Linn, 1995; Linn and Eylon, 2011; Litzinger et al., 2011). Meaningful collaborative experiences can also facilitate student learning about STEM content as students engage in research (Cortright et al., 2003; Johnson et al., 1998, 2007).
Research suggests that students can enhance their understanding when opportunities for reflection are embedded within the learning experience (Weinstein et al., 2000), despite few students reporting the spontaneous use of such strategies (Karpicke et al., 2009; National Research Council, 2015). For example, in engineering education, Svinicki and McKeachie (2011) reported that incorporating reflection steps and self-explanation prompts into
instruction led to improvements in students’ problem solving. In this way, UREs can promote the goal of developing STEM disciplinary knowledge by providing students with the opportunity to engage in reflection.
Moreover, to capture the nature of actual STEM research in UREs, instructors and mentors can broaden student understanding by noting and explicating their own frustrations when things do not go as planned, thereby highlighting the importance of iteration and refinement. For example, when instructors make their own thinking visible, they can reveal the wrong paths and complexities of software design (Clancy et al., 2003), the struggles involved in mathematical thinking (Schoenfeld, 2010), and the challenges of scientific reasoning (Clement, 2009). Used this way, UREs can promote an understanding of the research process in a way that lectures and explanations in traditional STEM course delivery cannot, as these approaches often articulate the outcome rather than the process that led to the insights from a line of research. Although in a traditional lecture course, faculty can emphasize the process and not just the outcomes, it is possible that this type of understanding can be solidified when students are active participants in the process. Situating students within a URE can allow students to get a sense of the process of conjecture, refinement, redesign, and reconceptualization involved in the research enterprise, while developing the requisite research skills (Johri and Olds, 2011; Koretsky et al., 2011; Litzinger et al., 2011).
Integrate Students into STEM Culture
In addition to promoting STEM disciplinary knowledge and practices, research experiences are intended to promote a sense of agency and identity as a STEM research professional by engaging students in the work and situating them in the disciplinary context. Several studies show that students can develop a sense of identity as a STEM professional by engaging in well-designed activities typical of STEM professionals. Activities that can promote a sense of agency include being involved in designing their own studies, choosing experimental methods, and collecting data that are of intrinsic interest; all these activities encourage autonomy and allow for a greater sense of project ownership (Corwin et al., 2015). As Corwin and colleagues noted, providing students with the opportunity to gain a sense of ownership may increase the students’ motivation to complete projects even when faced with challenges, which can further develop their sense of scientific self-efficacy.
Related to the development of agency and STEM identity, providing opportunities for students to be integrated into the STEM culture or providing them with a STEM experience that is sensitive to the students’ cultural background may be an additional aspect of this goal. As alluded to previously, students may have an ill-formed idea of what it means to do re-
search and therefore may not know what it means to be a STEM researcher. Academic enculturation—situating students in the social and environmental context of the research so that they can learn/acquire the values of the discipline—through UREs may help to shape not only student’s learning but also their identity as a STEM researcher (Mendoza et al., 2015; Prior and Bilbro, 2012).
Alternatively, dominant STEM culture may be uninviting to students from nondominant cultures. UREs allow students to “experience” research, sometimes in a new context, and might help them better understand and appreciate the work that is involved (Litzinger et al., 2011). For example, Visintainer and Linn (2015) found that individuals from nondominant cultures gained a sense of identity by participating in programs led by mentors who came from similar cultural backgrounds and imparted respect for engaging in STEM-based practices such as collecting data, analyzing data, and presenting their findings to high status individuals. That is, UREs could make STEM accessible by making the discipline-specific topics understandable and relevant to the learner and by providing a culturally aware environment (National Research Council, 2012). Designing an environment that communicates these understandings requires a culturally aware design team.
Substantial research illustrates that students often feel that the STEM disciplinary topics they encounter in classes are inaccessible and irrelevant to their lives (Barr et al., 2010). This is especially true for students from nondominant cultures who may have met fewer scientists than those from dominant cultures and who hold different value systems (Hurtado et al., 2010; Ong et al., 2011). Students have reported through surveys or interviews that mentors helped them learn how to pursue research problems and develop resilience to inevitable failures (Adedokun et al., 2012; Hernandez et al. 2013; Schwarz, 2012). When students embark on personally selected problems with uncertain outcomes and feel that their work is respected, they have the potential to learn a great deal about the nature of research and about their own identity as an investigator, which can create a sense of belonging to the STEM community of researchers (e.g., Johri and Olds, 2011; Pryor et al., 2007) and lead to persistence (Estrada et al., 2011).
Another important component associated with the goal of integrating students into the STEM culture is collaboration and teamwork. To address complex, systemic problems such as climate change, disease vectors, and the motility of organisms, multiple perspectives are needed. For such reasons, many programs of research are multidisciplinary, capitalizing on the multiple forms of experiences that can lead to innovative methods for solving complex problems. Learning from others with different experiences who can give hints and encouragement rather than providing immediate solutions is a hallmark of complex research programs (Johnson and Johnson,
1998; Linn and Hsi, 2000; Vygotsky, 1978). Although working in groups can be beneficial, groups often find communicating and collaborating difficult due to different cultural or methodological practices. To help better prepare students for working in multidisciplinary and diverse groups, one goal for UREs is to provide opportunities for collaboration. Hurtado and colleagues (2008) identified competencies for a multicultural world as including the abilities to interact with individuals from different social identity groups and to negotiate ethical decisions in situations characterized by inequality and conflict.
There has been a growing emphasis on engaging students in research and inquiry and how to make curricular changes that will best support this high-impact practice (Brew, 2013; Koretsky et al., 2011). The learning sciences provide a grounding for considering the instructional practices that allow for effective learning experiences (Brew, 2013; Johri and Olds, 2011; Litzinger et al., 2011). Many STEM disciplines have been using the ideas developed by learning sciences to ensure that the experiences undergraduate students receive while conducting research are optimally designed (Brownell and Kloser, 2015; Litzinger et al., 2011). However, it is not always clear to what degree existing UREs have been designed using the extent literature on pedagogy and the learning sciences. To follow up on our discussion of goals for students in the previous section (increasing retention and persistence in STEM, promoting STEM disciplinary knowledge and practices, and integrating students into STEM culture), the committee drew on the robust research base on how to support students’ learning in STEM and mapped these goals to the common elements of UREs. This exercise allowed us to articulate a set of design principles for UREs.
The design, implementation, and evaluation of UREs depend on the interactions among designers, instructors, researchers, evaluators, students, and instructional resources. The design team negotiates the goals for the URE, taking into consideration the systemic factors in the higher education setting (such as available resources, reward structure for faculty, and disciplinary certification programs). To gain some traction on how to think about the design and evaluation of UREs, the committee identified characteristics that typify UREs (see Chapter 2 for an in-depth discussion) and might distinguish them from other courses and experiences. These principles for design are listed in Figure 3-2 and grouped into four categories: (1) make STEM research accessible and relevant; (2) support students to learn from each other; (3) make thinking visible; and (4) promote autonomy. URE leaders need to assist undergraduates to integrate the experiences, activities, mentoring, and assignments they encounter as
they participate in UREs and to connect these experiences with their prior experiences and education. Consideration also needs to be given to how students will be assessed. Preliminary work by Brownell and Kloser (2015) has begun to explore this issue for course-based UREs (CUREs). This section explores how thinking about the four categories of characteristics can assist in URE design and how to foster knowledge integration for each learner.
Make STEM Research Accessible and Relevant
UREs can help students recognize the relevance of their STEM courses by situating the investigation in the context of a personally relevant, con-
temporary problem such as climate change, global health, human genetics, or earthquake safety (e.g., Jordan et al., 2014). UREs can make STEM accessible by illustrating the role of knowledge, culture, and identity in STEM and policy decision making (Barton, 1998; Keller, 2016; Lemke, 1990). Relevant topics motivate students to continue to explore the topic even after the course is completed (Wigfield et al., 2007). Designing UREs so students can explore a topic that is relevant to their lives can promote identity in STEM (Johri and Olds, 2011).
Understanding the underlying theories and concepts in a research project is essential for students to make sense of and engage in STEM practices (Thiry et al., 2012). Students may not have taken courses that support the concepts, topics, or ideas that underlie their URE projects. These students, therefore, may not recognize the importance of the research question or its relevance for their lives. Some students only begin to feel capable of understanding the work of the URE by the third semester of their URE placement (Feldman et al., 2013). An important role for instructors and mentors is to design the URE so the rationale for the research questions is accessible. This may involve activities to help students connect the research design and potential contributions of the URE project to their prior knowledge. It will also include explicitly clarifying for students what role they will play in moving the research project forward and how their contribution will fit in to the big picture of the research project.
UREs can make STEM disciplinary knowledge accessible by helping students build on their existing ideas. It is not sufficient for URE instructors or mentors to articulate accurate ideas and expect students to incorporate them into their understanding of the field. Instead, students benefit from making predictions to identify their prior knowledge. UREs can allow students to distinguish among their own diverse ideas as well as the new ideas by using evidence from experiments, observations, or other sources that they obtain during the URE. Research has identified promising ways to guide students to distinguish among ideas (e.g., Quintana et al., 2004).
To succeed in STEM, students need opportunities to organize often contradictory, fragmented, and disconnected ideas along with the new ideas they encounter. Knowledge that is organized and coherent is easier to remember because there are multiple links between items that can aid in recall. Organizing knowledge involves noticing patterns, relationships, and discrepancies among ideas (Reif and St. John, 1979). Moreover, when students develop integrated, organized understanding, they have knowledge that can be used to solve new problems.
Students often have difficulty applying their knowledge in a new context. One way to create the potential for transferring ideas about the process skills or competencies that are most important for UREs is to help students develop an understanding of the core concepts and patterns, in
addition to the requisite skills, that can serve as a structure for organizing knowledge (Bransford and Schwartz, 2009). Spending a lot of time studying material and practicing its application is not sufficient to promote transfer of knowledge and skills; what matters is how this time is spent (Bjork and Bjork, 2011). The goal is to spend time on activities that promote deeper learning. To start, students need complex, realistic problems that encourage extracting relevant information and analyzing it against prior knowledge. They need to apply the research process to new situations (Shaffer, 2012).
One way that designers can determine whether they have succeeded in making the topics of the URE accessible is by assessing the products that students prepare such as posters, journals, research reports, and presentations. Other forms of student success will require different assessments that measure understanding of the research process or of the nature of STEM. Students are more likely to produce products that feature integrated ideas and identify patterns in results or data when the problems they study are accessible and illustrate the process of linking and connecting ideas (Linn and Eylon, 2011).
A salient aspect of UREs is that they have the potential to promote autonomy. In a STEM research context, autonomy may be characterized as the ability to initiate research activities and carry them to completion by taking advantage of multiple resources including peers, experts, technologies, and media. This concept of autonomy is consistent with Hurtado and colleagues (2012, p. 50), who called for developing “habits of mind [that] involve the way students integrate different sources of knowledge.” UREs can promote autonomy by giving students the opportunity to make decisions about the problem to be studied, the research design, and the appropriate methodology to use (Bjork et al., 2013). Designers of UREs can carefully design tasks and opportunities for students to gradually develop skills that are necessary to promote autonomy (Brew, 2013).
As part of promoting autonomy, instructors can take advantage of reflection. By building a practice of reflecting on their evidence and identifying consistencies and open questions, students may develop autonomy. This is essential for achieving durable research understanding.
Linn’s (2006) knowledge integration framework calls for engaging students in distinguishing between their existing ideas and new ideas. In this process, students use many of the reasoning strategies desired in STEM fields, such as drawing on evidence and forming arguments to reach conclusions. Activities that require students to generate their own explanations of concepts or explain a concept to another person are thought of as revealing an element of reflection. Studies indicate that these “self-explanation” strat-
egies can enhance learning more than just having students read a passage or examine the diagrams in a textbook (National Research Council, 2012). To assess student ability to investigate research dilemmas autonomously, designers can examine the progress students make in UREs as reflected in the products they create, such as research reports or posters for meetings. Another approach is to build online miniprojects that could reveal student progress in developing these skills; some such assessments employ automated scoring, an advantage when increasing the size of a program (e.g., Liu et al., 2016; Quellmalz et al. 2012).
Learn from Each Other
Research increasingly involves collaboration and learning from others as problems become more and more complex (e.g., Cook-Deegan, 1994). Many argue that students learn more effectively when they collaborate (Brown and Campione, 1994; Linn and Hsi, 2000; Vygotsky, 1978). Yet collaboration is not universally efficient or effective for learning (Kollar et al., 2007; Webb, 1997). To benefit from collaboration, students often need to learn how to learn from each other. When students work together on well-designed learning activities, they establish a community of learners that provides cognitive and social support for the efforts of the community’s individual members. In such a community, students share the responsibility for thinking and doing. They can help each other solve problems by building on each other’s knowledge, asking each other questions, and suggesting ideas that an individual working alone might not have considered (Brown and Campione, 1994; Okita and Schwartz, 2013). By challenging each other’s thoughts and beliefs, they can compel the members of the group to be explicit about what they mean and to negotiate any conflicts that arise, which in turn fosters metacognition. Social interactions may also have a positive effect on motivation by making individuals feel they are contributing something to others (Schwartz, 1999). Facilitating interactions among various cultural groups could help improve student’s communication skills while also integrating students into the research enterprise (Hurtado et al., 2008). Supporting and promoting collaboration has potential for UREs (Brownell et al., 2015). However, orchestrating collaboration is difficult. Students must be able to respect the ideas of their peers, negotiate meaning, and guide peers who are less able.
Make Thinking Visible
Individual students come to UREs with a complex set of ideas stemming from their own cultural identity, previous academic experiences, and personal reflection. Students might have specific ideas about STEM-related
topics, but they might also have “knowledge in pieces.” That is, the ideas might be fragmented and contradictory (diSessa, 2000). As noted above, students may need to distinguish new ideas and prior knowledge.
An important step in helping students learn and gain a better understanding of the research enterprise is to ask them to make their ideas visible. When students are asked to articulate their existing ideas, they reveal to themselves and their mentors/instructors the current understanding that they have developed about a topic. Previous research has shown that student’s knowledge can be assessed by asking them to make predictions about phenomena. Students develop better conceptual understanding when they make predictions than when they do not (Linn and Songer, 1991; Mayer et al., 2003; White and Gunstone, 1992). In addition, the process of reflecting and explaining their reasoning often helps students recognize flaws in their own reasoning (Collins and Brown, 1988).
Encouraging students to make their thinking visible both when they generate explanations and when they revise them can promote knowledge integration. These activities can set in motion a process of revisiting STEM-specific issues when they arise in new contexts, such as news articles or public lectures. Autonomous learners sort out their existing ideas and integrate them with new ideas in order to continue to build coherent understanding. By practicing reflection regularly, students can develop the ability to monitor their own progress and to recognize new connections as they arise.
Reflection is common when STEM professionals maintain notebooks where they record results and identify trends. Instructors and mentors can encourage students to maintain notebooks and use them to make their thinking visible. They can ask students to include discussions of their struggles to conduct their project and the limitations of their work. In CUREs, instructors can include essay examinations rather than relying on multiple-choice questions to instill a practice of reflection. This approach has the advantage of being both part of the instruction and a source of insights into student progress (Lee et al., 2011).
Programs of undergraduate research are nested within multiple contexts. There are systemic factors—national and state policy, institutions, and departments and disciplines—that can have a top-down influence by promoting opportunities or placing constraints on UREs through reforms and funding. There are also more-local factors involved in the implementation of UREs—that is, designers (including faculty, mentors, and evaluators) and students.
As described in Chapter 2, UREs are heterogeneous, which is not surprising given the variation in systemic factors and the diverse views of
student learning held by the key actors. They vary on multiple dimensions. Programs of undergraduate research can differ in terms of leadership (i.e., who is responsible for the program), design, and duration. UREs also can vary in expectations or goals for students, mentoring provided, value for career trajectory (e.g., strengthen likelihood of graduate school admissions or industry employment, preparation as an informed citizen), and measured outcomes, as well as the population(s) served (e.g., STEM majors, nonmajors, historically underrepresented students, first generation students). Moreover, UREs can vary in how they are funded and how they are situated within the university. Given this variability, it can be challenging to cleanly categorize UREs and even more difficult to identify how many programs of any given type are being offered. (Chapter 2 provides a more in-depth discussion of program types.) In fact, data on the number of students who participate in UREs nationally is not systematically collected, although some funders do collect data on programs they sponsor.
Systemic factors include variation in institutional support (e.g., rare in community college, common in small liberal arts colleges), extramural funding, disciplinary expectations (e.g., common in chemistry and engineering, less common in mathematics, and rare in computer science), faculty motivation (e.g., improve instruction, make the laboratory experience more relevant and meaningful, meet funding requirement), and faculty rewards (e.g., no reward, release from course-teaching requirement, enhancement of research capability, value for promotion). In short, the substantial heterogeneity of UREs across multiple dimensions is due in part to the nature of the higher education system. These systemic factors interact with each other as shown in Figure 3-3. That is, national and state-level policies interact with institutional and/or departmental policies to shape opportunities and place constraints on UREs. A discussion of each of these three systemic factors and their impacts on UREs follows.
National and State Policy
As highlighted in Chapter 1, there have been many calls for reform focused on making undergraduate STEM education “more practical, relevant, engaging, and grounded in research on how people learn” (Laursen et al., 2010, p. 7). One of the major catalysts for this national-level reform was the Boyer Commission (Boyer Commission on Education of Undergraduates in the Research University, 1998), which issued a report calling for research-based learning to become the standard in undergraduate education, particularly at research universities. Moreover, national bodies have called for increasing opportunities that are student-centered and inquiry-based in STEM disciplines (Kuh, 2008; National Research Council, 1999; National Science Foundation, 1996).
Although these national-level calls for reform can encourage funding for undergraduate research, new initiatives can also shift research priorities and the types of projects that are funded, which can have substantial impacts on broader opportunities for students to engage in research. Several organizations, such as the National Science Foundation (NSF), the Howard Hughes Medical Institute (HHMI), and the National Institutes of Health, have developed funding initiatives specifically targeted at increasing access to UREs for more diverse students. An external review conducted by Russell and colleagues (2007) of NSF’s funding for undergraduate research suggested that engaging students in undergraduate research was associated
with positive outcomes, such as increasing the undergraduates understanding, confidence, and awareness of the importance of research.
For example, the National Institutes of Health has developed two initiatives geared toward increasing the participation of historically underrepresented groups in the biomedical sciences by providing them with access to resources and preparation for graduate-level work. The Maximizing Access to Research Careers/Undergraduate Student Training in Academic Research initiative1 provides support to undergraduate, honors-level, junior and senior students. In contrast, the Research Initiative for Scientific Enhancement (RISE) program aims to reduce the gap between underrepresented and non-underrepresented students in Ph.D. degree completions by providing support to institutions. This RISE funding can be used to pay salaries to undergraduates participating in research.2
HHMI provides funding through multiple mechanisms and encourages colleges and universities to build “capacity to effectively engage all students in science,” which includes transfer students from community colleges, first generation students, and historically underrepresented students.3 For example, HHMI has funded the development, implementation, and expansion of the Freshman Research Initiative at The University of Texas at Austin. Thousands of freshmen students have participated in this initiative since 2006 (Rodenbusch et al., 2016). Students in this program (40 percent of the incoming freshman class) join a “research stream” in which they engage in progressively more intense research experiences over time. For more information on this initiative, see Box 2-9 in Chapter 2.
NSF also has a portfolio dedicated to supporting UREs, called Research Experiences for Undergraduates (REU) that provides funding for programs and projects that encourage active research participation by undergraduate students.4 Over the years, REU has experienced increases in both the number of awards granted each year and the amount of money being awarded. For this report, the committee used the DIA2 tool5 to extract the number of awards and the award amount per year for REU grants from 1995 to 2015. Figure 3-4 depicts this gradual increase from 1995 through 2015 in number of awards (left side) and total award amount (right side); there is a relative plateau beginning around 2007 for both measures of funding level.
Although external funding provides essential resources, it also imposes
1 See https://www.nigms.nih.gov/Training/MARC/Pages/USTARAwards.aspx [February 2017].
2 See https://www.nigms.nih.gov/training/RISE/Pages/default.aspx [February 2017].
3 See http://www.hhmi.org/programs/undergraduate-science-education-grants [February 2017].
4 See http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5517&from=fund [February 2017].
5 The DIA2 tool is a public search tool that was developed with NSF funding to Purdue University. The tool currently accesses a database of more than 200,000 grants awarded by NSF from 1995 to present.
some constraints. Federal and state policy, including the length of time that individual grants for UREs are awarded, and the funding priorities of major foundations affect the kinds of undergraduate research that are offered at colleges and universities nationally. A large problem is nonrenewable funding that is available to launch and start UREs. To see sustained impacts,
there must be enough time elapsed that cohorts can progress through their college career, which takes a good deal of time after the initial funding is available, especially if there is a ramp-up/development stage that prevents the first cohorts from happening in the first 2 years of the grant award. This can make it difficult to show the impacts and secure additional funding.
In addition to the availability and kinds of funding for UREs (e.g., individual initiatives versus supplements to existing faculty grants), education priorities are established through national and state policy making, as well as through other policy priorities of the government that can affect the breadth and scope of such programs. For example, it is possible that recent emphasis on having students complete their degrees as quickly as possible could discourage institutions from supporting longer-term (e.g., multiple semesters) projects if they do not allow students to obtain credit toward graduation and if the time required to be engaged with them results in students taking fewer credits per semester. Policies that emphasize keeping tuition and fees as low as possible could discourage development of CUREs, which sometimes may be funded in part by an increase in student lab fees or by additional costs for enrolling in STEM courses compared with those in other disciplines. When a project is funded with external support, funding agencies determine the amount of resources given to support research and evaluation of programs or they might request periodic reports and dictate what type of information to be tracked and measured to demonstrate a programs’ success. Access to program evaluations may not be widely available unless published in peer-reviewed journals or disciplinary society publications.
Institutional initiatives, mission, and culture can impact the degree to which there is financial and logistical support for the development of UREs and how those activities may be structured. These institutional priorities, in turn, are influenced by national and state education policies and priorities. A study that examined the relationship between campus missions and the five benchmarks for effective educational practice (measured by the National Survey of Student Engagement) showed that certain programs, policies, and approaches may work better depending on the institution’s mission (Kezar and Kinzie, 2006). Other research on this topic shows that if institutions align policies and practices that support student success, then students are more likely to persist (Berger, 2001-2002; Kuh, 2001-2002).
Opportunities to participate in UREs may be limited at certain campuses due to those institutions’ mission, priorities, or funding sources. Institutional support for UREs may be less common in community colleges than at small liberal arts colleges and research-intensive universities. Uni-
versities with more research funding often have more research opportunities for undergraduates (Kezar and Kinzie, 2006); however, for large institutions that have more students, this might not translate into an increase in the number of opportunities for an individual student.
Institutions also provide the infrastructure and resources to support undergraduate research more generally. This may include providing space and assisting in procuring the requisite laboratory and field equipment, which might be shared among multiple departments and/or faculty. At a broader level, it might include the creation of an office of undergraduate research to facilitate the promotion and implementation of such programs. Moreover, the institution might sponsor campus-wide initiatives that support UREs by providing supplemental funds to students engaging in research (i.e., funds to help acquire necessary equipment or supplies) or in the dissemination of research at national disciplinary meetings or through a campus-sponsored event.
However, institutions can broaden or impede student participation in UREs through their faculty promotion and reward structures. In some institutions, involving students in URE programs might take away faculty time from other activities that are expected by the university’s promotion and reward system (i.e., publishable funded research). In other institutions, supporting undergraduates in research is an expected activity.
When individual institutions decide to expand participation in undergraduate research, they may do so through a variety of approaches. For example, some colleges or universities may make participation in at least one URE mandatory rather than optional for the student. This could be achieved by supporting the development of more course-based experiences to involve more undergraduates per mentor. It could also be achieved through partnering with other institutions of higher education, local or regional research organizations, or industries that conduct research and development. Decades-long partnerships between predominantly white institutions and historically black colleges and universities through undergraduate research programs are one example of such partnerships (Louis et al., 2015). Similarly, community colleges sometimes partner with baccalaureate-granting institutions to provide their students with access to faculty and facilities (Russell et al., 2007). Additional opportunities may exist through study abroad programs or with local, national, or international consortia.
Departments and Disciplinary Context
Academic departments play an important role in shaping the type of experiences that are available to the students in their program and the requirements for participating in UREs, especially as schools are moving toward a “culture of undergraduate research” (Merkel, 2001). The require-
ments for degree completion in terms of the types of courses one needs to take can have an impact on students’ options for research. Making a URE a graduation requirement increases participation, whereas numerous other requirements could likely decrease participation in research. An evaluation of the existing curriculum might spur departments to adapt or add courses to increase accessibility to UREs for their majors, and potentially also for nonmajors.
Each department’s decisions are influenced by its particular disciplinary culture and context. The likelihood of participating in UREs is dependent upon the specific area of STEM considered. For example, one national study sent out a web-based survey to all recipients of eight NSF-funded grants that included an undergraduate research component. Almost 15,000 students responded to the survey. Approximately 72 percent of students that majored in chemistry and 74 percent that majored in environmental science stated that they had participated in UREs, whereas 34 percent of students in mathematics and computer science stated they had such opportunities (Russell et al., 2007). These disciplinary differences may be driven in part by the various STEM disciplines promoting different kinds of knowledge, skill sets, and approaches. For example, some fields have different expectations for learning specific content because these fields prepare students for specific professional careers and/or require certification (e.g., engineering, business, computer science, and information science). For other fields, such as mathematics, the learning of content is not specifically tied to an occupation. For fields such as engineering, where the curriculum may lead to a career path in certain industries, participating in UREs that focus on the relevant knowledge may be important.
Faculty participation in UREs may be motivated by a desire to improve instruction, enrich the students’ experience in an existing lab/research experience, boost research productivity, or satisfy requirements necessary to receive funding. Success of UREs may depend upon administrators and institutional policies to support interested faculty, along with the resources and professional development to encourage/compensate faculty. Furthermore, facilities and time to allow faculty to properly engage undergraduate students in research are important (Shortlidge et al., 2016).
Disciplinary societies, professional societies, and national networks also play an important role in the national policy discussion and shape the context that supports UREs. Societies of STEM research professionals traditionally have served as a platform for leaders and members from their respective STEM fields and subspecialties to present their research and to discuss challenges and opportunities in their field. These meetings provide opportunities for professional development and provide networking opportunities among members at regional and national levels. Some also have sessions or entire conferences focused on education, in addition to those
that invite undergraduate researchers to present their research during poster sessions and/or talks. For example, the National Conferences on Undergraduate Research are meetings completely devoted to undergraduates sharing their own research.
Systemic Influences and the Dynamic Interplay
Institutions, departments, and individual faculty each impact the precise nature of UREs in multiple ways and at multiple levels. The physical resources available, including laboratories, field stations, engineering design studios, and testing facilities, can influence the design of the research question as well as the ability to access resources in the surrounding community (including other parts of the campus). Institutions with an explicit mission to promote undergraduate research may provide more time, resources (e.g., financial, support personnel, space, equipment), and recognition and rewards to URE-engaged departments and faculty than those institutions with another focus. The culture of the institution with respect to innovation in pedagogy and support for faculty development can impact the extent to which UREs are introduced or improved.
Departmental and institutional differences affect students’ access to undergraduate research (Katkin, 2003). Many reform efforts that begin in a single department are not broadly adopted across programs, in other departments, or across colleges/universities in the STEM disciplines. “Student advising, faculty professional development, student research mentoring, academic support programs, clear STEM-focused institutional articulation agreements, external partnerships with business and industry related to internships and other research experiences, and many other critical programs and areas that have been identified as central to student success are often overlooked within reform efforts” (Elrod and Kezar, 2015, p. 67). These conditions suggest that UREs may need support from the institutional level in order to become sustainable and widespread in an institution.
The goals for students participating in UREs are to increase retention/participation in STEM, promote STEM disciplinary knowledge and practices, and integrate students into STEM culture. These goals, coupled with the design principles—make STEM research accessible, help students learn from each other, make thinking visible, and promote autonomy—can set the stage for a robust experience that can help students generate deeper learning. This process begins by engaging students in research experiences that require students to do more than “know” something. Many research
experiences are intended to empower students to appreciate their potential as creative contributors to their chosen discipline. The degree to which UREs are designed using the existing educational literature on pedagogy and how people learn is not clear.
The heterogeneity of UREs as described in Chapter 2 stems from variability associated with the multiple systemic factors, goals, and design principles described in this chapter. National calls for reform efforts and opportunities for funding shape UREs on campus. However, institutions, departments, and faculty play a big role in creating the context that surrounds the URE. Local policies and culture can provide a supportive environment that promotes undergraduate research as a “normal” part of STEM undergraduate education. When there is alignment between the policies and culture, there may be an increase in the likelihood of sustaining a URE.
Adedokun, O.A., Zhang, D., Parker, L.C., Bessenbacher, A., Childress, A., Burgess, W.D., and Adedokun, B.O.A. (2012). Understanding how undergraduate research experiences influence student aspirations for research careers and graduate education. Journal of College Science Teaching, 42(1), 82-90.
Auchincloss, L.C., Laursen, S.L., Branchaw, J.L., Eagan, K., Graham, M., Hanauer, D.I., Lawrie, G., McLinn, C.M., Pelaez, N., Rowland, S., Towns, M., Trautmann, N.M., VarmaNelson, P., Weston, T.J., and Dolan, E.L. (2014). Assessment of course-based undergraduate research experiences: A meeting report. CBE–Life Sciences Education, 13(1), 29-40.
Barr, D.A., Matsui, J., Wanat, S.F., and Gonzalez, M.E. (2010). Chemistry courses as the turning point for premedical students. Advances in Health Sciences Education, 15(1), 45-54.
Barton, A.C. (1998). Reframing “science for all” through the politics of poverty. Educational Policy, 12(5), 525-541.
Berger, J.B. (2001-2002). Understanding the organizational nature of student persistence: Empirically-based recommendations for practice. Journal of College Student Retention, 3(1), 3-21.
Bjork, E.L., and Bjork, R. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. Pp. 56-64 in M.A. Gernsbacher, R.W. Pew, L.M. Hough, and J.R. Pomerantz (Eds.), Psychology and the Real World: Essays Illustrating Fundamental Contributions to Society. New York: Worth Publishers.
Bjork, R.A., Dunlosky, J., and Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417-444.
Blockus, L. (2016). Strengthening Research Experiences for Undergraduate STEM Students: The Co-Curricular Model of the Research Experience. Paper commissioned for the Committee on Strengthening Research Experiences for Undergraduate STEM Students. Board on Science Education, Division of Behavioral and Social Sciences and Education. Board on Life Sciences, Division of Earth and Life Studies. National Academies of Sciences, Engineering, and Medicine. Available: http://nas.edu/STEM_Undergraduate_Research_Apprentice.
Boyer Commission on Education of Undergraduates in the Research University. (1998). Reinventing Undergraduate Education: A Blueprint for America’s Research Universities. Stony Brook, NY: State University of New York at Stony Brook for the Carnegie Foundation for the Advancement of Teaching.
Bransford, J.D., and Schwartz, D.L. (2009). It takes expertise to make expertise: Some thoughts about why and how. Pp. 748-774 in K.A. Ericsson (Ed.), Development of Professional Expertise: Toward Measurement of Expert Performance and Design of Optimal Learning Environments. Cambridge, UK: Cambridge University Press.
Brew, A. (2013). Understanding the scope of undergraduate research: A framework for curricular and pedagogical decision-making. Higher Education, 66, 603-618.
Brown, A.L., and Campione, J.C. (1994). Guided Discovery in a Community of Learners. Cambridge, MA: The MIT Press.
Brownell, S.E., and Kloser, M.J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology. Studies in Higher Education, 40(3), 525-544.
Brownell, S.E., Hekmat-Scafe, D.S., Singla, V., Seawell, P.C., Conklin-Imam, J.F., Eddy, S.L., Stearns, T., and Cyert, M.S. (2015). A high enrollment course-based undergraduate research experience improves student conceptions of scientific thinking and ability to interpret data. CBE–Life Sciences Education, 14(2), ar21. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4477737/pdf/ar21.pdf [January 2017].
Clancy, M., Titterton, N., Ryan, C., Slotta, J., and Linn, M. (2003). New roles for students, instructors, and computers in a lab-based introductory programming course. ACM SIGCSE Bulletin, 35(1), 132-136.
Clement, N. (2009). Perspectives from research and practice in values education. Pp. 13-25 in T. Lovat and R. Toomey (Eds.), Values Education and Quality Teaching. Dordrecht, Netherlands: Springer.
Collins, A., and Brown, J.S. (1988). The computer as a tool for learning through reflection. Pp. 1-18 in M.M Sebrechts, G. Fisher, and P.M. Fisher (Eds.), Learning Issues for Intelligent Tutoring Systems. New York: Springer US.
Cook-Deegan, R.M. (1994). The Gene Wars: Science, Politics, and the Human Genome. New York: Norton.
Cortright, R.N., Collins, H.L., Rodenbaugh, D.W., and DiCarlo, S.E. (2003). Student retention of course content is improved by collaborative-group testing. Advances in Psychology Education, 27, 102-108.
Corwin, L.A., Graham, M.J., and Dolan, E.L. (2015). Modeling course-based undergraduate research experiences: An agenda for future research and evaluation. CBE–Life Sciences Education, 14, es1. Available: http://www.lifescied.org/content/14/1/es1.full.pdf+html [January 2017].
diSessa, A.A. (1996). Faculty opponent review: On mole and amount of substance: A study of the dynamics of concept formation and concept attainment. Pedagogisk Forskning i Sverige, 1(4), 233-243.
diSessa, A.A. (2000). ChangingMinds. Cambridge, MA: MIT Press.
Dolan, E. (2016). Course-Based Undergraduate Research Experiences: Current Knowledge and Future Directions. Paper commissioned for the Committee on Strengthening Research Experiences for Undergraduate STEM Students. Board on Science Education, Division of Behavioral and Social Sciences and Education. Board on Life Sciences, Division of Earth and Life Studies. National Academies of Sciences, Engineering, and Medicine. Available: http://nas.edu/STEM_Undergraduate_Research_CURE [February 2017].
Elrod, S., and Kezar, A. (2015). Increasing student success in STEM. Peer Review, 17(2). Available: https://www.aacu.org/peerreview/2015/spring/elrod-kezar [February 2017].
Estrada, M., Woodcock, A., Hernandez, P.R., and Schultz, P. (2011). Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology, 103(1), 206-222.
Feldman, A., Divoll, K.A., and Rogan-Klyve, A. (2013). Becoming researchers: The participation of undergraduate and graduate students in scientific research groups. Science Education, 97(2), 218-243.
Hernandez, P.R., Schultz, P., Estrada, M., Woodcock, A., and Chance, R.C. (2013). Sustaining optimal motivation: A longitudinal analysis of interventions to broaden participation of underrepresented students in STEM. Journal of Educational Psychology, 105(1). Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3838411/pdf/nihms512414.pdf [February 2017].
Hurtado, S., Eagan, M.K., Cabrera, N.L., Lin, M.H., Park, J., and Lopez, M. (2008). Training future scientists: Predicting first-year minority student participation in health science research. Research in Higher Education, 49(2), 126-152.
Hurtado, S., Eagan, K., and Chang, M. (2010). Degrees of Success: Bachelor’s Degree Completion Rates among Initial STEM Majors. Higher Education Research Institute at UCLA. Available: http://www.heri.ucla.edu/nih/downloads/2010%20-%20Hurtado,%20Eagan,%20Chang%20-%20Degrees%20of%20Success.pdf [January 2010].
Hurtado, S., Eagan, M.K., and Hughes, B. (2012). Priming the Pump or the Sieve: Institutional Contexts and URM STEM Degree Attainments. Paper presented at the annual forum of the Association for Institutional Research Annual Form [June 2-6, 2012], New Orleans, LA. Available: http://www.heri.ucla.edu/nih/downloads/AIR2012HurtadoPrimingthePump.pdf [June 2016].
Johnson, D.W., and Johnson, R. (1998). Cooperative learning and social interdependence theory. In R. Tindale, L. Heath, J. Edwards, E. Posavac, F. Bryant, Y. Suzrez-Balcazar, E. Henderson-King, and J. Myers (Eds.), Theory and Research on Small Groups (4), 9-36. New York: Plenum.
Johnson, D.W., Johnson, R.T., and Smith, K.A. (1998). Cooperative learning returns to college: What evidence is there that it works? Change, 30, 26-35.
Johnson, D.W., Johnson, R.T., and Smith, K.A. (2007). The state of cooperative learning in postsecondary and professional settings. Educational Psychology Review, 19(1), 15-29.
Johri, A., and Olds, B.M. (2011). Situated engineering learning: Bridging engineering education research and the learning sciences. Journal of Engineering Education, 100(1), 151-185.
Jordan, T.C., Burnett, S.H., Carson, S., Caruso, S.M., Clase, K., DeJong, R.J., Dennehy, J.J., Denver, D.R., Dunbar, D., Elgin, S.C.R., Findley, A.M., Gissendanner, C.R., Golebiewska, U.P., Guild, N., Hartzog, G.A., Grillo, W.H., Hollowell, G.P., Hughes, L.E., Johnson, A., King, R.A., Lewis, L.O., Li, W., Rossenzweig, F., Rubin, M.R., Saha, M.S., Sandoz, J., Shaffer, C.D., Taylor, B., Temple, L., Vazquez, E., Ware, V.C., Barker, L.P., Bradley, K.W., Jacobs-Sera, D., Pope, W.H., Russell, D.A., Cresawn, S.G., Lopatto, D., Bailey, C.P. and Hatfull, G.F. (2014). A broadly implementable research course in phage discovery and genomics for first-year undergraduate students. MBio, 5(1), e01051-13. Available: http://mbio.asm.org/content/5/1/e01051-13.full.pdf [February 2017].
Karpicke, J.D., Butler, A.C., and Roediger III, H.L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own?. Memory, 17(4), 471-479.
Katkin, W. (2003). The Boyer Commission Report and its impact on undergraduate research. New Directions for Teaching and Learning, 2003(93), 19-38.
Keller, C. (2016). Using STEM Case Studies to Prepare Today’s Students for Tomorrow’s Jobs. An Evaluation of Spark 101 Interactive STEM videos. Available: http://www.spark101.org/media/user_files/2016-03-07_SparkEvaluation-Final.pdf [February 2017].
Kezar, A.J., and Kinzie, J. (2006). Examining the ways institutions create student engagement: The role of mission. Journal of College Student Development, 47(2), 149-172.
Kollar, I., Fisher, F., and Slotta, J.D. (2007). Internal and external scripts in computer supported collaborative inquiry learning. Learning and Instruction, 17(6), 708-721.
Koretsky, M., Kelly, C., and Gummer, E. (2011). Student perception of learning in the laboratory: Comparison of industrially situated virtual laboratories to capstone physical laboratories. Journal of Engineering Education, 100(3), 540-573.
Kuh, G.D. (2001-2002). Organizational culture and student persistence: Prospects and puzzles. Journal of College Student Retention, 3(1), 23-39
Kuh, G.D. (2008). Excerpt from High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Available: https://www.aacu.org/leap/hips [April 2015].
Laursen, S., Hunter, A.-B., Seymour, E., Thiry, H., and Melton, G. (2010). Undergraduate Research in the Sciences: Engaging Students in Real Science. San Francisco, CA: Jossey-Bass.
Lee, Jr., J.M., Contreras, F., McGuire, K.M., Flores-Ragade, A., Rawls, A., Edwards, K., and Menson, R. (2011). The College Completion Agenda: 2011 Progress Report. College Board Advocacy & Policy Center. Available: http://media.collegeboard.com/digitalServices/pdf/advocacy/policycenter/college-completion-agenda-2012-progress-report.pdf [February 2017].
Lemke, J.L. (1990). Talking Science: Language, Learning, and Values. Norwood, NJ: Ablex Publishing Corporation.
Linn, M.C. (1995). Designing computer learning environments for engineering and computer science: The Scaffolded Knowledge Integration framework. Journal of Science Education and Technology, 4(2), 103-126.
Linn, M.C. (2006). The knowledge integration perspective on learning and instruction. Pp. 243-264 in R.K. Sawyer (Ed.) The Cambridge Handbook of the Learning Sciences. New York: Cambridge University Press.
Linn, M.C., and Eylon, B.-S. (2011). Science Learning and Instruction: Taking Advantage of Technology to Promote Knowledge Integration. New York: Routledge.
Linn, M.C., and Hsi, S. (2000). Computers, Teachers, Peers: Science Learning Partners. New York: Routledge.
Linn, M.C., and Songer, N.B. (1991). Cognitive and conceptual change in adolescence. American Journal of Education, 99(4), 379-417.
Litzinger, T.A., Lattuca, L.R., Hadgraft, R.G., and Newstetter, W.C. (2011). Engineering education and the development of expertise. Journal of Engineering Education, 100(1), 123-150.
Liu, O.L., Rios, J.A., Heilman, M., Gerard, L., and Linn, M.C. (2016). Validation of automated scoring of science assessments. Journal of Research in Science Teaching, 53(2), 215-233.
Louis, D.A., Phillips, L.L., Louis, S.L., and Smith, A.R. (2015). Historically black colleges and universities: Undergraduate research, mentoring and extending the graduate pipeline. Perspectives on Undergraduate Research & Mentoring, 4(1). Available: https://works.bepress.com/dave_louis/41/ [February 2017].
Mayer, R.E., Dow, G.T., and Mayer, S. (2003). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds? Journal of Educational Psychology, 95(4), 806-812.
Mendoza, N., Richard, J., and Dugat Wickliff, T. (2015). Enculturation of diverse students to the engineering sciences through first year engineering college experiences at a southwestern institution: An exploratory work in progress. Paper presented at the 7th First Year Engineering Experience (FYEE) Conference, Roanoke, VA. Available: http://fyee.org/2015/papers/5079.pdf [February 2017].
Merkel, C.A. (2001). Undergraduate Research at Six Research Universities: A Pilot Study for the Association of American Universities. California Institute of Technology. Available: https://pdfs.semanticscholar.org/ebeb/1024b664b2ee093daf7e09911fc1aa28959d.pdf [February 2017].
National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. (2005). Facilitating Interdisciplinary Research. Committee on Facilitating Interdisciplinary Research. Committee on Science, Engineering, and Public Policy. Washington, DC: The National Academies Press.
National Research Council. (1999). How People Learn: Bridging Research and Practice. M. Donovan, J. Bransford, and J. Pellegrino (Eds.). Committee on Learning Research and Educational Practice. Board on Behavioral, Cognitive, and Sensory Sciences, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
National Research Council. (2000a). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press.
National Research Council. (2000b). How People Learn: Brain, Mind, Experience, and School: Expanded Edition. Committee on Developments in the Science of Learning. Committee on Learning Research and Educational Practice. Board on Behavioral, Cognitive, and Sensory Sciences, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
National Research Council. (2006). America’s Lab Report: Investigations in High School Science. S.R. Singer, M.L. Hilton, and H.A. Schweingruber (Eds.). Committee on High School Science Laboratories: Role and Vision. Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
National Research Council. (2009). Learning Science in Informal Environments: People, Places, and Pursuits. Committee on Learning Science in Informal Environments, P. Bell, B. Lewenstein, A.W. Shouse, and M.A. Feder (Eds.). Board on Science Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
National Research Council. (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. S. Singer, N.R. Nielsen, and H.A. Schweingruber (Eds). Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research. Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
National Research Council. (2015). Reaching Students: What Research Says about Effective Instruction in Undergraduate Science and Engineering. N. Kober (author). Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
National Science Foundation. (1996). Shaping the Future: New Expectations for Undergraduate Education in Science, Mathematics, Engineering, and Technology (NSF 96-139). Washington, DC: National Science Foundation. Available: http://www.nsf.gov/pubs/stis1996/nsf96139/nsf96139.txt [February 2017].
Okita, S.A., and Schwartz, D.L. (2013). Learning by teaching human pupils and teachable agents: The importance of recursive feedback. Journal of the Learning Sciences, 22(3), 375-412.
Ong, M., Wright, C., Espinosa, L., and Orfield, G. (2011). Inside the double bind: A synthesis of empirical research on undergraduate and graduate women of color in science, technology, engineering, and mathematics. Harvard Educational Review, 81(2), 172-208.
Pfund, C. (2016). Studying the Role and Impact of Mentoring on Undergraduate Research Experiences. Paper commissioned for the Committee on Strengthening Research Experiences for Undergraduate STEM Students. Board on Science Education, Division of Behavioral and Social Sciences and Education. Board on Life Science, Division of Earth and Life Studies. National Academies of Sciences, Engineering, and Medicine. Available: http://nas.edu/STEM_Undergraduate_Research_Mentoring.
Prior, P., and Bilbro, R. (2012). Academic enculturation: Developing literate practices and disciplinary identities. Pp. 19-31 in M. Castell and C. Donahue (Eds.), University Writing: Selves and Texts in Academic Societies. Bingley, UK: Emerald Group Publishing Limited.
Pryor, J.H., Hurtado, S., Saenz, V.B., Santos, J.L., and Korn, W.S. (2007). The American Freshman: Forty Year Trends. Los Angeles: Higher Education Research Institute. Available: https://heri.ucla.edu/PDFs/40TrendsManuscript.pdf [February 2017].
Quellmalz, E.S., Timms, M.J., Silberglitt, M.D., and Buckley, B.C. (2012). Science assessments for all: Integrating science simulations into balanced state science assessment systems. Journal of Research in Science Teaching, 49(3), 363-393.
Quintana, C., Reiser, B.J., Davis, E.A., Krajcik, J., Fretz, E., and Golan Duncan, R., Kyza, E., Edelson, D., and Soloway, E. (2004). A scaffolding design framework for software to support science inquiry. Journal of the Learning Sciences, 13(3), 337-386.
Reif, F., and St. John, M. (1979). Teaching physicists’ thinking skills in the laboratory. American Journal of Physics, 47(11), 950-957.
Rodenbusch, S.E., Hernandez, P.R., Simmons, S.L., and Dolan, E.L. (2016). Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE–Life Sciences Education, 15(2), ar20. Available: http://www.lifescied.org/content/15/2/ar20 [February 2017].
Russell, S.H., Hancock, M.P., and McCullough, J. (2007). Benefits of undergraduate research experiences. Science, 316, 548-549.
Schoenfeld, A.H. (2010). How We Think: A Theory of Goal-Oriented Decision Making and Its Educational Applications. New York: Routledge.
Schwartz, D.L. (1999). The productive agency that drives collaborative learning. Pp. 197-218 in P. Dillenbourg (Ed.), Collaborative Learning: Cognitive and Computational Approaches. New York: Elsevier Science.
Schwarz, J. (2012). Faculty as undergraduate research mentors for students of color: Taking into account the costs. Science Education, 96, 527-542.
Shaffer, D.W. (2012). Models of situated action: Computer games and the problem of transfer. Pp. 403-432 in C. Steinkuehler, K. Squre, and S. Barab (Eds.), Games, Learning, and Society: Learning and Meaning in the Digital Age. New York: Cambridge University Press.
Shortlidge, E.E., Bangera, G., and Brownell, S.E. (2016). Faculty perspectives on developing and teaching course-based undergraduate research experiences. BioScience, 66(1), 54-62.
Svinicki, M., and McKeachie, W.J. (2011). McKeachie’s Teaching Tips: Strategies, Research, and Theory for College and University Teachers. Belmont, CA: Wadsworth, Cengage Learning.
Thiry, H., Weston, T.J., Laursen, S.L., and Hunter, A.B. (2012). The benefits of multi-year research experiences: Differences in novice and experienced students’ reported gains from undergraduate research. CBE–Life Sciences Education, 11(3), 260-272.
Visintainer, T., and Linn, M. (2015). Sixth-grade students’ progress in understanding the mechanisms of global climate change. Journal of Science Education and Technology, 24(2-3), 287-310.
Vygotsky, L. (1978). Interaction between learning and development. Readings on the Development of Children, 23(3), 34-41.
Webb, N.M. (1997). Assessing students in small collaborative groups. Theory into Practice, 36, 205-213.
Weinstein, C.E., Husman, J., and Dierking, D.R. (2000). Self-regulation interventions with a focus on learning strategies. Pp. 727-747 in M. Boekaerts, P.R. Pintrich, and M. Zeidner (Eds.), Handbook of Self-Regulation. San Diego, CA: Academic Press.
White, R., and Gunstone, R. (1992). Prediction-observation-explanation. Pp. 44-64 in Probing Understanding. New York: Routledge.
Wigfield, A., Eccles, J.S., Schiefele, U., Roeser, R.W., and Davis-Kean, P. (2007). Development of achievement motivation. In W. Damon and N. Eisenberg (Eds.), Handbook of Child Psychology: Social, Emotional, and Personality Development. New York: John Wiley & Sons, Inc.