Summary

Undergraduate research has a rich history, and many practicing researchers point to undergraduate research experiences (UREs) as crucial to their own career success. One of the most prominent opportunities for undergraduate research has been through the National Science Foundation’s Research Experiences for Undergraduates program, but many other funders (large and small) have contributed to the opportunities available. Organizations such as the Council on Undergraduate Research and the National Conferences on Undergraduate Research have provided a showcase for undergraduate work and a network for faculty to learn from each other about UREs.

There are many ongoing efforts to improve undergraduate science, technology, engineering, and mathematics (STEM) education that focus on increasing the active engagement of students and decreasing traditional lecture-based teaching. UREs have been proposed as an opportune way to actively engage students and may be a key strategy for broadening participation in STEM. Multiple reports have focused on the potential high impact of UREs and the often limited availability of the experiences.¹ These reports often call for an expansion in UREs to allow for greater access

___________________

¹ Three important examples of such reports are Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics from the President’s Council of Advisors on Science and Technology; High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter from the Association of American Colleges and Universities; and Science in Solution: The Impact of Undergraduate Research on Student Learning by David Lopatto and published by the Research Corporation for Science Advancement.

to a wider array of students. Current efforts are working to increase the number of students participating in UREs and to increase the diversity of those participants.

The National Science Foundation commissioned this study by the National Academies of Sciences, Engineering, and Medicine to examine what is known about UREs and, if possible, to identify best practices that should be applied to future UREs. The committee was also asked to discuss the needs of involved faculty and administrators, to examine costs and benefits, and to provide recommendations for research and practice. The committee approached its analysis of UREs by considering them as part of a learning system that is shaped by forces related to national policy, institutional leadership, and departmental culture, as well as by the interactions among faculty, other mentors, and students. The committee also considered UREs in the context of the goals for students and what research on learning says about how such experiences should be designed to reach those goals. Many existing studies that provide information on how students learn can inform URE designers.

DIVERSITY OF URES

The classic image of a URE is a student spending the summer working directly with a faculty member on a project related to that faculty member’s research, but UREs have diversified beyond this traditional apprentice model. Course-based undergraduate research experiences are becoming increasingly common. Students also participate in research via internships and co-ops, where they do academically relevant work outside of academia. In addition, undergraduate research can be part of wrap-around programs that may offer combinations of mentoring, scholarships, courses in study skills, and courses in research approaches and ethics. As well as these variations in structure, UREs can also differ in location (on campus or off campus, in a variety of settings) and rewards to students (e.g., academic year course credit, service credit, stipends). A discussion of the great variety of UREs and a definition of URE is provided in Chapter 2.

College students today are more diverse than in the past, and faculty and administrators implementing UREs need to consider how they include historically underrepresented students, first generation college students, STEM majors, non-STEM majors, beginning students, students enrolled in capstone experiences, and pre-service teachers.² Many of the more extensive

___________________

² Capstone experiences are large projects done by upper-level students that bring together multiple aspects of their undergraduate education. First generation students are the first generation in their family to attend college. Pre-service teachers are undergraduates preparing to become teachers in grades from kindergarten through 12th grade.

studies of UREs have focused on participation by historically underrepresented groups of students in a comprehensive program. Further research is needed to see whether the conclusions drawn from those studies can be applied more widely to other student populations and other types of UREs.

IMPLEMENTATION OF URES

The culture and values of campuses and departments affect how UREs are implemented and perceived. On some campuses, UREs are a prominent feature of undergraduate education for all students, whereas on other campuses, they are known (and hence available) only to a small pool of students. There are wide variations across departments and institutions in the degree to which faculty are expected to include undergraduates in research. Incentives for faculty to participate can be tied to traditions and attitudes, as well as to the potential for their participation to be considered in promotion and tenure decisions. These expectations and attitudes can influence the level of administrative support available to help faculty develop, implement, refine, and study UREs. Campus culture also impacts many more-practical issues, such as the availability of resources (e.g., space, equipment, libraries and journal access). The availability of external and internal funding can also affect the creation and sustainability of UREs. National networks, including disciplinary and educational societies, can play an important role in connecting faculty members with others with similar interests in a supportive “community of practice.” New UREs are often modeled on or adapted from existing UREs, and this raises issues about the best ways to learn from the experiences of others. These networking connections can be very important on campuses where teaching expectations are high and few faculty members have maintained an active research program of their own.

Mentoring is a key aspect of the research experience for many undergraduates. In addition to the mentoring done by faculty members, undergraduates are frequently mentored by instructors, postdoctoral fellows, graduate students, and even fellow undergraduates. Faculty engage in UREs in many ways. In addition to serving as mentors they generally make decisions about the structure and design of the URE, including making decisions about goals and evaluation. A URE program has the potential to drive faculty research and create synergy between the teaching and research responsibilities of individual faculty members. Faculty incentives and rewards for engaging in UREs vary across departments and institutions. The opportunity for faculty and other mentors to engage in relevant professional development also varies. Little research has been done on how working with undergraduates doing research or establishing a URE program impacts the professional life of the faculty.

WHAT IS KNOWN ABOUT URES

Studies of UREs have examined many potential outcomes of participation in UREs. They have looked at the impact on persistence and retention of students in a STEM major or STEM career, promoting understanding of STEM, and integration into the STEM culture. From the available evidence, the committee concludes that UREs impact graduation rates and retention in STEM, and they may increase students’ feelings of belonging and their confidence in understanding STEM content, data analysis, and the nature of experiments. In addition, there is a large body of literature available on how people learn that can be applied to UREs. However, the extent to which those designing and implementing UREs have explicitly relied upon the studies about UREs or on the knowledge of how students learn is unclear.

As the focus on UREs has grown, so have the questions about their impact. There is an emerging body of literature describing specific UREs and surveys of student participants, as well as unpublished evaluations that provide additional information about UREs. Although these sources provide a rich description of UREs, they do not currently answer questions about the ways that UREs lead to benefits to students and which aspects of UREs are most powerful. In addition, it is difficult to evaluate the costs of UREs because many schools seek to leverage already available resources or use in-kind donations, such as nonmonetized (uncompensated) faculty time, in building their URE program.

Taken together, there are many unanswered questions and opportunities for further investigation of the role of UREs in the undergraduate learning experience and the mechanisms by which UREs might support various student, faculty, and institutional goals. Different types of questions rely on different research methodologies, and attention to study design as UREs are planned will facilitate research on them. Carefully designed studies can enable the community to develop a more robust understanding of how UREs work for different students, why they work, and how to evaluate the reported outcomes for URE participants. Such studies need to be based on sound research questions and use valid methods to measure outcomes. The committee’s research agenda in Chapter 7 proposes specific areas where additional studies would be particularly informative.

To maximize the return on the investment in URE programs, it will be useful to collect additional data comparing programs to ascertain those design features that contribute to student success. Student success includes many different aspects, such as learning important content of a discipline, understanding practices of STEM researchers, and gaining a sense of belonging to the STEM enterprise; markers of success can be measured in both the short term (e.g., by grade point averages) and the long term (e.g., by career choice). Despite this need for additional research, much is known

that can inform the decisions and actions of the many interacting people and administrative units influencing UREs.

Practitioners designing or improving UREs can build on the experiences of colleagues and learn from both the literature about UREs and the research on how students learn. During the design process, practitioners should consider the goals of the students, goals of the program, goals of the faculty member, and goals of the campus. Other factors to consider include the available resources, how the program or experience will be evaluated or studied, and how to build in opportunities to improve the experience over time, based on new evidence. Analysis of the current offerings on campus can inform decisions and help create a culture of improvement in which faculty are supported in their efforts to continually refine UREs based on the evidence currently available and evidence that they and others generate in the future.

CONCLUSIONS AND RECOMMENDATIONS

Following its analysis of the available information, the committee reached consensus on a set of conclusions and recommendations. The conclusions and recommendations discussed in Chapter 9 are included in this summary. In addition, Chapter 7 identifies five additional recommendations for future research about UREs.

Conclusions

Conclusion 1: The current and emerging landscape of what constitutes UREs is diverse and complex. Students can engage in STEM-based undergraduate research in many different ways, across a variety of settings, and along a continuum that extends and expands upon learning opportunities in other educational settings. The following characteristics define UREs. Due to the variation in the types of UREs, not all experiences include all of the following characteristics in the same way; experiences vary in how much a particular characteristic is emphasized.

• They engage students in research practices including the ability to argue from evidence.
• They aim to generate novel information with an emphasis on discovery and innovation or to determine whether recent preliminary results can be replicated.
• They focus on significant, relevant problems of interest to STEM researchers and in some cases a broader community (e.g., civic engagement).
• They emphasize and expect collaboration and teamwork.

• They involve iterative refinement of experimental design, experimental questions, or data obtained.
• They allow students to master specific research techniques.
• They help students engage in reflection about the problems being investigated and the work being undertaken to address those problems.
• They require communication of results, either through publication or presentations in various STEM venues.
• They are structured and guided by a mentor, with students assuming increasing ownership of some aspects of the project over time.

Conclusion 2: Research on the efficacy of UREs is still in the early stages of development compared with other interventions to improve undergraduate STEM education.

• The types of UREs are diverse, and their goals are even more diverse. Questions and methodologies used to investigate the roles and effectiveness of UREs in achieving those goals are similarly diverse.
• Most of the studies of UREs to date are descriptive case studies or use correlational designs. Many of these studies report positive outcomes from engagement in a URE.
• Only a small number of studies have employed research designs that can support inferences about causation. Most of these studies find evidence for a causal relationship between URE participation and subsequent persistence in STEM. More studies are needed to provide evidence that participation in UREs is a causal factor in a range of desired student outcomes.

Taking the entire body of evidence into account, the committee concludes that the published peer-reviewed literature to date suggests that participation in a URE is beneficial for students.

Conclusion 3: Studies focused on students from historically underrepresented groups indicate that participation in UREs improves their persistence in STEM and helps to validate their disciplinary identity.

Conclusion 4: The committee was unable to find evidence that URE designers are taking full advantage of the information available in the education literature on strategies for designing, implementing, and evaluating learning experiences. STEM faculty members do not generally receive training in interpreting or conducting education research. Partnerships between those with expertise in education research and those with expertise in implementing UREs are one way to strengthen the application of evidence on what works in planning and implementing UREs.

Conclusion 5: Evaluations of UREs are often conducted to inform program providers and funders; however, they may not be accessible to others. While these evaluations are not designed to be research studies and often have small sample sizes, they may contain information that could be useful to those initiating new URE programs and those refining UREs. Increasing access to these evaluations and to the accumulated experience of the program providers may enable URE designers and implementers to build upon knowledge gained from earlier UREs.

Conclusion 6: Data at the institutional, state, or national levels on the number and type of UREs offered, or who participates in UREs overall or at specific types of institutions, have not been collected systematically. Although the committee found that some individual institutions track at least some of this type of information, we were unable to determine how common it is to do so or what specific information is most commonly gathered.

Conclusion 7: While data are lacking on the precise number of students engaged in UREs, there is some evidence of a recent growth in course-based undergraduate research experiences (CUREs), which engage a cohort of students in a research project as part of a formal academic experience.

Conclusion 8: The quality of mentoring can make a substantial difference in a student’s experiences with research. However, professional development in how to be a good mentor is not available to many faculty or other prospective mentors (e.g., graduate students, postdoctoral fellows).

Conclusion 9: The unique assets, resources, priorities, and constraints of the department and institution, in addition to those of individual mentors, impact the goals and structures of UREs. Schools across the country are showing considerable creativity in using unique resources, repurposing current assets, and leveraging student enthusiasm to increase research opportunities for their students.

Recommendations

Recommendation 1: Researchers with expertise in education research should conduct well-designed studies in collaboration with URE program directors to improve the evidence base about the processes and effects of UREs. This research should address how the various components of UREs may benefit students. It should also include additional causal evidence for the individual and additive effects of outcomes from student participation in different types of UREs. Not all UREs need be designed to undertake

this type of research, but it would be very useful to have some UREs that are designed to facilitate these efforts to improve the evidence base.

Recommendation 2: Funders should provide appropriate resources to support the design, implementation, and analysis of some URE programs that are specifically designed to enable detailed research establishing the effects on participant outcomes and on other variables of interest such as the consequences for mentors or institutions.

Recommendation 3: Designers of UREs should base their design decisions on sound evidence. Consultations with education and social science researchers may be helpful as designers analyze the literature and make decisions on the creation or improvement of UREs. Professional development materials should be created and made available to faculty. Educational and disciplinary societies should consider how they can provide resources and connections to those working on UREs.

Recommendation 4: Institutions should collect data on student participation in UREs to inform their planning and to look for opportunities to improve quality and access.

Recommendation 5: Administrators and faculty at all types of colleges and universities should continually and holistically evaluate the range of UREs that they offer. As part of this process, institutions should:

• Consider how best to leverage available resources (including off-campus experiences available to students and current or potential networks or partnerships that the institution may form) when offering UREs so that they align with their institution’s mission and priorities;
• Consider whether current UREs are both accessible and welcoming to students from various subpopulations across campus (e.g., historically underrepresented students, first generation college students, those with disabilities, non-STEM majors, prospective kindergarten-through-12th-grade teachers); and
• Gather and analyze data on the types of UREs offered and the students who participate, making this information widely available to the campus community and using it to make evidence-based decisions about improving opportunities for URE participation. This may entail devising or implementing systems for tracking relevant data (see Conclusion 4).

Recommendation 6: Administrators and faculty at colleges and universities should ensure that all who mentor undergraduates in research experiences

(this includes faculty, instructors, postdoctoral fellows, graduate students, and undergraduates serving as peer mentors) have access to appropriate professional development opportunities to help them grow and succeed in this role.

Recommendation 7: Administrators and faculty at all types of colleges and universities should work together within and, where feasible, across institutions to create a culture that supports the development of evidence-based, iterative, and continuous refinement of UREs, in an effort to improve student learning outcomes and overall academic success. This should include the development, evaluation, and revision of policies and practices designed to create a culture supportive of the participation of faculty and other mentors in effective UREs. Policies should consider pedagogy, professional development, cross-cultural awareness, hiring practices, compensation, promotion (incentives, rewards), and the tenure process.

Recommendation 8: Administrators and faculty at all types of colleges and universities should work to develop strong and sustainable partnerships within and between institutions and with educational and professional societies for the purpose of sharing resources to facilitate the creation of sustainable URE programs.

This page intentionally left blank.