WHAT IS KNOWN AND RECOMMENDATIONS FOR ACTION
As the number and diversity of out-of-school programs that support STEM learning continue to grow, it is becoming increasingly important for policy makers and funders to make informed decisions about which programs to support. The existing research provides important information that can help inform some of those difficult decisions. The committee offers three criteria for identifying and developing productive out-of-school STEM learning programs, and six recommendations for actions that policy makers, educators, and other stakeholders can take to support programs that reflect the criteria.
Criteria For Productive Programs
Young people (ages 5-18) develop an understanding of STEM concepts and skills through an iterative process across a wide array of learning experiences that take place in both out-of-school programs and in school.123 The iterative process of learning STEM requires policy makers to create funding streams and policies that encourage productive out-of-school STEM experiences and how to link them in order to create sets of coherent learning opportunities. Policy makers at the local, state, and national levels have different mechanisms available to them for achieving these goals, and they can each play a role in supporting such education reform.
Although opportunities to engage in STEM activities in out-of-school settings and programs is sometimes thought of as an optional enrichment opportunity, this perspective is not consistent with what is known about the outcomes of such settings and programs. Access to productive out-of-school opportunities that engage young people in authentic STEM experiences is a critical piece of the STEM learning ecosystem. Such out-of-school opportunities can support STEM learning independently from classroom learning, and they are particularly well suited to building interest in STEM and identity as a STEM learner.124
There is an increasing understanding of how to broaden and deepen access to quality out-of-school programs that support STEM learning and a growing awareness of the need to make it easier for families to engage their children.125 Clear evidence from summative and comparative program evaluations of what programs work best for whom and under what circumstances does not yet exist, but the field is taking steps to develop new and meaningful measurement strategies.
To support informed policy and program decision making we concluded that there are three criteria for identifying and developing productive out-of-school STEM learning programs. Together, the criteria represent the ways in which youth development, STEM learning in informal environments, and learning across settings intertwine to support productive STEM out-of-school programs that successfully engage young people in STEM learning and actively support inclusion and broaden participation by young people in STEM learning.
- Productive programs engage young people intellectually, socially, and emotionally. Productive out-of-school STEM programs provide young people with first-hand experiences with STEM phenomena and materials, engage them in sustained STEM practices, and are aligned with participants’ cultural resources and practices. In such programs, young people are engaged in first-hand, materials-rich, and place-based learning experiences that involve processes of scientific or engineering investigation and practice. Thus, productive out-of-school STEM programs engage young people in the processes of doing STEM in ways they find compelling and challenging and develop their interest, understanding, and commitment to continue engaging in STEM learning.
- Productive programs respond to young people’s interests, experiences, and cultural practices. Productive out-of-school STEM programs make STEM relevant to the questions that interest young people, support collaboration and leadership by young people, and train
- staff to support and build young people’s STEM activities and interest. Productive out-of-school STEM programs are also responsive to young people’s prior interests and experiences so that they can see STEM as meaningful and relevant to their own experiences and aspirations.
- Productive programs connect STEM learning in out-of-school, school, home, and other settings. Productive out-of-school STEM programs explicitly help young people make connections among STEM experiences in and across settings and programs, leveraging community resources and partnerships, and brokering ongoing opportunities to engage in STEM learning activities. Productive out-of-school programs also help young people to understand how what they experience and learn relates to learning in other settings, including school. Thus, productive out-of-school programs purposefully help young people, their parents, and others in the community capitalize on developing expertise and interests across time and setting.
Evaluating Programs and generating New Evidence
Generating evidence of productive out-of-school STEM programs is conceptually and practically complicated by the fact that STEM learning accumulates over time and across settings, change occurs at multiple levels (individual, program, and community), STEM learning is idiosyncratic, and the norms of out-of-school programs lead to practical barriers in administering assessments. There is a need for new conceptual tools and approaches to evaluation that can help generate hypotheses and theoretical accounts of STEM learning in out-of-school programs. Some of this work has begun: for example, there are efforts to develop common metrics and instruments to compare individual outcomes across a large number of programs, and there are efforts under way to develop innovative, unobtrusive approaches that are culturally responsive and honor the multiplicity of out-of-school program goals.
Policy makers, funders, and program leaders need to work together to sustain and expand a robust and iterative ecosystem of learning opportunities in schools and in out-of-school programs. We identified six actions that policy makers, program developers, and other stakeholders should take to support programs that reflect the criteria for identifying and designing productive out-of-school STEM learning programs:
- Understand the local conditions for creating an ecosystem of high-quality productive out-of-school STEM learning programs: Build a map and bridge the gaps.
- Design programs to achieve access, equity, continuity, and coherence: Connect young people with opportunities to learn.
- Support the use of creative and responsive approaches to evaluating the success of programs at the individual, program, and community levels: Support innovative evaluation approaches.
- Increase the professionalization of out-of-school program leaders and staff: Provide professional development.
- Strengthen the STEM learning infrastructure: Build an infrastructure that will last.
- Invest in research to improve our understanding of STEM learning in out-of-school programs: Explore how STEM learning ecosystems work.
The rest of this chapter elaborates on these important next steps.
BUILD A MAP AND BRIDGE THE GAPS
Mapping existing STEM learning resources and gaps is a critical first step in supporting a robust STEM learning ecosystem that can meet the interests and needs of all young people through a wide variety of intellectually compelling and culturally responsive programs.
Every community has a unique set of learning resources available to young people: they include natural settings, industries, universities, and local community-based and youth development organizations. As discussed in the second chapter of this report, productive programs provide compelling, responsive, and connected learning experiences in STEM.
To ensure that a wide variety of developmentally appropriate opportunities in a STEM learning ecosystem are available to all, there is a need for educational leaders to inventory existing resources, as well as gaps in opportunities, that both reinforce and expand on opportunities in schools. The resulting regional or community STEM learning map should guide program investments and help identify opportunities to leverage existing resources and experiences. Funders and policy makers should encourage program leaders to develop or review existing STEM learning maps to increase the potential for return on their investments and identify opportunities for partnerships.
CONNECT YOUNG PEOPLE WITH OPPORTUNITIES TO LEARN
To support equitable access and participation in out-of-school opportunities to learn STEM, there is a need to identify and train brokers or develop brokering mechanisms that can help families and young people, especially from groups historically underrepresented in STEM, to identify and access settings and programs that help young people “take the next step” in their STEM learning.
A robust STEM learning ecosystem is only effective in the long run if its many and varied opportunities are apparent and available to all school-age children in the community. Funders and policy makers should support efforts to develop brokers who can connect young people with STEM learning opportunities.
Creating connections among learning opportunities will require program managers to provide connections to other programs and opportunities for learning. As young people’s interests deepen or shift, adults need to identify and direct them to new programs or opportunities in which they can advance their learning and pursue out-of-school STEM experiences. Communities need brokers who understand the interests and needs of the young people in their communities and of the STEM learning opportunities available. Brokers can benefit from participation in regional networks that include other brokers and program leaders to enrich and connect opportunities in their communities.
SUPPORT INNOVATIVE EVALUATION APPROACHES
To evaluate out-of-school programs, the field needs innovative measures for program evaluation that will not impinge on the nature of out-of-school learning experiences, are culturally responsive, and are flexible enough to address a wide range of program goals.
A robust STEM learning ecosystem offers a wide variety of programs and opportunities that meet the varied needs of young people and has positive effects on individuals, programs, and communities. To better understand this ecosystem, education leaders, funders, and policy makers should support the development of innovative evaluation approaches that are valid in out-of-school STEM environments, are locally and culturally responsive, and honor the multiplicity of program goals. From an ecosystem perspective, measures need to take into account how young people learn over time; thus, longitudinal studies and innovations in assessment that account for development over time are essential.
A central principle for such novel approaches should be that they do not inadvertently formalize informal settings or disrupt young people’s learning experiences. In addition, there is a need for evaluations that yield rich descriptions of community contexts, program implementation, and learner experiences. Innovations from other fields, such as youth development, should be brought to the out-of-school STEM ecosystem to better investigate the characteristics and qualities of programs.
PROVIDE PROFESSIONAL DEVELOPMENT
To support productive and responsive teaching and learning in out-of-school settings and programs, program staff need opportunities to develop their ability to nurture young people’s interests and understanding of STEM content and practices.
The variety of out-of-school settings and programs that support STEM learning is facilitated by educators and other adults who come to the field with a wide array of prior experiences. Education leaders and program managers should support the professional activities of program staff by planning for and providing ongoing opportunities for professional reflection and learning in content, pedagogy, and instructional design. Professional development should integrate research and practice from multiple disciplines, such as formal education, social work, developmental psychology, urban studies, and similar fields. It is important that time and compensation for participating in professional development activities are provided. Policy makers and funders should invest in efforts to create entry-level and ongoing professional development mechanisms for staff of out-of-school programs.
BUILD AN INFRASTRUCTURE THAT WILL LAST
To develop an effective, sustainable infrastructure of STEM in out-of-school programs for all young people, funders, community leaders, and program leaders need to work together to identify areas for investment, expansion, or redirection.
Only a fraction of the need for programs outside of school is being met, and not all existing programs provide high-quality STEM learning opportunities. Programs are supported by a variety of funding sources, including volunteer organizations, private foundation grants, and local, state, and national agencies; some are fee for service and some are free. To sustain the high-quality programs that are available today and to gradually increase the nation’s capacity to meet existing and future needs, funders, community leaders, and program leaders need to develop a sustainable infrastructure to support long-term growth.
Funder networks should facilitate sharing and collaboration across programs, including both in-school and out-of-school efforts. Community networks should provide administrative support in such areas as professional development, evaluation, assessment, and brokering of opportunities. Networks for program leaders should share strategies for program design, staff development, and documentation of program effects. In building these professional networks and infrastructure, it is critical that they do not lead to a narrowing of possibilities for young people.
ECPLORE HOW STEM LEARNING ECOSYSTEMS WORK
To expand research-based knowledge about productive strategies to support STEM learning in out-of-school settings and programs, there is a need to invest in research that documents both the learning that occurs in individual programs and also how STEM learning develops across settings and over time.
To build on existing knowledge, policy makers and funders should invest in local research-practice partnerships that combine the wisdom of practice and understanding of local conditions and young people with expertise in research and evaluation, while recognizing the challenges to implementation and sustainability of program improvements. The work that is needed includes longitudinal studies of youth trajectories in STEM learning, studies that relate program strategies to learner experiences and outcomes, studies of how brokering local STEM learning opportunities can broaden participation in STEM, and studies that examine how formal and informal STEM learning program designs can reinforce and enrich one another. Also needed is comparative research into questions of how the strategies of different out-of-school STEM program affect participants’ experiences and outcomes and how community or regional contexts influence program implementation and quality.
