As we have established in the preceding chapters, the Science Activation (SciAct) Program supports a wide variety of science, technology, engineering, and mathematics (STEM) activities that engage diverse audiences in both formal and informal learning environments. Project activities and products across the portfolio include but are not limited to hands-on learning experiences, science instructional resources, professional learning opportunities, and data visualization tools. Given the breadth and the diversity of these interventions, the committee sought to organize its analysis of the portfolio’s work by identifying core themes that bridge the work of SciAct awardees with SciAct’s desired goals and objectives.
In order to identify these themes, the committee turned to evidence provided through SciAct’s presentations to the committee and other written documentation. For the PowerPoint slides used by Kristen Erickson in her presentation, see Appendix C on The National Academies Press Webpage at http://www.nap.edu/25569. While this report does not include an analysis of the 24 awarded projects themselves, the committee did review project-specific plans and evaluation reports in search of issues and concerns that were common across projects. In order to qualify as a core theme for this report, the committee needed to see evidence that (1) the issue in question was cited by SciAct as important to the work of the portfolio and (2) the majority of awardees reported implementing their projects with the issue in mind. The results of these investigations converged around four key themes that organize our analyses of the portfolio: STEM learning, leveraging—National Aeronautics and Space Administration (NASA) assets, networks, and broadening participation. It is important to note that these
four themes are not meant to replace SciAct’s current four objectives. They could very well serve as inspiration for future program planning; however, in the context of this report, these themes are used solely as a framework for the committee’s assessment.
Additionally, the committee wishes to note that its review of literature in the following sections is intended to surface major trends and theoretical considerations within each of its four core themes. There is, however, a body of scholarship specific to each programmatic intervention within the SciAct portfolio. Given the breadth and diversity of intervention types included in the SciAct portfolio, it is beyond the scope of this study to delve into these more specific literatures (e.g. teacher professional development, citizen science, development of museum exhibitions). Projects interested in understanding the theory and evidence behind their specific intervention type should consult these respective literatures as appropriate.
In this and the following chapters, the committee addresses each of the above four themes individually. For each theme, we begin by describing major principles in the relevant research literature and what this research suggests about how project activities might lead to desired outcomes. We then examine the collection of projects in the portfolio to explore whether they are collectively employing strategies that reflect the research evidence. We conclude the discussion of each theme with our observations on considerations for future planning as SciAct moves into Phase 2 of the program. In this chapter, we provide our analysis of two of the themes—STEM learning and leveraging NASA assets.
Projects in the SciAct portfolio employ a wide range of strategies to engage learners with STEM, and they vary widely in how explicitly they articulate the underlying logic of their project designs. While the committee does not believe that any single theory of learning should guide all of the projects in the SciAct portfolio, we do believe that it would be beneficial for projects to be explicit about their underlying assumptions about learning and how these assumptions inform the design of their activities. In addition, the design of projects needs to consider what the most current evidence on science learning suggests about effective strategies for supporting learning (also see Chapter 1 for a discussion of the current evidence base on STEM education standards and how people learn).
Principles to Consider for STEM Learning
Current research on STEM learning highlights some key principles that are important for the SciAct portfolio. First, becoming proficient in the dis-
ciplines that make up STEM is more than simply learning content. Instead, it involves engaging people in disciplinary practices, language, and tools in order for them to learn in, through, and about that discipline (National Academies of Sciences, Engineering, and Medicine [NASEM], 2018a). This perspective informs the design of A Framework for K–12 Science Education (National Research Council, 2012a) and the Next Generation Science Standards (NGSS Lead States, 2013) that are based on it. As noted in Chapter 1, this approach as articulated in the Framework calls for students to engage in science and engineering practices to build deeper understanding of the natural and built worlds. Effective learning environments provide opportunities for learners to pose questions, design solutions, gather data, make inferences from that data, build explanations and arguments, and engage in communication with others. By engaging in these practices, learners build more coherent understandings over time (McNeill, Katsh-Singer, and Pelletier, 2015; National Research Council, 2012a).
Second, research clearly demonstrates the pivotal role that learners’ interests, experiences, and concerns play in motivating their decisions to participate in learning activities (NASEM, 2018a). Foregrounding learners’ interests, their personal agency, and the knowledge they bring to science learning opportunities is also a powerful way to interrupt or reverse deficit-based perspectives on who can learn or participate in science (National Research Council, 2009). These theories offer an assets-based perspective on learning in which extended families, parents, and community elders can play a role in designing and evaluating how NASA assets are mobilized for public engagement with science.
Finally, STEM learning occurs across a broad range of settings, time-frames, and experiences (National Research Council, 2006, 2009). This constellation of experiences is often referred to as the “STEM learning ecosystem.” STEM learning ecosystems are emphasized in the 2018 National Science and Technology Council’s Committee on STEM Education (CoSTEM) Strategic Plan as a top strategy for improving STEM literacy in the nation and supporting diversity, equity, and inclusion in STEM, which complement the objectives of SciAct (National Science and Technology Council, 2018). STEM learning ecosystems are commonly conceptualized as the array of learning opportunities, both physical and virtual, available to members of a community. Elements of the ecosystem can include families; school districts; state, local, and tribal governments; the federal government and its facilities; libraries; museums and science centers; community colleges, technical schools, and universities; community groups and clubs; foundations and nonprofits; faith-based organizations; and businesses (see Figure 4-1) (National Science and Technology Council, 2018; Pinkard, 2019; Traphagen and Traill, 2014). In the view of the committee, this ecological perspective is particularly important when considering the broader
learning ecology in which the public engages with science, particularly for learners from communities who have historically been excluded from STEM learning pathways (Bevan, 2016; Ching et al., 2016; Ito et al., 2013; NASEM, 2018a; Pinkard, 2019).
Taken together, these evidence-based conceptualizations of learning mean that providing scientific content is a necessary but not sufficient condition when designing, evaluating, and seeking to improve environments for science learning (NASEM, 2018a). Characteristics of learners and the communities in which they live, learn, and work are also necessary and are equally important to consider. Even in settings where school attendance is compulsory, learning is elective and conditioned by the interests and needs of learners (e.g., Engeström, 1991; Kohl, 1992). This is certainly also true of informal learning environments that are the focus in much of the work of the NASA SciAct portfolio. Thus, it is important to pay careful attention to learner-centered theories that focus on interests and learner agency (Azevedo, 2011, 2018; National Research Council, 2009; Renninger and Bachrach, 2015). This includes identifying assets and background knowledge or beliefs that learners bring to opportunities to engage with science practices, and including these as design considerations (González, Moll, and Amanti, 2006; NASEM, 2018b; National Research Council, 2009).
In recent years, this approach has been extended to include aspects of human-centered and participatory design as a basis for creating and studying learning environments in collaboration with community stakeholders, who have traditionally been seen as recipients of (not participants in) design (e.g., Bang and Vossoughi, 2016; Gutiérrez and Jurow, 2016). These research methods and related findings can help SciAct awardees think about how designed activities are implemented and supported, and in turn, which processes of learning and teaching are likely to be productive as NASA assets are distributed and used in a larger learning ecosystem. In summary, in designing projects that are aimed at supporting STEM learning, it is important to be explicit about the assumptions that underlie the design of project activities; to engage in evidence-based practices for project implementation; to attend to the backgrounds and needs of participants and, when possible, to collaborate with participants as mutual stakeholders (rather than just mere recipients of an experience) in the design of the learning environment (NASEM, 2018b).
Representation of STEM Learning Within the Portfolio
As noted, projects in the portfolio vary widely in the strategies they use to connect to learners. This includes variation in the venue—characterized broadly as informal or formal or both—and in the kinds of activities designed for learners. The committee found inconsistencies in the characterization of each project as focused on informal or formal settings. Consequently, the committee decided to examine more closely the provided information on all 24 projects and employ its own categorization. From the committee’s process, the projects can be categorized as follows:
- formal learning settings—4 projects,
- informal learning settings—6 projects,
- informal and formal learning settings (with no connection between the formal and informal activities)—7 projects,
- informal and formal learning settings (with connection between the formal and informal activities)—6 projects, and
- no information or inappropriate information provided—1 project.
One-fourth of the current projects clearly articulate an ecological approach—making intentional connections across settings—to STEM learning. As for other projects, some learning across different learning environments may occur, but it does not appear to be an intentional goal.
Looking more closely at the specific activities of projects, there appear to be multiple strategies for connecting project design to desired learning outcomes in play as projects mobilize NASA assets. These include
- engaging learners to “get personal” with scientific concepts (e.g., bringing local perspectives on climate change into contact with global data and models);
- creating self-contained “kits” or take-home “backpacks” to support informal science learning activities in home or community settings;
- engaging with community participants across multiple generations, in order to build trust and show respect for local knowledge (important strategies for broadening participation in STEM);
- building directories (online databases) that allow educators or designers to find NASA subject matter experts (SMEs) with interests and backgrounds that match those of prospective learners (also an important strategy for broadening participation in STEM);
- creating online (i.e., computer-based) tutoring and game-like experiences for individual learners;
- utilizing SMEs to communicate science content as part of learning experiences that allow various audiences (i.e., educators, students, and the general public) to explore scientific discoveries; and
- delivering immersive learning experiences in both physical and virtual spaces.
The variety of strategies and settings described above suggests that a systematic study of (and support for) design practices that target specific learning goals and value different kinds of interaction with project stakeholders is warranted. It is unclear to the committee whether all SciAct awardees are using research and evidence in education to inform how their project activities will lead to the desired learning outcomes. For example, because online gaming, adaptive tutoring, and support for family interaction with museum or library exhibits are quite different settings for informal STEM learning, the approach that a project would employ to bring about a desired learning outcome would likely vary. Research and evidence in education can help shed light on best practices for supporting learning in each of these specific contexts.
In an effort to probe what project leaders might be learning from their work with SciAct, the committee asked each of the 24 awardees to reflect on insights and new understandings they developed related to the design of their projects. These project findings were an important source of evidence for the committee, because they indicate that awardees are making discoveries about how to design productively their projects to support learning. Findings across projects, both positive and negative, include the following:
- Despite high levels of interest, students’ family and work expectations sometimes limit their ability to participate in more intensive
- learning experiences such as internships; on the other hand, demand often exceeds capacity for access to more intensive activities, requiring the development of online alternatives.
- NASA SMEs use language and understand life on Earth in ways that are very different from what youth, teachers, or other adults understand, leading to the need for extended negotiations over meaning.
- Time and deliberate effort are required to build trust with local communities, and this is often necessary for increasing the participation of members from underrepresented groups (e.g., working with community elders in tribal communities).
- SciAct awardees/principal investigators experienced difficulty in finding a “sweet spot” or “niche” for learner engagement that aligned local interests with global perspectives on earth and space science (e.g., local versus global data and visualizations of climate change).
- SciAct awardees occasionally changed design strategies midstream to improve implementation (e.g., adding hands-on or constructive activities to increase interest and engagement with NASA visualizations), and users adapted designs to meet local needs (e.g., local variation in how best to use SMEs and “kit” materials).
- Unanticipated collaborations have developed across SciAct projects, sometimes extending the capacity of one partner (e.g., access to larger networks of teachers or other stakeholders) or leading to entirely new projects.
Across projects, the committee finds that while many projects cite STEM learning as a goal, there is often a lack of clarity as to how projects expect that their work will bring about desired learning outcomes. Moreover, even where those plans are specified, they are generally not clearly aligned with the most recent evidence on how to support learning. While many positive learning outcomes are emerging as noted above, the committee believes that projects could be better positioned to support participants’ learning if they more clearly delineate their expectations for the relationship between their work and desired learning outcomes.
Considerations for Future Planning
Based on our review of SciAct projects and presentations, the committee finds that the entire portfolio would benefit from more explicit use of research on learning to inform how NASA’s resources are mobilized in science teaching and learning activities. Variation across SciAct projects means that projects may adopt different approaches to supporting learning with different underlying assumptions about how learning works. Making these underlying assumptions more explicit can lead to greater clarity
about why and how the activities of the project are expected to bring about desired learning outcomes. This clarity can inform both the initial design of the activities and provide insight about how to improve outcomes. As discussed in Chapter 3, this explicit plan that links a project’s activities to its stated goals and objectives— otherwise known as a logic model—can help serve as both a guide for making design decisions as well as a framework for assessing a project’s success. In the final chapter of this report, the committee offers recommendations for how logic models—both at the project level and at the award level—can assist in clarifying and supporting the work of SciAct.
The committee realizes that these observations and the questions they raise about design, learning, and teaching science are complex. But we also see an opportunity to begin asking and answering these questions as the SciAct portfolio transitions into a second phase of NASA support. This would capitalize on informal collaborations across the SciAct network that appear to be emerging (see Chapter 5), and could strengthen and support a STEM learning ecology at the local, regional, or national scale.
While many scientific organizations and federal science agencies are well positioned to support learning about and through scientific practices, we noted in Chapter 1 that NASA brings a wealth of compelling assets (i.e., expert scientists and engineers, datasets, state-of-the-art technology, and data visualizations) developed in the context of the captivating human narratives about specific missions (see Box 4-1). The datasets and visualizations can be used as important tools for public engagement in scientific practices of modeling and computational thinking. In this section, we focus primarily on one of NASA’s biggest assets, its SMEs, as one example of how SciAct is leveraging NASA’s assets. At the end of this section, we offer considerations for future planning across different types of assets.
Principles to Consider for Using SMEs
The involvement of NASA’s SMEs in projects’ activities is an emphasized component of the SciAct Program. In her presentation to the committee, Director Kristen Erickson noted that one of the overarching goals of SciAct is to enable SMEs to share science with multiple audiences. This emphasis on involving SMEs in SciAct projects aligns with previous recommendations from the science-based profession and the federal government calling on scientists to engage more effectively with the public and policy makers, and for federal workers in STEM fields to volunteer their time and expertise toward improving STEM education (Berry, 2012; Leshner, 2012).
Research on broadening participation discusses the importance of reducing social distance between people who pursue science as a profession and people from communities that have historically been excluded from STEM fields and therefore have fewer role models and mentors readily available (Malone and Barabino, 2009; Nelson, 2009). Thus, the SMEs associated with NASA and its missions represent a significant resource for bridging this gap. Role models, mentors, and other social actors are beneficial in supporting people’s relationship to and interest in science, as well as supporting how to navigate and pursue science learning and practice (Jurow, Hall, and Ma, 2008). SMEs have important stories to tell about their own pathways into science, and even in short-term engagements, they can humanize science and what it is to be a scientist. Additionally, research
suggests that in the context of a nonexpert audience, narratives and storytelling are more effective than traditional logical-scientific communication (Green, 2006). They support increased comprehension, engagement, and interest, and reflect the predominant way that the majority of scientific information is disseminated to nonexperts through mass media formats (Dahlstrom, 2014). Through more structured learning activities, SMEs can serve not only as inspirations, but also as role models and mentors to learners in a wide variety of ways, from supervising student summer internships in NASA labs to providing content expertise for citizen science activities. SMEs can also play key nonpublic engagement roles, which include informing the design and focus of STEM programs.
Science communication strategies used by science-based professionals often assume that the listener or learner simply lacks knowledge and the task is to inform them—often called a deficit approach. This approach, however, does not reflect what learning scientists know to be effective (NASEM, 2017, 2018a). Rather than framing this work in ways that are deficit-based (e.g., SMEs coming to communities to disseminate information and share their experiences in a one-way direction), approaches that forge bidirectional relationships, whereby community members can equally inform and inspire scientists by sharing their own relevant experiences and questions, are more effective (NASEM, 2017). There is a small but growing body of research documenting the benefits of two-way communication and engagement, much of it in the social sciences (Frickel et al., 2010). Particularly for communities historically underrepresented in STEM, SciAct could play a unique role in brokering productive engagements for SMEs that move beyond traditional public relations and dissemination approaches to more substantive strategies that involve deeper engagement with communities of learners (Ching et al., 2016). Indeed, the involvement of NASA SMEs as a participant in dialogues about science has the potential to support the development of community science literacy (see Chapter 2 for a discussion of the different dimensions of science literacy).
Indeed, science communication training that is informed by current evidence on effective communication plays a critical role in developing science-based professionals who can engage effectively with public audiences. However, both a study looking at the views on science communication training held by members of the American Association for the Advancement of Science (AAAS), and a study on the quality of training delivered by science communication trainers revealed that training of this kind focuses primarily on developing specific skills, and is only loosely based on the social science research on effective communication (Besley, Dudo, and Storksdieck, 2015; Besley et al., 2016). Efforts to enhance the science communication skills of SMEs would do better not only to focus on developing skills for information transmission but also to emphasize the effectiveness of engagement activities
(i.e., public engagement can build trust and improve research and access to knowledge). Evidence indicates that when effective training and connection to appropriate audiences are included, scientists are appreciative and positive about the science communication training they receive. For example, scientists trained through the Portal to the Public (PoP) Program1 were more likely to display science communication and engagement abilities and skills that align with evidence-based practices in informal science education and science communication, and self-reported increases in motivation to do the work in their regular profession and interest in conducting more outreach and engagement activities, as a result of participation in the program (Storksdieck, Stylinski, and Canzoneri, 2017).
Representations of Asset Use in the Portfolio
SMEs greatly enrich the scientific currency of the SciAct Program. The descriptions of projects across the portfolio identify a number of ways that SMEs are involved in providing expertise and materials for use by a variety of audiences, including
- providing datasets to be put into forums for manipulation by public audiences;
- assisting with the development of curriculum materials and exhibition content;
- providing data to be converted to public-friendly visualizations;
- mentoring interns or online learners throughout project participation; and
- acting in mission-linked, scripted roles in video productions.
In most of these cases—if not all—SMEs are working with project staff who help ensure that complex scientific information is made accessible to participants. In these cases, SMEs generally do not bring (nor do they need) additional expertise in appropriate theories of learning or of science communication to their work.
When SMEs have direct engagement with public audiences, however, they may benefit from additional expertise in learning theories and science communication strategies in order to engage effectively with the audience.
1 Portal to the Public (PoP) is a program developed as a collaboration between Pacific Science Center, Explora, North Museum of Natural History and Science, and the Institute for Learning Innovation. The program trains and supports science-based professionals (“scientists”) in outreach and engagement activities, based on the premise that scientists should engage directly and through materials-rich, hands-on activities with family audiences in ways that reflect basic understanding on how people learn and how to engage audiences in discovery-based learning.
This might include strategies for leveraging the wide range of cultural assets that different groups bring to STEM that may be different from (or even challenge) dominant cultural norms of STEM in academia and the professions (Bevan, Calabrese Barton, and Garibay, 2018).
Currently, there is little documentation of how the SMEs within SciAct are prepared to be effective science communicators with the public. One project, however, did identify an explicit mechanism for providing training to SMEs in science communication by offering Stony Brook University’s Alan Alda training in science communication to the SMEs associated with the project. However, it is not clear how comprehensive the training was for all of the SMEs involved. Moreover, it was difficult to know whether or not this project represents an isolated case or if other projects are also concerned with the science communication skills of their SMEs. SciAct may want to consider strategic coordination of these efforts to support SMEs across its entire portfolio.
In terms of the use of assets beyond SMEs, SciAct projects are deploying a number of strategies. These strategies typically use some combination of mission-generated data or visualizations of these data, but there are also distinct uses of
- physical tools and associated methods of observation or measurement,
- live data or observational “feeds” that can be featured in teaching and learning activities,
- leveraging use of select NASA missions as internship sites for teachers or students (middle school or older), and
- using the occasion of a major event (e.g., solar eclipse) to engage public audiences.
It is clear to the committee that ambitious and innovative projects are under way. However, there is not yet a logic model that shows how NASA assets (i.e., content, SMEs, existing infrastructure) are being used to design activities (e.g., outreach events, educator professional development, and infrastructural resources), and also how these activities are organized to support teaching and learning in ways that are expected to lead to positive outcomes in STEM education.
Considerations for Future Planning
Exploring how NASA’s assets could be integrated into public engagement programs to deepen participants’ desired learning outcomes would be a major contribution of the SciAct Program. With respect to aligning global perspectives with local interests of learners, SciAct might further support the work of its awardees by creating shared tools that would enable
NASA’s global science data to be synthesized more easily and made modular and useful for local and regional investigations, insights, and applications. Across the current portfolio there are a number of efforts that have increased access to NASA’s resources, but greater attention could be given to ensuring that assets are intentionally employed to promote learning.
Additionally, by focusing on leveraging its unique and compelling assets within the broader science learning and education landscape, SciAct will be better positioned to demonstrate measurable impacts. For example, SciAct can support the creation of unique and engaging opportunities for learners of all ages to participate in scientific practices, including modeling, computational sciences, and other forms of inquiry. SciAct can also provide social networks, role models, and mentors that can support learners to deepen current STEM engagement experiences and seek out new pathways of engagement. These represent measurable ways for SciAct to diversify the range of opportunities to engage with STEM, in NASA-specific ways that can enrich the learning ecosystem (support its diversity, scale, and local adaptation).
Looking across projects in the SciAct portfolio, the committee could not identify a coordinated strategy across projects for how NASA assets are being used to design educational activities that have observable outputs (e.g., media products or citizen science efforts). Further, because projects were not universally making explicit the theory of learning behind their project design, it was also difficult to identify why these outputs might lead to STEM learning, other than through availability or exposure (i.e., very simple theories of learning or teaching). It is the view of the committee that the SciAct Program has yet to develop a logic model and theory of change (Funnell and Rogers, 2011) that can continually guide design and dissemination activities across the portfolio. A clear articulation of a logic model and theory of change could also help to develop an integrated understanding of how and why designed activities influence learning and teaching in the STEM education ecosystem at local, regional or national scale.
NASA’s assets are invaluable resources to support STEM learning in a variety of contexts, as illustrated by the number of ways they have been utilized within the SciAct portfolio. Some assets have been deeply integrated into the project activities to support STEM learning (e.g., data and visualization tools), whereas other assets are not being effectively mobilized as part of the current efforts. Utilizing SMEs to engage efficiently and effectively with learners is an emphasized priority of NASA SciAct and one of its predominant ways for connecting its content and discoveries to the public domain. However, engaging SMEs in effective science communica-
tion training is not pervasive across the portfolio, which has implications for the program’s potential to promote STEM learning and science literacy, and broaden participation. In cultivating effective learning environments, considerations that go beyond access and exposure need to be taken into account, including the underlying theory of learning; specific project design strategies; the specific communities being engaged; whether or not an ecological approach to learning is employed; and whether there is alignment with the national vison for science, engineering, and technology education (i.e., the Framework and the Next Generation Science Standards). Additionally, a greater understanding of how the assets and outcomes are connected (i.e., an articulated logic model and a theory of change) will enhance the potential impact of SciAct’s education activities in the broader STEM landscape.
CONCLUSION 9: While science, technology, engineering, and mathematics learning is in the foreground as a Science Activation goal, there is no explicit link between theories of learning and how the National Aeronautics and Space Administration assets are used (e.g., transmission models, inquiry-based practices). There is a range of design intervention strategies that are used across the portfolio. Each project uses different theories of learning in its project design and often that theory of learning is not made explicit.
CONCLUSION 10: Current research on learning emphasizes the importance of learner-centered and community-centered instructional design and practices. Awardees have had uneven success at mobilizing the National Aeronautics and Space Administration assets while also being responsive to the needs of learners and communities.
CONCLUSION 11: Given the portfolio’s emphasis on the value of subject matter experts, the portfolio lacks a coordinated effort to incorporate evidence-based practices in translating their expertise in developing and implementing educational materials and learning experiences (e.g., translating datasets, engaging in public outreach).