2

Vision and Objectives

As mentioned in Chapter 1 of this report, one major component of the committee’s statement of task was assessing Science Activation’s (SciAct’s) progress toward its stated goals. In order to do this, the committee first needed to fully understand how the National Aeronautics and Space Administration (NASA) is conceiving of the work of SciAct: that is, what the stated goals of SciAct are, and how those goals relate to the U.S. agenda for science, technology, engineering, and mathematics (STEM) education, as well as how well positioned SciAct is to address these goals. In this chapter, we consider how NASA articulates the goals and objectives of the SciAct portfolio, and offer insight into how the SciAct portfolio fits into the larger landscape of work in STEM education in the United States. This conversation will set the stage for our later discussions of the implementation of SciAct’s work, helping to provide a framework for our assessment of SciAct’s strengths and challenges.

SciAct’S CENTRAL VISION

In order to understand the role of NASA’s SciAct Program within the broader domain of STEM education in the United States, the committee looked to multiple sources of evidence for documented descriptions of NASA’s vision for SciAct. According to SciAct’s documentation, its vision is

Embedded in this statement are multiple assumptions about the role and value of NASA that are important to understand. First, it is clear from this statement that part of the impetus for engaging in STEM education related work is to provide a “return on the public’s investment” in NASA. In this way, SciAct is positioning itself as a mechanism for NASA to be in service to the public that goes beyond its science-related mandates. Along these same lines, embedded in this vision is the notion that NASA science can contribute in productive ways to science education work more broadly. NASA’s work has the potential to excite and engage the public in ways that can support STEM learning. The committee recognizes that these core tenets are central to how NASA understands the role and purpose of the SciAct portfolio, and it is within this context that NASA has devised its objectives for SciAct, designed its portfolio, and implemented its awards.

When the committee attempted to understand how SciAct sought to put this vision into practice, it turned to the SciAct Cooperative Agreement Notification (CAN), which describes the portfolio strategy. The CAN notes that part of the rationale for SciAct is to “increase the overall coherence of the Science Mission Directorate (SMD) science education program leading to more effective, sustainable, and efficient utilization of SMD science discoveries and learning experiences and to meet overall SMD science education objectives. Fundamental to achieving this outcome is to enable NASA scientists and engineers to engage more effectively with learners of all ages [emphasis added]” (National Aeronautics and Space Administration, 2015). Here, the committee notes another core tenet behind the current design and implementation of SciAct: Engaging NASA subject matter experts (SMEs, or “NASA scientists and engineers”) with learners is central to supporting NASA’s goals for science education, and it is essential that this engagement is effective or strategic toward the organization’s stated goals and objectives. When these tenets were viewed in concert with one another, the committee was able to see how NASA understands both its role in and obligation to supporting STEM education in the United States; that is, NASA is funded by the nation’s taxpayers, and as a result, it is incumbent upon the agency to support the nation’s commitments and goals as they relate to STEM. These public investments have led to an untold number of discoveries and innovations in the STEM fields, and one way that NASA can be of service to the public is to share

___________________

¹ See https://www.SMDEPO.org.

the fruits of these investments as a mechanism for supporting STEM education. NASA’s SMEs are viewed by the agency as one of the primary vehicles for facilitating education and engagement work. Taken together, these tenets form the foundation upon which the SciAct portfolio is designed, implemented, and evaluated.

SciAct’s Four Overarching Objectives

Resting on this foundation, SciAct has identified four primary objectives for the portfolio:

1. enable STEM education,
2. improve U.S. scientific literacy,
3. advance national education goals, and
4. leverage efforts through partnerships.

Further, SciAct documentation defines NASA assets and resources to include

• exciting science and engineering content that engages audiences and motivates them to learn more;
• SMEs, including scientists and engineers, who ask compelling scientific questions and then find ways to answer them within the environment of space;
• real-life participatory and experiential opportunities (which includes student collaborations, e.g., suborbital balloon experiments and other student launch opportunities); and
• other science programs in NASA’s infrastructure (e.g., GLOBE, Night Sky Network, etc.)

SciAct awardees are regarded as NASA partners who bring educational expertise in designing and delivering the mechanisms for target audiences to learn and understand science content related to the four disciplines of NASA’s SMD (heliophysics, earth science, planetary science, and astrophysics). For more detail on the ways in which the current SciAct portfolio addresses this task, see Chapter 3 of this report.

SciAct intends for the work of individual project awards to aggregate toward achieving the four objectives above; that is, no one project is supposed to accomplish all of these objectives on its own. In this section, we describe each of these objectives in greater detail, and consider each in light of SciAct’s resources and position in the STEM education landscape. Chapter 3 of this report will delve more deeply into how projects are positioning themselves to support these four objectives.

Objective 1: Enable STEM Education in All 50 States

The SciAct CAN describes this objective by noting that

The committee notes that this first objective is, in some ways, a restatement of SciAct’s vision statement: SciAct intends to support STEM education through the mobilization of NASA’s existing resources and assets. Practically, this means that awardees are tasked with translating NASA assets for educational use by creating new materials that can be used by learners or educators and incorporated into programs that will allow learners and students to engage with NASA data and SMEs. In fact, the importance of engaging SMEs is made explicit in that awardees are required to use at a minimum one SME and report monthly on SME involvement in their projects. For SciAct awardees, this objective is further defined as “enable STEM education in all 50 states,” an objective that prioritizes widespread implementation. The CAN calls out the need to balance this objective with meeting the needs of local communities.

Currently, SciAct measures its progress toward this objective by using geographic dissemination as a portfolio-wide metric to assess the reach of programs and products. Additionally, it counts the number of SMEs participating in the portfolio’s work. The committee notes that these metrics characterize the portfolio’s work; that is, they speak to the volume and distribution of SMEs engaged in the work, and they describe who is being reached through SME participation. These metrics do not, however, measure whether the awards are, in fact, enabling STEM education, because they do not capture the extent to which educational activities (not to mention teaching and learning) are actually occurring as a result of the awards or the engagement of SMEs. The committee notes that the phrasing of this objective makes it particularly challenging to quantify: What is meant by “enable STEM education”? Is STEM learning occurring as a result of participation in the projects? Or is this objective trying to capture the quantity of resources and assets NASA is disseminating? The committee observes that while the idea of enabling STEM education captures NASA’s vision for SciAct, it is too broad to be measurable with respect to the impact of the SciAct portfolio.

Objective 2: Improve U.S. Scientific Literacy

According to SciAct documentation, this objective is motivated by the notion that a scientifically literate population is necessary to support

the nation’s economic development, and that NASA has a role to play in improving the nation’s science literacy. In order to understand how SciAct expects to support U.S. science literacy, the committee investigated how awards self-report their own progress toward this objective.

SciAct uses one of its awardees (the continuation of the Science and Engineering Indicators data collection that tracks science literacy at the national level) to assess the portfolio’s progress toward this objective. This project furnished documentation that helped the committee understand how SciAct is operationalizing the term “science literacy.” Based on this evidence, the committee finds that SciAct understands science literacy as

Unfortunately, this metric is not connected to the articulated outcomes of the other individual SciAct projects and, further, does not measure some aspects of science literacy.

To the extent that SciAct positions the improvement of science literacy as one of its primary objectives, the committee feels it is important for SciAct to draw upon the most recent scholarly assessments of how science literacy is defined, enacted, and measured.² Recent research on science literacy posits that the term should encompass more than just basic knowledge of science facts. Indeed, contemporary definitions of science literacy have expanded to include understandings of scientific processes and practices, familiarity with how science and scientists work, a capacity to weigh and evaluate the products of science, and an ability to engage in civic decisions about the value of science. Although science literacy has traditionally been seen as the responsibility of individuals, individuals are nested within communities that are nested within societies—and as a result, individual science literacy is limited or enhanced by the circumstances of that nesting.

In accordance with Miller’s (1998) definition, there are three aspects of science literacy common to most applications of the term: content knowledge, understanding of scientific practices, and understanding of science as a social process. However, the committee notes that there are four additional aspects of science literacy that, while less common, provide important

___________________

² The text in this section draws heavily on Science Literacy: Concepts, Contexts, and Consequences (National Academies of Sciences, Engineering, and Medicine, 2016).

insight into how the term has been used: foundational literacy, epistemic knowledge, identifying and judging scientific expertise, and dispositions and habits of mind. Given this range of aspects, it is not surprising that there is no clear consensus about which aspects of science literacy are most salient. The committee notes that depending on the context, different aspects of science literacy may be more or less important or desirable.

Expanding contemporary perspectives on science literacy encompass the ways that broader social structures can shape an individual’s science literacy. Indeed, contemporary scholarship is beginning to push back against the common understanding that science literacy is or should be seen only as a property of individuals—something that only individual people develop, possess, and use. Research on individual-level science literacy provides invaluable insight, but it likely offers an incomplete account of the nature, development, distribution, and impacts of science literacy within and across societies. Societies and communities can possess science literacy in ways that may transcend the aggregation of individuals’ knowledge and accomplishments.

Science literacy can also be expressed in a collective manner; that is, resources are distributed and organized in such a way that the varying abilities of community members work in concert to contribute to their overall well-being. Community science literacy does not require that each individual attain a particular threshold of knowledge, skills, and abilities; rather, it is a matter of that community having sufficient shared capability necessary to address a science-related issue. Research in this field is still emergent, though documented cases of communities’ efforts to leverage collective science knowledge, skills, and abilities in pursuit of science-related policy outcomes abound: examples of these cases include, among many others, efforts by the LGBT community to impact AIDS treatment policy (Epstein, 1995), families collaborating to confirm the existence of cancer clusters (Brown, 1993), and community monitoring of the effects of fracking on local watersheds (Kinchy, Jalbert, and Lyons, 2014). Because community science literacy requires that communities organize and call upon a diversity of knowledge bases and skill sets present in their collective, research can now document the ways in which communities can capitalize on individuals’ respective strengths in attaining their goals.

In terms of measurement, research on science literacy at the individual level has largely assessed individuals’ knowledge using content knowledge assessments and measures of understanding of scientific principles administered through large public surveys. These widely used surveys have provided valuable insight into science knowledge, but constraints on length and demands for comparability over time and across nations mean that they may be limited in what they can capture about science literacy. Most of the literature on science literacy assesses the relationship between science knowledge and attitudes toward, perceptions of, and support for science,

but there is a growing body of literature on additional contemporary perspectives beyond literacy and these include science capital (at the individual and community levels), science agency and identity, and science engagement (Archer et al., 2014, 2015; Calabrese Barton and Tan, 2010; Carlone and Johnson, 2007; Vossoughi and Vakill, 2018). These newer frameworks have particular implications for the types of interventions that would be most effective for increasing public participation in and engagement with science, which are both articulated goals in the vision statement of NASA SMD. Additionally, this highlights the importance of measuring the full range of science literacy dimensions in order to reliably understand the impact of SciAct on U.S. science literacy.

In the context of SciAct, individual awardees do not consistently document outcomes related to the dimensions of science literacy identified by Miller (1998). Some projects track audience exposure to and/or experience with authentic NASA science about the Earth, the solar system, and the universe, which potentially contributes to science literacy but is just one facet of a complicated picture. Approximately half of the projects measure the change in content knowledge in the audiences engaged; only three projects attempt to gauge contextual issues, such as attitude, behavior, and identity, and none considers science literacy in the context of a community. Moreover, even if all awards were consistently measuring outcomes, the number of factors contributing to science literacy at the national level mean that it is extremely challenging to isolate SciAct as the sole contributing factor to whether or not Americans are becoming more or less science literate.

As the space agency of the United States, it is indeed laudable for NASA to use its unique and inspirational assets and SME’s to advance public science literacy. However, given both the multifaceted nature of the concept, as well as the unlikelihood that a national measure would be able to isolate SciAct’s impact, there are a number of concerns that prevent this objective from being fully actionable and measurable for the SciAct portfolio.

Objective 3: Advance National Education Goals

Because of NASA’s participation in the integrated work of federal agencies in supporting STEM education (Federal Science, Technology, Engineering, and Mathematics (STEM) Education Five-Year Strategic Plan [Committee on STEM Education, 2013]), it makes sense that NASA would endeavor to support the educational goals outlined in the plan. The SciAct CAN emphasizes supporting four priority areas from this strategic plan:

1. Improve STEM instruction: Prepare 100,000 excellent new K–12 STEM teachers by 2020, and support the existing STEM teacher workforce.

2. Increase and sustain youth and public engagement in STEM: Support a 50 percent increase in the number of U.S. youth who have an authentic STEM experience each year prior to completing high school.
3. Enhance STEM experience of undergraduate students: Graduate 1 million additional students with degrees in STEM fields over the next 10 years.
4. Better serve groups historically underrepresented in STEM fields: Increase the number of students from groups that have been underrepresented in STEM fields who graduate with STEM degrees in the next 10 years, and improve women’s participation in areas of STEM where they are significantly underrepresented.

In its review of project activities, the committee found that SciAct is pursuing this objective primarily through activities that

• provide professional development to inservice educators (15 projects);
• provide authentic science experiences for students or citizens (23 projects); and
• target specific underrepresented populations either through direct engagement or by providing resources and professional development to educators serving students from underrepresented populations (11 projects).

While SciAct is currently tracking how individual projects support some of the specific goals in the Committee on STEM Education (CoSTEM) reports, there is currently no mechanism to measure how the work of the portfolio is aggregating toward the constellation of CoSTEM goals. Moreover, given the diversity and breadth of these goals, attempting to measure SciAct’s progress would require a considerable investment of resources. As a result, the committee finds that while advancing national education goals broadly informs the SciAct Program vision and some project activities support this objective, this objective is not measurable with respect to the SciAct portfolio overall.

In 2018, CoSTEM released a new strategic plan with three new goals³

1. Build strong foundations for STEM literacy by ensuring that every American has the opportunity to master basic STEM concepts, including computational thinking, and to become digitally literate. A STEM-literate public will be better equipped to handle rapid

___________________

³ See https://www.whitehouse.gov/wp-content/uploads/2018/12/STEM-Education-StrategicPlan-2018.pdf.

1. technological change and will be better prepared to participate in civil society.
2. Increase diversity, equity, and inclusion in STEM and provide all Americans with lifelong access to high-quality STEM education, especially those historically underserved and underrepresented in STEM fields and employment. The full benefits of the nation’s STEM enterprise will not be realized until this goal is achieved.
3. Prepare the STEM workforce for the future—both college-educated STEM practitioners and those working in skilled trades that do not require a 4-year degree—by creating authentic learning experiences that encourage and prepare learners to pursue STEM careers. A diverse talent pool of STEM-literate Americans prepared for the jobs of the future will be essential for maintaining the national innovation base that supports key sectors of the economy and for making the scientific discoveries and creating the technologies of the future.

Because these new goals were delineated after the fact of SciAct’s creation, the current portfolio of awards was not assembled to respond to the most current CoSTEM report. As the program moves forward and projects are added to the portfolio, it is important to consider how SciAct might align with these new national goals.

Objective 4: Leverage Efforts Through Partnerships

This objective, unlike the other three top-level science education objectives, is programmatic and potentially strategic in nature. The most explicit rationale for including leveraging efforts through partnerships as a program objective is offered through reference to the CoSTEM report (Committee on STEM Education, 2013, p. 7). SciAct’s CAN includes the following quote from the report:

Essentially, the rationale referenced here points to the reality that the federal government is but one of many institutions that has a role to play in

promoting STEM education and that working independently of these other institutions would miss an opportunity to broaden the impact of federal investments.

In its review of SciAct documentation, the committee identified two broad goals that NASA appears to have for SciAct partnerships: disseminating NASA SMD assets and broadening participation in STEM.⁴ The committee turned to SciAct’s own documentation of the relationships across projects for greater clarity on how SciAct intends to leverage partnerships (see Figure 2-1).

Figure 2-1 depicts a partner ecosystem made up of content partners, listed on the left of the figure (e.g., NASA Heliophysics, NASA Earth); dissemination partners, listed in the center of the figure (e.g., PBS Learning Media, NISENet); audience partners listed center right on the figure (e.g., planetariums, Girl Scouts, libraries); and infrastructure partners, listed at the bottom of the figure (e.g., Night Sky Network, American Camp Association). This partner ecosystem suggests the particular forms of partnership valued in SMD—those that directly leverage and aim to disseminate NASA SMD assets. These kinds of partnerships involve connections between

___________________

⁴ In the following chapters, we discuss the issue of broadening participation—how it is understood in the SciAct portfolio, how it is measured, how the concept could be better operationalized—in depth. Despite the committee’s observation that broadening participation in STEM is one of SciAct’s stated (but unofficial) aims, it is worth noting that Figure 2-1 does not explicitly describe the ways that leveraging partnerships will lead to broadened participation.

content partners who work with dissemination partners who connect with audience partners, and potentially leverage existing infrastructure partners.

Leverage efforts through partnerships can be measured in multiple ways, particularly with more specific framing of partnership expectations going forward. Strategically, this objective is central to how the SciAct Program is envisioned and designed. However, SciAct is not currently using any portfolio-level mechanism to measure its progress toward this objective.

SUMMARY

At its core, the SciAct Program aims to bring unique NASA expertise and assets, including people, missions, products, data, and scientific results, to a diversity of learners effectively and efficiently. The SciAct awardees represent a critical piece of that vision by providing the educational expertise to translate NASA science to different types of learners and users. The committee applauds the aspirations of the SciAct Program overall and finds that three of the top-level science education objectives—enable STEM education, improve U.S. scientific literacy, and advance national education goals—describe ends to which the program wishes to contribute, including the desire to contribute to the larger education agenda of the federal government agencies. These objectives essentially inform the SciAct Program vision. The fourth objective, leverage efforts through partnerships, is a central programmatic objective and potentially a strategic goal that can be defined more specifically going forward.

CONCLUSION 1: The National Aeronautics and Space Administration has a unique role to play in the science, technology, engineering, and mathematics education landscape, but the current four objectives for the Science Activation Program are too broad and do not appropriately reflect that role.

CONCLUSION 2: The four current Science Activation Program objectives are general enough to inform a vision for the program, however they lack specific, actionable targets. As currently stated, the objectives are so broad that they obscure a clear understanding of how awardees’ contributions aggregate toward desired outcomes.

CONCLUSION 3: Improving science literacy at the national level is one of the four Science Activation (SciAct) Program objectives. We do not have evidence that there is a centrally agreed-upon definition of science literacy across the projects. While, the approach to measuring science literacy at the national level that SciAct is currently using reflects one approach to measuring science literacy, it does not fully reflect the most up-to-date conceptualizations of science literacy

This page intentionally left blank.