4
Analysis of NASA’s K-12 Education Portfolio

In this chapter we present our analysis of NASA’s portfolio in K-12 science, technology, engineering, and mathematics (STEM) education with particular attention to program design and effectiveness. The committee reviewed the seven core projects in the headquarters Office of Education Elementary and Secondary Program in depth: the Aerospace Education Services Project; NASA Explorer Schools; Digital Learning Network; Science, Engineering, Math and Aerospace Academy; the Education Flight Projects; Educator Astronaut Project; and the Interdisciplinary National Science Project Incorporating Research and Education Experience (INSPIRE).

The committee also reviewed some of the projects and activities in the Science Mission Directorate (SMD). Our review of the Science Mission Directorate projects was less detailed, as an in-depth review of such a large portfolio was beyond the scope of our study. The committee did believe it was necessary to give some attention to the SMD projects; however, because they represent approximately one-half of the agency’s funding in K-12 education. Including these projects in the review gave the committee a better overall perspective of the scope of the agency’s work at the precollege level. This chapter does not include analysis of individual SMD projects; however, we do discuss the general approach to education projects used in SMD and mention individual projects as examples.

The committee used several strategies for reviewing the seven core projects. We received briefings from NASA staff on each project, and we reviewed administrative documents, annual reports, and recent external evaluations. Committee members also drew on their knowledge of research in K-12 education regarding best practices in developing students’ inter-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 57
4 Analysis of NASA’s K-12 Education Portfolio I n this chapter we present our analysis of NASA’s portfolio in K-12 science, technology, engineering, and mathematics (STEM) education with particular attention to program design and effectiveness. The com- mittee reviewed the seven core projects in the headquarters Office of Educa- tion Elementary and Secondary Program in depth: the Aerospace Education Services Project; NASA Explorer Schools; Digital Learning Network; Science, Engineering, Math and Aerospace Academy; the Education Flight Projects; Educator Astronaut Project; and the Interdisciplinary National Science Project Incorporating Research and Education Experience (INSPIRE). The committee also reviewed some of the projects and activities in the Science Mission Directorate (SMD). Our review of the Science Mission Directorate projects was less detailed, as an in-depth review of such a large portfolio was beyond the scope of our study. The committee did believe it was necessary to give some attention to the SMD projects; however, because they represent approximately one-half of the agency’s funding in K-12 edu- cation. Including these projects in the review gave the committee a better overall perspective of the scope of the agency’s work at the precollege level. This chapter does not include analysis of individual SMD projects; however, we do discuss the general approach to education projects used in SMD and mention individual projects as examples. The committee used several strategies for reviewing the seven core projects. We received briefings from NASA staff on each project, and we reviewed administrative documents, annual reports, and recent external evaluations. Committee members also drew on their knowledge of research in K-12 education regarding best practices in developing students’ inter- 

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM ests in science, technology, engineering, and mathematics; curriculum and instruction; and professional development as a framework against which to compare NASA K-12 projects. This expert knowledge was critical for the committee analysis because of the limitations of existing project evalu- ations. These limitations are not unique to NASA but are reflected across many federal science agencies involved in STEM education: see the report of the Academic Competitiveness Council (U.S. Department of Education, 2007a); also see Chapter 5 for an in-depth discussion of evaluation. From its analyses of individual projects, the committee identified three areas in which NASA can improve the quality of its K-12 education pro- gram: project design and improvement, use of expertise in education, and the connection to the science and engineering in the agency. Before present- ing our analysis, we lay out the frameworks that guided that analysis. FRAMEWORK FOR BEST PRACTICE From its review of research and the members’ expertise, the committee identified three major topics that connect to NASA’s program goals and encompass most of the activities of the constituent projects: developing inter- est; curriculum and instruction; and professional development for teachers. For each of these topics, the committee identified major conclusions that can be drawn from the research evidence regarding principles for best practice. In the following section, we briefly review these principles, which are then used as a framework for the critique of the constituent projects. Developing and Sustaining Interest Inspiring, engaging, and sustaining the interest of teachers and students in STEM subjects is one of the main goals of NASA’s current education program, and is one of the greatest contributions that NASA can make to K-12 STEM education. The excitement generated by space flight and exploration puts NASA in a unique position to draw teachers and students into science, technology, engineering, and mathematics and related fields. However, of equal importance to the need to attract the interest of teachers and students is the need to sustain that interest over time and to link it to meaningful science content. Substantial research has been done on the development of students’ and teachers’ motivations and interests, with some attention to how to design learning experiences that are both engaging and that result in real learning. In this research, “interest” is defined as both a positive feeling for science and the predisposition to continue to engage in science (Hidi and Renninger, 2006). Interest, in this sense, includes the stored knowledge, stored values, and feelings that influence the engagement, questioning,

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO and activity of individuals (or groups of individuals). Interest has positive consequences for learning. For example, when people—both young and old—have a real interest in science, they are more likely to pose questions out of curiosity, seek out challenges, and use effective learning strategies (Barron, 2006; Csikzentmihayli, Rathunde, and Whalen, 1993; Engle and Conant, 2002; Kuhn and Franklin, 2006; Lipstein and Renninger, 2006; Renninger, 2000; Renninger and Hidi, 2002). Early on, interest may be primarily triggered or maintained by external experience. As interest develops and deepens, however, a person is more likely to initiate engagement and to generate and seek answers to questions about content (Renninger, 2000). NASA’s program in K-12 STEM educa- tion has the potential to trigger initial interest in students and teachers, as well as to provide experiences to deepen engagement for those who already have some initial interest. Two challenges for NASA in designing activities to “inspire and engage” are to attend to what is needed to translate initial excitement into a meaningful learning experience and a sustained, long- term interest and to support teachers in providing appropriate follow-up activities for an initial activity. Reaching and engaging students who are typically underrepresented in STEM fields is a challenge that many of NASA’s programs, particularly those managed by headquarters Office of Education, are designed to address. Although research on the most effective ways to bring underrepresented populations into STEM fields is thin, the evidence does suggest guidelines for best practice (BEST, 2004; Hall, 2007). One set of best practices was developed by the Building Engineering and Science Talent Initiative (BEST, 2004) through an expert review of programs. The practices include • Defined outcomes: Students and educational staff agree on goals and outcomes. Success is measured against the intended results. Outcome data provide both quantitative and qualitative informa- tion. Disaggregated outcomes provide a basis for research and continuous improvement. • Persistence: Effective interventions take hold, produce results, adapt to changing circumstances and persevere in the face of setbacks. Conditions that ensure persistence include proactive leadership, sufficient resources, and support at the district and school levels. • Personalization: Student-centered teaching and learning methods are core approaches. Mentoring, tutoring, and peer interaction are integral parts of the learning environment. Individual differences, uniqueness, and diversity are recognized and honored. • Challenging content: Curriculum is clearly defined and understood. Content goes beyond minimum competencies; relates to real-world applications and career opportunities and reflects local, state, and

OCR for page 57
0 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM national standards. Students understand the link between content rigor and career opportunities. Appropriate academic remediation is readily available. • Engaged adults: Adults provide support, stimulate interest, and create expectations that are fundamental to the intervention. Edu- cators play multiple roles as teachers, coaches, mentors, tutors, and counselors. Teachers develop and maintain quality interactions with students and each other. Active family support is sought and established. A flexible program structure and opportunities for students to work in groups and socialize are also important based on a literature review com- missioned by the committee (Hall, 2007). Curriculum and Instruction Many of NASA’s contributions in K-12 STEM education fall under the category of curriculum materials and instructional activities. NASA seeks to provide curricular support resources that “use NASA themes and con- tent to enhance student skills and proficiency in STEM disciplines, inform students about STEM career opportunities, and communicate information about NASA’s mission activities” (National Aeronautics and Space Admin- istration, 2006c). Science curricula, for the purposes of this discussion, are defined as having three components: curriculum standards, curriculum materials, and instructional activities. Curriculum standards are the learning goals estab- lished collectively by national standards, state science expectations (e.g., state standards, state core curriculums, state expected learning outcomes), and dis- trict science curriculum guidance (e.g., guidelines, blueprints, learning expec- tations). Curriculum materials include textbooks, materials or labs, videos and other audio-visual materials, and reading materials. Instructional activi- ties comprise the lesson plans, students’ laboratory and field experiences, and modeling activities. NASA’s work in K-12 STEM education focuses on cur- riculum materials designed to support NASA-related instructional activities. A teacher’s decision to incorporate those activities should be informed by the curriculum and standards that apply for the course in question. Curriculum standards lay out the science content and processes essen- tial for science literacy and preparation for STEM pursuits. They provide a blueprint for the development of essential knowledge and skills and cultivation of scientific habits of mind for all students. The key role of cur- riculum standards is to bring coherence, articulation, and focus to instruc- tion. Over the last 10–15 years there has been a movement toward creating standards at the national and state level that provide a framework to guide

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO educators at the local level (National Research Council, 2007b). NASA has recognized this movement and has taken steps in its work with schools to show how the materials the agency offers are aligned with national and state standards. In general, curriculum materials should, at a minimum, meet four criteria to be useful in improving student learning and achievement: 1. They should be aligned to the specific instructional objectives of the state and district standards. 2. They should be pedagogically sound. 3. They should be engaging and relevant. 4. They should be accurate in their presentation of scientific information. The National Science Education Standards suggest that “[e]ffective sci- ence curriculum materials are developed by teams of experienced teachers, scientists, and science curriculum specialists using a systematic research and development process that involves repeated cycles of design, trial teaching with children, evaluation, and revision” (National Research Council, 1995, p. 213). Research also shows that successful implementation of curriculum or of particular instructional activities and strategies usually requires some form of professional development for teachers. Indeed, increasing the effective use of high-quality instructional materials is at the center of many educa- tional reform efforts. The National Science Foundation’s Local Systemic Change in Mathematic and Science Program stressed the importance of the use of quality instructional materials with linked professional develop- ment. The evaluation of this program found that extensive use of even first rate instructional materials was effective only when linked to professional development targeted at teachers’ practice, investigation, problem-solving, and instruction (Banilower et al., 2006). Michael Lach, director of Mathematics and Science for the Chicago Public Schools, in his remarks to Congress on May 15, 2007, emphasized that professional development should focus not only on content, but also on effective instruction of that content. [A] picture emerges about the sort of work that isn’t very helpful. Curricu- lum development is one. We know from decades of instructional material development that writing curriculum is a complicated, difficult process. We know that robust curriculum is necessary but not sufficient for classroom improvement. In addition to strong materials, teachers need equipment, professional development workshops, coaching, and good assessments. . . . Collections of lesson plans, by themselves, are only a small piece of the puzzle. (Lach, 2007, p. 4)

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM Teacher Enhancement and Professional Development Professional development is clearly important for supporting effective implementation of the many curriculum resource materials developed by NASA for K-12 STEM education. Indeed, many projects incorporate activi- ties aimed at increasing teachers’ familiarity with NASA’s resources and providing them with guidance on implementation. Research on the effectiveness of combining teacher professional develop- ment with accepted best practices in the field provide clear guidelines for the design of quality professional development. For example, the recent report, Taking Science to School, identified several features of well-structured opportunities for teacher learning, including a focus on a specific content area, clear connections to the classroom and the curriculum being taught, and sustained support over time (National Research Council, 2007b). The research indicates that superficial coverage of topics that are unrelated to school priorities or to teaching practice, with little or no follow-up to support classroom implementation, are of limited value (DeSimone et al., 2002; Garet et al., 1999). Instead, sustained engagement with teachers over an extended period of weeks or months is required to effect lasting change in instruction and strengthen teachers’ confidence in their knowl- edge and teaching of science content (Rosenberg, Heck, and Banilower, 2005; Supovitz and Turner, 2000). NASA pursues a wide variety of projects and activities aimed at teacher support and professional development. NASA defines their professional development offerings as either of short or long duration. Short-duration activities are events for inservice educators that last less than 2 days. Long- duration activities last longer than 2 days or are offered over an extended period of time. The short-duration events are intended to meet the objective of engaging teachers, while the long-duration events are intended to meet the more demanding objective of educating teachers. A recent inventory of NASA’s education portfolio (Schwerin, 2006) catalogued 150 professional development activities for K-12 teachers across the headquarters Office of Education, the mission directorates and the centers. Of these, 53 percent (80 activities) were short duration as defined by NASA and 47 percent (70 activities) were long duration as defined by NASA. In the headquarters Office of Education, 13 percent (3 activities) were short duration and 87 percent (21 activities) were long duration. In the mission directorates and centers, 61 percent (77 activities) were short duration and 39 percent (49 activities) were long duration. Although the research evidence cited above calls into question the utility of short-term professional development, it is important to consider the purpose of a professional development opportunity when assessing the design. If an opportunity is intended merely to make teachers aware of

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO NASA resources and briefly acquaint them with what is available, a short- term program may be appropriate. However, it is inappropriate to label such an activity as a professional development program; rather, it should be called an informational meeting or some similar name. For activities that NASA defines as long duration, there is a differ- ent concern. The time for those activities is not commensurate with the extended engagement needed to support change in teacher practice: much of the “long-duration” activities with teachers should more properly labeled as intermediate in length. SEVEN CORE EDUCATION PROJECTS This section presents our analysis of the seven core projects in the Office of Education Elementary and Secondary Program, drawing on the framework presented above. For each project, the committee identifies both its strengths and areas for improvement. As a setting for this analysis, a summary of the major goals and intended outcomes (if specified) for each project are presented in Box 4-1. Aerospace Education Services Project The Aerospace Education Services Project (AESP), which was estab- lished 45 years ago, is designed to provide customized opportunities for showcasing NASA-related curriculum materials and activities in formal and informal settings with educators in the states and U.S. territories. To carry out the program, NASA, through the AESP contractor, employs a corps of aerospace education specialists who are former teachers and are required to have at least 5 years of classroom teaching experience in grades 4 through 12. These specialists are assigned to a NASA center and travel to provide services to the schools or teachers in the designated region. There are cur- rently 23 specialists. Typically, specialists respond to requests for services and programs from interested parties, such as school groups, districts, teachers, or administrators. According to a 2004 evaluation report (Horn and McKinley, 2004), about 62.5 percent of the specialists’ time is spent either preparing for or making school-site presentations. The specialists are also responsible for mapping NASA materials against the science and mathematics standards of the states in their region—a map that is intended to inform teachers which activities will help them “meet” a particular standard. The remainder of the time is spent on travel, leave, and personal professional development activities (Horn and McKinley, 2004). Recently, the project has been significantly revised to provide the infra- structure needed for a newer education effort, the NASA Explorer Schools (NES). Aerospace education specialists are now called on to provide or

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM BOX 4-1 Goals and Intended Outcomes: NASA Core K-12 Education Projects Aerospace Education Services Project (AESP) Provide customized professional development opportunities that educate inservice and preservice teachers that are aligned to their states’ standards, to gain rigorous and relevant content understanding for teaching in the STEM disci- plines and how they relate to NASA research and development. Build the nation’s workforce by engaging K-12 students and families in educational opportunities using the NASA mission, the STEM disciplines, and research- based teaching. Support and nurture state and national partnerships with education agencies, pro- fessional organizations, and informal education entities to collaborate STEM literacy and awareness of NASA’s mission. Support family participation in the NASA mission. Support the NASA Office of Education and NASA pathfinder initiatives to provide compelling experiences for educators and students that increase interest in STEM coursework and careers. NASA Explorer Schools (NES) (includes the Digital Learning Network [DLN]) Goal 1: Provide all students the opportunity to explore science, technology, engi- neering, mathematics, and geography. Goal 2: Provide educators with sustained professional development, unique STEM-based teaching, and collaborative tools. Goal 3: Build strong family involvement within NES. Outcome: Increase student knowledge about careers in science, technology, engineering, mathematics, and geography. Outcome: Increase student ability to apply STEM concepts and skills in meaning- ful ways. Outcome: Increase the active participation and professional growth of educators in science. Outcome: Increase the academic assistance for and technology use by educators in schools with high populations of underserved students. Outcome: Increase family involvement in children’s learning. Science, Engineering, Mathematics and Aerospace Academy (SEMAA) Inspire a more diverse student population to pursue careers in STEM-related fields.

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO Engage students, parents, and teachers by incorporating emerging technologies. Educate students by utilizing rigorous STEM curricula that meet national math- ematics, science, and technology standards and encompass the research and technology of NASA’s four mission directorates. Education Flight Projects (EFP) Develop and provide NASA-unique experiences, opportunities, content, and resources to educators to increase K-12 student interest and achievement in STEM disciplines. Develop and facilitate a Network of Educator Astronaut Teachers (NEAT)-like group of highly motivated educators. Build internal and external partnerships with formal and informal education com- munities to create unique learning opportunities and professional development experiences. Educator Astronaut Project (EAP) Develop and provide NASA-unique experiences, opportunities, content, and resources to educators to increase K-12 student interest and achievement in STEM disciplines. Develop and facilitate a Network of Educator Astronaut Teachers (NEAT)-like group of highly motivated educators. Build internal and external partnerships with formal and informal education com- munities to create unique learning opportunities and professional development experiences. Interdisciplinary National Science Project Incorporating Research and Education Experience (INSPIRE) Attract and retain students in STEM disciplines. SOURCE: Information from NASA’s 2006 project plans and personal commu- nication, Shelley Canright, outcome manager, Elementary and Secondary and e-Education Programs.

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM support teacher training or student activities for NES and to support the development and implementation of school “action plans” for the use of NASA units and materials. In fact, specialists report that they now spend about 60 percent of their time working with NES and another 10 percent with the digital learning network (DLN), which is part of NES. The rest of their time is allocated for non-NES schools and teachers (20%) and on NASA center-related programming (10%) (Horn and McKinley, 2006). Project Evaluations AESP was the subject of a 3-year external evaluation in 2001–2004 (Horn and McKinley, 2004) and a small follow-up evaluation in 2006 (Horn and McKinley, 2006). In the 3-year evaluation, a variety of methods (surveys, interviews, site visits, presentations, review of documents, and the NASA Education Evaluation and Information System [NEEIS]) were used to gather data from a provider and client group in order to address 19 evaluation questions. The evaluation concluded that AESP provides good support to NASA projects in raising awareness of the available resources and services. How- ever, many schools and teachers remain unaware of AESP services. In addi- tion, specialists most often engage in activities that generate immediate interest but do not necessarily have long-term effects in terms of education reform and improvement and curriculum enrichment. Although there was enthusiasm from participants for AESP presentations, all respondents indi- cated that the residual effect of the program is relatively low. The evaluation raised the concern that the project might be limited because of its adher- ence to an “in-person” presentation model, rather than incorporating more distance learning technology. The supplementary 2006 evaluation (Horn and McKinley, 2006) used case studies of sites selected as exemplary, surveys, and analysis of NEEIS data to address six evaluation questions. The report provides good insight into activities at these sites, but there is no solid evidence of impact. The evaluation details AESP’s role in supporting other NASA education programs (Horn and McKinley, 2006). Requests by NASA programs for support services from AESP personnel are frequent, and requests also come from schools and educators. In fact, particularly with the extra load of the NES, requests have become so frequent that the aerospace education spe- cialists are not able to deliver all needed services in a face-to-face manner; thus, they have begun to use the DLN to reach schools, particularly the NASA Explorer Schools, through the Internet and videoconferencing.

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO Project Strengths One of the strengths of the project is its responsiveness to clients in providing services and other types of support through a network of regionally based specialists. Another is the use of former teachers as the NASA educators. This approach provides a group of knowledgeable former teachers who have some understanding of school systems and of the instruc- tional needs of students. The geographic distribution of these educators allows each AESP specialist to become knowledgeable about the state stan- dards for the two or three states they serve. The ability of the specialists to engage the regional educational system and form local or regional partnerships is critical for ensuring that NASA’s activities are used in an effective way as part of school science and math- ematics instruction. The specialists are particularly important in rural states or states without NASA centers that may otherwise have little access to NASA activities and materials. Areas for Improvement The distributed model also has a potential weakness. The quality of the services delivered regionally appears to depend heavily on the individual specialists and the relationships a particular specialist is able to build with local educational organizations, districts, and schools. In this respect, a high turnover rate for specialists, which was noted in the 2006 evaluation report, is a problem. In addition, the specialists’ role in the NASA Explorer Schools appears to be limiting the amount of time for them to work in other schools (Horn and McKinley, 2006). The committee is concerned about the ability of specialists to remain abreast of newly emerging NASA science and technology related to NASA missions. A yearly workshop, the current means for updating specialists on new developments, seems insufficient for keeping them truly up to date. Specialists need immediate links to the science, scientists, technology, and engineers in the agency in order to be able to effectively communicate cur- rent science and engineering developments and information to teachers and students. In the committee’s view, the stated objectives for the project are too broad, and, therefore, potentially misleading. Those objectives closely follow the overall objectives for the Elementary and Secondary Program, with little specification to make them more appropriate to AESP’s scope and target audiences. In addition, the breadth and lack of structure in the project has led to a lack of stability in the focus and sustainability of specific project goals, and there is little evidence of any sustained effects on teachers’ professional development. There are some teachers who, by

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO NEAT members are expected to serve as NASA education advocates by engaging their schools and communities across the country in the agency’s education services and informing them of NASA resources. They participate in a one-time, 2–3 day professional development workshop to provide them with a background for this work. NEAT members are responsible for devel- oping their own local opportunities for sharing NASA information and resources. The Teaching from Space Education Office is planning to review the design of the NEAT in order to determine how to make it more robust and inclusive of other teachers. They are also interested in determining the best approach to selecting teachers to be part of NEAT.2 Project Evaluation EAP has not yet been externally evaluated; it is a high priority for the office that oversees the project. Project Strengths EAP has the potential to inspire many students through participation in the education downlinks and the design challenge. NEAT appears to have been formed in response to the strong interest in maintaining a link to NASA expressed by many teacher applicants who were not selected to become astronauts. This was a creative response to the desire to capitalize on valuable public interest and could provide another mechanism for dis- seminating NASA’s materials and information. Areas for Improvement In its current form, it is not clear how NEAT will be leveraged to dis- seminate NASA’s materials and information, both generally and in conjunc- tion with flights of educator astronauts. Examining how NEAT members could best be used, or how links to other projects, such as AESP, might be developed, would be useful. Because the project is so new and because the first flight of an educator astronaut took place in summer when schools were not in session, it is impossible to accurately assess the project’s impact. Interdisciplinary National Science Project Incorporating Research and Education Experience The Interdisciplinary National Science Project Incorporating Research and Education Experience (INSPIRE), which is in a formative stage, is a 2Personal communication, Cynthia MacArthur and Edward Pritchard, June 2007.

OCR for page 57
0 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM replacement for a former program NASA SHARP. INSPIRE is a three- tiered project designed to maximize student participation and involvement in STEM subjects and to strengthen and enhance the STEM pipeline from middle school through high school and to the undergraduate level. Tier I is junior explorers (grades 9 and 10); tier II is junior guest researchers (grades 11 and 12); and tier III is collegiate interns (rising college freshmen and sophomores). INSPIRE is still in the planning stages; a pilot phase began in summer 2007. INSPIRE is designed to provide critical STEM pathways for eligible students, with special emphasis on underrepresented and underserved groups. Students will be exposed to STEM experiences and encouraged to consider graduate studies in STEM fields. It is also hoped that INSPIRE will provide a public benefit by incorporating parent and community participation through program activities that inform and engage the public in NASA’s exploration vision. INSPIRE will offer research experiences, short courses, workshops, and seminars for students. Project Evaluation The project is still in the design and planning stages. Project Strengths Although INSPIRE is not yet implemented, the committee commissioned a paper to review the research literature on projects designed to engage underrepresented students in STEM subjects and compare best practice to INSPIRE’s design. (Hall, 2007) The author concludes that the INSPIRE model mirrors much of best practice about teaching and learning STEM sub- jects in out-of-school time, including such program elements as mentoring, family involvement, inquiry-based learning, and hands-on activities. Areas for Improvement Hall (2007) also provides suggestions to move the design closer to best practice. The author encourages the incorporation of hands-on activi- ties and suggests that INSPIRE make use of existing informal education organizations, such as Boys and Girls Clubs and faith-based organizations. For example, INSPIRE might use youth organization staff as cofacilitators, adapting effective procedures from youth organizations and creating similar learning environments or spaces that have proven successful for those orga- nizations. The author particularly cautions against activities in INSPIRE that might too closely resemble more formal school learning experiences. Rather, activities should be delivered in a way that provides youth with opportunities for choice, independence, flexibility, and social experiences.

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO Finally, the author points out that INSPIRE staff might consider calling on high school guidance staff and science educators to refer students who might otherwise be overlooked as INSPIRE candidates because they are not motivated by STEM subjects as taught in traditional classrooms. CROSS‑CUTTING ISSUES Through the committee’s analyses of the seven core projects, three cross-cutting issues emerged: improving the process for program and project design and improvement; drawing on outside expertise in education; and maintaining a connection to the science and engineering in the agency. It is not the committee’s intent to imply that NASA gives no attention to these issues: each of them is discussed in NASA’s new strategic plan for education. Rather, the committee seeks to emphasize the importance of these issues as a means to improve and bring more coherence to the agency’s work in precollege STEM education. Improving the Process for Project Design and Improvement One of the most important cross-cutting issues is the need for a more intentional approach to the design and continuous improvement of projects. NASA appears to have already recognized this issue, as evidenced in the emphasis on a portfolio approach in the strategic framework and recent efforts to review projects. Taking a portfolio approach seriously will entail using strategies the agency has not consistently used in the past. First, the agency might benefit from further articulation of a strategy for K-12 activities across the agency and the role of the Elementary and Secondary Program specifically. Currently, the Elementary and Secondary Program is charged with contributing to outcome 2, “attract and retain stu- dents in STEM disciplines through a progression of educational opportuni- ties for students, teachers, and faculty” and is integrated into the “engage” and “educate” categories of the strategic education framework. Both this outcome and the two categories are broad. A more detailed analysis of NASA’s assets, the needs of the K-12 system, and research-based strategies for achieving the stated goals for K-12 education in the agency is needed. Next, NASA needs to sharpen goals and objectives for individual projects so that they better reflect the scope and specific activities of the projects, rather than the broad overall goals of the headquarters Office of Education. As currently stated in the administrative plans for the core projects submitted to the office, the broad goals and objectives of the Elementary and Secondary Program have often been used as a substitute for individual project goals. Moreover, the projects have not consistently attempted to provide project-specific goals and objectives that would be

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM closely aligned with project activities. For example, AESP nominally targets all of the Elementary and Secondary Program objectives and NES targets five of the six; see Table 4-1. Likewise, a portfolio approach requires thoughtful planning across projects, informed by knowledge of best practice in education. The process should begin with project design, with attention to how projects comple- ment each other and how they capitalize on NASA’s strengths. Special atten- tion should be given to the question of when it is appropriate for NASA to take the lead on projects and when it is appropriate to develop partner- ships. NASA also needs to have a systematic approach, based on educa- tional value, for determining which projects that originate from centers or TABLE 4-1 Objectives for Seven Core Education Programs EFP and NESa Objectives AESP SEMAA EAP INSPIRE Provide short-duration professional X X development to engage teachers Provide long-duration professional X X X X development to educate teachers Provide curricular support resources that X X X • use NASA themes and content to enhance student skills and proficiency in STEM • inform students about STEM career opportunities • communicate information about NASA mission activities Student involvement: provide K-12 X X X X X students with authentic first-hand opportunities to participate in NASA mission activities, thus inspiring interest in STEM disciplines and careers Dissemination X X Coordination X X NOTES: AESP = Aerospace Education Services Project; SEMAA = Science, Engineering, Mathematics and Aerospace Academy; NES = NASA Explorer Schools; EFP = Education Flight Projects; EAP = Educator Astronaut Project; and INSPIRE = Interdisciplinary National Science Project Incorporating Research and Education Experience. aIncludes DLN (Digital Learning Network).

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO missions contribute to the portfolio and can be supported and when a new project is needed to address an emerging area of interest. Periodic review of projects in order to evaluate whether they have maintained their focus and are reaching their intended audience is also critical. To this end, there is a need to have a process for continuous project improvement and periodic “culling”—refinement of the portfolio. This culling should be done intentionally, with input from experts in education, and based on data provided by projects and through external evaluations (see Chapter 5). The criteria for culling and refining projects should be care- fully developed and should reflect the objectives for the overall portfolio. One potential challenge for the K-12 education program is to achieve a balance between projects that achieve a broad reach and those that foster deep engagement with the science and engineering content of the agency. The committee agrees that NASA has an important role to play in both sorts of activities. However, the two kinds of projects require very different designs and deployment of resources. There is also a need to reconsider project design as the needs of the educational community change and particularly as new technology becomes available. For example, AESP and DLN do not appear to capitalize suffi- ciently on emerging technologies. Programs that were designed around old technology or old approaches need to evolve as educational practice evolves and as new technologies emerge. For example, the emergence of standards- based approaches in STEM education necessitated a response from NASA projects, and AESP, SEMAA, and NES have made efforts to adjust to those new approaches. In developing projects, it is also important to consider the investment required to accomplish intended goals and whether that level of invest- ment is sustainable across the life of the projects. For example, NES is an expensive project that also draws resources from existing NASA projects in ways that are not obvious in the budget. Despite these high investments, the project still does not provide the levels of funding that are necessary for whole school reform (Mundry, 2007). In addition, NES relies heavily on AESP for support, and there is evidence in evaluations that the broader function of AESP is being negatively affected as a result. Drawing on Outside Expertise in Education The design and implementation of NASA’s K-12 STEM education pro- grams and projects should be informed by the substantial knowledge base in the cognitive and learning sciences and education. Such expertise is not a typical qualification for agency staff, since NASA is primarily a science and engineering agency. Thus, expertise in education must be intentionally brought into the agency’s precollege projects through a variety of means.

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM Hiring education staff with appropriate expertise is one avenue. An example of this approach is the position of AESP specialist. The use of former teachers provides a qualified group of individuals who understand school systems and the realities of classrooms. The regional distribution of educators allows each AESP educator to become expert in the state standards for two or three states. Yet even these specialists may still lack expertise in curriculum development or professional development strategies, which are not areas of expertise for most classroom teachers. The committee also identified two other methods for increasing the involvement of individuals with expertise in education: partnerships and expert review. Both of these are already in use in some education projects and might be considered for wider use in the future. Partnerships Partnerships are already used in some of NASA’s education projects, and cultivation of partnerships and sustainability are part of the overarching philosophy described in the 2006 strategic framework. The former Office of Space Science explicitly called out partnerships as a basic operational principle: “Base all of OSS’s E/PO (education and public outreach) efforts on collaborations between the scientific and education communities thereby drawing upon and marrying the appropriate expertise of the two communi- ties” (Rosendahl, Sakimoto, Pertzborn, and Cooper, 2004). This emphasis has been carried forward in the Science Mission Directorate and is reflected in its guide (National Aeronautics and Space Administration, 2006b). One major criterion for education and public outreach grants is partnership sustainability, and the guide emphasizes that projects and activities “require the active involvement of the research team and participation partners with appropriate expertise.” This involvement might include expertise in cogni- tion and the learning sciences; design of effective instruction, curricula, and professional development; or evaluation. Partnerships can be used successfully to accomplish a variety of objec- tives, including development of curriculum materials, dissemination of materials, and support for professional development. Examples of success- ful partnerships include the partnership between EarthKAM and TERC to support educational use of images (see above) and a partnership between the SMD forums and Lawrence Hall of Science to develop space science GEMS guides. In fact in a recent summative evaluation of the education and public outreach effort of the Office of Space Science (Gutbezahl, 2007), several projects were identified as having developed exemplary resources for formal education; those projects included partnerships. See Box 4-2 for descriptions of these and other partnership projects. Similarly, in many

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO BOX 4-2 Examples of High-Quality NASA Partnership Projects in Education GEMS Guides. These guides engage students in direct experience and experi- mentation to introduce essential, standards-based principles and concepts. Clear step-by-step instructions enable all teachers to be successful presenting the activities. GEMS units offer effective, practical, economical, and schedule-friendly ways to provide high-quality science and math learning to all students. Informa- tion about GEMS can be found at http://www.lawrencehallofscience.org/gems/ aboutgems.html. Mars Student Imaging Project. Teams of students in grades 5 through college sophomore level work with scientists, mission planners, and educators to image a site on Mars using the visible wavelength camera onboard the Mars Odyssey spacecraft. The curriculum was developed to align with national science education standards and fit with existing science curricula. More information about the Mars Student Imaging Project can be found at http://msip.asu.edu/. Sun-Earth Day. A series of programs and events occur throughout the year and culminate with a celebration on or near the spring equinox (“Sun-Earth Day”). These programs are supported by a variety of resources, including a website, print resources, and various multimedia products. More information about Sun-Earth Day can be found at http://sunearthday.nasa.gov/. Modeling the Universe. A suite of hands-on activities and inquiries is related to current models for the origins and evolution of the universe. These activities are shared with 8th–12th grade teachers at workshops at which the teachers receive content and pedagogical training, as well as classroom-ready materials supporting each activity. After completing the workshop, teachers have access to a webpage and wiki, which contain additional materials and support. More information about Modeling the Universe can be found at http://cfa-www.harvard. edu/seuforum/mtu/. NOTE: All the websites cited were current as of November 2007. cases, the NASA materials and activities that the committee judges as having the highest quality were those developed in the context of partner- ships between NASA scientists and other personnel and existing educa- tional organizations. Developing partnerships is also a strength of the AESP specialists. The ability of these specialists to engage the educational system and form local partnerships is important for ensuring that NASA’s activities are used in an effective way as part of school science and mathematics instruction.

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM Use of partnerships does not seem to be consistent across headquarters Office of Education projects, nor is it clear that there are consistent methods for determining which partners are most appropriate or have the best fit in terms of expertise for a given project. For AESP specialists, the extent of partnerships appears to depend very much on the characteristics of the indi- vidual and the relationships he or she is able to build with local educational organizations, districts, and schools. In this respect, a high turnover rate for specialists, which was noted in an external evaluation of the project, is a problem. Partnerships can be particularly useful in the design of curriculum materials, but they are not consistently used by individual projects such as DLN and NES. Without partnerships and careful design, curriculum support resources are often ineffective and difficult to integrate with exist- ing curricula. This concern was echoed in testimony provided on May 15, 2007, to the House Subcommittee on Research and Science Education by George Nelson, director of Science, Mathematics, and Technology Educa- tion at Western Washington University and a former astronaut. In answer to a question about how lack of coordination might hinder federal agencies from making an impact, Nelson noted: “There is a huge inventory of poorly designed and under-evaluated mission-related curriculum (posters, lesson plans and associated professional development) rarely used in classrooms and with no natural home in a coherent standards-based curriculum.” Nelson did identify the GEMS guides as exemplary. NASA has not consistently tapped partners for expertise in the design and planning of projects. This is perhaps the most critical time for partner- ships. NASA should explore mechanisms to bring in this expertise early. NASA should consider which kinds of projects the agency is well positioned to initiate and which projects are better suited to partnership in which the agency plays a value-added role. Finally, projects designed to develop students’ interest in and knowl- edge of engineering might be of particular value because engineering does not usually receive attention in the K-12 curriculum. The agency could seek out partners and resources to leverage its contributions in this area. Expert Review Peer-reviewed competition and expert review is another mechanism by which expertise in education can be brought to bear on projects and pro- grams. Again, tapping outside expertise was an operating principle for the Office of Space Science: “Use outside advice from the scientific, educational, and minority communities in the planning, development, implementation, and assessment of all our education and outreach activities” (Rosendahl et al., 2004). Expertise can play a role on several levels. In competitions,

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO expert panels provide an important filter for determining which proposals have the most educational merit. Expert review of curriculum materials or project design is another mechanism for maintaining quality (see Chapter 5 for a discussion of expertise in evaluation). It is not clear whether expert review is consistently used in the seven core projects reviewed by the committee. In mission competitions in the SMD, the basic design of a project is part of what is evaluated in the competition. However, the current projects in the headquarters Office of Education were not selected through a competitive process and were not subjected to a rigorous expert review. The projects themselves also do not consistently use expert review by educators or by knowledgeable scientists and engineers in the design of their activities and materials. For example, the menu of modules provided through DLN has not undergone review by outside experts. It also appears that curriculum materials developed by SEMAA and by NES do not con- sistently undergo any kind of external review. The headquarters Office of Education is in the process of developing a mechanism for expert review of curriculum support materials. Currently, NASA produces a number of curriculum support materials that incorporate a variety of instructional activities for students, as evidenced in the large catalogues listing available materials.3 The current formal review process was developed by the Office of Earth Science and adapted by the Office of Space Science and is now coordinated by the Science Mission Directorate. The review is based on the assumption that materials have been field tested and have undergone formative evaluation prior to submission for review. The review is based on relevance to NASA’s mission and education goals, scientific accuracy, educational value (pedagogy), effectiveness of presenta- tion, documentation, ease of use, and power to engage and/or inspire the target audience. Products are reviewed by a panel of five to seven experts, including classroom teachers, education specialists, informal educators, and scientists. The reviews are conducted under contract by the Insti- tute for Global Environmental Strategies (IGES), a nonprofit education organization. Reviews occur on a twice yearly cycle, in May–August, and December–March. The headquarters Office of Education is currently studying the feasi- bility of a more frequent, rolling schedule for reviews, due in part to the demands that arise from increasing use of Internet and web-based activities. 3 See, for example: for space science educators, http://www.nasa.gov/audience/foreducators/ index.html; for NASA Central Operation of Resources for Educators, http://education.nasa. gov/edprograms/core/home/index.html; and for NASA Space Science Education Resource Directory, .

OCR for page 57
 NASA’S ELEMENTARY AND SECONDARY EDUCATION PROGRAM The process is being tested in collaboration with the Exploration System Mission Directorate and the Space Operations Mission Directorate. Given the challenge of designing effective curriculum resources, use of a review system is necessary to ensure the quality of materials. Furthermore, in conjunction with expansion of the current system, it would be worth- while to consider developing a mechanism for culling existing materials that may not have originally undergone rigorous review. As part of a review system, NASA needs a set of criteria for determining the kinds of topics or learning goals that are most appropriate to develop. For example, the committee agrees with the SMD guidelines (which in turn originated with the Office of Space Science) that it is not appropriate for NASA to develop materials that target basic concepts in science and mathematics that are not clearly tied to the science and engineering in the agency. One activity that might warrant more attention by the agency is the development and dissemination of materials and activities that offer students and teachers an opportunity for first-hand experience with the processes of science and engineering design. Emerging research on how to design effective laboratory experiences of this sort indicate that they should: have clear learning outcomes in mind; be thoughtfully sequenced into the flow of instruction; integrate learning science content with learning about the processes of science; and incorporate ongoing student reflection and discussion (National Research Council, 2006). Connection to Science and Engineering Work in NASA The third cross-cutting issue the committee identified was the impor- tance of consistently connecting NASA’s work in precollege education to the science, engineering, and exploration carried out by the agency. The committee recognizes, however, that maintaining this focus in all of NASA’s K-12 activities presents challenges for those projects not directly linked to science or engineering missions. One such challenge is how to keep education field staff, such as the AESP specialists, SEMAA staff, or educator astronauts, apprised of NASA’s current work and related education resources. AESP makes an effort to update staff through yearly workshops, but the committee does not believe that this is sufficient. In addition, solid knowledge of the underlying sci- ence and engineering concepts is critical for the staff, and it is unclear how this depth of content knowledge is maintained. The use of the Internet and other technology to facilitate ongoing professional development might be one way to help address this challenge. A second challenge is how to respond to demands from partnering schools to provide more support for basic science and mathematics that

OCR for page 57
 ANALYSIS OF NASA’S K- EDUCATION PORTFOLIO are not necessarily linked to space science. There is evidence of this kind of pressure from schools in both NES and SEMAA. In such cases, NASA needs to be judicious in how to respond. For example, developing very general units on forces and motion or on ratio and proportion that are only super- ficially tied to the agency’s science and engineering activities through choice of examples is inappropriate. However, even when development might be tied directly to NASA-related experiences, such as the process of designing a spacecraft, partnerships should be used, and schools should be referred to other individuals or organizations who can more appropriately work with the demands of the general K-12 STEM curriculum. This is admittedly a difficult line to walk; however, in the context of limited resources for educa- tion at NASA, it is important to figure out how to do so.