Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
4 Analysis of NASAâs K-12 Education Portfolio I n this chapter we present our analysis of NASAâs portfolio in K-12 s Â cience, technology, engineering, and mathematics (STEM) education with particular attention to program design and effectiveness. The com- mittee reviewed the seven core projects in the headquarters Office of Educa- tion Elementary and Secondary Program in depth: the Aerospace Education Services Project; NASA Explorer Schools; Digital Learning Network; ÂScience, Engineering, Math and Aerospace Academy; the Education Flight ÂProjects; Educator Astronaut Project; and the Interdisciplinary National Â Science P Â roject Incorporating Research and Education Experience (INSPIRE). The committee also reviewed some of the projects and activities in the Science Mission Directorate (SMD). Our review of the Science Mission Directorate projects was less detailed, as an in-depth review of such a large portfolio was beyond the scope of our study. The committee did believe it was necessary to give some attention to the SMD projects; however, because they represent approximately one-half of the agencyâs funding in K-12 edu- cation. Including these projects in the review gave the committee a better overall perspective of the scope of the agencyâs work at the precollege level. This chapter does not include analysis of individual SMD projects; however, we do discuss the general approach to education projects used in SMD and mention individual projects as examples. The committee used several strategies for reviewing the seven core projects. We received briefings from NASA staff on each project, and we reviewed administrative documents, annual reports, and recent external evaluations. Committee members also drew on their knowledge of research in K-12 education regarding best practices in developing studentsâ inter- 57
58 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM ests in science, technology, engineering, and mathematics; curriculum and instruction; and professional development as a framework against which to compare NASA K-12 projects. This expert knowledge was critical for the committee analysis because of the limitations of existing project evalu- ations. These limitations are not unique to NASA but are reflected across many federal science agencies involved in STEM education: see the report of the Academic Competitiveness Council (U.S. Department of Education, 2007a); also see Chapter 5 for an in-depth discussion of evaluation. From its analyses of individual projects, the committee identified three areas in which NASA can improve the quality of its K-12 education pro- gram: project design and improvement, use of expertise in education, and the connection to the science and engineering in the agency. Before present- ing our analysis, we lay out the frameworks that guided that analysis. FRAMEWORK FOR BEST PRACTICE From its review of research and the membersâ expertise, the committee identified three major topics that connect to NASAâs program goals and encompass most of the activities of the constituent projects: developing inter- est; curriculum and instruction; and professional development for teachers. For each of these topics, the committee identified major conclusions that can be drawn from the research evidence regarding principles for best practice. In the following section, we briefly review these principles, which are then used as a framework for the critique of the constituent projects. Developing and Sustaining Interest Inspiring, engaging, and sustaining the interest of teachers and students in STEM subjects is one of the main goals of NASAâs current education program, and is one of the greatest contributions that NASA can make to K-12 STEM education. The excitement generated by space flight and exploration puts NASA in a unique position to draw teachers and students into science, technology, engineering, and mathematics and related fields. However, of equal importance to the need to attract the interest of teachers and students is the need to sustain that interest over time and to link it to meaningful science content. Substantial research has been done on the development of studentsâ and teachersâ motivations and interests, with some attention to how to design learning experiences that are both engaging and that result in real learning. In this research, âinterestâ is defined as both a positive feeling for science and the predisposition to continue to engage in science (Hidi and R Â enninger, 2006). Interest, in this sense, includes the stored knowledge, stored values, and feelings that influence the engagement, questioning,
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 59 and activity of individuals (or groups of individuals). Interest has positive consequences for learning. For example, when peopleâboth young and oldâhave a real interest in science, they are more likely to pose questions out of curiosity, seek out challenges, and use effective learning strategies (Barron, 2006; Csikzentmihayli, Rathunde, and Whalen, 1993; Engle and Conant, 2002; Kuhn and Franklin, 2006; Lipstein and Renninger, 2006; Renninger, 2000; Renninger and Hidi, 2002). Early on, interest may be primarily triggered or maintained by external experience. As interest develops and deepens, however, a person is more likely to initiate engagement and to generate and seek answers to questions about content (Renninger, 2000). NASAâs program in K-12 STEM educa- tion has the potential to trigger initial interest in students and teachers, as well as to provide experiences to deepen engagement for those who already have some initial interest. Two challenges for NASA in designing activities to âinspire and engageâ are to attend to what is needed to translate initial excitement into a meaningful learning experience and a sustained, long- term interest and to support teachers in providing appropriate follow-up activities for an initial activity. Reaching and engaging students who are typically underrepresented in STEM fields is a challenge that many of NASAâs programs, particularly those managed by headquarters Office of Education, are designed to address. Although research on the most effective ways to bring underrepresented populations into STEM fields is thin, the evidence does suggest guidelines for best practice (BEST, 2004; Hall, 2007). One set of best practices was developed by the Building Engineering and Science Talent Initiative (BEST, 2004) through an expert review of programs. The practices include â¢ Defined outcomes: Students and educational staff agree on goals and outcomes. Success is measured against the intended results. Outcome data provide both quantitative and qualitative informa- tion. Disaggregated outcomes provide a basis for research and continuous improvement. â¢ Persistence: Effective interventions take hold, produce results, adapt to changing circumstances and persevere in the face of setbacks. Conditions that ensure persistence include proactive leadership, sufficient resources, and support at the district and school levels. â¢ Personalization: Student-centered teaching and learning methods are core approaches. Mentoring, tutoring, and peer interaction are integral parts of the learning environment. Individual differences, uniqueness, and diversity are recognized and honored. â¢ Challenging content: Curriculum is clearly defined and understood. Content goes beyond minimum competencies; relates to real-world applications and career opportunities and reflects local, state, and
60 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM national standards. Students understand the link between content rigor and career opportunities. Appropriate academic remediation is readily available. â¢ Engaged adults: Adults provide support, stimulate interest, and create expectations that are fundamental to the intervention. Edu- cators play multiple roles as teachers, coaches, mentors, tutors, and counselors. Teachers develop and maintain quality interactions with students and each other. Active family support is sought and established. A flexible program structure and opportunities for students to work in groups and socialize are also important based on a literature review com- missioned by the committee (Hall, 2007). Curriculum and Instruction Many of NASAâs contributions in K-12 STEM education fall under the category of curriculum materials and instructional activities. NASA seeks to provide curricular support resources that âuse NASA themes and con- tent to enhance student skills and proficiency in STEM disciplines, inform students about STEM career opportunities, and communicate information about NASAâs mission activitiesâ (National Aeronautics and Space Admin- istration, 2006c). Science curricula, for the purposes of this discussion, are defined as having three components: curriculum standards, curriculum materials, and instructional activities. Curriculum standards are the learning goals estab- lished collectively by national standards, state science expectations (e.g., state standards, state core curriculums, state expected learning outcomes), and dis- trict science curriculum guidance (e.g., guidelines, blueprints, learning expec- tations). Curriculum materials include textbooks, materials or labs, videos and other audio-visual materials, and reading materials. Instructional activi- ties comprise the lesson plans, studentsâ laboratory and field experiences, and modeling activities. NASAâs work in K-12 STEM education focuses on cur- riculum materials designed to support NASA-related instructional activities. A teacherâs decision to incorporate those activities should be informed by the curriculum and standards that apply for the course in question. Curriculum standards lay out the science content and processes essen- tial for science literacy and preparation for STEM pursuits. They provide a blueprint for the development of essential knowledge and skills and cultivation of scientific habits of mind for all students. The key role of cur- riculum standards is to bring coherence, articulation, and focus to instruc- tion. Over the last 10â15 years there has been a movement toward creating standards at the national and state level that provide a framework to guide
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 61 educators at the local level (National Research Council, 2007b). NASA has recognized this movement and has taken steps in its work with schools to show how the materials the agency offers are aligned with national and state standards. In general, curriculum materials should, at a minimum, meet four c Â riteria to be useful in improving student learning and achievement: 1. They should be aligned to the specific instructional objectives of the state and district standards. 2. They should be pedagogically sound. 3. They should be engaging and relevant. 4. They should be accurate in their presentation of scientific information. The National Science Education Standards suggest that â[e]ffective sci- ence curriculum materials are developed by teams of experienced teachers, scientists, and science curriculum specialists using a systematic research and development process that involves repeated cycles of design, trial teaching with children, evaluation, and revisionâ (National Research Council, 1995, p. 213). Research also shows that successful implementation of curriculum or of particular instructional activities and strategies usually requires some form of professional development for teachers. Indeed, increasing the effective use of high-quality instructional materials is at the center of many educa- tional reform efforts. The National Science Foundationâs Local Systemic Change in Mathematic and Science Program stressed the importance of the use of quality instructional materials with linked professional develop- ment. The evaluation of this program found that extensive use of even first rate instructional materials was effective only when linked to professional development targeted at teachersâ practice, investigation, problem-solving, and instruction (Banilower et al., 2006). Michael Lach, director of Mathematics and Science for the Chicago Public Schools, in his remarks to Congress on May 15, 2007, emphasized that professional development should focus not only on content, but also on effective instruction of that content. [A] picture emerges about the sort of work that isnât very helpful. Curricu- lum development is one. We know from decades of instructional material development that writing curriculum is a complicated, difficult process. We know that robust curriculum is necessary but not sufficient for classroom improvement. In addition to strong materials, teachers need equipment, professional development workshops, coaching, and good assessments.Â .Â .Â . Collections of lesson plans, by themselves, are only a small piece of the puzzle. (Lach, 2007, p. 4)
62 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM Teacher Enhancement and Professional Development Professional development is clearly important for supporting effective implementation of the many curriculum resource materials developed by NASA for K-12 STEM education. Indeed, many projects incorporate activi- ties aimed at increasing teachersâ familiarity with NASAâs resources and providing them with guidance on implementation. Research on the effectiveness of combining teacher professional develop- ment with accepted best practices in the field provide clear guidelines for the design of quality professional development. For example, the recent report, Taking Science to School, identified several features of well-Âstructured opportunities for teacher learning, including a focus on a specific content area, clear connections to the classroom and the curriculum being taught, and sustained support over time (National Research Council, 2007b). The research indicates that superficial coverage of topics that are unrelated to school priorities or to teaching practice, with little or no follow-up to support classroom implementation, are of limited value (DeSimone et al., 2002; Garet et al., 1999). Instead, sustained engagement with teachers over an extended period of weeks or months is required to effect lasting change in instruction and strengthen teachersâ confidence in their knowl- edge and teaching of science content (Rosenberg, Heck, and Banilower, 2005; ÂSupovitz and Turner, 2000). NASA pursues a wide variety of projects and activities aimed at teacher support and professional development. NASA defines their professional development offerings as either of short or long duration. Short-duration activities are events for inservice educators that last less than 2 days. Long- duration activities last longer than 2 days or are offered over an extended period of time. The short-duration events are intended to meet the objective of engaging teachers, while the long-duration events are intended to meet the more demanding objective of educating teachers. A recent inventory of NASAâs education portfolio (Schwerin, 2006) catalogued 150 professional development activities for K-12 teachers across the headquarters Office of Education, the mission directorates and the centers. Of these, 53 percent (80 activities) were short duration as defined by NASA and 47 percent (70 activities) were long duration as defined by NASA. In the headquarters Office of Education, 13 percent (3 activities) were short duration and 87 percent (21 activities) were long duration. In the mission directorates and centers, 61 percent (77 activities) were short duration and 39 percent (49 activities) were long duration. Although the research evidence cited above calls into question the utility of short-term professional development, it is important to consider the purpose of a professional development opportunity when assessing the design. If an opportunity is intended merely to make teachers aware of
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 63 NASA resources and briefly acquaint them with what is available, a short- term program may be appropriate. However, it is inappropriate to label such an activity as a professional development program; rather, it should be called an informational meeting or some similar name. For activities that NASA defines as long duration, there is a differ- ent concern. The time for those activities is not commensurate with the extended engagement needed to support change in teacher practice: much of the âlong-durationâ activities with teachers should more properly labeled as intermediate in length. SEVEN CORE EDUCATION PROJECTS This section presents our analysis of the seven core projects in the Office of Education Elementary and Secondary Program, drawing on the framework presented above. For each project, the committee identifies both its strengths and areas for improvement. As a setting for this analysis, a summary of the major goals and intended outcomes (if specified) for each project are presented in Box 4-1. Aerospace Education Services Project The Aerospace Education Services Project (AESP), which was estab- lished 45 years ago, is designed to provide customized opportunities for showcasing NASA-related curriculum materials and activities in formal and informal settings with educators in the states and U.S. territories. To carry out the program, NASA, through the AESP contractor, employs a corps of aerospace education specialists who are former teachers and are required to have at least 5 years of classroom teaching experience in grades 4 through 12. These specialists are assigned to a NASA center and travel to provide services to the schools or teachers in the designated region. There are cur- rently 23 specialists. Typically, specialists respond to requests for services and programs from interested parties, such as school groups, districts, teachers, or administrators. According to a 2004 evaluation report (Horn and McKinley, 2004), about 62.5 percent of the specialistsâ time is spent either preparing for or making school-site presentations. The specialists are also responsible for mapping NASA materials against the science and mathematics standards of the states in their regionâa map that is intended to inform teachers which activities will help them âmeetâ a particular standard. The remainder of the time is spent on travel, leave, and personal professional development activities (Horn and McKinley, 2004). Recently, the project has been significantly revised to provide the infra- structure needed for a newer education effort, the NASA Explorer Schools (NES). Aerospace education specialists are now called on to provide or
64 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM BOX 4-1 Goals and Intended Outcomes: NASA Core K-12 Education Projects Aerospace Education Services Project (AESP) Provide customized professional development opportunities that educate inservice and preservice teachers that are aligned to their statesâ standards, to gain r Â igorous and relevant content understanding for teaching in the STEM disci- plines and how they relate to NASA research and development. Build the nationâs workforce by engaging K-12 students and families in educational opportunities using the NASA mission, the STEM disciplines, and research- based teaching. Support and nurture state and national partnerships with education agencies, pro- fessional organizations, and informal education entities to collaborate STEM literacy and awareness of NASAâs mission. Support family participation in the NASA mission. Support the NASA Office of Education and NASA pathfinder initiatives to provide compelling experiences for educators and students that increase interest in STEM coursework and careers. NASA Explorer Schools (NES) (includes the Digital Learning Network [DLN]) Goal 1: Provide all students the opportunity to explore science, technology, engi- neering, mathematics, and geography. Goal 2: Provide educators with sustained professional development, unique STEM-based teaching, and collaborative tools. Goal 3: Build strong family involvement within NES. Outcome: Increase student knowledge about careers in science, technology, engineering, mathematics, and geography. Outcome: Increase student ability to apply STEM concepts and skills in meaning- ful ways. Outcome: Increase the active participation and professional growth of educators in science. Outcome: Increase the academic assistance for and technology use by educators in schools with high populations of underserved students. Outcome: Increase family involvement in childrenâs learning. Science, Engineering, Mathematics and Aerospace Academy (SEMAA) Inspire a more diverse student population to pursue careers in STEM-related fields.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 65 Engage students, parents, and teachers by incorporating emerging technologies. Educate students by utilizing rigorous STEM curricula that meet national math- ematics, science, and technology standards and encompass the research and technology of NASAâs four mission directorates. Education Flight Projects (EFP) Develop and provide NASA-unique experiences, opportunities, content, and r Â esources to educators to increase K-12 student interest and achievement in STEM disciplines. Develop and facilitate a Network of Educator Astronaut Teachers (NEAT)-like group of highly motivated educators. Build internal and external partnerships with formal and informal education com- munities to create unique learning opportunities and professional development experiences. Educator Astronaut Project (EAP) Develop and provide NASA-unique experiences, opportunities, content, and r Â esources to educators to increase K-12 student interest and achievement in STEM disciplines. Develop and facilitate a Network of Educator Astronaut Teachers (NEAT)-like group of highly motivated educators. Build internal and external partnerships with formal and informal education com- munities to create unique learning opportunities and professional development experiences. Interdisciplinary National Science Project Incorporating Research and E Â ducation Experience (INSPIRE) Attract and retain students in STEM disciplines. SOURCE: Information from NASAâs 2006 project plans and personal commu- nication, Shelley Canright, outcome manager, Elementary and Secondary and e-Education Programs.
66 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM support teacher training or student activities for NES and to support the development and implementation of school âaction plansâ for the use of NASA units and materials. In fact, specialists report that they now spend about 60 percent of their time working with NES and another 10 percent with the digital learning network (DLN), which is part of NES. The rest of their time is allocated for non-NES schools and teachers (20%) and on NASA center-related programming (10%) (Horn and McKinley, 2006). Project Evaluations AESP was the subject of a 3-year external evaluation in 2001â2004 (Horn and McKinley, 2004) and a small follow-up evaluation in 2006 (Horn and McKinley, 2006). In the 3-year evaluation, a variety of methods (surveys, interviews, site visits, presentations, review of documents, and the NASA Education Evaluation and Information System [NEEIS]) were used to gather data from a provider and client group in order to address 19 evaluation questions. The evaluation concluded that AESP provides good support to NASA projects in raising awareness of the available resources and services. How- ever, many schools and teachers remain unaware of AESP services. In addi- tion, specialists most often engage in activities that generate immediate interest but do not necessarily have long-term effects in terms of education reform and improvement and curriculum enrichment. Although there was enthusiasm from participants for AESP presentations, all respondents indi- cated that the residual effect of the program is relatively low. The evaluation raised the concern that the project might be limited because of its adher- ence to an âin-personâ presentation model, rather than incorporating more distance learning technology. The supplementary 2006 evaluation (Horn and McKinley, 2006) used case studies of sites selected as exemplary, surveys, and analysis of NEEIS data to address six evaluation questions. The report provides good insight into activities at these sites, but there is no solid evidence of impact. The evaluation details AESPâs role in supporting other NASA education programs (Horn and McKinley, 2006). Requests by NASA programs for support services from AESP personnel are frequent, and requests also come from schools and educators. In fact, particularly with the extra load of the NES, requests have become so frequent that the aerospace education spe- cialists are not able to deliver all needed services in a face-to-face Âmanner; thus, they have begun to use the DLN to reach schools, particularly the NASA Explorer Schools, through the Internet and videoconferencing.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 67 Project Strengths One of the strengths of the project is its responsiveness to clients in providing services and other types of support through a network of regionally based specialists. Another is the use of former teachers as the NASA Âeducators. This approach provides a group of knowledgeable Âformer t Â eachers who have some understanding of school systems and of the instruc- tional needs of students. The geographic distribution of these educators allows each AESP specialist to become knowledgeable about the state stan- dards for the two or three states they serve. The ability of the specialists to engage the regional educational system and form local or regional partnerships is critical for ensuring that NASAâs activities are used in an effective way as part of school science and math- ematics instruction. The specialists are particularly important in rural states or states without NASA centers that may otherwise have little access to NASA activities and materials. Areas for Improvement The distributed model also has a potential weakness. The quality of the services delivered regionally appears to depend heavily on the individual specialists and the relationships a particular specialist is able to build with local educational organizations, districts, and schools. In this respect, a high turnover rate for specialists, which was noted in the 2006 evaluation report, is a problem. In addition, the specialistsâ role in the NASA Explorer Schools appears to be limiting the amount of time for them to work in other schools (Horn and McKinley, 2006). The committee is concerned about the ability of specialists to remain abreast of newly emerging NASA science and technology related to NASA missions. A yearly workshop, the current means for updating specialists on new developments, seems insufficient for keeping them truly up to date. Specialists need immediate links to the science, scientists, technology, and engineers in the agency in order to be able to effectively communicate cur- rent science and engineering developments and information to teachers and students. In the committeeâs view, the stated objectives for the project are too broad, and, therefore, potentially misleading. Those objectives closely f Â ollow the overall objectives for the Elementary and Secondary Program, with little specification to make them more appropriate to AESPâs scope and target audiences. In addition, the breadth and lack of structure in the project has led to a lack of stability in the focus and sustainability of specific project goals, and there is little evidence of any sustained effects on Â teachersâ professional development. There are some teachers who, by
68 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM their own testimony, have found the opportunities offered by the program valuable, but this seems to be more the result of good choices by individual dedicated âcustomersâ rather than a consequence of good project design. A system for setting priorities for services might be useful to ensure a broader base of âcustomers,â rather than relying solely on a customer-initiated, first-come, first-served approach (which may serve the same few teachers year after year). Finally, it appears that the basic design of the project hasnât changed in 40 years and, as noted above, remains structured mainly around personal contact (although with some recent forays into other approaches because of the workload). Personal contact is indeed critical for building relationships and networks; however, NASA should also explore how information and communication technology such as the Internet can be used to disseminate materials, connect to schools, and improve and increase communication in general. Such use of information and communications technology could both leverage and extend the impact of face-to-face sessions. NASA Explorer Schools The NASA Explorer Schools (NES), launched in 2003, consist of 3âyear partnerships between NASA and selected schools, with a focus on under- served and underrepresented populations in grades 4â9. The project focuses on the whole school. As of 2007, there are 200 schools currently desig- nated as NES. They are in all 50 states, with at least one school in each state. Overall funding is managed at NASA headquarters, but the project is administered through center personnel, particularly the AESP staff and NES teams. Each school teamâcomposed of four or five people including a school administrator and three or four teachers or specialistsâworks with NASA support to develop and implement a 3-year action plan for how to work with NASA resources to address local challenges in STEM educa- tion. By policy, the project and its school-level action plans consider only NASA-developed education materials, and a primary job of the AESP is to inform the teachers about these materials. Consequently, the action plans mainly serve as a catalogue that identifies which NASA materials can best be used in specific classes. During the 3-year partnerships, the project provides summer profes- sional development workshops for teams of teachers and administrators, as well as ongoing professional development during the school year. Students have opportunities to participate in research, problem solving, and design challenges relating to NASAâs missions. Schools receive $17,500 in grants to support the purchase of technology tools, online services, and inservice support for the integration of technology that engages students in STEM learning. Because of the role of the Digital Learning Network in the NES,
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 69 a significant expenditure is often to bring full videoconferencing capability to each school site. The NES project itself does not have the funding to provide all of the needed services to NES schools. Many are provided through other NASA K-12 programs, such as AESP or SEMAA, or through other organizations. AESP specialists, for example, work directly with each school and are the most important mechanism for NES team members to receive onsite profes- sional development training; assistance with such special events as family nights; special training with technology tools, such as videoconferencing; remote control of telescopes and robotics; and collaboration with projects that require extensive preparation (e.g., the Reduced Gravity Project). Project Evaluation An ambitious evaluation plan for NES, only partly realized, is currently in motion. The initial plan for evaluation used formative evaluation as part of a âdesign experimentâ approach (McGee, Hernandez, and Kirby, 2003). That approach combines project design and evaluation through a series of attempts to address a problem where different hypotheses about how the problem might be solved are tested and modified until the best solution is identified. In addition, an experimental design for summative evaluation was proposed. However, the design experiment approach was not consis- tently used, and the experimental design for summative evaluation of sites completing the project proved unworkable (Hernandez, McGee, Reese, Kirby, and Martin, 2004a, 2004b). In addition, there were problems with sampling and approach to analysis that undermined the strength of the data and called any interpretations into question (Lawrenz, 2007). In spite of these drawbacks, the fourth evaluation brief on the project provides some insight into the project; however, it does not provide clear evidence of effectiveness (Davis, Palak, Martin, and Ruberg, 2006). For example, the evaluators identified the following factors that contribute to successful implementations of NES: teamwork of the field center staff; responsiveness to the schoolsâ needs and goals; ongoing communication with the schools; multiple forms of communication (i.e., e-mail, telephone, face-to-face); and frequent visits by the NES field center staff to a school. The evaluators noted that these factors map closely onto findings from the literature on school improvement which were also identified in a paper commissioned by the committee (Mundry, 2007). In addition, it appears that the project has been most effective when the schools were already engaged in an effective school reform process and thus could delineate their needs and goals in the context of that process. The evaluation at the close of the first year suggests that one of the most difficult elements of the NES approach is supporting school teams to develop strategies for
70 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM incorporating NASAâs materials with clear objectives for learning and teaching in mind. Teachersâ self-report data included in the fourth evaluation brief (Davis et al., 2006) indicate that they have enjoyed the training activities and that their content knowledge and teaching strategies have been influenced by the professional development opportunities. Studentsâ self-report data indicated that the project has had a positive effect on their content knowledge of and interest in STEM subjects. However, no direct observations of instruction or objective measures of teachersâ and studentsâ content knowledge were collected. In addition, no comparison groups were used, and project par- ticipants were not tracked longitudinally. Project Strengths NES is an ambitious project with commendable goals. The focus on disadvantaged students and schools is laudable, though working with such schools adds challenges, as many often lack resources even for basic STEM programs. The NES model recognizes the need for long-term professional development and cooperation between administrators and teachers to achieve meaningful and lasting change. Of the seven core headquarters projects, only NES offers teacher enhancement opportunities that last a week or more. The initial format of these sessions was mostly a series of multiple, short informational sessions about NASA curriculum enhance- ment resources, but it appears that at least in some regions the project staff have been responding to formative evaluation input. The result has been that more of the offerings in summer 2007 provided in-depth learning opportunities for teachers and work with partners who have the experience needed to deliver such experiences. Areas for Improvement Although the project model acknowledges the need for a sustained engagement with teachers in order to change instruction, the committee questions whether the scope of the work required to support NES schools is appropriate for the agency. Does the project model capitalize appropri- ately on NASAâs key strengths and resources? Is a focus on a relatively small number of schools distributed across all states the appropriate way for a federal science agency to try to affect education? Is the project model effective in improving instruction and student learning? First, the committee notes that NASAâs scientists and other personnel do not, in general, have the deep knowledge of education required to under- take whole school improvement. Improving teaching and learning in STEM subjects for a whole school requires a coherent schoolwide approach to
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 71 science and mathematics curricula, not all of which link directly to NASAâs missions. Developing materials on basic science content (not directly related to space science or exploration) for individual schools is not an appropri- ate activity for NASA. Furthermore, a coherent science program cannot be achieved by simply drawing on existing NASA materials. Second, NES involves 200 schools, representing less than 1 percent of elementary schools nationwide. Although deep engagement with small numbers of teachers and students can be an effective strategy for some p Â rojects in NASAâs precollege portfolio, support for basic STEM educa- tion that is not obviously connected to NASAâs science, engineering, and exploration missions seems well beyond the scope of work in precollege education that is appropriate to the agency. Third, NES is an expensive project that draws resources from existing NASA programs in ways that are not obvious in the budget. For example, despite high investments, the project does not provide the level and length of support necessary for successful whole school reform (Mundry, 2007). In addition, NES relies heavily on AESP for support, and there is evidence in evaluations that AESPâs broader function is being negatively affected by this work by its staff. Finally, there is no compelling evidence that NES consistently results in improved teaching and learning in participating schools. Some schools affiliated with the NES project showed increased performance, but NES was not the only initiative in place in these schools. In fact, the evaluation results suggest that creating meaningful and comprehensive changes in teaching and learning in STEM subjects was an ongoing challenge for the NES project. Digital Learning Network The Digital Learning Network (DLN) has been a component of NES, but the agency is considering making it a stand-alone project. DLN pro- vides videoconferencing and webcasting capabilities to allow teachers and their students to participate in live lectures and demonstrations with NASA personnel. The project began in 2004 with three hub sites (NASA Glenn Research Center in Cleveland, OH; NASA Johnson Space Center in H Â ouston, TX; and NASA Langley Research Center in Hampton, VA). Two-way audio- and videoconferencing systems that are based on either H.320 or H.323 standards are compatible with DLN. For participa- tion in a webcast, one computer with Internet connectivity and an optional projection device or a computer lab and Internet connectivity are neces- sary. Student interaction is possible in a chat forum or by e-mail. To use DLN, teachers select from an inventory of topics and schedule a time for participating in a conference. Most events are available only through videoÂ conferencing (though some are also available through a webcast).
72 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM Project Evaluation In the fall of 2005, project staff and an outside evaluator began devel- oping a method for analyzing DLN modules and the effects of the modules on teachers and students. The team developed a content assessment that consisted of multiple choice questions related to the concepts in microÂ gravity covered in the module. They also developed a general rubric that listed the criteria on which to rate modules. The rubric identifies several dimensions on which to rate modules using a 4-point Likert scale: description of the scheduler, developmental appro- priateness, focus question, objectives, national standards, degree of student inquiry, prelesson, videoconference interactivity, videoconference content, videoconference graphics and video, postlesson, assessment, vocabulary, and resources. A module must receive a score of 3 or 4 points in each cat- egory in order to be reviewed. The process for revising modules that do not achieve the necessary score was not described in the report. Limited docu- mentation supporting the rubric was presented. The report does not provide adequate information about the reliability or validity of the rubric. The DLN staff has made an effort to update the list of modules offered through elimination of some modules and creation of new ones. Decisions about what to eliminate were made on the basis of the frequency of requests by teachers, whether presenters were still available to present the module, and the staffâs judgment regarding quality. Project Strengths DLN has the potential to allow students to interact with NASA scientists and engineers. The project has been expanded to include webcasts and some podcasts, which take better advantage of new information and communica- tions technology. The staffâs efforts to update the offerings and develop a review system for modules are commendable and represent an important step toward quality control and continued improvement of the project. Areas for Improvement It is important to review and cull the existing modules, but the most recent effort to do so was based on user demand, with only weak standards for assessing the educational merits of the modules. In addition, it is not clear whether the accuracy of the scientific content in the modules was reviewed. Future reviews should focus on the educational merits (effective pedagogy) and also examine the scientific content of the modules. In the long run, a module design process that includes educational expertise and not just the interests of an individual presenter is preferable.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 73 The project should continue to explore ways to expand DLNâs offer- ings to take better advantage of current information and communications technology, with particular attention to maximizing the cost-effectiveness of the project. For example, expanding the project through creation of additional facilities for videoconferencing does not seem as cost effective as expanding the use of webcasts. Science, Engineering, Mathematics and Aerospace Academy The Science, Engineering, Mathematics and Aerospace Academy (SEMAA) Project is designed to reach K-12 students who are traditionally underrepresented in STEM careers. SEMAA was initially established as a joint venture between NASAâs Glenn Research Center and Cuyahoga Com- munity College (in Ohio). Initially, SEMAA sites were selected by the center without competition; as of fiscal year 2001, however, sites are only added as a result of open competition or a congressional earmark. The most recent request for proposals for SEMAA sites was released in fiscal 2007 and will support three new sites. The long-term plan is to select three new and rotate off three existing SEMAA sites each year. Sites are expected to continue SEMAA operations beyond NASA funding, supported by financial and in- kind contributions provided by other STEM education stakeholders. SEMAA consists of three major components: NASA K-12 curriculum enhancement activities, family cafÃ©, and aerospace education laboratories. The programs are run by K-12 certified teachers that the SEMAA conÂtractor employs and trains as instructors. The curriculum enhancement activities are designed to use hands-on, inquiry-based K-12 activities that connect to research from NASAâs mission directorates. Because of the history of the project, the content connects most closely to the engineering and explora- tion activities in the missions and does not encompass research in the Sci- ence Mission Directorate. SEMAA students participate in the curriculum enhancement activities for a total of 36 hours each year, (21 hours during the academic year and 15 hours during the summer, with the exception of grades K-2 that participate 27 hours each year). In the original design, and at most sites, this program occurs after school or on Saturdays. One site has chosen to incorporate the activities into the schoolâs regular schedule. The family cafÃ© is an interactive forum that provides information and opens a dialogue between families, local education officials, and other com- munity stakeholders. It puts families in touch with local resources and helps them gain an understanding and appreciation of what their children are learning in the classroom. The family cafÃ© incorporates three forums: family focus groups, family nights, and home-based family initiatives. Family focus groups take place concurrently with the programâs academic year student sessions and provide up to 21 hours of participation for parents or adult
74 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM family members of SEMAA students each year. Family nights, typically 1-3 hours in length, are designed to be fun, learning events that bring SEMAA students and their parents or adult family members together to work on hands-on, STEM-related projects. Home-based family initiatives are hands- on, STEM-focused activities for SEMAA students and their parents or adult family members to work on at home. An aerospace education laboratory is an electronically enhanced, com- puterized laboratory that serves as a training facility for preservice and inservice teachers for curriculum enhancement activities. It engages Âstudents in real-world challenges, relative to both aeronautics and microgravity s Â cenarios. It houses aerospace hardware and software including an advance flight simulator; a laboratory-grade, research wind tunnel; and a working, short-wave radio receiver and hand-held GPS (global positioning system) for aviation. Costs for a SEMAA site are $375,000 for the first year ($200,000 for setting up an aerospace education laboratory and $175,000 for opera- tions), and $125,000 for the subsequent 2 years. After the initial 3Â years, sites are expected to develop partnerships and raise their own money to sustain the work. The 2006 fourth quarter report on the project indicates that just over $1 million financial and in-kind matching funds were raised during the fourth quarter for operations (NASA, 2006c). Project Evaluation SEMAA underwent a summative evaluation in 2001 that covered 1992â2001 (Benson, Penick, and Associates, 2001). The evaluation was based on analyses of program documentation and parent surveys. ÂStatistics on participants reported in the fourth quarter report for 2006 indicate that the project is largely reaching the intended audiences: of the 19,069 participants, 74 percent were African American, 6 percent were Hispanic, and 5 percent were Native American; 41 percent were from families with incomes below the poverty line. On the basis of records of participation, the evaluation concluded that SEMAA is meeting its goals for reaching underrepresented students. Parent surveys indicated that studentsâ interest in science had increased, and they also reported that studentsâ performance in STEM subjects had improved. However, the evaluation did not provide any objective data on studentsâ performance. The evaluators concluded that SEMAA is highly successful. They suggested that the project should develop a plan to conduct long-term tracking. In addition, the evaluation indicated that Hispanic students were underrepresented in the program, chiefly as a consequence of the limited geographical distribution of sites.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 75 Project Strengths Overall, SEMAA is meeting its goals, well-matched to some of NASAâs strengths, of inspiring students in STEM subjects. However, the project model might more appropriately be labeled informal education, rather than formal K-12 education. The project is reaching the intended audiences, and participants, both students and parents, are satisfied with their experiences (Benson, Penick, and Associates, 2001). The family cafÃ© is a strong com- ponent of the project and aligns with research on effective programs for middle school students that suggest family connections are an important part of learning (Westmoreland and Little, 2006). The SEMAA contractor has done an outstanding job in helping sites develop ongoing partnerships and leveraging project funding by raising matching funds. Areas for Improvement Participation in SEMAA indicates that it is reaching most of its intended audience, except for the relatively low participation of Hispanic students. Thus, the project needs to consider ways to increase the participation of Hispanic students. The committee questions whether the aerospace education laboratories use up-to-date technology and whether having one at each SEMAA site is cost effective in terms of the projectâs intended outcomes. For example, computer simulations might offer an alternative and less expensive flight simulator experience. The content of the curriculum enhancement activities should connect with research and activities across all four of the mission directorates. Currently, content related to the Science Mission Directorate is not well represented. There does not appear to be a plan to periodically review and update the activities presented in this program, and such updates are needed. Education Flight Projects Education Flight Projects (EFP) are a collection of projects targeted to elementary and secondary teachers and students and to informal education organizations and institutions. The projects are intended to offer hands-on experiences for students; they are implemented through the agencyâs flight platforms such as the international space station and the space shuttle. EFP was officially established in 2003, bringing together several existing projects. Beginning in 2006, EFP was to be overseen by the Teaching from Space Education Office at Johnson Space Center.
76 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM Three activities under ELP are linked to the international space station (ISS): EarthKAM, amateur radio on ISS (ARISS), and education downlinks. EarthKAM, established in 1996, enables students and educators to visually investigate and analyze the earthâs surface from the unique perspectives of space. It utilizes a digital camera on the international space station, which transmits images of the earth. Students can request images based on their classroom investigations. The EarthKAM camera flew five space shuttle flights and is now on the space station. ARISS is an organization that was formed to design, build, and oper- ate ham radio equipment on the international space station. It was created in 1996 when delegates from major national radio organizations and from the Radio Amateur Satellite Corporation in eight nations involved with the space station signed a memorandum of understanding. Through ARISS, students gain experience in telecommunications using amateur radio tech- nology to speak directly with the crew of the space station. Education downlinks, established in 2001 are live, 20-minute, video sessions during which students and educators interact with the crew of a mission as the crew answers questions and performs educational demon- strations. Prior to the event, student participants are expected to study the space station and its onboard science activities and develop questions to ask the crew. Usually, two education downlinks occur each month. The sessions are hosted by people in the formal and informal education com- munities, NASA centers and education programs, and the space stationâs international partners. Live in-flight education downlinks, which have one- way video (from the space station) and two-way audio, are broadcast live on NASA Television. To participate in EarthKAM, ARISS, or the in-flight downlinks, schools must submit a proposal that describes how the EFP activity will be inte- grated in the classroom and the intended learning outcomes. The proposals are then evaluated on the basis, first, of educational value, and, second, on whether the timing is possible given the flight schedule. These projects have not been widely publicized partly because of their limited Âcapacity. How- ever, EarthKAM has recently increased capacity and is trying to expand its reach to a broader audience. Currently, many of the educators who apply have had previous contact with NASA through AESP, NES, or the centers. Another part of EFP are suborbital flight platforms, which will pro- vide various opportunities for students to engage in activity-based learn- ing through suggesting projects for the educational rocket initiative, student experimental module-balloons, student experiment module-sonde, FreeSpace, and sounding rockets.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 77 Project Evaluation EarthKAM was externally evaluated by Education Development ÂCenterâs Center for Children and Technology in 2006 (Ba and Sosnowy, 2006). The evaluation was intended to examine the project in light of NASAâs educa- tion goals and provide strategic recommendations for future directions. The evaluation was conducted over 3 months using qualitative methodology to obtain an in-depth understanding of the status of the program implementa- tion and its impact on participants. The evaluators conducted face-to-face and telephone interviews with project staff and participants; conducted a site visit; and reviewed relevant online and print documents and data from the agencyâs central database for education, NEEIS. The data from the evaluation are limited as only four teachers were interviewed, and the data from NEEIS were not readily available in formats that allowed for data analyses. The evaluation provides a detailed description of how the project is implemented and includes a set of conclusions and recommendations. The evaluators stress that a mechanism for systematically documenting the program, for both formative and summative evaluation purposes, is needed. They also note a number of potential areas of improvement for the p Â roject, including strengthening training for teachers to use the program and website, expanding and updating the curriculum resources available, and improving the reach of the project. Project Strengths EFP activities have the potential to provide very powerful experiences that engage students with STEM subjects. First-hand interaction with data, such as the EarthKAM images, and direct conversations with astronauts can also be a mechanism for building insight about the nature of science, engineering, and space exploration. ARISS is likely an exciting project for a small group of ham radio operators across the world, though it cannot be clearly defined as an educational program. Areas for Improvement With the possible exception of EarthKAM, EFP activities appear to reach only a small fraction of educational institutions in the United States. Even EarthKAM does not currently serve the maximum number of schools the project can accommodate. This lack of coverage may indicate the need for better dissemination of information about the project. The external evaluation indicates that the partnership with TERC (a nonprofit organiza- tion based in Cambridge, MA) to support outreach activities was successful;
78 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM however, this effort was hampered by budget cuts in 2006. In contrast with the potential to expand EarthKAM, ARISS opportunities are limited, and there is a rigorous process for reviewing of proposals. Currently, ARISS appears to reach a small audience, many of whom are not in the United States, and few schools are part of this network. At this time it appears that EFP does not provide enough support for teachers to help them understand how best to use project experiences. It would be useful to have data on how teachers and students use the resources and what steps lead to most effective use. Educator Astronaut Project The Educator Astronaut Project (EAP), established in 2003, trains outstanding teachers to become members of the Astronaut Corps. To date, 190Â teachers have been identified as the top tier of program applicants, and they have been made members of the Network of Educator Astronaut Teachers (NEAT). Three were selected to receive astronaut training, and the first educator astronaut, Barbara Morgan, participated in a flight in August 2007 before school was in session. Starting in 2006, the EAP is being over- seen by the Teaching from Space Education Office at the Johnson Space Center (previously managed by the Office of Education). The EAP encompasses several activities. Educator astronaut Ârecruitment/ selection activities guide the recruitment of outstanding educators to join the Astronaut Corps. The first recruitment took place in 2004, and the next recruitment may take place in 2008. EAP provides support for the actual flight of an educator astronaut, including the development, planning, inte- gration, and implementation of education activities during the premission, mission, and postmission phases. The educator astronaut is involved in activities at all of these phases. The premission activities for the August 2007 flight included materials on the website describing the flight and preliminary projects, such as the design of a pennant that will fly on the space shuttle. During the flight, B Â arbara Morgan participated in three interactive downlinks. Students also had the opportunity to participate in a challenge to design a model of a growth chamber that might be used on the moon. In conjunction with this activity, the shuttle carried an education payload of several million basil seeds. Teachers could request seeds that flew on the shuttle to plant in stu- dentsâ growth chambers. Morgan set up two small chambers on the space station and discussed the design challenge during the downlinks. â ersonal P communication, Cynthia MacArthur and Edward Pritchard, NASA project man- agers for EAP, June 2007.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 79 NEAT members are expected to serve as NASA education advocates by engaging their schools and communities across the country in the agencyâs education services and informing them of NASA resources. They participate in a one-time, 2â3 day professional development workshop to provide them with a background for this work. NEAT members are responsible for devel- oping their own local opportunities for sharing NASA information and resources. The Teaching from Space Education Office is planning to review the design of the NEAT in order to determine how to make it more robust and inclusive of other teachers. They are also interested in determining the best approach to selecting teachers to be part of NEAT. Project Evaluation EAP has not yet been externally evaluated; it is a high priority for the office that oversees the project. Project Strengths EAP has the potential to inspire many students through participation in the education downlinks and the design challenge. NEAT appears to have been formed in response to the strong interest in maintaining a link to NASA expressed by many teacher applicants who were not selected to become astronauts. This was a creative response to the desire to capitalize on valuable public interest and could provide another mechanism for dis- seminating NASAâs materials and information. Areas for Improvement In its current form, it is not clear how NEAT will be leveraged to dis- seminate NASAâs materials and information, both generally and in conjunc- tion with flights of educator astronauts. Examining how NEAT members could best be used, or how links to other projects, such as AESP, might be developed, would be useful. Because the project is so new and because the first flight of an educator astronaut took place in summer when schools were not in session, it is impossible to accurately assess the projectâs impact. Interdisciplinary National Science Project Incorporating Research and Education Experience The Interdisciplinary National Science Project Incorporating Research and Education Experience (INSPIRE), which is in a formative stage, is a Personal communication, Cynthia MacArthur and Edward Pritchard, June 2007.
80 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM replacement for a former program NASA SHARP. INSPIRE is a three- tiered project designed to maximize student participation and involvement in STEM subjects and to strengthen and enhance the STEM pipeline from middle school through high school and to the undergraduate level. TierÂ I is junior explorers (grades 9 and 10); tier II is junior guest researchers (grades 11 and 12); and tier III is collegiate interns (rising college freshmen and sophomores). INSPIRE is still in the planning stages; a pilot phase began in summer 2007. INSPIRE is designed to provide critical STEM pathways for eligible students, with special emphasis on underrepresented and underserved groups. Students will be exposed to STEM experiences and encouraged to consider graduate studies in STEM fields. It is also hoped that INSPIRE will provide a public benefit by incorporating parent and community participation through program activities that inform and engage the public in NASAâs exploration vision. INSPIRE will offer research experiences, short courses, workshops, and seminars for students. Project Evaluation The project is still in the design and planning stages. Project Strengths Although INSPIRE is not yet implemented, the committee commissioned a paper to review the research literature on projects designed to engage underrepresented students in STEM subjects and compare best practice to INSPIREâs design. (Hall, 2007) The author concludes that the INSPIRE model mirrors much of best practice about teaching and learning STEM sub- jects in out-of-school time, including such program elements as mentoring, family involvement, inquiry-based learning, and hands-on activities. Areas for Improvement Hall (2007) also provides suggestions to move the design closer to best practice. The author encourages the incorporation of hands-on activi- ties and suggests that INSPIRE make use of existing informal education organizations, such as Boys and Girls Clubs and faith-based organizations. For example, INSPIRE might use youth organization staff as cofacilitators, adapting effective procedures from youth organizations and creating similar learning environments or spaces that have proven successful for those orga- nizations. The author particularly cautions against activities in INSPIRE that might too closely resemble more formal school learning experiences. Rather, activities should be delivered in a way that provides youth with opportunities for choice, independence, flexibility, and social experiences.
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 81 Finally, the author points out that INSPIRE staff might consider calling on high school guidance staff and science educators to refer students who might otherwise be overlooked as INSPIRE candidates because they are not motivated by STEM subjects as taught in traditional classrooms. CROSS-CUTTING ISSUES Through the committeeâs analyses of the seven core projects, three cross-cutting issues emerged: improving the process for program and Âproject design and improvement; drawing on outside expertise in education; and maintaining a connection to the science and engineering in the agency. It is not the committeeâs intent to imply that NASA gives no attention to these issues: each of them is discussed in NASAâs new strategic plan for education. Rather, the committee seeks to emphasize the importance of these issues as a means to improve and bring more coherence to the agencyâs work in precollege STEM education. Improving the Process for Project Design and Improvement One of the most important cross-cutting issues is the need for a more intentional approach to the design and continuous improvement of projects. NASA appears to have already recognized this issue, as evidenced in the emphasis on a portfolio approach in the strategic framework and recent efforts to review projects. Taking a portfolio approach seriously will entail using strategies the agency has not consistently used in the past. First, the agency might benefit from further articulation of a strategy for K-12 activities across the agency and the role of the Elementary and Secondary Program specifically. Currently, the Elementary and Secondary Program is charged with contributing to outcome 2, âattract and retain stu- dents in STEM disciplines through a progression of educational opportuni- ties for students, teachers, and facultyâ and is integrated into the âengageâ and âeducateâ categories of the strategic education framework. Both this outcome and the two categories are broad. A more detailed analysis of NASAâs assets, the needs of the K-12 system, and research-based strategies for achieving the stated goals for K-12 education in the agency is needed. Next, NASA needs to sharpen goals and objectives for individual p Â rojects so that they better reflect the scope and specific activities of the projects, rather than the broad overall goals of the headquarters Office of Education. As currently stated in the administrative plans for the core projects submitted to the office, the broad goals and objectives of the Elementary and Secondary Program have often been used as a substitute for individual project goals. Moreover, the projects have not consistently attempted to provide project-specific goals and objectives that would be
82 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM closely aligned with project activities. For example, AESP nominally targets all of the Elementary and Secondary Program objectives and NES targets five of the six; see Table 4-1. Likewise, a portfolio approach requires thoughtful planning across projects, informed by knowledge of best practice in education. The process should begin with project design, with attention to how projects comple- ment each other and how they capitalize on NASAâs strengths. Special atten- tion should be given to the question of when it is appropriate for NASA to take the lead on projects and when it is appropriate to develop partner- ships. NASA also needs to have a systematic approach, based on educa- tional value, for determining which projects that originate from Âcenters or TABLE 4-1â Objectives for Seven Core Education Programs EFP and Objectives AESP SEMAA NESa EAP INSPIRE Provide short-duration professional X X development to engage teachers Provide long-duration professional X X X X development to educate teachers Provide curricular support resources that X X X â¢ use NASA themes and content to enhance student skills and proficiency in STEM â¢ inform students about STEM career opportunities â¢ communicate information about NASA mission activities Student involvement: provide K-12 X X X X X students with authentic first-hand opportunities to participate in NASA mission activities, thus inspiring interest in STEM disciplines and careers Dissemination X X Coordination X X NOTES: AESP = Aerospace Education Services Project; SEMAA = Science, Engineering, Mathematics and Aerospace Academy; NES = NASA Explorer Schools; EFP = Education Flight Projects; EAP = Educator Astronaut Project; and INSPIRE = Interdisciplinary National Science Project Incorporating Research and Education Experience. aIncludes DLN (Digital Learning Network).
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 83 m Â issions contribute to the portfolio and can be supported and when a new project is needed to address an emerging area of interest. Periodic review of projects in order to evaluate whether they have maintained their focus and are reaching their intended audience is also critical. To this end, there is a need to have a process for continuous Âproject improvement and periodic âcullingâârefinement of the portfolio. This c Â ulling should be done intentionally, with input from experts in education, and based on data provided by projects and through external evaluations (see Chapter 5). The criteria for culling and refining projects should be care- fully developed and should reflect the objectives for the overall portfolio. One potential challenge for the K-12 education program is to achieve a balance between projects that achieve a broad reach and those that foster deep engagement with the science and engineering content of the agency. The committee agrees that NASA has an important role to play in both sorts of activities. However, the two kinds of projects require very different designs and deployment of resources. There is also a need to reconsider project design as the needs of the educational community change and particularly as new technology becomes available. For example, AESP and DLN do not appear to capitalize suffi- ciently on emerging technologies. Programs that were designed around old technology or old approaches need to evolve as educational practice evolves and as new technologies emerge. For example, the emergence of standards- based approaches in STEM education necessitated a response from NASA projects, and AESP, SEMAA, and NES have made efforts to adjust to those new approaches. In developing projects, it is also important to consider the investment required to accomplish intended goals and whether that level of invest- ment is sustainable across the life of the projects. For example, NES is an expensive project that also draws resources from existing NASA projects in ways that are not obvious in the budget. Despite these high investments, the project still does not provide the levels of funding that are necessary for whole school reform (Mundry, 2007). In addition, NES relies heavily on AESP for support, and there is evidence in evaluations that the broader function of AESP is being negatively affected as a result. Drawing on Outside Expertise in Education The design and implementation of NASAâs K-12 STEM education pro- grams and projects should be informed by the substantial knowledge base in the cognitive and learning sciences and education. Such expertise is not a typical qualification for agency staff, since NASA is primarily a science and engineering agency. Thus, expertise in education must be intentionally brought into the agencyâs precollege projects through a variety of means.
84 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM Hiring education staff with appropriate expertise is one avenue. An example of this approach is the position of AESP specialist. The use of former teachers provides a qualified group of individuals who understand school systems and the realities of classrooms. The regional distribution of educators allows each AESP educator to become expert in the state standards for two or three states. Yet even these specialists may still lack expertise in curriculum development or professional development strategies, which are not areas of expertise for most classroom teachers. The committee also identified two other methods for increasing the involvement of individuals with expertise in education: partnerships and expert review. Both of these are already in use in some education projects and might be considered for wider use in the future. Partnerships Partnerships are already used in some of NASAâs education projects, and cultivation of partnerships and sustainability are part of the overÂarching philosophy described in the 2006 strategic framework. The former Office of Space Science explicitly called out partnerships as a basic operational principle: âBase all of OSSâs E/PO (education and public outreach) efforts on collaborations between the scientific and education communities thereby drawing upon and marrying the appropriate expertise of the two communi- tiesâ (Rosendahl, Sakimoto, Pertzborn, and Cooper, 2004). This emphasis has been carried forward in the Science Mission Directorate and is reflected in its guide (National Aeronautics and Space Administration, 2006b). One major criterion for education and public outreach grants is partnership sustainability, and the guide emphasizes that projects and activities ârequire the active involvement of the research team and participation partners with appropriate expertise.â This involvement might include expertise in cogni- tion and the learning sciences; design of effective instruction, curricula, and professional development; or evaluation. Partnerships can be used successfully to accomplish a variety of objec- tives, including development of curriculum materials, dissemination of materials, and support for professional development. Examples of success- ful partnerships include the partnership between EarthKAM and TERC to support educational use of images (see above) and a partnership between the SMD forums and Lawrence Hall of Science to develop space science GEMS guides. In fact in a recent summative evaluation of the education and public outreach effort of the Office of Space Science (Gutbezahl, 2007), several projects were identified as having developed exemplary resources for formal education; those projects included partnerships. See Box 4-2 for descriptions of these and other partnership projects. Similarly, in many
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 85 BOX 4-2 Examples of High-Quality NASA Partnership Projects in Education GEMS Guides. These guides engage students in direct experience and experi- mentation to introduce essential, standards-based principles and concepts. Clear step-by-step instructions enable all teachers to be successful presenting the activities. GEMS units offer effective, practical, economical, and schedule-friendly ways to provide high-quality science and math learning to all students. Informa- tion about GEMS can be found at http://www.lawrencehallofscience.org/gems/Â aboutgems.html. Mars Student Imaging Project. Teams of students in grades 5 through college sophomore level work with scientists, mission planners, and educators to image a site on Mars using the visible wavelength camera onboard the Mars Odyssey spacecraft. The curriculum was developed to align with national science education standards and fit with existing science curricula. More information about the Mars Student Imaging Project can be found at http://msip.asu.edu/. Sun-Earth Day. A series of programs and events occur throughout the year and culminate with a celebration on or near the spring equinox (âSun-Earth Dayâ). These programs are supported by a variety of resources, including a website, print resources, and various multimedia products. More information about Sun-Earth Day can be found at http://sunearthday.nasa.gov/. Modeling the Universe. A suite of hands-on activities and inquiries is related to current models for the origins and evolution of the universe. These activities are shared with 8thâ12th grade teachers at workshops at which the teachers receive content and pedagogical training, as well as classroom-ready materials supporting each activity. After completing the workshop, teachers have access to a webpage and wiki, which contain additional materials and support. More information about Modeling the Universe can be found at http://cfa-www.harvard. edu/seuforum/mtu/. NOTE: All the websites cited were current as of November 2007. cases, the NASA materials and activities that the committee judges as h Â aving the highest quality were those developed in the context of partner- ships between NASA scientists and other personnel and existing educa- tional organizations. Developing partnerships is also a strength of the AESP specialists. The ability of these specialists to engage the educational system and form local partnerships is important for ensuring that NASAâs activities are used in an effective way as part of school science and mathematics instruction.
86 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM Use of partnerships does not seem to be consistent across headÂquarters Office of Education projects, nor is it clear that there are consistent methods for determining which partners are most appropriate or have the best fit in terms of expertise for a given project. For AESP specialists, the extent of partnerships appears to depend very much on the characteristics of the indi- vidual and the relationships he or she is able to build with local educational organizations, districts, and schools. In this respect, a high turnover rate for specialists, which was noted in an external evaluation of the project, is a problem. Partnerships can be particularly useful in the design of curriculum materials, but they are not consistently used by individual projects such as DLN and NES. Without partnerships and careful design, curriculum support resources are often ineffective and difficult to integrate with exist- ing curricula. This concern was echoed in testimony provided on May 15, 2007, to the House Subcommittee on Research and Science Education by George Nelson, director of Science, Mathematics, and Technology Educa- tion at Western Washington University and a former astronaut. In answer to a question about how lack of coordination might hinder federal agencies from making an impact, Nelson noted: âThere is a huge inventory of poorly designed and under-evaluated mission-related curriculum (posters, lesson plans and associated professional development) rarely used in classrooms and with no natural home in a coherent standards-based curriculum.â Nelson did identify the GEMS guides as exemplary. NASA has not consistently tapped partners for expertise in the design and planning of projects. This is perhaps the most critical time for partner- ships. NASA should explore mechanisms to bring in this expertise early. NASA should consider which kinds of projects the agency is well positioned to initiate and which projects are better suited to partnership in which the agency plays a value-added role. Finally, projects designed to develop studentsâ interest in and knowl- edge of engineering might be of particular value because engineering does not usually receive attention in the K-12 curriculum. The agency could seek out partners and resources to leverage its contributions in this area. Expert Review Peer-reviewed competition and expert review is another mechanism by which expertise in education can be brought to bear on projects and pro- grams. Again, tapping outside expertise was an operating principle for the Office of Space Science: âUse outside advice from the scientific, educational, and minority communities in the planning, development, implementation, and assessment of all our education and outreach activitiesâ (Rosendahl et al., 2004). Expertise can play a role on several levels. In competitions,
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 87 expert panels provide an important filter for determining which proposals have the most educational merit. Expert review of curriculum materials or project design is another mechanism for maintaining quality (see ChapterÂ 5 for a discussion of expertise in evaluation). It is not clear whether expert review is consistently used in the seven core projects reviewed by the committee. In mission competitions in the SMD, the basic design of a project is part of what is evaluated in the competition. However, the current projects in the headquarters Office of Education were not selected through a competitive process and were not subjected to a rigorous expert review. The projects themselves also do not consistently use expert review by educators or by knowledgeable scientists and engineers in the design of their activities and materials. For example, the menu of modules provided through DLN has not undergone review by outside experts. It also appears that curriculum materials developed by SEMAA and by NES do not con- sistently undergo any kind of external review. The headquarters Office of Education is in the process of developing a mechanism for expert review of curriculum support materials. Currently, NASA produces a number of curriculum support materials that incorporate a variety of instructional activities for students, as evidenced in the large catalogues listing available materials. The current formal review process was developed by the Office of Earth Science and adapted by the Office of Space Science and is now coordinated by the Science Mission ÂDirectorate. The review is based on the assumption that materials have been field tested and have undergone formative evaluation prior to submission for review. The review is based on relevance to NASAâs mission and education goals, scientific accuracy, educational value (pedagogy), effectiveness of presenta- tion, documentation, ease of use, and power to engage and/or inspire the target audience. Products are reviewed by a panel of five to seven experts, including classroom teachers, education specialists, informal educators, and scientists. The reviews are conducted under contract by the Insti- tute for Global Environmental Strategies (IGES), a nonprofit education organization. Reviews occur on a twice yearly cycle, in MayâAugust, and DecemberâMarch. The headquarters Office of Education is currently studying the feasi- bility of a more frequent, rolling schedule for reviews, due in part to the demands that arise from increasing use of Internet and web-based activities. â See,for example: for space science educators, http://www.nasa.gov/audience/Âforeducators/ index.html; for NASA Central Operation of Resources for Educators, http://education.nasa. gov/edprograms/core/home/index.html; and for NASA Space Science Education Resource Directory, <http://teachspacescience.stsci.edu/cgi-bin/ssrtop.plex>.
88 NASAâS ELEMENTARY AND SECONDARY EDUCATION PROGRAM The process is being tested in collaboration with the Exploration System Mission Directorate and the Space Operations Mission Directorate. Given the challenge of designing effective curriculum resources, use of a review system is necessary to ensure the quality of materials. Furthermore, in conjunction with expansion of the current system, it would be worth- while to consider developing a mechanism for culling existing materials that may not have originally undergone rigorous review. As part of a review system, NASA needs a set of criteria for determining the kinds of topics or learning goals that are most appropriate to develop. For example, the committee agrees with the SMD guidelines (which in turn originated with the Office of Space Science) that it is not appropriate for NASA to develop materials that target basic concepts in science and mathematics that are not clearly tied to the science and engineering in the agency. One activity that might warrant more attention by the agency is the development and dissemination of materials and activities that offer students and teachers an opportunity for first-hand experience with the p Â rocesses of science and engineering design. Emerging research on how to design effective laboratory experiences of this sort indicate that they should: have clear learning outcomes in mind; be thoughtfully sequenced into the flow of instruction; integrate learning science content with learning about the processes of science; and incorporate ongoing student reflection and discussion (National Research Council, 2006). Connection to Science and Engineering Work in NASA The third cross-cutting issue the committee identified was the impor- tance of consistently connecting NASAâs work in precollege education to the science, engineering, and exploration carried out by the agency. The committee recognizes, however, that maintaining this focus in all of NASAâs K-12 activities presents challenges for those projects not directly linked to science or engineering missions. One such challenge is how to keep education field staff, such as the AESP specialists, SEMAA staff, or educator astronauts, apprised of NASAâs current work and related education resources. AESP makes an effort to update staff through yearly workshops, but the committee does not believe that this is sufficient. In addition, solid knowledge of the underlying sci- ence and engineering concepts is critical for the staff, and it is unclear how this depth of content knowledge is maintained. The use of the Internet and other technology to facilitate ongoing professional development might be one way to help address this challenge. A second challenge is how to respond to demands from partnering schools to provide more support for basic science and mathematics that
ANALYSIS OF NASAâS K-12 EDUCATION PORTFOLIO 89 are not necessarily linked to space science. There is evidence of this kind of pressure from schools in both NES and SEMAA. In such cases, NASA needs to be judicious in how to respond. For example, developing very general units on forces and motion or on ratio and proportion that are only super- ficially tied to the agencyâs science and engineering activities through choice of examples is inappropriate. However, even when development might be tied directly to NASA-related experiences, such as the process of designing a spacecraft, partnerships should be used, and schools should be referred to other individuals or organizations who can more appropriately work with the demands of the general K-12 STEM curriculum. This is admittedly a difficult line to walk; however, in the context of limited resources for educa- tion at NASA, it is important to figure out how to do so.