The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 67
Appendix D Program Evaluation Information Provided by the Agencies 67

OCR for page 67
68 Program Goals Evaluation Criteria for Success Successes USGS National Cooperative • Train students in geologic • Survey of participants and • Number of students • Program benefits participants Geologic Mapping Program mapping providers educated in geologic mapping (EdMap) • Help fund academic techniques research • Prepare students for real-world careers in the geosciences • Create collaborations between academic institutions, state geological surveys, and the USGS • Contribute to national geologic mapping efforts USGS/NAGT Cooperative • Provide an opportunity to • Surveys of participants and • Number of participants • Excellent interns are placed in Summer Field Training partner a highly able intern providers • Success of participants in research units Program with a quality science mentor earth science careers • Program quality is recognized by to work on a meaningful USGS and earth science academic research project community • Present research results at professional meetings or in refereed publications USGS Youth Internship • Stimulate ongoing interest • Reports by participants • Fraction of participants • Most available slots are filled Program in science among college and providers that obtained permanent jobs • Many interns accept positions at undergraduates • Program satisfaction USGS • Build a pool of well- prepared new college graduates who could fill vacancies in the USGS USGS Hydrologic Technician • Stimulate ongoing interest • Reports by participants • Fraction of participants • Most available slots are filled Internship Program in water science among and providers that obtained permanent jobs • Many interns accept positions at college undergraduates • Program satisfaction USGS • Build a pool of well- prepared new college graduates who could fill vacancies in the USGS

OCR for page 67
NSF Earth Sciences • Support research • External, formal evaluation • Number of applicants to • Interest of undergraduate students Research Experience for participation by in 2006 REU sites Undergraduates (REU) undergraduate students • Reports by providers • Number of former Program participants in an earth science career NSF Opportunities for • Increase participation • External evaluation by an • None specified • New degree programs at minority- Enhancing Diversity in in the geosciences by expert consulting firm using serving institutions (MSIs) the Geosciences (OEDG) African Americans, a logic model • New partnerships between MSIs Program Hispanic Americans, Native • External reviews every 3 and 4-year and research universities Americans, Native Pacific years by a visiting committee • Better understanding how to Islanders, and persons with • Reports by providers recruit and retain minorities in earth disabilities science • Increase the perceived • Slight increase in number of relevance of the geosciences undergraduate majors among broad and diverse segments of the population NSF Earth Sciences • Recognize beginning • External reviews every 3 • None stated • A high fraction of participants are Postdoctoral Fellowships investigators of significant years by a visiting committee in tenure-track positions potential • Reports by participants on • Participants are active • Provide these individuals research advances scientifically with research and education • Information on career • Some participants have become experience that will establish development of former mentors them in positions of fellows leadership in the scientific community continued 69

OCR for page 67
70 Program Goals Evaluation Criteria for Success Successes NSF Geoscience Education • Improve the quality and • External reviews every 3 • Widespread use of models • Incubation of the Digital Library (GeoEd) Program effectiveness of formal years by a visiting committee for understanding geoscience for Earth System Education and informal geoscience • Reports by providers content and strategies for • Incorporation of big ideas that all education at all educational developing spatial and citizens should know into the next- levels systems-thinking skills generation science standards • Increase the number of • Number of peer-reviewed • Creation of geoscience educators students pursuing geoscience publications who use best practices from STEM education and career paths • Continued research support education research to strengthen • Broaden participation of formal and informal education traditionally underrepresented groups in the geosciences • Promote public engagement in Earth system science NSF/NOAA/NASA Global • Improve student • Internal evaluation (not • None specified • A large number of K–16 students Learning and Observations understanding of specified) are engaged worldwide to Benefit the Environment environmental and Earth • A large community of teachers (GLOBE) Program system science across the with improved pedagogy and content curriculum knowledge • Contribute to scientific understanding of Earth as a system • Build and sustain a global community of students, teachers, scientists, and citizens • Engage the next generation of scientists and global citizens in activities to benefit the environment NSF Geoscience Teacher • Improve the quality of ESSEA project: ESSEA project: ESSEA project: Training (GEO-Teach) geoscience instruction, • Internal evaluation based • Resources developed and • Institutionalization of effective Program primarily at middle and high on data from course site, how much they have been professional development and school levels participant surveys and used training methods over many products, and provider • Effect of courses on universities surveys and products learner’s knowledge of Earth • Funding to create more modules system science and how to

OCR for page 67
teach it • Effect of the program on faculty and their institutions TESSE project: TESSE project: TESSE project: • External evaluation • Extent to which project • Identification of challenges planned based on participant goals, activities, and in extending a professional surveys, focus groups, phone outcomes are understood by development model from research interviews, observation of providers and participants institutions to HBCUs activities, and standardized • Extent to which program teacher assessment activities met stated goals • Extent to which teacher opportunities improved their understanding of earth science and ability to teach it • Success of processes and structures to improve collaboration DOE Office of Science • Encourage the • Program goals, outputs, Long-term outcomes include • Program is too young to Graduate Fellowship (SCGF) development of the next and outcomes defined in a • Pursuit of a career in DOE determine. The first cohort is nearing Program generation of scientific and logic model Office of Science-related the end of the 3-year support period technical talent in the United • Reports by providers and research in academia, a DOE States who will pursue participants laboratory, or industry careers in research critical to • Longitudinal tracking of the Office of Science mission participants • External review of WDTS portfolio in 2010 • External reviews every 3 years by a visiting committee continued 71

OCR for page 67
72 Program Goals Evaluation Criteria for Success Successes DOE Summer of Applied • Introduce students in • Reports by participants • Rate of continuation to • None specified Geophysical Experience geophysics and related fields and providers graduate school and careers (SAGE) Program to “hands-on” geophysical in geophysics exploration and research • Support and feedback from • Apply research and industry participants and geophysical exploration visitors methods to solve specific • Presentations at problems professional meetings and publications DOE Science Undergraduate • Encourage undergraduate • Program goals, outputs, Near-term outcomes include • More than 90 percent of Laboratory Internships students to pursue STEM and outcomes defined in a • Completion of STEM participants report that the internship (SULI) Program careers logic model degree helped prepare them for a STEM • Pre- and postparticipation • Pursuit of advanced degree career surveys of participants in STEM • More than three-quarters of • Longitudinal tracking of • Preparedness for STEM participants (2003-2011) are participants career pursuing a career in STEM • External review of DOE • Knowledge of DOE laboratories’ execution of the mission science and program technology • External review of WDTS portfolio in 2010 Long-term outcomes include • External reviews every 3 • Pursuit of a career in DOE years by a visiting committee Office of Science-related research in academia, a DOE laboratory, or industry DOE Community College • Encourage community • Program goals, outputs, Near-term outcomes include • None specified Internship (CCI) Program college students to enter and outcomes defined in a • Completion of STEM technical careers relevant to logic model degree the DOE mission • Pre- and postparticipation • Pursuit of certification in surveys of participants STEM • Longitudinal tracking of • Preparedness for STEM participants career • External review of DOE • Knowledge of DOE laboratories’ execution of the mission science and program technology

OCR for page 67
• External review of WDTS Long-term outcomes include portfolio in 2010 • Pursuit of a career in DOE • External reviews every 3 Office of Science-related years by a visiting committee research in academia, a DOE laboratory, or industry NASA Gravity Recovery None specified. Proposed • Reports by providers • Number of teacher and • None specified and Climate Experiment outcomes are: • External evaluations are student participants (GRACE) programs • Conduct GRACE activities planned • Field trip opportunities for with 500 teachers with students priority given to educators • Increase in knowledge of from underrepresented areas global climate change or with underrepresented • Student interest in STEM audiences careers • Increase the knowledge of 500 secondary teachers about the Earth’s history, Earth system science, and global climate change • Increase the knowledge of 500 students about earth science and global climate change • Make available selected activities of “changing mass = changing earth” programs through the University of Texas Center for Space Research Web site • Incorporate NASA satellite data and resources into the activities continued 73

OCR for page 67
74 Program Goals Evaluation Criteria for Success Successes USDA 4-H Environmental • Increase awareness of • Internal evaluation (not • None specified • 4-H enrollment Education/Earth Science science among youth specified) • Professional development has programs • Improve science skills and positive impact on performance and knowledge among youth satisfaction • Increase awareness of • High-quality programs opportunities to contribute to • Creation of a new research society using science skills database • Increase life skills among • Participants excel beyond their youth peers USDA Agriculture and Food • Increase technical • None yet • Number of participants • Participants complete their degree Research Initiative (AFRI) competency in priority projects area(s) to ensure that the United States remains globally competitive in the knowledge age USDA National Institute • Strengthen the ability • Reports by providers • None specified (project • None specified (project of Food and Agriculture of the nation’s scientific dependent) dependent) Fellowships Grant Program community to meet the current and future challenges facing agriculture, forestry and food systems • Develop the technical and academic competence of doctoral candidates • Develop the research independence and teaching credentials of postdoctoral scientists EPA Greater Research • Support undergraduate • External review by a • Number of awards • None specified Opportunities (GRO) study in environmentally visiting committee in 2009 • Number of completed Undergraduate Fellowship related fields • Reports by participants degrees Program • Topic area distribution • Professional success of former fellows

OCR for page 67
EPA Science to Achieve • Defray the cost of study • External review by a • Number of awards • None specified Results (STAR) Graduate leading to advanced degrees visiting committee in 2009 • Number of completed Fellowship Program in environmental science • Reports by participants degrees fields • NRC review of STAR in • Topic area distribution 2003 • Professional success of former fellows NOAA Educational Cooperative Science Centers: Cooperative Science Centers: Cooperative Science Centers: Cooperative Science Centers: Partnership Program (EPP) • Increase the number • External review of • Annual number of students • None specified With Minority Serving of educated, trained, cooperative science centers in from underrepresented Institutions and graduated students, their third year communities who are trained particularly from • NRC review of education and graduate in NOAA underrepresented program in 2010 mission sciences communities in STEM fields • Surveys of providers and • Annual number of students that directly support NOAA’s participants who are trained and graduate mission in NOAA mission sciences • Increase collaborative • Number of students research efforts between completing experiential NOAA scientists and opportunities at NOAA researchers at NOAA EPP facilities cooperative science centers • Number of students who are hired by NOAA; NOAA contractors; other environmental, natural resource, and science agencies; academia; and the private sector • Number of collaborative research projects undertaken between NOAA and EPP cooperative science centers in support of NOAA operations • Number of students and faculty who participate in and complete postdoctoral-level research programs in support of the NOAA mission continued 75

OCR for page 67
76 Program Goals Evaluation Criteria for Success Successes • Number of peer-reviewed papers published in NOAA mission sciences by faculty, postdoctoral fellows, and students sponsored by NOAA EPP • Funds leveraged with NOAA EPP funds • Number of participants engaged in NOAA mission-relevant learning opportunities Graduate Sciences Program: Graduate Sciences Program: Graduate Sciences Program: Graduate Sciences Program: • Increase the number • NRC review of education • Number of graduate • None specified of educated, trained, program in 2010 sciences program students and graduated students, • Surveys of providers and hired by NOAA particularly from participants underrepresented communities in STEM fields that directly support NOAA’s mission Undergraduate Scholarship Undergraduate Scholarship Undergraduate Scholarship Undergraduate Scholarship Program: Program: Program: Program: • None specified • Increase the number • NRC review of education • Number of undergraduate of educated, trained, program in 2010 scholarship students who and graduated students, • Surveys of providers and are educated, trained, and particularly from participants graduate in NOAA mission underrepresented sciences communities in STEM fields • Number of undergraduate that directly support NOAA’s scholarship students who mission pursue graduate work in NOAA mission sciences

OCR for page 67
• Number of undergraduate scholarship students who are hired by NOAA, NOAA contractors, and other natural resources and science agencies NPS/GSA Geoscientists-in- • Provide on-the-job • Reports by providers • Completion of unmet earth • Program has grown from 20 the-Parks Program earth science training for science projects in NPS units positions per year in 2007 to more America’s youth • Communication of earth than 100 positions per year in 2012 • Build technical capacity science to the public and for parks park staff who are not • Enhance the public’s geologists understanding of earth • Program growth science NPS/NAGT Geoscience- • Provide cooperative • Participant feedback • None specified • Teachers gain career enhancement Teachers-in-Parks Program exchange of learning and opportunities scientific research between • Program increases the recognition the park, local earth science of NAGT as a leader in promoting teachers, and communities earth science education • Advance educational and • Teachers sign up as park interpretive opportunities at volunteers and return to the park the park after completing their internship • Develop a lifelong network • Products developed during the with local communities, internship can be used by park staff schools, and the park for other educational purposes • Teachers continue using the park as an educational tool or destination • Students become more familiar with the park as an educational resource and more aware of the need to protect park resources • Teachers and students become stewards of parks and the environment continued 77

OCR for page 67
78 Program Goals Evaluation Criteria for Success Successes NPS National Fossil Day • Promote public awareness • Survey of participants • None specified • Rapid expansion of partners and stewardship of fossils embracing the mission • Foster a greater • Positive feedback from partners appreciation of the scientific and the media and educational value of fossils Smithsonian Leadership • Improve K–12 science • External reviews of the • Increased student • Statewide implementation of and Assistance for Science education program and its components achievement program (e.g., Washington state Education Reform (LASER) are planned LASER) Program NOTE: DOE = Department of Energy; EPA = Environmental Protection Agency; ESSEA = Earth System Science Education Alliance; GSA = Geological Society of America; HBCUs = historically Black colleges and universities; MSI = minority-serving institution; NAGT = National Association of Geoscience Teachers; NASA = National Aeronautics and Space Administration; NOAA = National Oceanic and Atmospheric Administration; NPS = National Park Service; NSF = National Science Foundation; STEM = science, technology, engi- neering, and mathematics; TESSE = Transforming Earth Systems Science Education; USDA = U.S. Department of Agriculture; USGS = U.S. Geological Survey; WDTS = Office of Workforce Development for Teachers and Scientists.