Strengthening High School Chemistry Education Through Teacher Outreach Programs: A Workshop Summary to the Chemical Sciences Roundtable (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Activities by Nonprofit and For-Profit Organizations Major Points in Chapter 6 The Bayer Corporation’s Making Science Make Sense program supports outreach to teachers and other activities designed to foster a well-educated workforce and a scientifically literate public. The Achieving Student Success Through Excellence in Teaching program provides high-quality teaching materials and professional development for elementary school and middle school teachers throughout Pennsylvania. The American Chemical Society sponsors several programs for high school chemistry teachers and can have a substantial influence on high school chemistry through its affiliates, local sections, and clubs. The Hach Scientific Foundation offers scholarships for chemistry majors and for people working in chemistry-related fields who intend to become chemistry teachers, and it supports training programs for current teachers. The grants program of the Howard Hughes Medical Institute emphasizes evaluations of the education programs it supports and wide dissemination of the results of assessments and of information about successful programs. Surveys and interviews conducted with representatives of 37 foundations active in science education found that foundations support a broad range of activities but, with several important exceptions, have gathered few data about the effectiveness of the programs they fund. THE BAYER CORPORATION here in the United States,” said McCourt. It is designed “to help teachers teach and to help students learn the way that Since the 1960s, scientists, engineers, and other employ- scientists do.” ees of the Bayer Corporation have been volunteering in local The program, which has been recognized by several schools to enhance science education. Most of these efforts national awards, extends from preschool to graduate school were ad hoc, said Bayer’s Bridget McCourt, until 1995, and beyond and addresses two populations of students. First, when all of the company’s educational efforts were brought it seeks to prepare a well-educated workforce that can fill together to form the Making Science Make Sense program. high-level jobs, including jobs in scientific and engineering “It is our premier corporate social responsibility program 29 

30 STRENGTHENING HIGH SCHOOL CHEMISTRY EDUCATION research. Second, it is directed toward producing a scien- classroom. Together, the three components of Making Sci- tifically literate public so that Bayer can be a successful ence Make Sense “form a comprehensive and integrated business. “When we’re either establishing a new site in the program that is driving results and impacting lives.” community or having a crisis in the community, we need As Bayer has expanded into new communities, it has [the public] to be based in science. . . . We need them to brought the program to new places. Branches of Bayer have a basic understanding of chemistry, biology, physics, outside the United States also have been instituting versions and other science fields so that they’re . . . educated voter[s] of the program in their home countries, including Japan, and neighbor[s].” Colombia, Italy, India, and the United Kingdom. In addition, Science education is particularly well suited to fostering Bayer has encouraged the involvement of other corporations the kinds of skill people need in today’s world, said McCourt. in educational initiatives, in part by sponsoring forums on It teaches creative thinking, critical thinking, team building, science education. A 2006 forum on educational diversity and adapting to change. Science education “is not just about held in Washington, DC, for example, attracted more than educating the next generation of scientists, engineers, and 150 STEM industry and organization representatives, federal mathematicians. It’s about equipping students with the skills and state government officials, and others in the nonprofit that they’ll need in whatever job they go into.” and science education fields. Making Science Make Sense has three components. The As with many of the other programs discussed at the first is systematic science education reform. Schools need workshop, assessment of the Making Science Make Sense help to move beyond traditional teaching approaches toward program is difficult, McCourt acknowledged. The company inquiry-based learning. For that reason, Bayer projects sup- requires projects supported through the program to perform port the national science education standards and incorporate assessments and provide the company with progress reports. substantial professional development for teachers. For many Bayer also has supported efforts to assess the state of science classrooms, McCourt observed, inquiry-based learning education in the United States, including an annual survey involves a “complete shift” in the way teachers are teaching on STEM programs, policies, and practices. and the way students are learning. “All of corporate America has a role to play in improving The second component of the program is public education science education and science literacy across the country,” and outreach. Led by former astronaut Mae Jemison, this said McCourt. “We believe that Making Science Make Sense component of the program has featured a variety of part- is an effective program for our corporation, and we are com- nerships on both the national and the local levels. Through mitted to continuing the program in the years to come.” these partnerships, Bayer has been able to learn where the During the question-and-answer session, Ken White from company’s efforts are needed and how those efforts can help. Brookhaven pointed out that if studies demonstrate the value For example, a partnership with the American Chemical of programs such as Making Science Make Sense, other Society has led to efforts to address teacher development and companies might be influenced to initiate and participate in diversity in science, technology, engineering, and mathemat- outreach efforts. McCourt responded that the initial success ics (STEM) education. This partnership resulted in a recent of a program can drive future successes, especially when set of three-day workshops on green chemistry for high volunteers come back into the workplace and describe their school teachers. It also created new internships in Texas that accomplishments to others. However, she also noted that give disadvantaged students an opportunity to experience one challenge she faces is to maintain the continuity of the chemistry careers through hands-on summer internships. In volunteer effort over time. Half of the Bayer workforce has addition, a partnership with the Carnegie Science Center in joined the company just in the past five years. “I have to Pittsburgh has led to a program in which high school stu- continually reorient people to the program, introduce them dents are trained and employed to do scientific experiments to it, explain to them what it is,” she said. A great advantage and field trips both during the school year and during the at Bayer is that the leadership of the company supports the summer with small groups of elementary students. Since program and encourages employees to participate. “It’s not its inception in 2000, this program has been reaching about seen as a detraction from their position but rather as an addi- 250 elementary school students per week during the school tion to their role in the company.” year and nearly 1,000 students on average in the summer, and each of the high school students who has participated in ASSET: ACHIEVING STUDENT SUCCESS THROUGH the program has gone on to college, often as the first person EXCELLENCE IN TEACHING from his or her family to do so. The third component of the program is based on employee One program that the Bayer Corporation has supported, volunteerism. As has been the case throughout Bayer’s along with other funders, is Achieving Student Success involvement with education, volunteers continue to work Through Excellence in Teaching (ASSET). It was created in with individual students, teachers, and schools to bring Pennsylvania in 1994 as an independent, nonprofit, educa- meaningful and enjoyable scientific experiences into the tional reform initiative dedicated to continuously improving 

ACTIVITIES BY NONPROFIT AND FOR-PROFIT ORGANIZATIONS 31 the abilities of teachers and the performance of students. Its engage in multiday workshops over more than one year, usu- vision is to be a leader in developing and implementing effec- ally focusing on one curriculum module each year. tive, innovative programs, products, and practices that align The program has contracted with Horizon Research, teaching to learning. It is focused on kindergarten through Inc., to do evaluation research, including comparing student eighth grade, which is essential to establish “a strong foun- scores with the amount of professional development teach- dation for you at the high school level,” said its executive ers have undertaken. The results show that students whose director Reeny Davison. ASSET has become “the leading teachers participated in three days of professional develop- science education organization in classrooms throughout ment scored significantly higher than students of teachers Pennsylvania.” who participated in two days or less. Furthermore, student The hypothesis behind the program is that high-quality achievement was greater the second time the teachers imple- materials and high-quality professional development will mented a module to which they had been exposed during an produce more effective teachers and better-performing stu- ASSET workshop. dents. It has employed standards-based curriculum materials, Davison called for cooperation among programs to centralized materials support, assessment, and involvement address the full range of problems facing teachers and stu- of the administration and communities to create a national dents. “There can’t be too many of us,” she said. “The time model for effective science education reform. The program for competition is over. It is all about collaboration.” has drawn heavily on materials and methods developed by the National Science Resources Center, which is a joint THE AMERICAN CHEMICAL SOCIETY project of the Smithsonian Institution and the National Acad- emies. The program also uses materials from other sources, Mary Kirchhoff of the American Chemical Society (ACS) such as the Full Option Science System from the Lawrence briefly described some of the activities undertaken by ACS Hall of Science. “As a nonprofit, we are free to become what to improve high school chemistry education. ACS conducts teachers need us to be. If teachers don’t need us, we will go a number of summer workshops, including a three-day resi- out of business.” dential workshop on green chemistry (partially sponsored by ASSET’s Materials Support Center purchases standards- the Bayer Corporation) and a five-day workshop on bringing based materials and stores, cleans, refurbishes, and distrib- chemistry into the community. “One of the things that struck utes those materials, in some cases with hands-on assistance me throughout the workshops is how much the teachers from its corporate sponsors. Schools choose the kits they appreciate the opportunity to talk with each other,” Kirchhoff want to use, which range across the earth, life, and physical said. Teachers from different kinds of schools were able to sciences as well as technology and engineering. “Like a good describe both the particular challenges they faced and the business, we give our districts choices,” said Davison. “We issues common to all teachers. don’t tell them what they have to order. They order what’s Other ACS activities provide training for teachers of right for their curriculum and for their teachers.” advanced placement (AP) and international baccalaureate ASSET also supports professional development in the courses in chemistry and offer workshops for middle school form of teachers’ teaching teachers. “When you have another science teachers and their supervisors. A new edition of the teacher standing in front of you, there is instant credibility, book Chemistry in the National Science Education Standards because they can say that when I did this I found that this addresses standards and provides models for meaningful trick helped.” learning in high school chemistry classrooms. In 2001 the program transitioned to a fee-for-service The ACS has looked periodically at the idea of forming organization, which required that it continually develop a stand-alone high school chemistry teachers association. new products and services for teachers, in part through Although the idea has not gained traction in the past, said partnerships with private organizations. In 2006 the State of Kirchhoff, she planned to bring it up again with the society’s Pennsylvania launched the “It’s Elementary” initiative and Committee on Education. “Out of 160,000 members of the arranged with ASSET to expand its program throughout the ACS, only a couple of thousand are high school chemistry state. “No one in the Pennsylvania Department of Educa- teachers. Clearly, they are not finding the value that we could tion or the Governor’s Office designed the program,” said be providing to them.” Davison. “We got to design it, coordinate it, and implement ACS has large networks of members, local sections, stu- it according to the things that we have learned in the last 10 dent affiliates, and high school chemistry clubs, all of which years.” ASSET would like to become a professional develop- can have an influence on high school chemistry education. ment center that teachers can rely on in a standards-based Where resources are not available in a particular school or environment. ASSET is currently serving 164 school districts, 6,392 teachers, and slightly more than 180,000 students. Teachers S. L. Bretz, ed. 2007. Chemistry in the National Science Education Standards, Second edition. Washington, DC: American Chemical Society. 

32 STRENGTHENING HIGH SCHOOL CHEMISTRY EDUCATION district, the ACS can step in and provide a service directly or worked in a chemistry-related field and are looking to go foster a partnership that could meet the needs that exist. into teaching. This $6,000 scholarship can be used at any The ACS also has been working with organizations in college or university in the country as long as the student higher education such as the National Association of State has been accepted into a program to work toward a master’s Universities and Land Grant Colleges to improve chemistry in education. A $3,000 scholarship is offered for part-time education, including the education of undergraduates who students who continue to work or support a family. “We’ve become chemistry teachers. had scholars ranging from their early 20s to their early 60s,” said Hach. With just three full-time staff members, the foundation THE HACH SCIENTIFIC FOUNDATION provides universities with the criteria for the chemistry major Clifford Hach was a chemist who worked on the Man- scholarship, and the universities administer the scholarships, hattan Project in the 1940s and started the Hach Company, usually through the chemistry department. The foundation which was an analysis, instrumentation, and water chemistry chooses the second-career scholars itself, with advertise- firm. Located originally in a one-room building in Ames, ments in chemistry publications to inform potential recipi- Iowa, the company grew and moved to Colorado in the ents about the program. 1970s. In the early 1980s, it formed the Hach Scientific The foundation also has decided to provide in-service sup- Foundation to provide scholarships to future chemists. The port to chemistry teachers, so it has begun a program to offer foundation became fully funded when Hach died and the a $1,500 grant to any chemistry teacher who would like to company was sold. improve his or her teaching skills. A one-page application on Several years ago, Clifford Hach’s grandson Bryce Hach, the foundation’s Web site (http://www.hachscientificfounda- who is executive director of the foundation, decided to drive tion.org/home.shtml) asks how teachers are going to use the to each of the scholarship recipients and ask them why they funds and how the impact of the funding will be measured. chose to study chemistry. “I was a biology major myself, “We want to make the application process as simple as we and I wasn’t the greatest chemistry student in the world. I can,” said Hach. Later, teachers write a one-paragraph sum- thought chemistry was really hard, so I was curious,” Hach mary of how the grant was used that is posted, by state, on said. “At least 90 percent of them said that the number one the Web site. influence that led them into chemistry was a really good With very little advertising, the foundation received more high school chemistry teacher.” That led Hach to give greater than 200 applications in the first two-and-a-half months of consideration to the importance of these teachers. Only about the program and was able to grant 178 requests in more than a quarter of high school chemistry teachers have a degree in 40 states. The program “was far more exciting and far more chemistry, Hach said, and less than half of them minor in involved than we ever thought it would be, and we’re really the subject. Motivated by these observations, the foundation glad to continue the program. Certainly it shows that there’s decided to broaden its involvement in chemistry education. a lot of untapped potential here.” In the 2007-2008 academic year, the foundation began In northern Colorado, the foundation runs a program to offering scholarships to chemistry majors who plan to go bring together almost all of the chemistry teachers in four into teaching. At least two $6,000 scholarships are provided local school districts to engage in workshops organized at each of the land grant universities in the country, which around the Process Oriented Guided Inquiry Learning, or ensures that the program will have a national reach. The POGIL, approach. The program begins with a three-day scholarships are available for undergraduates at any level, workshop, followed by subsequent one-day and two-day including undergraduates who want to spend extra time in a workshops. The foundation pays for substitutes while teach- university to take education classes. The foundation wantsto ers attend the workshops. POGIL “transforms the chemistry reach students who are thinking about going into research, class from a passive learning environment to an active one,” industry, the pharmaceutical industry, medical school, or said Hach. “Students have to teach each other. They work in other destinations and get them thinking about teaching. small groups. They’re doing real research. They have to take It wants to “create teachers where there otherwise weren’t the onus of education on themselves.” any.” Two chemistry education Ph.D. students are doing their In addition, the foundation has created a second-career dissertations on the impacts of these workshops on learning. chemistry teacher scholarship program for people who have Initial assessments have demonstrated a 15 to 20 percent increase in the grades of students whose teachers have participated in workshops and a 15 to 20 percent increase In January 2009, the Hach Foundation announced that it plans to transfer in students’ going on to higher levels of chemistry. “The the foundation’s funds and assets to the American Chemical Society (ACS) results from this program will be available on our Web site to administer the grants described in this section. For more information see: as soon as they’re formally released,” said Hach. “Everything Raber, L. 2009. Philanthropy: ACS Receives Hach Funds. Chemical and is going to be transparent to the public.” Engineering News 87(4):7. 

ACTIVITIES BY NONPROFIT AND FOR-PROFIT ORGANIZATIONS 33 THE HOWARD HUGHES MEDICAL INSTITUTE college entry rates, and focus groups. “In science education we have learned that an arsenal [of assessment strategies] is The Howard Hughes Medical Institute (HHMI) is primar- really what’s needed. . . . Assessment has to be very creative, ily a biomedical research organization, funding more than and you have to be willing to do many different types.” 300 scientists and their associates in research laboratories The other program she described was at Occidental Col- across the United States. However, HHMI also has a grants lege, which had the goal of improving high school biology program that supports precollege science education, along and chemistry students’ laboratory instruction by enhancing with undergraduate and graduate education and research. In their teachers’ knowledge and classroom application of mod- the most recent precollege competition in 2007, $22.5 mil- ern instruments, techniques, and experiments. The program lion in grants were awarded over five years to 31 institutions consisted of 13 experiments developed and tested in class- to engage in educational outreach. rooms by a steering committee of about a dozen high school HHMI has learned a number of valuable lessons since the teachers and five college staff members. The experiments, grants program was initiated in 1987, according to HHMI’s which conformed to the science framework for California Patricia Soochan. The first is to assume nothing and assess public schools, employed a biochemical focus to enhance everything. Assessments should be done early, often, and and bridge the biology and chemistry curricula. Each teacher comprehensively and should be quantitative as well as quali- who participated in the program attended a two-week sum- tative. “Assessment should be used to adjust the program as mer institute focused on the details of the experiments. necessary and make sure the grantee is on the right track,” Activities reflected the background of the experiments and said Soochan, “not . . . wait to the end to show the foundation instruments, hands-on practice with the experiments using that the grant has worked.” both inquiry-based and traditional instructional models, and HHMI also emphasizes dissemination, both of the results pedagogical discussions of how to incorporate the experi- of assessments and of successful programs. Networking with ments into the curriculum at different levels. others helps to ensure that useful models are replicated and The program also used high school students selected by mistakes are avoided. Publishing the results of assessments their teachers from the previous year’s classes and trained helps to disseminate results widely. to operate the specialized instruments and equipment. The From 1988 to 2008, HHMI’s grants to undergraduate students then assisted in the classroom during the labs. Par- institutions totaled $767 million, and 22 percent of that ticipating teachers generally used a specific experiment with amount—about $170 million—went to precollege and other three to five classes, with many teachers using it for all of outreach activities. Those grants have served about 85,000 their classes. From 1992 to 1995, teachers conducted more teachers from preschool to high school in programs lasting than 38,000 student experiments. The experiments also were more than two weeks, with many more served in shorter adapted to a wide variety of other settings, ranging from AP programs. The precollege programs are very heterogeneous, classes to other science classes. said Soochan. Most focus on biology, but some focus on A statistical analysis of the responses on student question- chemistry, physics, and other areas of science. They range naires suggested a significant positive change in students’ from 10-week summer research experiences to workshops attitudes toward science and toward the equipment. A that meet periodically during the school year. Among the survey of teaching assistants indicated that their involvement features characterizing successful programs have been increased their interest in majoring in science as under- involving teachers in the early stages of program conception graduates and their interest in a science teaching career. and development, treating teachers as partners, incorporat- Furthermore, survey results strongly suggested that teachers ing educational standards, using master teachers, provid- experienced significant growth in their knowledge of biol- ing continued resources such as undergraduate teaching ogy and chemistry concepts and the use and theory of the assistants and equipment libraries, encouraging networking, instrumentation underlying the experiments. The positive providing subsequent experiences, and including support impact of the program on teacher content knowledge and for evaluation. classroom activities was strongly substantiated by the direct Soochan described two examples. A grant to Emory observations of the program’s outside evaluator. University supported teams of middle and high school sci- ence teachers, graduate students, and undergraduates on a year-long project to create model inquiry-based curriculum WHAT ARE FOUNDATIONS DOING? materials that are aligned with the Georgia and national Given that foundations support a wide variety of educa- standards. From 2003 to 2007, teams that included 48 teach- tion reform efforts, Sandra Laursen and Heather Thiry at ers implemented 166 new units in 150 classes of more than 4,000 students. Evaluation of the program included surveys of teachers, audits of lesson plans, measures of student C. Craney, A. Mazzeo, and K. Lord. 1996. A high school-collegiate performance, reviews of student portfolios, comparison of outreach program in chemistry and biology delivering modern technology in a mobile van. Journal of Chemical Education 73(7):646-650. 

34 STRENGTHENING HIGH SCHOOL CHEMISTRY EDUCATION the University of Colorado at Boulder, with support from Support for K-12 systems, the fourth category, can go the Camille & Henry Dreyfus Foundation, set out to learn to schools, to districts, to partnerships, or for policy devel- more about the outlooks and practices of foundations. Their opment and implementation. Also, a fifth miscellaneous approach was to ask foundations that support activities in category includes activities such as employee volunteerism, secondary chemistry education a series of questions: What special events, and projects such as film or web projects. do you do? What evidence do you have about how it works? The researchers attempted to attach dollar amounts to What do you conclude from the evidence? How does the these activities, but the range of programs and activities made evidence shape your practice? this impossible, especially given that education accounts for First they analyzed the Web sites and available publica- about 25 percent of all philanthropic giving. Nevertheless, tions of 37 foundations identified as key players in science by establishing these categories, the study sought to examine education. Then they conducted surveys and in-depth activities that the foundations deemed important. “Our idea interviews with 16 selected foundations. They divided the was to look at these activities as a way of saying, What do activities supported by foundations into five broad categories people think works?” (Figure 6.1). In the first category—direct support for stu- From the broad analysis of 37 foundations, Laursen and dents—they placed scholarships and competitions. Examples Thiry discovered that corporate foundations tended to sup- include competitions “that have students inventing things port different activities than private foundations. Corpora- or solving problems,” said Laursen, who summarized the tions tended to fund student scholarships and competitions study’s findings at the workshop, or scholarships “for high and small classroom grants to entrepreneurial teachers. They school students to do summer research or to have some kind also tended to target their home communities. Private foun- of extra learning experiences.” dations were more likely to engage with districts, systems, The second category—classroom support—includes or policy. Both supported teacher professional development, programs directed at teachers or individual classrooms, such which they see as a high-leverage strategy. as grants for equipment, programs to develop curricula, or Informal education was popular with both types of foun- professional development for teachers. The third category— dations, partly because it did not directly involve school informal education—includes all activities beyond the K-12 systems. “You don’t have to deal with all that bureaucracy, educational system, such as support for museums, science all those state standards, and all those rules,” said Laursen. centers, summer camps, and after-school programs. “They see K-12 systems as difficult, as too big a ship to turn.” Informal education is also a way to inspire and motivate students and build their interest in science. It is difficult to measure the impacts of these activities, but “I think we all believe and have seen examples in our own lives about how that works.” 70% Laursen and Thiry hoped that, in their interviews with Informal science 60% foundation representatives, they would uncover stores of Students data about the effectiveness of programs that had not been Percent of Foundations (of 37) 50% K-12 classrooms analyzed. This turned out not to be the case. “They are busy. 40% They are on the road. These people . . . are doing a lot of K-12 systems 30% Other good things. [But] that mine of data doesn’t for the most 20% part exist.” On the contrary, the researchers found that fairly few data 10% are collected and that the sources of information are mainly 0% grantees’ reports and site visits. Most of the information is Technology Scholarships Equipment Classroom Teacher PD Competencies Curriculum School-wide District-wide Employee Informal ed Partnerships Events Projects Policy about the populations served and the activities conducted, with uneven internal evaluation and little external evaluation. A B C D E F G H I J O K L M N Most foundations know what happens to whom, but they know little about whether, how, or why it works. However, Figure 6.1  The percentage of foundations engaged in supporting said Laursen, the researchers talked with very insightful New 6.1 secondary science education was highest for informal science and program officers and found very interesting initiatives under lowest for activities focused specifically on students in classrooms. way. SOURCE: Laursen, S., & Thiry, H. (2008, January). What Do We From these interviews, the researchers culled a number Know about What Works? Review of US Foundations’ Programs in of “best practices” in grant making. These practices are Secondary Chemistry Education. (Report to the Camille & Henry “experienced people’s advice, but not necessarily evidence- Dreyfus Foundation) Boulder, CO: University of Colorado at Boul- based advice,” said Laursen. “They have gone out and have der, Ethnography & Evaluation Research. seen things and have watched things and have paid attention 

ACTIVITIES BY NONPROFIT AND FOR-PROFIT ORGANIZATIONS 35 to similarities and differences. They don’t necessarily have for smaller foundations, but these organizations may be able data in hand.” to use common and shared evaluation tools. For example, the In setting directions, foundations should draw on the Noyce Foundation is compiling the surveys, interviews, and research literature, on national reports, and on observed other methods that are publicly available to study the impact trends. They should seek to have an impact through either on students from informal science education experiences. breadth or depth. “They are making strategic choices. Do we Once these instruments have been identified, gaps can be spread our resources over a wider area and go for impact by located and tools can be supplied to grantees for use in evalu- having lots of people participate, or do we go for depth in ating projects. In contrast, the Burroughs Wellcome Fund is a smaller area or a smaller targeted project?” Foundations developing the capacity of its grantees to evaluate their own also are evaluating their own work to set future directions. work. The fund has an evaluation team that leads workshop, The data drive them in directions that they might not have does consultations, and coaches grantees on how to identify considered before. goals, measure progress toward those goals, analyze data, General elements of strong project design include build- and draw broad conclusions across projects. The evaluation ing stakeholder support, beginning with a needs assessment, work is supported by a tax of about 1 percent on each of the using the research literature, involving scientists and engi- grantees. “Across all of their grants, this adds up to enough neers, and addressing sustainability up front. “What happens money to fund this kind of effort.” when the foundation money ends?” Foundation officers were Laursen also cited a tool developed by her colleagues interested in seeing plans that had a longer-term vision of Elaine Seymour and Tim Weston called Student Assessment how to keep programs going once funding is gone. of Their Learning Goals (SALG). It is a publicly accessible Best practices in teacher professional development assessment tool that faculty can use to ask students what they include aligning content with the curricula teachers are gained from a course and what aspects of a course helped using in class, aligning with state and national science them learn. It is online and free, with core questions and standards, strengthening teachers’ content knowledge while optional additions, at http://www.salgsite.org. The instru- linking to pedagogy, incorporating follow-up in profes- ment has about 12,000 users so far who have customized sional development, providing time to reflect and network, their versions. A similar instrument, also available on the and modeling and discussing effective teaching and learn- SALG Web site, is the Undergraduate Research Student Self- ing methods. Evaluating teacher professional development Assessment (URSSA), which is a research-based technique is not straightforward, Laursen observed, partly because the for assessing what students get from doing undergraduate desired effects are far downstream, but evaluation efforts research. are necessary. Chemists need to apply evidence-based methods in their Laursen and Thiry found several intriguing examples education work as well as their science, Laursen concluded. of foundations that were trying to improve their evaluation They need better evidence about what works to shape the practices. Accountability ensures that foundations can learn design and implementation of projects, to guide the choices from the activities they support. “As one foundation officer of projects to fund, and to learn from their own and other’s said, in the end the board is going to look at you and say, mistakes and successes. They need to think about their objec- ‘Well, what happened?’” Sometimes knowledge can be tives and how to measure progress toward those objectives generalized from one program across a range of programs so at the beginning of a project, not at the end. Funders and that general principles can be distilled. Having some sense program developers alike have an interest in sharing pro- of the impact of a program can be motivating for funders cesses and tools for evaluating the outcomes of educational and practitioners and can engage each in further activity. As outreach efforts. Laursen pointed out, other researchers have speculated that In the question-and-answer session, Tom Keller of the the use of good evaluation could multiply the payoffs from National Academies’ Board on Science Education noted foundation resources at least severalfold. that the National Science Foundation has just released a The Bill & Melinda Gates Foundation has established framework for informal science education. It is a good start- an entire evaluation office and has set up metrics for the ing point for anyone interested in evaluating such programs, schools it is supporting. This may not be a realistic strategy he said.