Summary of the Workshop

In his introductory remarks, Louis Gross (University of Tennessee), chair of the workshop planning group, explained what he and the group saw as the different ways to interpret the workshop title, “Integrating Education in Biocomplexity Research.” The group chose that title because of its multiple meanings, recognizing the benefits of approaching the workshop from several viewpoints. One view is that a larger audience would be educated about the science of biocomplexity, another is that biocomplexity researchers themselves would learn about approaches to educational research. Mechanisms for communicating with students and the public about biocomplexity can be enhanced by education research. Gross emphasized the wealth of knowledge that educators have to offer scientists. The field of education has its own research community, and principal investigators (many of whom also consider themselves educators) can tap into that research to learn how people learn about science—for example, the findings of research on learning (see Appendix E).

Throughout the workshop, examples of how research and education might be integrated were highlighted. Seven case studies and several hypothetical scenarios were discussed, including scenarios of how researchers might develop education projects directed toward target audiences, such as postdoctoral researchers, graduate and undergraduate students, K-12 students and educators, students in professional programs (law, medicine, journalism, and so on), policy-makers, nonscience professionals, and people associated with the informal education community (museums, aquariums,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 5
Summary of the Workshop In his introductory remarks, Louis Gross (University of Tennessee), chair of the workshop planning group, explained what he and the group saw as the different ways to interpret the workshop title, “Integrating Education in Biocomplexity Research.” The group chose that title because of its multiple meanings, recognizing the benefits of approaching the workshop from several viewpoints. One view is that a larger audience would be educated about the science of biocomplexity, another is that biocomplexity researchers themselves would learn about approaches to educational research. Mechanisms for communicating with students and the public about biocomplexity can be enhanced by education research. Gross emphasized the wealth of knowledge that educators have to offer scientists. The field of education has its own research community, and principal investigators (many of whom also consider themselves educators) can tap into that research to learn how people learn about science—for example, the findings of research on learning (see Appendix E). Throughout the workshop, examples of how research and education might be integrated were highlighted. Seven case studies and several hypothetical scenarios were discussed, including scenarios of how researchers might develop education projects directed toward target audiences, such as postdoctoral researchers, graduate and undergraduate students, K-12 students and educators, students in professional programs (law, medicine, journalism, and so on), policy-makers, nonscience professionals, and people associated with the informal education community (museums, aquariums,

OCR for page 5
Outline of Ideas and Themes Generated During the Workshop Collaborating with others with complementary talents is potentially quite valuable, but requires mutual benefits that exceed costs or the benefits of working alone, and requires careful facilitation, logistics and modeling. Researchers can benefit from the knowledge educators have to offer (e.g., the American Association for the Advancement of Science education materials, education researchers). If researchers are going to contribute to teaching, they need to understand teachers’ constraints, use mutually respectful language, share work equitably, etc. Scientists and those they might collaborate with through education share many things in common. Teachers and scientists share a passion for learning. They both must deal with a public that sometimes follows them with blind faith, and at other times questions their motives. Journalists and scientists share curiosity laced with skepticism and need to see evidence, a belief that the truth exists and that it is imperative to find and communicate it. Education researchers, assessment specialists, and scientists share a focus on questions, hypotheses, careful methods, peer review, etc. and so on). According to NSF guidelines, researchers need not limit themselves to universities or even educational institutions in complying with Criterion 2, but can reach out to all parts of society—science affects everyone. Several presenters of case studies and some planning group members offered suggestions for integrating education and research drawn from their specific experiences. Their suggestions were based on extensive experience with education projects. The projects themselves are described here as case studies, and several are treated in Appendix D, which presents information on evaluation and assessment. Most of the case studies describe projects targeted to particular audiences (such as undergraduates or museum visi

OCR for page 5
As a corollary to integrating education into research, we can work to integrate research into the education work we do. This was highlighted by the comments of Keesing, Levitan, and Ebert-May. Involving nonscientists in research is a means of providing valuable professional development opportunities, e.g., for teachers (Carvellas) and journalists (Kastens), as well as for future scientists (Manduca). Clear guidelines exist for designing such research experiences, at least for young scientists (Manduca) and teachers (Carvellas). Undergraduate curriculum reform, such as the University of Michigan example, might be one of the most logical ways of linking research and education but numerous barriers exist to giving such efforts the time, collaboration, and attention required. Indeed, one would think that the undergraduate arena should be the first place to look for ways of infusing the latest research into teaching, creating models for application in other arenas. There is a useful multiplier effect from working with the teachers of teachers or journalists (e.g., Kastens). It is imperative to have the same high standard of excellence for the education component as for the research component. Allowing education work to be voluntary for researchers was seen as essential for achieving this goal, at least at one of the institutions highlighted in the workshop summary (Woods Hole Oceanographic Institution). Assessment and evaluation are imperative in considering the effectiveness of an educational component of a project. tors), but many of the comments will be helpful when applied to other groups. The box above provides an outline of important ideas and themes explored during the workshop. PRINCIPLES OF RESEARCH APPLIED TO EDUCATION PROJECTS Herb Levitan, of the NSF Division of Undergraduate Education, asked workshop attendees to think of education projects with a perspective that parallels that of scientific research. He began by asking the attendees to indicate what they believe are the core principles of research. Attendees

OCR for page 5
discussed their ideas in small groups and then offered their answers to the audience at large. Themes of various principles among the attendees’ responses included the joy of discovery, working with others, breaking down disciplinary walls, integrity and rigor of research, and sharing the scientific experience with students. Levitan proposed, in line with what the attendees had identified as essential principles of research, that there are four principles that guide research, and that these principles should also be applied to projects that integrate education and research. He proposed that these efforts should Be original and break new ground. The best research is that which builds on the efforts of others, explores unknown territory, and risks failure. Provide opportunities for professional development. Research provides opportunities for personal growth for all who are actively involved. More-experienced researchers may act as mentors or trainers of those with less experience—the “learners.” Learners gain confidence and stature among peers as they gain proficiency in a field. Provide opportunities for collaboration and cooperation. Because the most interesting and important problems and questions are usually complex and multidisciplinary, researchers with diverse and complementary perspectives and experiences often collaborate. Provide opportunities for work that results in a product. The expectation of all research is that the outcomes will be communicated and available to an audience beyond those immediately involved in the research activity. That can occur via peer-reviewed publication or via patents or commercial products. The value of the research will then be measured by the impact of its product—how widely cited or otherwise used it is. GETTING STARTED FORMING COLLABORATIONS Cathryn Manduca, of Carleton College, gave advice based on her experiences with the Keck Geology Consortium. “While collaboration is regarded as a valuable experience, it is also a costly one. It takes time. It takes money. It takes a strong base of communication. To be worthwhile, a collaboration should take place only when working together as a group is better than working alone as individuals.” In her keynote address to workshop attendees, Patricia Morse, of the University of Washington, echoed Manduca’s advice that collaborations

OCR for page 5
should be formed only when they will yield more to the participants than would acting alone and noted that the needs of all parties in the collaboration must be considered. “Both sides have expectations that need to be thoroughly considered.” Morse also offered guiding principles to consider when forming a collaboration. In order to achieve quality outcomes, she advised that collaborators should “be very careful to choose high-quality participants with strong backgrounds.” According to Morse, one way to foster collaborations across expertise lines would be to “include experts from the field of education in meetings geared toward principal investigators and connect the relevant principal investigators with each other. For example, someone working with butterflies could approach NSF to get connected with other researchers in a specific field.” Morse concluded, “Successful collaborations should be celebrated, and participants in a collaboration should be given time to reflect on their experiences and possibly work with their project mentors to plan their next steps.” Morse also cautioned against harboring common misconceptions regarding education and research. She noted three misconceptions in particular, first that teaching is intuitive, or that instructors often assume that the way they learned is the way to learn. This attitude ignores the wealth of research in cognitive sciences. Secondly, she noted the misconception that undergraduates can’t do research, despite the fact that some scientists’ best work is done at a very early age. The third misconception she noted was that scientists can’t understand “education-ese,” or that they don’t have time to learn about what education experts have to offer. She suggested that avoiding these misconceptions and instead looking toward solutions would aid collaborators in their efforts to integrate research and education. Susan Singer, of Carleton College, suggested that a collaboration should be considered as something that does not necessarily revolve around the principal investigator. “Those who are interested in collaborations should consider research projects with both undergraduate science students and education students, that is, being partners in the education process and creating a culture that encourages an exchange of ideas about teaching that parallels the culture of exchange of ideas dealing with our own research. This type of exchange deals with professional development, so education and research are fully integrated.” John Farrington, of the Woods Hole Oceanographic Institution, offered ideas for facilitating relationships in a collaboration based on his experiences at Woods Hole. One such approach that is now under way is

OCR for page 5
what Farrington called a reverse workshop, in which teachers educate scientists and those involved in informal education. To design such a workshop, one could have master teachers, informal educators, or cognitive psychologists teach scientists about curriculum standards, expectations, or advances in research on how people learn. Additionally, senior faculty or research scholars who have some experience in collaborative efforts between scientists and educators can act as mentors in such programs. Farrington emphasized the need for openness and patience in forming a collaboration. “Keep diversity needs in mind throughout the process, programs, and activities. Maintain patience and persistence leavened with appropriately aggressive goals and approaches.” Angelo Collins, of the Knowles Science Teaching Foundation, noted the importance of logistics in forming a collaboration. Logistics can be one of the most serious problems: time, place or distance, and expense can cause unnecessary hurdles in a project. Collins explained that education and science have different cultures and that part of what a school-science partnership attempts is to create a new culture that is a blend of the two. “It is a point to keep in mind that scientists have more resources and status than teachers. But even more pressing in the age of standards testing is the level of accountability that teachers face. An analogous situation for principal investigators might be if the local newspaper published on the front page, not their research grant or publications, but the number of citations of their publications, something over which they have no control—and if, on the basis of those data, it were decided whether they would get salary increases, stay in their departments, or keep their jobs at all. That kind of accountability is what teachers are facing, and it would be smart to keep this in mind in forming a partnership.” Collins suggested that teachers and scientists working together must pay attention to who talks and who listens and who is doing the routine work. To show respect for one another, it is important to have an equitable distribution of both ideas and work assignments. One workshop participant likened such understanding of cultural differences to the same kind of understanding that would be needed at a stakeholders’ meeting—one can’t assume that the same tacit knowledge is shared by all. Collins encouraged celebration among collaborators—they should look on informal social gatherings as necessary for forming bonds that facilitate working together. Patricia Morse suggested that collaborators share leadership duties and

OCR for page 5
responsibilities. She mentioned her experience that anyone given the appropriate resources can function as a leader if there are shared values among the members of a community (or collaboration). This type of behavior is very different from the common hierarchical structure of the university. CONSIDERING A TARGET AUDIENCE In considering how to engage members of the public in an understanding of science, Kastens suggested that “researchers ask themselves why they think that the public should care about their research. Questions can be asked of people in specific situations to identify the kinds of information that will be important to them. Why would a researcher want various kinds of people to know about his or her work, and what details would they want him or him to know? A voter? A parent shopping for a family’s groceries? Property developers? An elderly person newly diagnosed with cancer? A Senate staffer? Any of those could be part of a target audience, and education projects aimed at them would be different from one another.” Manduca pointed out that the target audience of a project must also be considered in the dissemination of the project results. “Results should be communicated with the intended audience in mind and how that audience might receive the results—in written form, via the Internet, or by some other means.” Students in other professional programs would benefit from exposure to science, and providing in-depth experiences with science before graduation can provide a useful background to students going into teaching, law, medicine, or even the clergy. Teachers are a relatively well-understood constituency for integrating research and education, but other professions would be worthy of attention from principal investigators. An attorney with research experience in environmental science will make a better environmental lawyer. A physician with knowledge of environmental impacts on health will view his or her practice of medicine more broadly. Clergy with exposure to biomedical science and research will make better-informed spiritual leaders. As one workshop participant noted, elected officials often have a frighteningly limited understanding of controversial issues involving scientific knowledge. An increased exposure to science would result in better-informed public officials to the benefit of their constituencies.

OCR for page 5
Case Study 1 Cathryn Manduca, Carleton College—Designing Research Experiences for Undergraduates The Keck Geology Consortium involves the coordination of students and faculty from the 12 member institutions in a four-week summer research experience. The W.M. Keck Foundation, the National Science Foundation, the Exxon Educational Foundation, the American Association of Petroleum Geologists Foundation, and 12 liberal-arts member institutions fund the consortium. The consortium is a group of small geoscience departments in predominantly undergraduate, liberal-arts institutions that cooperate to improve geoscience education through research. The primary activity of the consortium is to sponsor projects involving faculty and undergraduate students in a collaborative effort to solve geoscience problems. For more information about the consortium, see http://keck.carleton.edu/. The overall structure of the consortium involves matching three faculty members with nine students. Over 4 weeks in the summer, students work together in groups on several projects in a variety of subjects and design individual projects for themselves. By the end of the summer experience, students are expected to have the necessary data from their projects to look at a scientific question in depth. Their results are discussed with an on-campus mentor from the Consortium who works collaboratively with faculty members from other institutions. This format allows students to experience a WHAT CONSTITUTES AN EFFECTIVE UNDERGRADUATE RESEARCH PROJECT? In designing an educational component and integrating it into an undergraduate research initiative, one of the first steps is to identify the elements needed for a successful project. Manduca offered detailed advice on forming and implementing an education or outreach project on the basis of her experience with the Keck Geology Consortium (see Case Study 1). According to Manduca, the first step in designing an undergraduate research experience must be clear delineation of the goals of the program. Once the goals are understood and embraced, decisions about how to design the educational experience will flow naturally from the goal. Manduca outlined the Keck Geology Consortium’s two main sets of goals (one for student education, and the other for faculty professional development).

OCR for page 5
breadth of topics and a depth of knowledge in a particular subject. At the end of the academic year, students present their results at an annual symposium, which may be held at their own academic institution or some other site. “Within the overall framework, faculty members are free to design group projects with any structure they think would best serve their interests,” Manduca noted. “Students who have gone through the consortium experience have given witness to its impact on them. They have reported gaining an understanding of scientific inquiry; in-depth, integrated, self-directed learning in their field of interest; technical, interpersonal, and communication skills valued by graduate schools and employers; and a test of their career interests. As one student reported, ‘This experience is unparalleled by anything else I have ever done.’” The impact on faculty can also be tremendous. Faculty members report gaining resources and ideas for teaching, increased content knowledge, and new research interests and techniques.     Note: The Keck Geology Consortium is funded by the W.M. Keck Foundation, NSF, Exxon Educational Foundation, American Association of Petroleum Geologists Foundation, and 12 liberal-arts member institutions (Amherst College, Beloit College, Carleton College, Colorado College, Franklin & Marshall College, Pomona College, Smith College, Trinity University, Washington and Lee University, Whitman College, Williams College, and the College of Wooster). Student Education: Help students to develop intellectual, technical, and personal skills. The research experience should enhance students’ intellectual growth and give them technical and personal skills that they would not have developed otherwise. Encourage and test career interests. Give students a variety of opportunities to experience work in a field so that they can determine whether they want to pursue further study or a career in that field. Faculty Professional Development: Encourage interactions. Interactions could be among faculty from different institutions. This can be especially helpful for researchers working

OCR for page 5
in small departments in small institutions whose opportunities for such interactions might otherwise be few. Enhance research. This usually means ultimately generating results that are published. In defining an expanded set of goals for a research experience, one should consider the needs of all stakeholders—students, faculty, the institution, and so on. Institutional goals may include building connections to industry or other universities or otherwise gaining exposure that might not be possible without a collaborative relationship. Constraints will always need to be considered in designing a project, just as there are constraints in designing an experiment to test a particular hypothesis. Manduca suggested that in both, one must first identify the goals of the project. The consortium faculty wanted their students to Do science—from project design to public presentation of results. Study a problem in detail. Learn specific research techniques. Develop and experience the empowerment that comes from collaboration, writing, and speaking skills. Gain confidence, both personally and as researchers. Test career interests. Manduca reported that when students in the consortium were asked what their goals were, they named goals similar to those laid out by the faculty described above. Undergraduate students wanted to Do science in a particular subdiscipline (to test career and intellectual interests). Apply classroom learning to work on a real problem. Gain job skills or graduate-school credentials. Work in groups. Gain confidence. Manduca defined four steps of designing student research experiences, each with its own set of issues or concerns as follows.

OCR for page 5
1. Define the Problem The issues involved in this step constitute an overview of the entire research plan: ensuring student ownership of a problem; finding a meaningful and well-defined problem; finding a project that can be done within the constraints of time, equipment, logistics, and funding; aligning the problem with laboratory priorities and research plans; and discerning the level of knowledge and preparation that students bring to the research experience. Manduca and various participants identified strategies to help students to address those issues: Guiding them through the research literature and mentoring them in developing a project that suits their interests. Introducing a problem and then helping them to choose from a list of possible projects. Allowing the whole group to collaborate in choosing projects. Assigning a project to a student according to the student’s knowledge and expertise level. 2. Develop the Research Plan Manduca put forth several questions for research students to consider at this step in the development of their project: Will planned experiments respond to the hypothesis? Is the project feasible with respect to time, equipment, and personnel costs? Can the students learn the necessary techniques and interpret the results? Does their plan address goals established by faculty and students? Does the plan maximize the experience for all of the students? One strategy for developing the students’ research plan includes the proposal writing and review cycle (with students acting as peer reviewers for each other). In some cases, it may work best for students to develop plans that incorporate faculty-defined standard protocols for data collection and analysis. 3. Collect and Interpret Data According to Manduca, issues to consider with respect to students’ collecting and interpreting data include the identification of meaningless

OCR for page 5
Case Study 6 Museums Cary Sneider, Boston Museum of Science— “Nowcasting” Project The field of weather, weather forecasting, or meteorology is one to which anyone can relate. Most people are curious about the local weather on any given day, and many rely heavily on television meteorologists to give them information. “So,” Sneider explains, “along with a group of meteorologists and atmospheric scientists, the museum designed a program called ‘nowcasting.’ Nowcasting means making a prediction about how the weather will be in a particular location in a few hours, or at most, a day.” Many people will look at the Doppler radar, look at the last few hours of activity in their particular area, see how rain is progressing, and make a prediction as to whether they’ll see rain at their own homes. Of course, there are only a limited number of sources of Doppler radar data, and they vary widely at times, making it interesting to contrast different sources but also making it difficult to rely on one source for an accurate prediction of whether it will rain in one specific neighborhood. Weather is more complicated than Doppler radar can indicate, especially to atmospheric scientists who develop the processes and the instrumentation to make predictions about the weather. “The idea is not to have visitors walk away from an exhibit saying, ‘Gee, all I have to do is check the Doppler radar, and that’s what these meteorologists get paid for’”, said Sneider. There is a component of the project that, in more detail, explains mathematical models and To facilitate the relationship between researchers and schoolteachers, Carvellas recommended that “scientists work with teachers to see what kinds of information will fit well within the curriculum and not assume that just anything provided to teachers will be useful or possible to integrate into the classroom.” Many teachers have had unpleasant experiences with scientists who initially offer help, but do not follow through and work to sustain a long-term partnership. Issues related to partnership formation are discussed in the summary section “Getting Started Forming Collaborations.”

OCR for page 5
the deeper philosophic issues that meteorologists and atmospheric scientists address, such as whether it is even possible to make a precise weather prediction. The goal of the exhibit is to help people to learn about the nature and process of science through weather. Meteorologists collect data from various sources, draw patterns, form hypotheses, and, as the weather system moves, test their hypotheses. Many also run mathematical models using computers to find out what might happen. Like other scientists, meteorologists will make statistical arguments to form their predictions. This method of doing science is explained in the context of a phenomenon that affects everyone everyday. There is a room in the Museum of Science that is three stories tall and has two large balls on two columns—a van der Graaf generator. This generator produces sparks that are 20 feet long and is used to demonstrate the production of lightning. With additional funding, there will be a feeling of a whole storm in the room during the presentations. Presentations like this get people’s attention and draw them in to specific exhibits around the room that illustrate various aspects of nowcasting. In addition to the exhibits, there are a number of interlocking programs. There are teacher workshops and a Web site where people can do nowcasting. A number of area schools participate in the WeatherNet project. Students in these schools collect data and share them via the Internet. Some of the students will be in the exhibit areas on the weekends and during the summer to interpret their data to the public. (See Appendix D for an evaluation of this project.) COMMUNITY OUTREACH—EDUCATION PROJECTS OUTSIDE THE EDUCATIONAL SYSTEM Cary Sneider, of the Museum of Science in Boston, Massachusetts (see Case Study 6) spoke of the advantages of having scientists interact with the public through informal education venues. The public can benefit from the expertise of the scientists and their knowledge of both scientific history and modern applications. However the scientific information must be presented in an accessible format. “The informal education arena—science centers, zoos, arboreta, and so on—offer a diverse audience of visitors. Many are voters—parents or grandparents bringing children to the museum. Classes

OCR for page 5
of schoolchildren also come through with teachers and chaperones, providing a challenging opportunity to reach a wide variety of people who can take the knowledge they gained from an exhibit into various aspects of society.” On a much larger scale than individual principal investigators can approach, public outreach programs such as those outlined by Cary Sneider and Kim Kastens, of Columbia University (see Case Study 7), can provide ideas to scientists for education projects. WORKING WITH JOURNALISTS AND OTHER GROUPS THAT INFLUENCE THE PUBLIC Journalists communicate with adults, and adults make decisions. Compared with K-12 or even undergraduate audiences, journalists (or attorneys, physicians, or clergy members) deal in an immediate way with adults who make decisions about how society is run. They reach adults by warning them that a decision has to be made or a vote has to be cast or money has to be spent. Scientists and journalists share many values. Both are strongly driven by curiosity, but a curiosity that is laced with skepticism and a “show-me” attitude. Kastens believes that both are driven by a sense that searching for the truth is important—that the truth exists and it is imperative to find it and communicate it to people. HOW TO WORK WITH JOURNALISTS One possibility is to work with journalists and journalism educators to expand the public’s understanding of science. Another strategy is to think about the potential value of a research experience for people who are on a preprofessional track that will not lead them into science. Researchers could approach the journalism schools at their own universities about developing a course in science writing with a journalism professor or even acting as guest professors in this type of course. The Society of Environmental Journalists and the National Association of Science Writers have panels and lists of experts through whom researchers can gain exposure to journalism professionals who are interested in science. Researchers also could meet with the science or environmental writers of their newspapers and take it upon themselves to help better understand and appreciate the nature of science and the kinds of issues and problems that science can and cannot address.

OCR for page 5
Case Study 7 Kim Kastens, Columbia University, Lamont-Doherty Earth Observatory Environmental Journalism Program Kim Kastens oversees a 2-year environmental journalism program at Columbia University. Students who have an undergraduate background in science and a demonstrated writing ability are recruited to the program. Students spend 1 year on a master’s-level research project and investigate how science is communicated to the public. In their second year, they complete a journalism master’s program so that they graduate with two master’s degrees—one in journalism and one in environmental and earth sciences. The goal of the program is to prepare journalists who have both the scientific background and the communications skills to inform the public about insights, discoveries, and the environment in ways that are interesting and accurate. Journalists sometimes find it strange to undertake a scientific research project, but it is consistent with the scientific community’s emphasis on integrating research and education. It also gets future journalists to think about processes of science and how the scientific community works. Through this process, students come to understand better the process of floundering around at the boundary between the known and the unknown. By learning skills and techniques in one particular subdiscipline, they begin to understand that research is both a craft and a process. They experience the thrills and challenges of generating original data and thereby come to appreciate the ambiguities and complexities in a field of study. Kastens noted that “although it has not been difficult to find researchers willing to mentor these students—researchers do want to work with young minds if the structure is right, considering their own constraints—a challenge has come when researchers try to lure students away from the program and into the laboratory for a PhD track. Researchers must respect the fact that there are other legitimate career goals for which an exposure to science is beneficial—that people don’t need to end up as research scientists to make it worth while.” The program works with the National Association of Black Journalists, the Native American Journalists Association, and the National Hispanic Journalists Association to recruit minority-group members into this profession. These associations provide funds for fellowships, allowing minority-group journalists to attend meetings of the Society of Environmental Journalists. That is important because the coverage of science and the environment is often poor in media that are targeted to minority populations. (See Appendix D for an evaluation of this project.)

OCR for page 5
ASSESSING THE PROGRESS AND EFFICACY OF PROJECTS Diane Ebert-May, of Michigan State University, spoke to the workshop audience about assessment and evaluation of projects. She began by challenging the audience to think of assessment in terms of what kinds of evidence they would find acceptable for measuring progress and outcomes in the research components of their projects. “Assessment is data collection with a purpose,” she said. “In research projects, principal investigators collect data with the purpose of answering a question or hypothesis. In education projects, data are collected to answer questions about student learning and instruction. If education projects are considered in the same way as research projects, assessment must be done at appropriate intervals throughout the project.” When making decisions about assessing learning, she explained, one should consider how a similar assessment would be done for scientific work. Ebert-May then turned to what she defined as the parallels of assessment in research and education: Observations and questions are asked that are meaningful, interesting, and fundable. Questions form the basis of assessment. Data collected are aligned with a question about a problem. When researchers use the wrong tool or the wrong process, they end up with meaningless data. Instruments and techniques are used that are accepted in the field and that stand up to peer review. Results are explained in the context of a question. Ideas are peer reviewed for merit, publication, dissemination, and funding. Assessment of learning poses a challenge, but it is possible within the context of the science disciplines and with knowledge from the social sciences. Collins spoke about evaluation and assessment in her presentation, “Model for Successful Partnerships with K-12 Educators (Science for Early Adolescence Teachers)” (Science FEAT: http://www.serve.org/Eisenhower/FEAT.html). Science FEAT was a 3-year teacher-enhancement program for middle-school teachers of science based at Florida State University and supported by NSF. Sixty-five middle-school teachers in northern Florida and southern Georgia completed the program. Science FEAT received the 1995 In

OCR for page 5
novation in Teaching Science Teachers Award from the Association for the Education of Teachers in Science and was recognized by the Florida Postsecondary Education Planning Commission as an exemplary initiative program in mathematics-, science-, and technology-related education. Collins and her colleague at Science FEAT, Sam Spiegel, carried out a formative assessment of the program with data analysis and teacher interviews. They developed a model to evaluate science-school partnerships that was based on three main considerations: Cognitive aspects of the partnership. Why is the program worth the teachers’ time? What makes this knowledge worth while? How does it align with the purpose of schooling? How does it align with state or national standards? Variable expertise. When multiple communities come together to talk about improving opportunities for students, they should avoid jargon. One way to get around jargon is to draw out a concept map to identify where there might be gaps in understanding. Concept maps are two-dimensional, hierarchic representations of concepts and of relationships between concepts that model the structure of knowledge possessed by a learner or expert. The theory of learning that underlies concept mapping recognizes that all meaningful learning builds on the learner’s existing relevant knowledge and the quality of its organization.1 How people learn. The National Academies 2000 report How People Learn (http://www.nap.edu/catalog/9853.html?se_side) elucidates aspects of learning. Students come to experiences with prior knowledge. Humans are viewed as goal-directed agents who actively seek information. They come to formal education with a range of prior knowledge, skills, beliefs, and concepts that significantly influence what they notice about the environment and how they organize and interpret it. This, in turn, affects their abilities to remember, reason, solve problems, and acquire new knowledge. Valuable learning experiences, by definition, are meaningful. A 1   Free software that aids in the construction of concept maps is available at www.cmap.coginst.uwf.edu. (Source: Learning and Understanding, National Research Council, 2002.)

OCR for page 5
corollary is that hands-on or concrete experiences precede learning in the abstract. Understanding implies a rich and useful network of knowledge. Facts alone, although necessary, are not sufficient for full understanding of a subject. Students must know the “why” behind the facts to truly understand. Learning communities are venues to try out new ideas. Although students must have the intellectual, physical, and practical tools to accomplish their assigned tasks before working in a group can be productive, a group is an opportunity to try out ideas. Does my idea help me describe, explain, or predict the phenomenon with which I am dealing? Reflection is necessary for analysis. If active learning is what we want for students, they must have time to reflect and analyze information that is presented to them. Evaluation is an important part of any research or educational project. In addition to the work of the students, the program itself should be assessed. Some ways to do this are formative evaluations (evaluating programs while they are forming or happening) for program elements (based on opinions of students and faculty), tracking papers and talks (how many, where, and so on), and gathering statistics on students regarding what they did or did not gain from the program, and whether they reached their own goals. These kinds of evaluation may become more and more important. Research experiences are expensive, so researchers will have to be able to demonstrate the value of specific experiences if they are to continue. PUTTING IT ALL TOGETHER: AN OVERVIEW OF WHY EDUCATION PROPOSALS ARE UNIQUE A brief presentation by David Mogk, of Montana State University, titled “What’s Different About Education Proposals?” was based on A Guide for Proposal Writing prepared by the NSF Directorate for Education and Human Resources (http://www.nsf.gov/pubs/1998/nsf9891/nsf9891.htm). Mogk emphasized that any good proposal begins with a clear idea of goals and objectives and a sense of why the proposed project will be a substantial improvement over current practice. “Proposals should be innovative within their contexts, describe resources that will be needed, refer to prior work, and, where possible, present evidence of preliminary work by the principal

OCR for page 5
investigators. Education proposals also should address goals that are specific to education and human-resource development. Target audiences need to be clearly identified, and collaborations and coalitions necessary to complete the project successfully (e.g., between scientists, science educators, and developers of instructional materials) should be described in detail. Prospective applicants should seek advice from the program officer or access the abstracts of recently funded projects and contact their principal investigators.” Review of proposals at NSF are considered according to two criteria: intellectual merit and broader impacts. Questions about the intellectual merit of an education proposal might include these: Does the project have potential for improving student learning of important principles of science, mathematics, engineering, or technology?” Is the project informed by research in teaching and learning, current pedagogical issues, what others have done, and relevant literature? Does the project design consider the background, preparation, and experience of the target audience? Does the project have the potential to provide fundamental improvements in teaching and learning through effective uses of technology? Is the project led by and supported by the involvement of capable faculty (and where appropriate, practicing scientists, mathematicians, engineers, technicians, teachers, and student assistants), who have recent and relevant experience in education, in research, or in the workplace? Is the project supported by adequate facilities and resources, and by an institutional and departmental commitment? More information on implementation of the NSF broader-impacts review criterion can be found at http://www.nsf.gov/od/opp/opp_advisory/oaccrit2.htm. Mogk offered the following examples of how this criterion could be applied to education proposals (NSF 98-91): To what extent will the results of the project contribute to the knowledge base of activities that enhance student learning? Are the proposed course, curriculum, faculty or teacher professional development, experiential learning, or laboratory activities integrated into the institution’s academic program?

OCR for page 5
Are the results of the project likely to be useful at similar institutions? What is the potential for the project to produce widely used products? Does the project address the current and future needs of industry for technicians? Will the project result in solid content and pedagogical preparation of faculty and teachers? Does the project effectively address one or more of the following objectives: ensure the highest quality education for those students planning to pursue STEM [science, technology, engineering, and mathematics] careers? increase the participation of women, underrepresented minorities, and persons with disabilities? provide a foundation for scientific, technological, and workplace literacy? Are plans for evaluation of the project appropriate and adequate for the project’s size and scope? With respect to the final point on evaluation, Mogk noted that, although evaluation and dissemination plans are essential for education proposals, program officers report that they are often the weakest parts of education proposals. Evaluation plans will provide information as the project is developing and determine whether the overall project has met the investigator’s scientific and pedagogic expectations. Dissemination is at the core of all education projects, and it is essential that information about the success and content of a project be communicated to other scientists and educators. Researchers and educators alike should anticipate and plan for changes in current educational venues. As Farrington noted, “future learning environments are unknown, but we must anticipate the need for multimedia tools and new formats for the next-generation equivalent of the great textbooks.” John Jungck added “Researchers should look to be capturing revolutions in science education. It has been said that science education in the 21st century will have to be integrative, multivariate, multi-level, and multidimensional.”

OCR for page 5
As biocomplexity researchers or others grapple with the new challenges of incorporating educational components into their research, they can look to the advice of the workshop presenters. Overall, as framed by Levitan, if researchers would consider the development of education projects in the same ways they develop research projects, they could more easily identify and reach their goals. In a presentation of his summative thoughts on the workshop, John Jungck put forth several questions to consider when striving to integrate research and education, “Has background work been done? Has the education research that is relevant to a project been considered; are there related projects that NSF, NIH (National Institutes of Health) or USDA (U.S. Department of Agriculture) funded? Have the available resources in terms of curricular materials, laboratories, classroom exercises, and software, been tapped? What do the students or audience expect? Many students are adults; they are taxpayers; they are putting a great deal of effort into their education—we don’t want to waste their time. Is there enough time in the project’s schedule for them to accomplish their goals? The definition of ‘colleague’ should be expanded to include the students and/or the audience. Researchers should respect the recipients of their knowledge, what these recipients know already, and the diversity of their backgrounds and talents.” Levitan’s ideas for collaboration were echoed by many at the workshop, whose suggestions covered a wide array of potential collaborative sources. From the interdisciplinary nature of biocomplexity to the interactions involving scientists with undergraduates, elementary students, K-12 teachers, lawmakers, journalists, or others, workshop presenters continually pointed to the benefits of establishing, fostering, and maintaining relationships with other scientists who are committed to improving education and with those who have specific educational or related expertise. The workshop planning group aimed to provide attendees and those who read this summary with tools for integrating education into research, and this summary is structured with that goal in mind. As Lou Gross described the intent of the workshop and of this summary, he indicated his thoughts on the role of researchers in the scientific community and society at large: they are uniquely positioned to learn about their world and how it works and to share this knowledge with society for the good of all.

OCR for page 5
This page in the original is blank.