3
Science Literacy: Implications for Assessment

Every assessment is designed to measure a construct. Science achievement, Newtonian mechanics, and understanding inquiry are examples of constructs that might be measured by a science assessment. A necessary first step in the development of any assessment is defining the construct that it is intended to measure. In this chapter we discuss the construct of science literacy, which many organizations—the National Research Council (1990, 1996), the National Science Teachers Association (1992), the American Association for the Advancement of Science (1989, 1993), and the National Science Board (1983)—have identified as a goal for K–12 science education. Given this goal, K–12 science assessment should be designed to measure how well and to what degree students are gaining the knowledge, understanding, and skills that are necessary for science literacy.

This chapter discusses three elements of science literacy that are widely represented in state science standards, some of the challenges they pose for assessment design, and ways that research on learning might help states in addressing those challenges.

SCIENCE LITERACY

While the definitions of science literacy that have been proposed by professional societies and others vary in their specifics, three elements are commonly found in most state science standards:

  • knowledge of science content,

  • understanding science as a way of knowing, and

  • understanding and conducting scientific inquiry.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 38
Systems for State Science Assessment 3 Science Literacy: Implications for Assessment Every assessment is designed to measure a construct. Science achievement, Newtonian mechanics, and understanding inquiry are examples of constructs that might be measured by a science assessment. A necessary first step in the development of any assessment is defining the construct that it is intended to measure. In this chapter we discuss the construct of science literacy, which many organizations—the National Research Council (1990, 1996), the National Science Teachers Association (1992), the American Association for the Advancement of Science (1989, 1993), and the National Science Board (1983)—have identified as a goal for K–12 science education. Given this goal, K–12 science assessment should be designed to measure how well and to what degree students are gaining the knowledge, understanding, and skills that are necessary for science literacy. This chapter discusses three elements of science literacy that are widely represented in state science standards, some of the challenges they pose for assessment design, and ways that research on learning might help states in addressing those challenges. SCIENCE LITERACY While the definitions of science literacy that have been proposed by professional societies and others vary in their specifics, three elements are commonly found in most state science standards: knowledge of science content, understanding science as a way of knowing, and understanding and conducting scientific inquiry.

OCR for page 38
Systems for State Science Assessment Other aspects of science literacy are also important, but they are not included in this discussion because they are not often mentioned in state science standards or assessments. These include, among other things, the history of science, scientific habits of mind, science in social and personal perspectives, and the nature of the scientific enterprise. Knowledge of Science Content A strong foundation of science content knowledge is a necessary component of the ability to think scientifically. The ability to plan a task, to notice patterns, to generate reasonable arguments and explanations, and to draw analogies to other problems—all key elements of science literacy—are dependent on factual knowledge (National Research Council, 1999a). A review of both state and national standards and benchmarks calls attention to the considerable breadth of content knowledge in the natural sciences that K– 12 students are expected to attain. For example, the National Science Education Standards (National Research Council, 1996) includes eight dimensions of science content: Inquiry, Physical Science, Biological Science, Earth and Space Science, Unifying Concepts and Processes, Science and Technology, Science in Social and Personal Perspectives, and History and Nature of Science. The authors of the NSES indicate that “the standards are a complete set of outcomes for students … [and that] the implementation of these standards cannot be successful if only a subset of the content standards is used (such as implementing only the subject matter standards for physical, life, and earth science)” (p.103). The framework for organizing curriculum put forth in the Benchmarks for Science Literacy (American Association for the Advancement of Science (AAAS), 1993) describes 12 topical areas: Nature of Science, Nature of Mathematics, Nature of Technology, The Physical Setting, The Living Environment, The Human Organism, Human Society, The Designed World, The Mathematical World, Historical Perspectives, Common Themes, and Habits of Mind. The authors used five major criteria in determining what should be included as science content in their recommendations. These are utility, social responsibility, intrinsic value of the knowledge, philosophical value, and childhood enrichment (AAAS, 1989). Although these documents include a considerable body of content knowledge, they also emphasize that students are expected to understand science principles and be able to apply their science knowledge, not just absorb it. To do this, students cannot learn science as a series of facts, formulas, and procedures disconnected from any context. Organizing Knowledge Scientific knowledge has been characterized as hierarchical and highly organized, with many connections and interrelationships among ideas. Scientists do

OCR for page 38
Systems for State Science Assessment not just mentally store long lists of facts, procedures, formulas, or even connections; rather, they have a mental map of the major concepts within a discipline that guides the way new information is used and assessed. For example, when asked how they would solve various problems, professional physicists used major principles of physics, such as Newton’s laws, to classify them and devise solutions. Individuals with less expertise used superficial features, such as isolated memories related to inclined planes or individual formulas, to classify the problems and consider ways to respond (Larkin 1981; Chi, Feltovich, and Glaser, 1981; Chi, Glaser, and Rees, 1982). As Bransford, Brown, and Cocking (NRC, 2000b) have observed, knowing is less the accumulation of facts than the capacity to integrate knowledge, skills, and procedures in responding to new situations and addressing new tasks. Thus, it is important for students to develop a structure for organizing what they learn so that it is accessible when it is needed. One way to help them develop this structure is to organize science instruction in much the same way that expert scientists organize their knowledge—around the organizing principles, or big ideas, of the discipline. Big ideas are central to a scientific discipline and have broad explanatory scope. They are the source of coherence among the various concepts, theories, principles and explanatory schemes within a discipline. They also provide insight into the development of the field, and provide links between disciplines. Big ideas can be understood in progressively more sophisticated ways as students gain in cognitive abilities and experiences. Big ideas underlie the acquisition and development of concepts central to a discipline and lay the foundation for continual learning. Organizing information around core principles helps students see similarities and patterns across scientific ideas and disciplines, enabling them to understand that the principles that underpin one scientific discipline also apply to others. For example, the ideas of scale and structure, models, stability and change, systems and interactions, and energy are all applicable in the biological, physical, and earth and space sciences. However, they are generally taught and retaught as separate topics and related only to the discipline under consideration at the time. This approach may hinder students from making important connections that could help them integrate new learning more effectively. In Chapter 4, we discuss the need for content standards to be organized around big ideas and for the curriculum, instruction, and assessment that are aligned with standards to be organized this way as well. Context and Access to Knowledge Research that compares the performance of experts and novices demonstrates that experts are good at knowing which knowledge is relevant to a particular task, but novices are not. Expert knowledge is conditionalized; that is, it is organized and linked to a specification for when it might be useful (Simon, 1980;

OCR for page 38
Systems for State Science Assessment Glaser, 1992). Students’ knowledge is not. In fact, there is evidence to suggest that students’ knowledge is context bound, that is, it is tied to the context of the original learning. Bransford (1979) showed that students’ knowledge is so tied to the context of learning that including items on a final examination, with no clue as to the textbook chapter with which they are associated, creates problems for students (even those who answered the same types of questions correctly on unit tests). That is because students do not know what information is relevant for solving them. Experts know when and how to use their knowledge because they have had multiple experiences with applying it across related contexts. To conditionalize their knowledge, students need to have multiple experiences applying the same principle in different contexts. The abilities to apply a principle to an unfamiliar problem, to combine ideas that were originally learned separately, and to use knowledge to construct new products are evidence that robust understanding has been achieved (Hoz, Bowman, and Chacham, 1997; Perkins, 1992). The concept of applying what is learned in one context to others is frequently referred to as transfer (see Mestre, 2005, for a discussion of some contemporary views on this topic). Helping students develop an understanding of when and how to use what they know is an important key to the development of science literacy. Yet, many science assessments fail to help teachers and students assess the degree to which the student’s knowledge is conditionalized and rarely ask students to demonstrate that they know when, where, and how to apply what they know. Science as a Way of Knowing Each of the sciences has its own unique way of knowing, but all scientists share certain basic beliefs and attitudes about what they do. They approach their work with the belief that the world is understandable, that scientific ideas are subject to change yet durable over time, and that science involves the collection of verifiable evidence. All scientists ask questions about what happens in the world around them. Scientists share the goal of explaining the phenomena they observe and making predictions about what will happen in the context of their observations. To make these predictions and explanations, scientists develop detailed explanations of how the world works. The hallmark of any scientific theory is that it can explain current and previous observations and helps scientists predict new events. For instance, the theory of plate tectonics provides a detailed explanation of the origin of the ocean basins. It also explains other, related phenomena such as earthquakes and volcanic activity, which allows scientists to make valuable predictions about possible future events. Scientists create theories through careful, systematic study and observation, and they base their work on the assumption that the world has order and is understandable. Although scientists and philosophers of science agree that there is no one single scientific method, and although different scientific fields use

OCR for page 38
Systems for State Science Assessment various approaches and methods, empirical verification of theories is a critical aspect of science. Scientists continually test their theories by subjecting them to new empirical challenges. When empirical evidence does not support claims, the underlying theories will change. Science, then, is not about finding absolute truth, but rather about constructing theories that provide better means of predicting and explaining phenomena. Scientists use data and existing hypotheses, theories, models, and principles to create logical, consistent explanations of what they observe. To be classified as scientific, observations, measurements, explanations, and conclusions must be verifiable by other scientists. Thus, the understandings that result from scientific investigations are modified or changed with new observations and the further testing of ideas. For instance, chemists once thought that atoms were small, indivisible spheres. However, this model of the atom could not explain later findings, made possible through observations of macroscopic phenomena such as spectra. Chemists thus replaced the concept of the indivisible atom with that of an atom that has subcomponents. The replacement of old hypotheses and theories with new ones illustrates the dynamic nature of science. Assessment of students’ understanding of this aspect of science literacy should focus on ascertaining whether students can use their knowledge of science content to reason, make and justify predictions, develop explanations, and revise explanations in light of additional information. Inquiry Scientific inquiry is difficult to define and different organizations have taken slightly different approaches in describing it. In general, scientific inquiry can be thought of as the set of skills and approaches that scientists use in conducting their work.1 Conducting inquiry allows students to experience the ways in which scientists study the world and encourages an understanding of the nature of science and scientific knowledge. Central to inquiry is a view of science as an ongoing cyclical process of constructing and modifying ideas and models through the systematic gathering of evidence, the application of logical argument, and the questioning of assumptions, procedures, and conclusions. Scientifically literate adults should understand both how scientific evidence is obtained and how it is used to support explanations. Both the National Science Education Standards and the Benchmarks for Science Literacy recommend that students both understand and develop the skills related to inquiry, although the 1   Inquiry is also used to refer to the activities of students in classrooms in which they develop knowledge and understanding of scientific ideas, as well as an understanding of how scientists study the world. The National Science Education Standards includes inquiry as both a content area and an instructional strategy.

OCR for page 38
Systems for State Science Assessment former places more emphasis on students’ acquisition of the ability to conduct an inquiry, while the latter emphasizes the importance of students knowing about inquiry (Kouba and Champagne, 2002). Both the Benchmarks for Science Literacy and the National Science Education Standards suggest that before students graduate from high school they should have the opportunity to conduct a scientific investigation from start to finish, from identifying the question to presenting the results of the investigation and responding to criticism. The majority of state science content standards include science inquiry as an important aspect of what students should learn. However, state standards (like their national counterparts) vary somewhat in their approaches to inquiry, with some states focusing on developing students’ abilities to conduct inquiry, while others focus on developing students’ understanding and appreciation for the process of scientific inquiry as practiced by scientists. Still others require that students demonstrate competence in both ways (see Box 3-1). Despite these differences there is general agreement that including inquiry in the science curriculum gives students not only an understanding of what scientists have accomplished but also how they have learned what they know. The way in which a state’s standards describe what students are expected to know and be able to do relative to inquiry is important, as it influences what is BOX 3-1 Illustrative State Standards for Scientific Inquiry The following state standard emphasizes that students understand and appreciate the nature of inquiry. Example #1 A student should possess and understand the skills of scientific inquiry. Example #2 All students will develop problem-solving, decision-making, and inquiry skills, reflected by formulating usable questions and hypotheses, planning experiments, conducting systematic observations, interpreting and analyzing data, drawing conclusions, and communicating results. Example #3 Students understand the processes of scientific investigation and design, conduct, communicate about, and evaluate such investigations. SOURCE: Adapted from multiple state standards documents.

OCR for page 38
Systems for State Science Assessment taught, what should be expected from teachers and students, and what should be taken as evidence that students have attained the defined goal. Considering the abilities that comprise inquiry, such as observing, controlling variables, hypothesizing, thinking critically, and developing well-reasoned arguments further illustrates its complexity. While there is little question that inquiry should be measured in state science assessments (if it is included in the state content standards), each state will need to link decisions about which elements of inquiry will be measured, and how they will be measured, to their standards. Assessment developers will need explicit guidance from states on what students are expected to know and be able to do relative to inquiry. Does the state have the goal that graduates will be able to use methods of scientific inquiry to develop new scientific knowledge, or that they will have an understanding of inquiry that enables them to make well-founded decisions about scientific issues that affect their daily lives? A precise understanding of what is meant by science inquiry in a state’s standards is a necessary condition for developing tasks to assess it; if a state does not make it explicit, the items and tasks included in the state assessment program will come to define its meaning to teachers and students (Champagne and Kouba, 1996; Gummer and Champagne, 2005). We note than many abilities associated with inquiry can be assessed using paper-and-pencil test items, including: identifying questions that can be answered through scientific investigations; developing descriptions, explanations, predictions, and models using evidence; thinking critically and logically to link evidence and explanations; recognizing and analyzing alternative explanations and models; and communicating and defending a scientific argument. While it is possible for paper-and-pencil tests to provide a snapshot of students’ abilities in these areas, these snapshots may miss the mark by not addressing the iterative nature of inquiry and the revisions in thinking that occur as scientific inquiry unfolds. Few large-scale assessments, even performance assessments, probe in detail the fundamental ways that individuals process and use information in tasks that require extended lines of reasoning (Baxter, Elder, and Glaser, 1996; Quellmalz, 1984). Moreover, studies such as the Validities of Science Inquiry Assessments study lend empirical support to claims that many science inquiry standards, such as formulating scientific explanations or communicating scientific understanding, cannot be adequately measured using a multiple-choice format (see Quellmalz and Haertel, 2004). It is also the case that paper-and-pencil items cannot assess students’ ability to conduct a scientific inquiry from beginning to end. This process entails generating a question, designing the approach, running trials, gathering and analyzing

OCR for page 38
Systems for State Science Assessment the data, writing a report, presenting the results to others, and responding to criticism. The ability to complete this complex task may best be evaluated by teachers or others observing students as they are engaged in sustained investigations (Neill and Medina, 1989; Raizen and Kaser, 1989; Baron, 1990). It is possible, with careful planning, to incorporate these kinds of evaluations into the states’ science assessment system. For example, in New York teachers administer a standardized classroom inquiry assessment, score student work using rubrics that include samples of student work, and report scores as part of the state science assessment program.2 In other states—Connecticut for example—a separate test that is developed by the state is administered in the classroom after the students have engaged in inquiry activities (see Box 3-2). State assessment systems that include a classroom assessment component, or that are based almost completely on teacher-led assessments (for example, Nebraska), have the advantage of being able to measure the conduct of inquiry in the classroom over time with multiple tasks and opportunities for observing student growth in understanding. Maine and Vermont, two states that have developed multilevel assessment systems, collect information about students’ ability to inquire through classroom- and district-level assessment (see web sites). Assessing students’ understanding of inquiry as it is described in the state standards is important in itself, and is also critical to alignment. If alignment between a state’s science standards and its assessments is to be sustained, as is required both by the No Child Left Behind Act and the principles of standards-based education, inquiry must be assessed. If the state standards require that students be able to conduct an investigation, then building such opportunities into a state science assessment system is important. On the other hand, if the standards require only that students demonstrate an appreciation for the role inquiry plays in the work of scientists, or that students demonstrate an understanding of the nature of inquiry, including opportunities to assess students as they conduct an investigation could be less important. States should look to their standards for guidance on the role inquiry should play in their state science assessment system. DEVELOPMENTAL NATURE OF SCIENCE LEARNING While individuals who study learning do not believe that there is a single trajectory that all students follow, they recognize that some scientific ideas and concepts have to be learned so that more sophisticated understandings can be built on them. For example, before students can understand that organisms get energy from oxidizing their food, they must understand that energy can change 2   Information about the New York assessment is available at: http://www.emsc.nysed.gov/ciai/mst.html.

OCR for page 38
Systems for State Science Assessment BOX 3-2 Assessing Experimentation To assess student’s ability to use inquiry skills, Connecticut requires students to participate in a hands-on laboratory activity several weeks prior to the written test. This performance task asks students to design and carry out their own experiment to solve a problem and write about their results in an authentic format. Students are not scored on their actual performance on this task at the state level. Rather, teachers are encouraged to score their own students’ work and provide students with feedback about their performance. On the written test, students are given follow-up questions that relate directly to the hands-on task. These questions are scored at the state level and become part of the student’s score on the science portion of the CAPT. CAPT Science Performance Task: Soapy Water Grade 10 Scenario: Local water treatment plants often remove environmentally harmful impurities, such as soap, from wastewater before returning it to the environment. One way to remove soap from water is to have it react with other substances. When these reactions occur, a solid called a precipitate is sometimes formed. The precipitate can be filtered out of the water. Student’s Task: The students will design and conduct an experiment to explore the use of several substances in removing soap from water. During this activity they will work with a lab partner (or possibly two partners). The students must keep their own individual lab notes because after they finish, they will work independently to write a lab report about the experiment. The materials listed below should be provided for each lab group. It may not be necessary for the students to use all of the equipment that is provided. You may use additional materials or equipment if they are available. Powdered soap Table salt Epsom salt Sugar 4 paper cups 8 clear plastic cups 4 white plastic spoons/stirring rods Graduated cylinder Access to tap water Access to a balance Access to a clock or watch with a second hand Paper towels for cleanup Splash-proof goggles and apron for each student 4 test tubes Test tube rack Test tube brush Parafilm (to cover test tubes) Marking pencil 5 paper cones 5 pieces of filter paper Ruler Scissors 1 beaker Labeling dots

OCR for page 38
Systems for State Science Assessment SOAPY WATER Item 1 This item assess students’ understanding of conclusions drawn from scientific investigations and factors that affect their validity. The results of the experiment seem to indicate that soap has been removed from the water by the Epsom salt; however, this conclusion should be questioned. The color of the filtrate and presence of a precipitate do not mean that all of the soap has been removed. The group did not include a control in their experiment for comparison purposes. The groups also could have performed a shake test on the filtrate to see if suds formed, indicating the presence of soap. It is also unclear if important variables that affect the validity of the conclusion have been controlled in the experiment. CAPT Framework Experimentation: Draw valid conclusions and discuss their validity. Item 2 This item assess students’ understanding of what constitutes a complete experimental design. In this case, students do not have all of the information they need to replicate the experiment. Students are given some information, such as the substances added to the soapy water, and general procedures that were followed. However, other important information, such as the amount of soap added to the water, the amount of soapy water added to each cup, and the amount of each substance added to the cups, is needed. CAPT Framework Experimentation: Design and conduct appropriate experiments. Item 3 This item assess students’ understanding of what makes an appropriate control in an experiment. In Group B’s experiment, an appropriate control would have been a cup containing 50 mL of soapy water in which nothing was added. The control should have been filtered and the filtrate shaken, just as with the other samples. The control would improve the experiment because it would serve as a basis of comparison to determine if any of the substances removed soap from the water. The control would show if filtering alone removes soap. CAPT Framework Experimentation: Design and conduct appropriate experiments. Item 4 This item assess students’ understanding of what constitutes an appropriate experimental design. Group B’s experiment is somewhat better although each experiment has its flaws. Group B specifies the amount of each material used (soapy water, salts, sugar) and uses a shake test as a quantitative measure of soap left in the water. Neither group included a control in their experiment, neither performed multiple trials, and it is not clear if all variables have been controlled in either experiment. (NOTE: Downloaded from http://pals.sri.com/pals/tasks/9-12/SoapyWater/admin.html. Connecticut is revising its science assessment and may no longer require students to participate in this assessment activity. Information about the new assessment can be found on the Connecticut Department of Education’s web site.)

OCR for page 38
Systems for State Science Assessment from one form to another. This concept can be represented in the form of learning progressions. Learning progressions are descriptions of the successively more sophisticated ways of thinking about an idea that follow one another as students learn.3 A learning progression lays out in words and examples what it means to move toward more expert understanding in an area of interest. Ideally, learning progressions should be based on research about how competence develops in the domain; however, for many aspects of science learning the research literature is incomplete. Thus, research findings may need to be supplemented with the experience of expert teachers and others with knowledge of how students learn science. In such cases, basic principles of cognition and learning that can be applied more generally, such as the importance of how scientists organize and retrieve their knowledge in approaching new questions and solving new problems, can be used to develop the learning progressions. More than one path leads to competence. The pathways that individual students follow depend on many things, including the knowledge and experience that they bring to the task, the quality of the instruction that supports their learning, and the nature of the specific tasks that are part of the experience. Nonetheless, some paths are followed more often than others. Using these typical paths as a foundation for describing learning and the ways in which deeper understanding develops can provide the basis for developing learning progressions. It also can provide clues about the types of assessment tasks that will elicit evidence to support inferences about student achievement at different points along the progression. Prior Knowledge Contemporary theories of learning emphasize that learning is a process of constructing understanding that involves ongoing revision and reorganization of current thinking as new knowledge is acquired. Thus, one very important aspect of science knowledge that should be considered is students’ prior knowledge. To focus their instruction, teachers need a clear idea of the depth of knowledge, skills, and experiences their students bring to the classroom. Teachers must draw out and work with the prior understandings that students have. To do this, the teacher must actively inquire into students’ reasoning, creating tasks and opportunities in which students’ thinking can be revealed. Ongoing classroom assessment conducted prior to and during instruction can help teachers develop instructional strategies that link new knowledge to existing knowledge. For example, asking students to describe their reasoning as they tackle tasks is a strategy that provides 3   Learning progressions have been referred to by many different names, including progress variables, learning trajectories, progressions of developmental competence, and profile strands.

OCR for page 38
Systems for State Science Assessment insight into their thinking and the ways they are using what they have previously learned. It also provides teachers and students with opportunities to correct any misunderstandings. The assessment of prior knowledge is most usefully accomplished in the classroom where students can receive timely feedback and further instruction that can help to reconstruct their alternate or naïve conceptions so that learning can proceed. Large-scale state and district tests can also help identify students’ alternate conceptions, but because the results of these assessments come too late to assist students in reconstructing their flawed beliefs, they may be more useful for improving future instruction than for helping current students learn. IMPLICATIONS FOR ASSESSMENT Practice and Feedback The domain of science is complex and multifaceted, requiring sustained effort and focused instruction for learning to progress. Students need multiple opportunities to practice what they have learned and to receive timely feedback with which to adjust their learning strategies—to reinforce successful ones and to modify and refine unsuccessful ones (Senge, 1990; Shepard, 2000; Sylvester, 1995). Classroom assessment strategies that provide timely feedback are an important tool for this purpose and should be included in any science assessment system, even when the results are not used as part of the state testing system for accountability purposes. Because different assessment strategies tap into different aspects of students’ knowledge and understanding, students should be provided with multiple opportunities to get feedback. For example, during a unit on cell structure, students might be asked to participate in an oral examination after completing a reading assignment and receive feedback from the teacher on their understanding of key concepts. As part of the same unit, the teacher may use a check sheet during microscope work to assess skills and share the results with the student. Laboratory data records might be evaluated individually as students are working on investigations, and the teacher can provide immediate feedback to students as they work. A short multiple-choice test on identifying and naming cell parts could also be given, graded, and discussed in class. Students could be asked to prepare concept maps illustrating the relationship between cell structure and function, and also be asked to explain their thinking to a small group. The group could give feedback, and each student could perform a self-assessment of the quality of his or her concept map. The class might develop a rubric that will be used to score an essay question comparing prokaryotic and eukaryotic cell structure that is part of a unit test. The results of the unit test would be returned to the student in a timely manner. This variety of measures provides students and teachers with a richer

OCR for page 38
Systems for State Science Assessment picture of what students know and are able to do in multiple contexts, and provides students with the feedback they need to progress. Assessing Science Literacy In a content-rich domain such as science, selecting the specific content to include as part of any one assessment activity is always a challenge. Even teachers cannot assess every aspect of learning that is important, but these decisions should not be made lightly. Assessment and learning are so tightly linked that both students and teachers likely will refine their expectations for student learning only to the outcomes that are assessed. This is particularly true when the results earned on any one test are valued more than the information provided by the others. If the goal of science education is to develop science literacy, then science assessments, and the standards on which they are based, must be consistent with that goal and must reflect the intellectual and cultural traditions that characterize the practice of contemporary science. Science assessments that reflect what is valued in the domain should include opportunities for students to demonstrate their science literacy by asking them, for example, to read and interpret scientific articles as they might appear in newspapers and the popular press, or to interpret graphs and charts and to use the information to support a claim. Science assessment that reflects the practice of science should focus not on the retention of discrete knowledge of facts or procedures but on assessing students’ abilities to use scientific theories to explain phenomena, to make predictions in light of evidence, and to apply their science-related knowledge in approaching new and unfamiliar situations. Assessments designed with these ideas in mind might, for example, ask students to describe, using Newton’s law, why seat belts should be used in cars, rather than asking them to state Newton’s law of motion. Such an assessment might ask them to explain why veins and arteries have particular properties rather than asking students to list the properties of arteries, or to use what they know to create an artificial artery and to justify why particular features were included in the model or not. In other words, such assessments will focus on assessing students’ ability to use what they know—to show evidence of transfer. Box 3-3 contains two questions from an examination given to first-year physics students (Mazur, 1997). The first question requires students to think scientifically and apply what they know about circuits, while the second question requires only that they use a frequently used formula to calculate an answer. Assessment strategies that rely most heavily on questions such as the one in the second example send a message that application is not as important as memorization. Science assessment should include opportunities for students to demonstrate their reasoning and conceptual understanding; their ability to build and revise logically consistent explanations using theories and evidence; and their ability to justify and explain their answers. Science assessment should be designed so that

OCR for page 38
Systems for State Science Assessment BOX 3-3 Memorization Versus Understanding Below are two assessment questions on DC circuits: one conceptual (top) and one conventional (bottom). The questions were given on a physics examination at Harvard University in 1991. Although the second question required more advanced calcualtions than the first, students performed better on the second question. 1. A series circuit consists of three identical light bulbs connected to a battery as shown here. When the switch S is closed, do the following increase, decrease, or stay the same? The intensities of bulbs A and B The intensity of bulb C The current drawn from the battery The voltage drop across each bulb The power dissipated in the circuit 5. For the circuit shown, calculate α the current in the 2-Ω resistor and (b) the potential difference between points P and Q. SOURCE: Mazur (1997, p. 5). Reprinted by permission of Pearson Education, Inc., Upper Saddle River, NJ.

OCR for page 38
Systems for State Science Assessment students are asked to perform tasks such as using theories, principles, and models to link data to claims; communicating and defending scientific explanations; and critiquing the reasoning in arguments in which fact and opinion are intermingled and the conclusions do not follow logically from the evidence. To assess students’ abilities to use what they have learned about science, states will need to use a variety of assessment approaches including, but not limited to, well-designed multiple-choice questions, open-ended items, performance assessments, and classroom assessment that can provide opportunities for students to demonstrate deeper understanding and complex skills that cannot easily be captured by time-limited tests, regardless of their quality. An assessment system provides opportunities to gather information about students’ understanding and abilities using a variety of sources, both proximate and distal from instruction, and to combine and reconcile results to paint a richer picture of student achievement. In such a system, not only could content knowledge be assessed more completely but it could also be assessed in ways that indicate whether students can apply their knowledge and reasoning to situations similar to those they will encounter outside the classroom, as well as to situations that are similar to how scientists work. QUESTIONS FOR STATES In designing a science assessment system that is consistent with the goals of science literacy and the ways in which students develop their understandings and abilities relative to science, states should ask themselves the following questions. Question 3-1: Does the state’s science assessment system target the knowledge, skills, and habits of mind that are necessary for science literacy? For example, does it include items, tasks, or tests that require students to describe, explain, and predict natural phenomena based on scientific principles, laws, and theories; understand articles about science; distinguish questions that can be answered scientifically from those that cannot; evaluate the quality of information on the basis of its source; pose and evaluate arguments based on evidence; and apply conclusions appropriately? Question 3-2: Does the state’s science assessment system reflect current scientific knowledge and understanding? For example, does the state have in place mechanisms to ensure that all of the measures that comprise the assessment system are scientifically accurate? Question 3-3: Does the state’s science assessment system measure students’ understanding and ability to apply important scientific content knowledge and scientific practices and processes? For example, does it include a focus on assessing

OCR for page 38
Systems for State Science Assessment students’ understanding of the big ideas of science as opposed to recall of isolated facts, formulas, and procedures? Question 3-4: Has the state conducted an independent review of its content standards to ensure that they articulate both the skills and the content knowledge students need to achieve science literacy? Question 3-5: Does the state’s science assessment system reflect contemporary understandings of how people learn science? Question 3-6: Is the state’s science assessment system consistent with the nature of scientific inquiry and practice as it is outlined in the state standards? For example, are opportunities built into the assessment system to assess students’ abilities to conduct extended scientific investigations, if such abilities are included in the state’s science standards?