The Assessment of Science Meets the Science of Assessment Summary of a Workshop (1999) / Chapter Skim
Currently Skimming:
The Assessment of Science Meets the Science of Assessment
The Assessment of Science Meets the Science of Assessment
Pages 1-28
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
In many states, science education reforms are linked to broader efforts to improve the entire educational system and to set high standards for student learning. Assessment has become an increasingly critical part of these reforms.
|
From page 2... ...
They debated the role of assessment in the systemic reform of science education. The con ~ ference also featured a poster session in which participants could see firsthand some of the new science assessment tools being used in several schools.
|
From page 3... ...
The teachers can use the feedback to revise their classroom practices, and the students can use the feed back to monitor their own learning. This purpose, often called formative assessment, is also receiving greater attention with the spread of new teaching methods.
|
From page 4... ...
Because of this, existing tests might not adequately support reforms in science education and might discourage teachers from adopting elective strategies for teaching advanced knowledge and skills. The developers of new assessment approaches are trying to overcome the limitations of multiple choice tests by broadening the available types of assessment tasks and methods and by more directly tying them to classroom practices.
|
From page 5... ...
In contrast to traditional assessment, which often takes place at the end of learning when it is too late to make a difference, many innovative assessments encourage teachers and students to engage in continuous cycles of problem solving, reflection, discussion, and revision. PERFORMANCE ASSESSMENT Many of the alternative approaches presented at the conference fall within the broad category of performance assessment, which refers to any assessment that requires students to demonstrate their knowledge or skills by creating an answer or product.
|
From page 6... ...
One effort to grapple with the challenges of portfolio assessment was presented by Elizabeth Stage, co-director of the New Standards Project science initiative. In this project, several states and urban districts are working together to design an assessment system that will measure how well students are doing compared with the voluntary content standards emerging from national professional organizations in science, math, and other disciplines.
|
From page 7... ...
To help teachers and students understand what high quality student work looks like, the New Standards Project is collecting pieces of authentic work that exemplify high perfor mance at different grade levels, along with explanations of how that work was evaluated. To make the point that students in all types of classroom situations can meet high standards, the project is trying to amass diverse examples of high quality work from a variety of classrooms with different instructional programs.
|
From page 8... ...
They require less student writing than some other types of performance assessments, and often cost less to administer and score. Students can collaborate on constructing concept maps, which means the maps can be used to assess teamwork skills, such as leadership, decision making, and communication, as well as individual skills.
|
From page 9... ...
As described by Robert Glaser of the University of Pittsburgh, the aim is to produce an assessment that Alec tively measures complex cognitive skills, such as the ability to integrate knowledge and make inferences, select and execute strategies for solving problems, adjust one's own per formance, and offer coherent explanations for the strategies used. Under this approach, a test developer determines the construct validity of a test by pay ing attention to its cognitive validity, that is, by looking at whether the test is a valid measure of both the content knowledge and the cognitive skills required by a particular domain, and whether it engages students in the kinds of cognitive activities that underlie competent performance.
|
From page 10... ...
Glaser and Baxter are studying several trial state performance assessments and have conducted observations of students as they complete science tasks and solve problems in different ways and contexts. The research goal is to determine whether these tests actually measure the cognitive capabilities associated with various levels of student achievement, whether the tasks accurately reflect the performance objectives, and whether the scores reflect the quality of the cognition exhibited by test takers.
|
From page 11... ...
And concept maps also show promise for wider use. Scoring Issues The degree to which there is consistency in scoring is a primary concern related to the reliability of performance assessments.
|
From page 12... ...
It is necessary to provide raters with extensive training or the specific assessment task and with continuing supervisiona type of training that is not feasible for raters of portfolio assessments because the tasks are not the same across students. Comparisons with Other Methocls Analyses of reliability often compare alternative assessments with multiple-choice tests in the same domain.
|
From page 13... ...
O'Neil obtained a correlation of r =.7 between concept maps and an essay based task on the same topic; Ruiz Primo obtained an average r of .57 between concept map scores and multiple choice tests. NEED FOR SCAFFOLDING As discussed by Maryellen Harmon, some of the first science performance assessments designed for TIMSS were quite open ended, providing little guidance to students so that they could design their own strategies and apply their learning in new ways.
|
From page 14... ...
In nearly every country, students who could solve problems correctly on the TIMSS performance tasks had considerably more difficulty trying to explain why something happened or how they figured out the answer. Good reading, writing, and communication skills are indispensable to the practice of science, as reflected by their inclusion in voluntary science education standards.
|
From page 15... ...
Advocates in the United States similarly argue that $50 per student for performance assessment is not an unreasonable investment to procruce a testing system that works better. A main argument for pursuing performance assessment is its ability to encourage teachers and students to engage in more desirable classroom practices (discussed at length in the next section)
|
From page 16... ...
Other conference participants asserted that there is still little proof that performance assessment connects to higher student learning. Because of this lack of evidence and the limited utility of performance assessment for high-stakes decisions, they find it difficult to justify its extra costs, especially if the same basic news about student achievement can be collected with less-expensive assessments.
|
From page 17... ...
The ability to integrate knowledge is another critical skill in science, enabling people to become better problem solvers and apply science in a variety of contexts throughout their lives. Marcia Linn at the University of California, Berkeley, has sought to identify a repertoire of assessments and activities that engage students in knowledge integration, which she defines as a dynamic process of linking, connecting, distinguishing, organizing, and reflecting on models of scientific phenomena.
|
From page 18... ...
Several projects discussed at the conference use assessment expressly as a vehicle for the professional development of teachers. Margaret Brown found that, with the right kind of professional development, it is possible to encourage teachers to use formative assessment more often and more effectively.
|
From page 19... ...
Jan Hawkins of the Center for Children and Technology described the Center's projects to help teachers learn to make reliable and useful judgments about student work. Teachers in one project judged three kinds of student performance tasks: a final product, an oral ~9
|
From page 20... ...
According to conference presenters, teacher based assessment projects help teachers become more systematic in collecting their observations and more reliable in judging student work. By seeing and discussing exemplary work, they begin to internalize standards and adjust instruction based on their assessments.
|
From page 21... ...
Scaffolding for Teachers The projects discussed in this section suggest that teachers need appropriate scaffolding, too. This means providing teachers with a great deal of support and guidance during the early stages, as they are learning to use new forms of assessment, followed by diminishing support as teachers gain competence in the new techniques.
|
From page 22... ...
The ambiguity, vague ness, and wide variation in standards make it difficult to construct compatible assess meets. As noted by Eva Baker of the University of California, Los Angeles, the standards being adopted by states tend to describe broad areas of content emphasis and general cognitive demands, so that an array of performance tasks can be inferred from any given standard.
|
From page 23... ...
Some students used creative responses to avoid addressing the mathematics. The reactions of the college students show the challenges involved in designing practical tasks with literacy demands that test math and science, and the complexities of scoring diverse student responses.
|
From page 24... ...
For example, some performance assessments do not yield a score that can be aggregated. Some participants made the point that formative assessments are designed primarily to assess what is taught in the classroom and inform revisions in teaching, and are less concerned with testing a child's overall knowledge and skills.
|
From page 25... ...
Curriculum and Accountability Another unresolved issue is how to allow for differences in a curriculum, such as a voca tional education curriculum, within a common accountability system, and whether we should even have a common system. Many science and math education reformers emphasize advanced academic courses for all students, although an applied math and science curriculum might work better for some students.
|
From page 26... ...
In many states and school districts, debates over assessment have been subject to increased scrutiny and involvement of policymakers and the public. Although an open process is imporrant, it sometimes has made these debates more complex and politically charged, allowing researchers and assessment designers "few protected enclaves in which to try and fail," as Eva Baker noted.
|
From page 27... ...
1 performance assessments mean. For instance, performance assessment results are typi cally reported in relation to public goals or standards, rather than in terms of the norma tive comparisons that the public is more accustomed to, so new reporting systems need to be designed that make sense to the typical parent.
|
From page 28... ...
Teachers could play a stronger role in assessment development to ensure that new approaches are credible and feasible. All three groups must communicate with the larger polity of parents, policymakers, and the public to build support for changes that will have a lasting positive impact on classroom practice.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.