INTRODUCTION

Part III sets forth the foundations for educational assessment in terms of contemporary scientific understanding of the nature of human cognition and methods of measurement. These two bodies of knowledge jointly provide a set of principles and methods for assessment design and use. As in any design activity, the scientific foundations provide direction and constrain the set of choices and possibilities, but they do not prescribe the exact nature of the design, nor do they preclude ingenuity to achieve a final product. Design is always a complex process that, while guided by theory and research, involves optimatization under a series of practical constraints outside the realm of science. Thus the design is influenced in important ways by the purpose of the assessment (e.g., to assist learning, measure individual attainment, or evaluate a program), the context in which it will be used (classroom or large-scale), and practical constraints (e.g., resources and time). The following chapters explore issues of how the foundations play out in the design of real assessment situations. A variety of existing assessments are described to illustrate the points.

Chapter 5 presents features of a new approach to assessment design that capitalizes on the scientific advances described in Part II. Using the assessment triangle defined in Chapter 2 as a framework, we discuss various aspects of design—including identification of the targets for assessment, item and test design, validation, reporting, and fairness—always focusing on how a cognitive approach to design would differ from current approaches.

Chapter 6 broadens the discussion beyond assessment to explore the interactions of assessment with curriculum and instruction; how assessments could best be used to support learning, first in classroom contexts and second in large-scale contexts; and the need for systems of multiple assessments that would work together to help achieve a common set of learning goals.

Chapter 7 considers the role current and future information technologies could play in realizing the full potential of the new kinds of assessment the committee envisions. Technology is enabling the assessment of a wider array of performances and simultaneously changing the nature of learning environments and the potential targets of assessment. The opportunities as well as the challenges are considered.

Two kinds of suggestions are presented in these chapters. Some relate to how scientific developments in the foundational areas of cognition and measurement could improve assessment. Others involve changes that could be made to the educational system to accommodate effective use of these assessments. Many of the suggestions in this latter area are consistent with the scientific developments, but those developments themselves are not sufficient to determine how educational systems should function. Political and societal decisions come into play with regard to organizational changes, resource implications, and investment priorities, and the committee recognizes that these are issues on which knowledgeable people may disagree.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 175
Knowing What Students Know: The Science and Design of Eduacational Assessment INTRODUCTION Part III sets forth the foundations for educational assessment in terms of contemporary scientific understanding of the nature of human cognition and methods of measurement. These two bodies of knowledge jointly provide a set of principles and methods for assessment design and use. As in any design activity, the scientific foundations provide direction and constrain the set of choices and possibilities, but they do not prescribe the exact nature of the design, nor do they preclude ingenuity to achieve a final product. Design is always a complex process that, while guided by theory and research, involves optimatization under a series of practical constraints outside the realm of science. Thus the design is influenced in important ways by the purpose of the assessment (e.g., to assist learning, measure individual attainment, or evaluate a program), the context in which it will be used (classroom or large-scale), and practical constraints (e.g., resources and time). The following chapters explore issues of how the foundations play out in the design of real assessment situations. A variety of existing assessments are described to illustrate the points. Chapter 5 presents features of a new approach to assessment design that capitalizes on the scientific advances described in Part II. Using the assessment triangle defined in Chapter 2 as a framework, we discuss various aspects of design—including identification of the targets for assessment, item and test design, validation, reporting, and fairness—always focusing on how a cognitive approach to design would differ from current approaches. Chapter 6 broadens the discussion beyond assessment to explore the interactions of assessment with curriculum and instruction; how assessments could best be used to support learning, first in classroom contexts and second in large-scale contexts; and the need for systems of multiple assessments that would work together to help achieve a common set of learning goals. Chapter 7 considers the role current and future information technologies could play in realizing the full potential of the new kinds of assessment the committee envisions. Technology is enabling the assessment of a wider array of performances and simultaneously changing the nature of learning environments and the potential targets of assessment. The opportunities as well as the challenges are considered. Two kinds of suggestions are presented in these chapters. Some relate to how scientific developments in the foundational areas of cognition and measurement could improve assessment. Others involve changes that could be made to the educational system to accommodate effective use of these assessments. Many of the suggestions in this latter area are consistent with the scientific developments, but those developments themselves are not sufficient to determine how educational systems should function. Political and societal decisions come into play with regard to organizational changes, resource implications, and investment priorities, and the committee recognizes that these are issues on which knowledgeable people may disagree.

OCR for page 175
Knowing What Students Know: The Science and Design of Eduacational Assessment Five key features of a new approach to assessment design serve as the organizing themes for this chapter: A model of cognition and learning, or a description of how people represent knowledge and develop competence in a subject domain, is a cornerstone of the assessment development enterprise. Unfortunately, the model of learning is not made explicit in most assessment development efforts, is not empirically derived, and/or is impoverished relative to what it could be. To increase the chances of collecting evidence that supports the types of inferences one wants to draw, the design and selection of assessment tasks, along with the procedures for evaluating students’ responses, should be guided jointly by the cognition and interpretation elements of the assessment triangle. An assessment should be more than a collection of tasks that work well individually. The utility of assessment information can be enhanced by considering how to design and/or select tasks so that the information derived from them can be combined to support the desired inferences. The process of construct validation during test design should rest, in part, on evidence that tasks actually tap the cognitive content and processes intended. Although reporting of results occurs at the end of an assessment cycle, assessments must be designed from the beginning to ensure that reporting of the desired types of information will be possible and effective. When results are reported, well-delineated descriptions of learning in the domain are key to their effectiveness for communicating about student performance. Fairness in testing is defined in many ways, but at its core is the idea of comparable validity: a fair assessment is one that yields comparably valid inferences from person to person and group to group. One way of thinking about fairness is to take into account individual learners’ instructional histories when designing an assessment.