In the course of this study, the committee returned again and again to several ideas of central importance to the development of high-quality assessment instruments. Although these themes are not the only important concepts in the field of assessment, they are given special emphasis in this report, which will be read by many people outside the field. The central themes are: (1) defining purpose; (2) selecting content; (3) avoiding bias; and (4) ensuring fairness.
Any assessment instrument can only assess a small part of what a person or group of people knows, believes, or can do. Thus, before starting the design process, it is important to define the purpose of the assessment. Although an assessment may serve more than one purpose, the most effective assessments are designed to serve only one purpose; different purposes all but imply different kinds of assessments. Completely different designs would be used, for instance, to test how well museum-goers understand the lessons of a technology exhibit and to determine how well graduates of a school of education have been prepared to teach technology to elementary school students.
A designer must first establish what test takers will be expected to know about technology and what they should be able to demonstrate that they know. For students, these questions have often been answered in the form of standards. ITEA (2000) has developed content standards for K–12 students that address technological literacy. AAAS (1993) and NRC (1996) have developed national science education standards that include references to technological literacy. However, because none of these technology-related standards has been widely accepted or incorporated into education programs in the United States, the issue of assessment design can be very complicated.
In the K–12 setting, researchers have identified a number of purposes for assessments.
In the K–12 setting, researchers have identified a number of purposes for assessments, ranging from program evaluation and instructional planning to pupil diagnosis (e.g., Brandt, 1998; McTighe and Ferrara, 1996; Stiggins, 1995). Assessments of technological literacy have two primary purposes in the K–12 setting: (1) to provide a measure of what students and teachers know about technology and how well they are