critical-thinking and decision-making dimension. Assessing technology-related capability, which includes the ability to use a design process, is more difficult than gauging knowledge, and only a few methods have been tried for assessing it, partly because this tends to be very expensive, at least for large-scale application. Nevertheless, assessing the capability dimension is crucial. Only a few instruments encourage higher order thinking (critical thinking and decision making), although a goal of all types of learning is to encourage thinking that considers uncertainty and requires nuanced judgment, rather than just factual recall.
One step common to the design of assessments is the development of a framework.
One step common to the design of assessments is the development of a framework that describes the cognitive and content components of the proposed assessment. The framework often suggests the relative emphasis on each area of content, depending on the age of the test population and other factors. The conceptual underpinnings of the framework can be represented visually as a two-dimensional matrix, which serves as a blueprint for the more detailed phases of assessment design, the development of test specifications, and, ultimately, the development of test items.
The committee developed a sample assessment matrix (Figure ES-2) modeled after conceptual frameworks developed for the National Assessment of Educational Progress (NAEP) for closely related subjects (e.g., science and mathematics) (NAGB, 2002, 2004). With one modification, the matrix includes the three dimensions of technological literacy described in Technically Speaking—knowledge, capabilities, and