Technology and Assessment Thinking Ahead: Proceedings from a Workshop (2002) / Chapter Skim
Currently Skimming:
6 How Computer-Based Technology Can Disrupt the Technology of Testing and Assessment
6 How Computer-Based Technology Can Disrupt the Technology of Testing and Assessment
Pages 63-78
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 63... ...
in other cases, the technology itself can increase the efficiency of testing. And in still other cases, computer-based technology provides opportunities to radically transform testing and assessment.
|
From page 64... ...
in a series of randomized experiments, this mode of administration effect has ranged from an effect size of about .4 to just over 1.0. in practical terms, the mode of administration found in the first study indicated that when students accustomed to writing on computer were forced to use paper and pencil, only 30 percent performed at a "passing" level; when they wrote on computer, 67 percent "passed." In a second study, the difference in performance on paper versus on computer for students who could keyboard approximately 20 words a minute was larger than the amount students' scores typically change between grade 7 and grade ~ on standardized tests.
|
From page 65... ...
This iterative process occurs until the test-taker's ability estimate stabilizes. Computer-adaptive testing complicates the item selection and delivery process, but because it is usually able to obtain an ability estimate based on a smaller set of items, it is more efficient than traditional paper-based tests.
|
From page 66... ...
As a result, these items provide very indirect and crude insight into examiners' cognitive processes. Similarly, while the educational community uses tests for a variety of purposes including diagnosing students' strengths and weaknesses; measuring achievement, aptitude, and ability; assessing the impact of instruction on student learning; and examining the quality of education students receive within a school, district, state or even country, test experts have long argued that the current technology of testing should be applied to meet a single purpose at a time.
|
From page 67... ...
Information collected as students interact with the learning system could also be used to diagnose student learning styles, common errors or tendencies, and misconceptions. Once these elements are identified, the systems could not only help teachers intervene immediately, but also could help structure future instruction in a way that is compatible with the students' learning style.
|
From page 68... ...
has launched several initiatives involving medical simulations. While the main purpose of these initiatives is to develop medical and surgical simulators to efficiently and more effectively train Army medics, these simulators are providing unique opportunities to assess kinesthetic abilities, content knowledge, and medical decision-making skills.
|
From page 69... ...
From an assessment perspective, the simulator enables unique approaches to diagnostics and mastery testing. As the simulator provides opportunities for the recruit to practice new procedures or introduces new complications, the system can identify tendencies such as inserting an instrument at a dangerously steep or flat angle or inserting instruments too deep or shallow.
|
From page 70... ...
Novice Trainee Filbert Figure 6-3 Motion trajectories of novice, trainee, and experienced surgeon. SOURCE: Images taken from Metrics for Objective Assessment of Surgical Skills Workshop summary draft report, 2001.
|
From page 71... ...
At times, the learning system presents textual or graphical information to explain concepts and provides students with access to various tools and pieces of information via menu selections. In addition, the system often asks students to demonstrate their understanding via written responses to specific questions, multiple-choice questions, and, most often, modifying different aspects of genetic codes to create dragons with specific traits or to determine how a trait suddenly appeared in a generation of dragons.
|
From page 72... ...
. Figure 6-6 Generalizing from guided explorations to rules of genetics in BioLogica.
|
From page 73... ...
These challenges fall into three broad categories: technical, political, and practical. Technical Challenges The first major technical challenge involves figuring out which information collected by these systems is most useful for a given purpose and then deciding how to combine this Disrupting the Technology of Testing and Assessment 73
|
From page 74... ...
Political Challenges Currently, political and educational leaders strongly embrace large-scale and high-stakes testing, and educational accountability appears to be the top priority shaping our educational system. But political and education leaders appear deaf to the calls for the incorporation of multiple measures into these school accountability systems.
|
From page 75... ...
, more closely scrutinize the reliability of scores given to such work, return data from multiple measures in more useful formats, share information and student work with a wider base of constituents, and provide a forum to account for school programs and strategies. investing now in developing web-based accountability systems that broaden the definition of educational accountability will better set the stage for replacing external state-mandated achievement tests with assessments that are integrated with learning systems.
|
From page 76... ...
The potential positive impacts that integrated learning and assessment systems could have on teaching and learning, coupled with the vast amount of technical work that must be done to develop these new methodologies, make it imperative that the educational community follow the military's lead by investing now in developing disruptive applications of computerbased technology to the technology of testing and assessment. 76 Technology and Assessment: Thinking Ahead
|
From page 77... ...
National Board on Educational Testing and Public Policy Statements, 2~. Madaus, G.F., Scriven, M S., & Stufflebeam, D.~., (1983~.
|
From page 78... ...
(1997~. Testing writing on computers: An experiment comparing student performance on tests conducted via computer and via paper-and-pencil.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.