Ripley admitted that he and his colleagues in the United Kingdom often feel torn between offering a “reductionist” test (presenting a task so that virtually all students will be familiar with it) and elevating the test (trying to raise the minimum expectation for students). Because his agency’s mission is to design “high-stakes” assessments that offer “a very similar test experience for all students” around the country, it is important to try to minimize any bias in such environments.
Similarly, Katz pointed out that ETS—in using e-mail, for example, in its testing—“tries to come up with something generic” that will likely resemble whatever a student is used to. Moreover, in echoing a major point from his talk, he noted that “we are focusing not so much on the technology but on what people are doing with the information that is presented.” Still, he acknowledged, “it is hard to avoid some aspect of bias.”
Ripley added that administration of tests in a digital environment might actually reduce bias. QCA wanted to know “which students, in which categories of need, we would exclude if we went down a digital front—a screen route—for formulating tasks.” So it did a study, completed in 2004, “Our top-line conclusion was that we were enabling more students to access the tasks on screen than if they were on paper,” said Ripley. “So we are certainly not doing more harm than in paper-based tests. And I would argue that we are facilitating engagement, not preventing engagement, with the test.”
Heidi Schweingruber of the National Academies’ Board on Science Education raised the issue of ICT embeddedness in content areas—an often-mentioned idea during the workshop—and noted that it did not seem to be reflected in the discussion of assessments. Ripley acknowledged that so far “this has been a challenge for us. Our tests look rather like standardized ICT lessons, or business applications of ICT, and not even school-based applications of ICT.” But the omission has been noted, he said, and about two years ago his agency began development work in this area. Colleagues are making progress, he suggested, though “the material is not yet ready to show publicly or to use in any of our test administrations.”