Uncommon Measures Equivalence and Linkage Among Educational Tests (1999) / Chapter Skim
Currently Skimming:
3 Challenges of Linking to NAEP
3 Challenges of Linking to NAEP
Pages 48-60
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 48... ...
Although states were unwilling to pay for most services, two-thircis said they would pay to develop linkages between their state assessments anc3 NAEP (Levine et al., 19981. Several states have already begun partial linkages: three states compare one component of their assessment program to NAEP; two link at least one component of their state results to the NAEP results; anc3 one links at least one component of its assessment to the NAEP scale (Bond et al., 1998~.
|
From page 49... ...
Rather, NAEP re' ports the aggregate, or collective, performance of students anc3 it floes so in two ways scale scores anc3 achievement levels: scale scores provide information about the distribution of student achievement for groups anc3 subgroups in terms of a continuous scale; achievement levels are used to characterize student achievement as basic, proficient, or acivancec3, using ranges of performance established for each gracle. The National Assessment Governing Board (NAGB)
|
From page 50... ...
If a student is assessed with six blocks of NAEP items, it is likely that his or her location on the NAEP scale and his or her assignment to a NAEP achievement-level category will be estimated with reasonable precision. If the test to be equated to NAEP and the conditions of administration and use of the test are sufficiently similar to those of NAEP, a stucient's score on the linked test likely might then be used to estimate his or her NAEP achievement-level placement with acceptable precision.
|
From page 51... ...
Although the regression approach makes statistical sense, it raises problems if the results are intenclec3 to be used for policy purposes. Linking Large-Scale Assessments Many studies have compared the aggregate NAEP results with similar results from state-level assessments, from international assessments, or from other testing programs (see e.g., National Center for Education Statistics, 1998; Pashley anc3 Phillips, 1993; Linn anc3 Kiplinger, 19951.
|
From page 52... ...
The resulting projection of the {AEP scale to the NAEP scale could be checked by comparing actual NAEP results for the United States as a whole with the predictions of NAEP results from the lAEP results of the U.S. sample that hac3 taken the lAEP.
|
From page 53... ...
Also, 62 percent were preclictec3 by the link to score in the top half of the international sample; in fact, 57 percent scored in the top half internal tionally. The NAEP-TIMSS link for the 8th~gracle science assessment inclicatec3 that 16 percent of the students would be classified in the top 10 percent internationally, based on NAEP results; on the actual TIMSS,20 percent of the students scored at that level.
|
From page 54... ...
A much more specialized kind of equipercentile equating was clevelopec3 in the early l990s between the Kentucky Instructional Results Information System (KIRIS) anc3 the NAEP achievement-level categories (Kentucky Department of Eclucation, 19951.
|
From page 55... ...
The North Carolina Department of Public Instruction linked mathematics on its comprehensive academic testing program with the NAEP scale (Williams et al., 19951. The state's enc3-of-gracie tests were closely aligned with the state curriculum, but not with NAEP, so content alignment was not close.
|
From page 56... ...
When the unreliability of NAEP results for individuals is coupled with differences between NAEP and state assessments and differences in the aciministration conditions, the resulting links cannot be expected to yield precise links. These threats to linkage cannot be overcome by simply using the same test takers on both assessments.4 fit is important to keep the magnitude of this problem in the proper perspective, i.e., by comparing the degree of spread of the confidence interval after the linkage with the
|
From page 57... ...
llem Format Distribution Like its content, the format of the NAEP assessment is clerivec3 through a national consensus process anc3 is unlikely to match precisely the format of particular state or commercial tests. The proposed format distribution of the Voluntary National Tests, for example, is 80 percent multiple~choice items anc3 20 percent constructec3-response items, come pared with the approximately 50~50 distribution of NAEP items across these format categories.
|
From page 58... ...
Although corresponding differences were less extreme for other NAEP assessments, they were substantial. These findings suggest that the problem of congruence between the abilities anc3 knowledge suggested by students' performances on the items of a linked test anc3 the abilities anc3 knowledge ciescribec3 by the NAEP achievement levels will be exacerbated to the extent that the item format distribution of a linked test differs markedly from that of NAEP.
|
From page 59... ...
Students' improvements over time on a high stakes statewide test are unlikely to be mirrored by commensurate gains on NAEP unless the statewide accountability test is congruent to its count terpart NAEP assessment in terms of content, format, anc3 skill clemancis. Linking to NAEP Achievement Levels In abolition to the challenges posed for linking presented by the unique features of NAEP's design, the use of NAEP achievement levels as a way of reporting results from a linked test pose other challenges.
|
From page 60... ...
In abolition, the validity of the placement of incliviclual students into NAEP achievement-level categories on the basis of their performances on a linked test couIc3 also be assessed. Again, a public report of the cletailec3 results of a linking study wouIc3 permit comparison of the performance of incliviclual students on the linked test and the descriptions of their know)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.