Uncommon Measures Equivalence and Linkage Among Educational Tests (1999) / Chapter Skim
Currently Skimming:
2 Technical Aspects of Links
2 Technical Aspects of Links
Pages 20-47
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 20... ...
Because the technical aspects of testing are unfamiliar to many reac3ers of this report, analogies with measuring temperature may be useful. For one thing, like test results, temperatures are reported on scales that are somewhat arbitrary, such as the 32-212 degrees of the Fahrenheit scale and the 0-500 scale for the National Assessment of Educational Progress (NAEP)
|
From page 21... ...
on statistical analyses of the score distributions on the tests or test forms being linked. Various stucly clesigns can be usec!
|
From page 22... ...
In linear equating, the scores on one test are acJjustec3 so that the mean anc3 stanciarc3 deviation of the scores are the same as the mean anc3 stanciarc3 deviation of the other test. Equipercentile equating adjusts the entire score distribution of one test to the entire score distribution of the other, so that scores at the same percentile on two different test forms are equivalent.
|
From page 23... ...
Procedures that match distributions using scores are called statistical moderation links; procedures that match clis' tributions using judgments are called social moderation links. Social moderation generally relies on information external to the testing situa' tion.
|
From page 24... ...
For example, when a new edition of a test is introclucec3 into a product line, a test publisher will establish links between the new edition anc3 the old one so that results obtained from the two tests can be comparec3. For example, CTB/McGraw Hill linked the California Test of Basic Skills with its newer TerraNova test; Harcourt Brace Educational Measurement linked the Stanforc3 Achievement Test 8th Edition with the Stanforc3 Achievement Test 9th Edition; Riverside Publishing linked the {owe Tests of Basic Skills M with earlier editions of the test.
|
From page 25... ...
The proposed study would have enabled a local school district to assess student achievement with any of a list of acceptable commercial tests; the tests would be linked to each other as were the tests in the Anchor Test Study. Results on the achievement tests, aggregated for the students in a school, would be used to rate the performance of each school.
|
From page 26... ...
Strict equating requires that it should be a matter of indifference to a test taker which of two equated test forms is used. The lore of equating holds that test takers who have a choice of two forms anc3 who expect to do poorly shoulcl, if they are willing to take a chance, choose the less reliable form, for there is a greater chance of getting an unrealistically high score by chance on that form, because of its larger margin of error.
|
From page 27... ...
For example, in both Kentucky and North Carolina, where the relationship between the state assessment and NAEP has been examined, there is evidence that scores have improved somewhat more on the state assessments than on NAEP. This effect suggests that as the activities in the classroom become more in line, over time, with the state's intended curriculum, and more aligned with the constructs measured by a state's assessment, the state assessment will become, in effect, easier for each successive cohort of students.
|
From page 28... ...
To link a comprehensive state achievement test to the NAEP scale for mathematics so that the more frequently administered state tests could be used for purposes of monitoring progress of North Carolina students with respect to national achievement standards.
|
From page 29... ...
The following were considered important factors in establishing a strong link: content on the North Carolina Test was closely aligned with state curriculum and NAEP's was not; student performance was affected by the order of the items in their test booklets; motivation or fatigue affected performance for some students.
|
From page 30... ...
To examine the accuracy of linking statewide test results to NAEP by comparing the results of four states' assessment programs with the NAEP results for those states. To pilot test a method for obtaining accurate links between the International Assessment of Educational Progress (IAEP)
|
From page 31... ...
form of the California Achievement r 1 Test tor their state testing program, state results had to be converted to a common scale. This scale was developed by the publisher of the California Achievement Test series.
|
From page 32... ...
Study Purpose Comparing the NAEP Trial State Assessment Results with the IAEP International Results (Beaton and Gonzalez, 1993) Linking to a Large-Scale Assessment: An Empirical Evaluation (Bloxom et al., 1995 To determine how American students compare to foreign students in mathematics, and how well foreign students meet the mathematics standards of the National Assessment Governing Board (NAGB)
|
From page 33... ...
Translated IAEP scores into NAEP scores by aligning the means and standard deviations for the two tests. Transformed the IAEP scores for students in the IAEP samples in each participating country into equivalent NAEP scores.
|
From page 34... ...
Study Purpose The Potential of Criterion Referenced Tests with Projected Norms (Behuniak and Tucker, 1992) To determine if norm-referenced scores could be provided, for the purpose of Chapter 1 program evaluation, by linking the Connecticut Mastery Test, a criterionreferenced test closely aligned with state curriculum, and a national "off-the-shelf" norm-referenced achievement test.
|
From page 35... ...
It was possible to develop a link between the MAT 6 and the Connecticut Mastery Test that accurately predicted Normal Curve Equivalent scores for the MAT 6 from the Connecticut Mastery Test, but no good validity checks were used. The linking function changed somewhat over time and the authors believed that this divergence would continue because teachers were gearing instruction to state standards which were more closely aligned with the Connecticut Mastery Test than the MAT 6.
|
From page 36... ...
Study Purpose Linking Statewide Tests to the To investigate the adequacy of linking National Assessment of statewide standardized test results to the Educational Progress: Stability of National Assessment of Educational Progress Results (Linn and Kiplinger, 1995)
|
From page 37... ...
The standardized test results were converted to the NAEP scale using the 1990 data and resulting conversion tables were then applied to the 1992 data. Examined content match between standardized tests and NAEP and reanalyzed data using subsections of the standardized tests and NAEP.
|
From page 38... ...
Study Purpose Using Performance Standards to Link Statewide Achievement Results to NAEP (Waltman, 1997) To investigate the comparability of performance standards obtained by using both statistical and social moderation to link NAEP standards to the Iowa Tests of Basic Skills (ITBS)
|
From page 39... ...
A socially moderated linkage was obtained by setting standards independently on the ITBS using the same achievement-level descriptions used to set the NAEP achievement levels. An equipercentile procedure was used to establish a statistically moderated link.
|
From page 40... ...
To address the need for clear, rigorous standards for linkage; to provide the foundation for developing practical guidelines for states to use in linking state assessments to NAEP; and to demonstrate that it is important for educational policy makers to be aware that linkages that support one use may not be valid for another. To compare the Maryland State Performance Assessment Program (MSPAP)
|
From page 41... ...
Linked scores had a 95 percent confidence interval of almost 2.0 standard deviations, which were not sufficiently accurate to permit reporting individual student proficiency on NAEP based on the state assessment score. Links differed noticeably by minority status and school district, in all four states.
|
From page 42... ...
. will result in inaccurate predictions of what NAEP performance wouIcl be based on the results of the later state assessments.
|
From page 43... ...
The link was established using reported results from the 1995 administration of TIMSS in the United States and the 1996 NAEP and matching characteristics of the score distributions for the two assessments. Validated the linking functions using data provided by states that participated in both state-level NAEP and state-level TIMSS but were not included in the development of the original linking function.
|
From page 44... ...
An example might be: "We want to compare mean student performance in two districts using different assessments. We would like the stanciarc3 error of the district B mean projected onto the scale used by district A to be less than .05 stanciarc3 deviations." Another example might be: "We want to know whether an incliviclual student taking assessment B is achieving at or above the proficient level on NAEP, and we want this classification to be correct for at least 95 percent of the students who take assessment B." i]
|
From page 45... ...
The power to detect key differences anc3 the stanciarc3 error of relevant estimates should be cieter' mined in advance. Once a linkage study has been clesignec3 anc3 the data have been collected, several analyses should be performed: · Examine the data for outliers that could distort results using scatterplots or other means.
|
From page 46... ...
If the relationship is linear anc3 the score distributions have approximately the same shape, linear linking can be used; otherwise, equiperentile linking is preferred. If the projection method is chosen, ordinary regression methods can be used.
|
From page 47... ...
Even if two tests appear to measure similar things to content experts and "pass" a careful statistical evaluation, it is important to explicitly examine the stability of a linkage across the important sub' groups of test takers and to check its stability over time.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.