Executive Summary
In November 1997 Congress requested that the National Research Council study the feasibility of developing a scale to compare (“link”) scores from existing commercial and state tests to each other and to the National Assessment of Educational Progress (NAEP) (see P.L. 105-78). The Committee on Equivalency and Linkage of Educational Tests is carrying out the requested study. This report presents the committee 's general findings and conclusions; it will be followed by the committee 's more detailed final report in fall 1998.
The request for the study arose in the context of congressional debate about the proposed voluntary national tests. The committee assumes that the development of a method to link and compare the full array of existing tests is seen as a possible substitute for the development of new national tests. Although our findings and conclusions may be relevant to certain technical issues in the debate over the national tests, we take no position on their overall technical or policy merits.
We have reviewed empirical evidence of two basic types: on the diversity in content, usage, and purposes of educational testing in the United States and on statistical and other technical aspects of creating valid linkages among various types of educational tests.
Our findings from this review raise fundamental questions about the feasibility of linkage as envisioned by the committee's charge. Although it may be technically possible to establish links between tests that are highly similar in design, format, content emphasis, difficulty level, and intended use, those conditions do not apply to the increasingly diverse and complex array of state and commercial tests in the nation 's 50 states and more than 15,000 school districts. In addition, those tests differ substantially from NAEP, which is designed specifically to provide scores for groups of students and not for individual students and which differs in other fundamental ways from almost all state and local tests.
Thus, we have reached two principal conclusions:
Comparing the full array of currently administered commercial and state achievement tests to one another, through the development of a single equivalency or linking scale, is not feasible.
Reporting individual student scores from the full array of state and commercial achievement
tests on the NAEP scale, and transforming individual scores on these various tests and assessment into the NAEP achievement levels is not feasible.
This interim report synthesizes the committee's review of empirical evidence that supports our conclusions. The final report will provide additional detail on the diverse landscape of educational testing in the United States, the special problems of establishing valid linkages with NAEP, and the implications of prior and ongoing research on technical aspects of linkage and equivalency. The final report will also consider the possibility of establishing criteria with which to evaluate the feasibility of linking subsets of the diverse and increasing number of tests currently used in the nation's schools.