Equivalency and Linkage of Educational Tests Interim Report (1998) / Chapter Skim
Currently Skimming:
Findings
Findings
Pages 13-32
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 13... ...
Offthe~shelf commercial tests anc3 tests that are custom clevelopec3 for states are increasingly constructed as mixec3-mociel assessments that contain different types of items, including multiple~choice items anc3 various kinds of open~enciec3 questions for which students construct their own responses by filling in a blank, solving a problem, writing a short answer, writing a longer response, or completing a graph or diagram (see, e.g., Shavelson, Baxter, and Pine, 1992~; Coloraclo, Connecticut, North Carolina, and Maryland are examples of states with mixec3-moclel assessments. Some item types are very useful for testing student recall of factual material (a claim often macle for certain types of multiple choice items)
|
From page 14... ...
DIVERSITY AND MULTIPLICITY OF TESTING PROGRAMS Educational testing in the United States is diverse, reflecting the nation's history of state anc3 local control over education policy. The number anc3 variety of existing state anc3 commercial tests pose formidable barriers to developing a single linking scale.
|
From page 15... ...
The complexity of linking even a small subset of existing tests could quickly render the task infeasible. For example, if the goal is to link just 15 different state assessments, it would be necessary to construct comparisons of more than 100 potential pairs of tests; each pair would require data collection, statistical analyses, anc3 empirical validation (see also Los Angeles County Office of Education, 1997~.
|
From page 16... ...
These moclifications in NAEP make it complicated to maintain stable linkages with state anc3 commercial assessments, which are themselves evolving, anc3 would minimize the validity of inferences from the linkages. TEST USES AND EFFECTS ON TEACHER AND STUDENT BEHAVIOR Many states use assessments for multiple purposes related to educational improvement, such as program evaluation, curriculum planning, school performance reporting, anc3 student diagnosis (U.S.
|
From page 17... ...
Credit by Examination Direct Writing Assessment, Direct Mathematics Assessment Iowa Test of Basic Skills, Test of Achievement Proficiency Stanford Achievement Test 8 Iowa Test of Basic Skills Form K, Test of Achievement Proficiency Custom developed Custom developed No mandated statewide testing program, approximately 99 percent of all districts participate in the Iowa Test of Basic Skills on a voluntary basis Custom developed Custom developed California Achievement Test 5 Custom developed Illinois Goals Assessment Program Indiana StatewideTesting for Educational Progress Plus Kansas Assessment Program (Kansas University Center for Educational Testing and Evaluation) Kentucky Instructional Results Information System Louisiana Educational Assessment Program Maine Educational Assessment (Advanced Systems in Measurement Inc.)
|
From page 18... ...
Grade 11 High School Proficiency Test, Grade 8 Early Warning Test New Mexico High School Competency Exam, Portfolio Writing Assessment, Reading Assessment for Grades 1 and2 Occupational Education Proficiency Examinations, Preliminary Competency Tests, Program Evaluation Tests, Pupil Evaluation Program Tests, Regents Competency Tests, Regents Examination Program, Second Language Proficiency Examinations North Carolina End of Grade Fourth-, Sixth-, Ninth-, and Twelfth-Grade Proficiency Tests Oklahoma Core Curriculum Tests Reading, Writing, and Mathematics Assessment Writing, Reading, and Mathematics Assessment Health Performance Assessment, Mathematics Performance Assessment, Writing Performance Assessment Basic Skills Assessment Program Stanford Achievement Test 9, Custom developed Has a voluntary state assessment program Customized off the shelf Tennessee Comprehensive Assessment Program (TCAP) Achievement Test Grades 2-8, TCAP Competency Graduation Test, TCAP Writing Assessment Grades 4, 8, and 11 Texas Assessment of Academic Skills, Texas End-of-Course Test Core Curriculum Assessment Program New Standards reference exams in math, Portfolio assessment in math and writing Literacy Passport Test, Degrees of Reading Power, Standards of Learning Assessments, Virginia State Assessment Program
|
From page 19... ...
Customized off-the-shelf assessments result from modifications to a commerical test publisher's existing product. SOURCE: Data from 1997 Council of Chief State School Officers Fall State Student Assessment Program Survey 19981.
|
From page 20... ...
20 TABLE 2 Student Testing: Diversity of Purpose EQUIVALENCY AND LINKAGE OF EDUCATIONAL TESTS State Decisions for Students Decisions for Schools Instructional Purposes Alabama High school graduation School performance Student diagnosis or placement; reporting improve instruction; program evaluation Alaska School performance Improve instruction reporting Arizona School performance Student diagnosis or placement; reporting improve instruction; program evaluation Arkansas School performance Student diagnosis or placement; reporting improve instruction; program evaluation California Student diagnosis or Coloradoa placement Student diagnosis or placement Connecticut Student diagnosis or Awards or recognition; Student diagnosis or placement; placement school performance improve instruction; reporting program evaluation Delaware Student diagnosis or placement; improve instruction; program evaluation Florida High school graduation Improve instruction; program evaluation Georgia High school graduation School performance Student diagnosis or placement; reporting improve instruction; program evaluation Hawaii High school graduation Awards or recognition; Student diagnosis or placement; school performance improve instruction; reporting program evaluation Idaho School performance Improve instruction reporting Iowaa Illinois Accreditation Indiana Awards or recognition; Student diagnosis or placement; school performance improve instruction; reporting program evaluation Kansas School performance; Student diagnosis or placement; reporting; accreditation improve instruction; program evaluation Kentucky Awards or recognition Improve instruction; program evaluation Louisiana Student promotion; Awards or recognition; Student diagnosis or placement; high school graduation school performance improve instruction; reporting program evaluation Maine Student diagnosis or Improve instruction; placement program evaluation
|
From page 21... ...
FINDINGS TABLE 2 (Continued) 21 State Decisions for Students Decisions for Schools Instructional Purposes Maryland High school graduation School performance Student diagnosis or placement; Massachusetts reporting; skills guarantee; improve instruction; accreditation program evaluation School performance Improve instruction reporting Michigan Student diagnosis or Awards or recognition;Improve instruction; placement; endorsed school performanceprogram evaluation diploma reporting; accreditation Minnesotan Mississippi High school graduation School performanceStudent diagnosis or placement; reporting; skills guarantee;improve instruction; accreditationprogram evaluation Missouri School performanceImprove instruction; reporting; accreditationprogram evaluation Montana Improve instruction; program evaluation Nebraskan Nevada High school graduation School performanceImprove instruction; reporting; accreditationprogram evaluation New Hampshire Improve instruction; program evaluation New Jersey High school graduation School performanceStudent diagnosis or placement; reporting; accreditationimprove instruction New Mexico High school graduation School performanceStudent diagnosis or placement; reporting; accreditationimprove instruction; program evaluation New York Student diagnosis or School performanceImprove instruction; placement; student reportingprogram evaluation promotion; honors diploma; endorsed diploma: high school -- r - - ~ 0 - - - graduation North Carolina Student diagnosis or Improve instruction; placement; student program evaluation Promotion; high school graduation North Dakota Student diagnosis or Student diagnosis or placement; placement improve instruction; program evaluation Ohio High school graduation Awards or recognition;Improve instruction; school performanceprogram evaluation reporting Oklahoma School performanceStudent diagnosis or placement; reporting; accreditationimprove instruction; program evaluation continued
|
From page 22... ...
Iowa does not administer a statewide assessment. SOURCE: Data from 1996 Council of Chief State School Officers Fall State Student Assessment Program Survey.
|
From page 23... ...
These problems are attributable in part to the tests themselves, but linkage magnifies them anc3 increases the risk of unfair inferences about incliviclual achievement. REPORTING RESULTS IN TERMS OF NAEP ACHIEVEMENT LEVELS Linking other tests to NAEP raises the possibility of reporting incliviclual student scores on state anc3 commercial tests in terms of the NAEP achievement levels.
|
From page 24... ...
24 EQUIVALENCY AND LINKAGE OF EDUCATIONAL TESTS assessment scores and NAEP scores; to potential differences in linking functions by different subgroups; and to other unicientifiec3 sources of measurement error. The committee has not been able to conduct a thorough study of parental and public reaction to this kind of scenario, but we caution that one of the more important putative purposes of linkageprovicling clear and relevant information about the performance of incliviclual students might be severely uncierminec3 by the need to report information which, in order to be faithful to the underlying statistics, must be ambiguous in its meaning.
|
From page 25... ...
Projecting to the To link a A total of 2,824 students from 99 A satisfactory linking was NAEP Scale: comprehensive state schools was tested using 78 items obtained for statewide statistics Results from the achievement test to from a short form of the North as a whole that were accurate North Carolina the NAEP scale for Carolina End-of-Grade Test enough to predict NAEP means End-of-Grade mathematics so that and two blocks of released or quartile distributions with Testing Program the more frequently 1992 NAEP items that were only modest error. (Williams et al., administered state embedded in the test.
|
From page 26... ...
Since none determine whether the state-to comparing the results of the four states used exactly state differences were due to the of four states' the same form of the California different testis) , the moderate assessment programs Achievement Test for their state content alignment, the with the NAEP testing program, state results motivation of the students, or results for those had to be converted to a the nature of the student states.
|
From page 27... ...
Translated IAEP scores into high average IAEP scores. NAEP scores by aligning the Different methods of linking means and standard deviations the IAEP and NAEP can for the two tests.
|
From page 28... ...
Connecticut study. It was possible to develop a Mastery Test, a For a relevant population, link between the MAT 6 and criterion-referenced calibrated the items from the the Connecticut Mastery test closely aligned two instruments in a given Test that accurately with state subject as a single IRT predicted Normal Curve curriculum, and a calibration then used the Equivalent scores for the national "off-the- results to calibrate the tests.
|
From page 29... ...
rather than the equating state academic The standardized test results function for females. performance and the were converted to the NAEP Linking standardized tests to national performance scale using the 1990 data and NAEP using equipercentile levels measured by resulting conversion tables equating procedures is not NAEP.
|
From page 30... ...
standards independently on for students at the advanced the ITBS using the same level, two-thirds or more were achievement-level descriptions classified differently. used to set the NAEP Cut-scores on the ITBS scale, achievement levels.
|
From page 31... ...
district, in all four states. state assessments to Participating states used Students with the same state NAEP; and to different assessments in their assessment score would be demonstrate that it state testing programs.
|
From page 32... ...
TIMSS. Validated the linking functions using data provided by states that participated in both state level NAEP and state-level TIMSS but were not included in the development of the original linking function.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.