Skip to main content

Designing a Market Basket for NAEP Summary of a Workshop (2000) / Chapter Skim
Currently Skimming:

6 Using Innovations in Measurement and Reporting: Reporting Percent Correct Scores
Pages 34-39

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 34...
... NAEP's sponsors believe that public understanding could be further improved by releasing a large number of sample items, summarizing performance using percent correct scores, and tying percent correct scores to achievement level descriptions. Since nearly everyone who has passed through the American school system has at one time or another taken a test and received a percent-correct score, most people could be expected to understand scores like 90%, 70%, or 50%.
Read the entire page →
From page 35...
... A simple adding up of total points would give twice as much weight to the barely correct constructed response item as to an entirely correct multiple-choice item. This might be reasonable if the constructed response questions required a level of skill much higher than the multiple-choice questions, such that a score of two on the former actually represented twice as much skill as a score of one on the latter.
Read the entire page →
From page 36...
... Attention should be focused on making progress in educational achievement, and the statistic should enable evaluation of the extent to which students have progressed. DISCONNECT WITH PUBLIC PERCEPTIONS OF "PROFICIENT" One plan for the NAEP percent correct scores is to report them in association with the NAEP achievement levels.
Read the entire page →
From page 37...
... Although associating percent correct scores with an achievement level might increase public understanding of NAEP, many people would still be tempted to convert the scores to letter grades, and their conversions might not be accurate. Richard Colvin offered his perspective as an education reporter for the Los Angeles Times On its own, a percent correct score is only slightly more meaningful than a scale score.
Read the entire page →
From page 38...
... Many discussants maintained that they did not see the benefits of the simpler metric. DOMAIN REFERENCED REPORTING During his comments on technical and measurement considerations, Don McLaughlin, chief scientist for the American Institutes of Research, reminded participants that the desired inferences about student achievement are about the content domain, not about the set of questions on a particular test form.
Read the entire page →
From page 39...
... The expected percents correct can be calculated for any given scale score using IRT methods and the estimated item parameters of the sample of test questions (see Bock et al., 19971. Bock further explained the concept of domain referenced reporting saying: FA]
Read the entire page →

Key Terms

  • constructed response
  • correct scores
  • correct score
  • correct metric
  • market basket

  • referenced reporting
  • domain referenced
  • response questions
  • achievement level
  • public understanding

  • response items
  • response item
  • correct scale
  • scale score
  • reporting results

  • reporting percent
  • average percent
  • increase public
  • top score
  • represent performance

  • estimated item
  • test questions
  • using percent
  • letter grades
  • test takers

  • score using
  • intuitive appeal
  • level category
  • simpler percent
  • naep percent


    • This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
    More information on Chapter Skim is available.