Grading the Nation's Report Card Research from the Evaluation of NAEP (2000) / Chapter Skim
Currently Skimming:
2 Families of Items in the NAEP Mathematics Assessment
2 Families of Items in the NAEP Mathematics Assessment
Pages 5-43
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 5... ...
Item families can serve as an illustration of how to more fully understand and describe levels of students' understanding by examining students' responses across a set of related items. The paper presents a brief overview of the NAEP mathematics framework developed for the 1996 assessment, the idea of families of items and how they have appeared in previous NAEP assessments, a rationale for the development of a sample family of items around the topic of number patterns at the fourth-grade level, and the family of items itself based on released NAEP pattern items and other items developed for this paper.
|
From page 6... ...
Families of Items in NAEP The 1996 NAEP mathematics framework document recommends that the assessment include sets of related tasks, called families of items, to "measure the breadth and depth of student knowledge in mathematics" (National Assessment Governing Board, 1994:5~. The framework describes two types of item families: a vertical family and a horizontal family.
|
From page 8... ...
An example of a scaffolded item pair from the 1992 NAEP mathematics assessment appears in Box 2-2. This item pair, administered to students in the grade 4 sample and classified by NAEP assessment developers in the content strand Algebra and Functions, assessed students' understanding of a number relationship involving an arithmetic operation.
|
From page 10... ...
The two kinds of related items just discussed, however, by no means match the definition of families of items in NAEP. And perhaps there is no reason to expect that they would, given that those related items appeared on the 1992 NAEP mathematics assessment and given that the directive about the assessment including item families was first prescribed in the 1996 version of the mathematics framework.
|
From page 11... ...
In recent NAEP mathematics assessments, then, there has been little implementation of the notion of families of items, although the 1996 framework document makes a case for their inclusion. What might a family of items look like?
|
From page 12... ...
The first part contains a discussion of the importance of number patterns in the elementary mathematics curriculum and of the appropriateness of this topic as the basis for an item family in NAEP. The purpose of the next part is to justify the use of released NAEP mathematics items as the basis for the family of items and the limitations inherent in this method.
|
From page 13... ...
There is evidence that patterns, particularly numerical patterns, were a topic assessed in NAEP. Within the Algebra and Functions content strand at grade 4, recent NAEP mathematics assessments have included items that assess informal algebraic thinking through patterns and relationships.
|
From page 14... ...
However, because released pattern items and related performance data on those items were available from NAEP, it seemed reasonable to use these single items along with appropriate supplemental items to form a sample family of items. The advantage of this method of constructing a sample family of items is that the sample family uses items that have already appeared on a NAEP assessment, and we know how students performed on them.
|
From page 15... ...
also involve nonconstant increases, results from those items can provide additional evidence about this misunderstanding. Some might argue that such information about students' understanding and misunderstandings of numerical patterns is already available from the NAEP mathematics assessment results.
|
From page 16... ...
ITEM 6 Nonconstant increase in a complex, problem solving setting FIGI~E 2-1 Progression of concepts in the number pattern family of items in Appendix 2B.
|
From page 17... ...
CONCLUDING COMMENTS The recommendation in the 1996 framework document about including families of items represents a positive direction for future NAEP mathematics assessments. The inclusion of item families can increase NAEP's potential to provide important information about the depth of students' knowledge in a particular content strand and across strands.
|
From page 18... ...
It is hoped that future NAEP assessments will include item families that reflect the intentions for the special sets of items in the 1996 NAEP mathematics framework document and that NAEP reports will take full advantage of the information provided by these related item sets. REFERENCES Armstrong, B.E.
|
From page 19... ...
In Results from the Seventh press Mathematics Assessment of the National Assessment of Educational Progress, E.A. Silver and P.A.
|
From page 20... ...
Stylianou in Students' performance on extended constructed response tasks. In Results from the press Seventh Mathematics Assessment of the National Assessment of Educational Progress, E.A.
|
From page 21... ...
r~ Loot A.... ~ 1: |
From page 22... ...
Your class is planing to have a Butterfly Booth. Your class has a lot to do to get ready far the Science Fair.
|
From page 23... ...
Use your ruler to measure the wingspans of the other two butterflies on the sheet, the Black Swallowtail butterfly and the Common Blue butterfly, to the nearest centimeter. Black Swallowtail Wingspan: centimeters Common Blue Wingspan: centuneters Did you use the calculator on this questions O Yes O No ~3 - l':
|
From page 24... ...
What is the greatest number of Common Blue butterflies that can be stored in the case below? (When you put butterflies in the case, you can't stack ~em.
|
From page 25... ...
Hov may stole cases Would you need to store 28 Common Blue butters Isle: Use ~g~ wordy or Ibex to ~1^ boa you got your asker. Did you use Me cunctator ~ Is Despond O As 23 No
|
From page 26... ...
26 FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT 4. The children who visit your booth are going to build models of butterflies.
|
From page 27... ...
PATRICIA ANN KENNEY 5. A fourth-~ade class needs ~ leaves each day to feed its 2 caterpillars.
|
From page 28... ...
Your class has decided to have a banner that will be 130 centuneters long. This banner will have a repeating pattern of one Monarch butterfly followed by two Black Swallowtail butterflies, as shown here.
|
From page 29... ...
PATRICIA ANN KENNEY If you need more room for your work, use the space below. Did you use the calculator on this question?
|
From page 31... ...
would best serve the purpose of determining the floor effect. The other versions are presented here as additional examples of simple pattern items based on a single-digit, constant increase between consecutive numbers.
|
From page 33... ...
In particular, the original NAEP item asked for three numbers in the pattern based on a given constant increase of 50 students. The results showed that just over half the fourth-grade students gave completely correct responses.
|
From page 36... ...
Perhaps both versions could be pilot-tested to answer this question, with only one version included in the item family.
|
From page 37... ...
Because the notion of nonconstant increases is likely to be difficulty for some fourth-grade students, basing the nonconstant increases on the set of odd numbers could make the item more accessible. Also, the operation used to create the pattern is again simple addition.
|
From page 40... ...
As for Item 4 in the family, this item has the potential to provide additional evidence of the misunderstanding about nonconstant increases. Despite the attempt at scaffolding, students could still change to a constant increase and answer 25 pounds or some other number based on a constant increase in weight.
|
From page 43... ...
Solving the problem requires students to identify the rule that underlies the pattern of nonconstant increases. The version for the item family begins with an introductory question about the number of dots in the fourth step as a way to introduce students to the problem.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.