Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 5
Families of Items in the
NAEP Mathematics Assessment
Patricia Ann Kenney
This paper discusses families of items in the National Assessment of Educa-
tional Progress (NAEP) mathematics assessment and presents a sample family of
items for grade 4. Item families can serve as an illustration of how to more fully
understand and describe levels of students' understanding by examining students'
responses across a set of related items. The paper presents a brief overview of the
NAEP mathematics framework developed for the 1996 assessment, the idea of
families of items and how they have appeared in previous NAEP assessments, a
rationale for the development of a sample family of items around the topic of
number patterns at the fourth-grade level, and the family of items itself based on
released NAEP pattern items and other items developed for this paper.
OVERVIEW OF THE 1996 NAEP MATHEMATICS FRAMEWORK
The 1996 NAEP mathematics assessment used a framework (National
Assessment Governing Board, 1994) that was influenced by ideas presented in
Curriculum and Evaluation Standards for School Mathematics of the National
Council of Teachers of Mathematics (NCTM, 1989~. The framework sampled
the content domain of mathematics using five strands: Number Sense, Proper-
ties, and Operations; Measurement; Geometry and Spatial Sense; Data Analysis,
Statistics, and Probability; and Algebra and Functions. Additional dimensions of
the framework included mathematical abilities (Conceptual Understanding,
Procedural Knowledge, Problem Solving) and mathematical power (Reasoning,
Connections, Communication). In addition to defining the content, ability, and
power dimensions along which to assess students' knowledge and understanding
OCR for page 6
6
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
of mathematics, the NAEP framework document included recommendations con-
cerning other aspects of the assessment, such as the distribution of items across
the content strands for grades 4, 8, and 12; the use of calculators and
manipulatives; and design considerations for special sets of items to appear on
the test. One of these special sets of items, called "families of items," is described
in more detail below.
Families of Items in NAEP
The 1996 NAEP mathematics framework document recommends that the
assessment include sets of related tasks, called families of items, to "measure the
breadth and depth of student knowledge in mathematics" (National Assessment
Governing Board, 1994:5~. The framework describes two types of item families:
a vertical family and a horizontal family. A vertical family includes items or
tasks that measure students' understanding of a single important mathematics
concept in a content strand (e.g., numerical patterns in algebra) but at different
levels, such as giving a definition, applying the concept in both familiar and
novel settings, and generalizing knowledge about the concept to represent a new
level of understanding. A horizontal family of items involves assessment of
students' understanding of a concept or principle across the various content
strands in NAEP within a grade level or across grade levels. For example, the
concept of proportionality can be assessed in a variety of content strands, such as
number properties and operations, measurement, geometry, probability, and alge-
bra. The framework also suggests that a family of items could be related through
a common mathematical or real-world context that serves as a rich problem
setting for the items.
Although the notion of item families was first articulated in the 1996 math-
ematics framework document, sets of related items have appeared in prior NAEP
assessments. However, these sets exhibited few characteristics of either horizon-
tal or vertical item families. Instead, the relationships between items involved
such features as a common stimulus (e.g., a table, chart, or graph), a rudimentary
form of scaffolding in which one item draws on information from the preceding
iambs), or a common context. Each of these relationships is discussed next.
The item set in Box 2-1 is an example of the first kind of relatedness in that
the items share a common stimulus. These three items were administered to
students in the grade 4 sample in 1992 and were classified by NAEP assessment
developers in the content strand Numbers and Operations (as the strand was
called then). While working on these items, students were permitted to use
simple four-function calculators. A table that displayed the number of points
earned from two school events for each of three classes served as the common
stimulus for the items. The first item asked for a comparison based on adding the
points earned by each class, comparing the total points per class, and selecting the
class that earned the most points; the second item required students to obtain the
OCR for page 7
PATRICIA ANN KENNEY
7
OCR for page 8
8
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
correct total number of points from the Mathathon column; and the third item
involved a within-class comparison concerning the number of points earned for
each event.
Because the items in Box 2-1 were related in only a cursory way (i.e., a table
was used as a common stimulus), looking at performance across them reveals
little about students' ability to select appropriate data from a chart and their use of
arithmetic operations on those data. Performance results were not very different
across items, and the results appeared to be related only to the item type; that is,
students' performance was higher on the multiple-choice item (66 percent cor-
rect) than on either of the short constructed-response items (52 and 49 percent,
respectively). It is well documented that, on the NAEP mathematics assessment,
student performance is higher on multiple-choice items than on constructed-
response items (Dossey et al.,l993; Silver et al., in press). With respect to the
item set in Box 2-1, it is reasonable to suspect that guessing could be contributing
to the higher performance on the first question (a multiple-choice item), but
because the items are not related in other ways it is difficult to interpret perfor-
mance among them on the basis of important mathematical concepts and
. .
pnnclp es.
Another way that sets of mathematics items in NAEP were related involved
an attempt to scaffold items; that is, a particular item is based on important and
purposeful ways on one or more items that precede it. For example, the answer
from the first item is used again in later items, or the first item presents a simple
concept that is elaborated on in the items that follow (e.g., the "superitems" as
described in Collis et al., 1986, and Wilson and Chavarria,1993~. An example of
a scaffolded item pair from the 1992 NAEP mathematics assessment appears in
Box 2-2. This item pair, administered to students in the grade 4 sample and
classified by NAEP assessment developers in the content strand Algebra and
Functions, assessed students' understanding of a number relationship involving
an arithmetic operation. In the first item, students were asked to identify the rule
used to transform the numbers in column A to those in column B. and its com-
panion item required them to use that same rule to generate another number in the
pattern. Thus, the pair illustrates a simple form of scaffolding in which the first
item provides important information to be used in the second item.
NAEP results show that 42 percent of the fourth-grade students chose the
correct rule (divide the number in column A by 4) in the first item. However, as
expected for constructed-response items, performance was lower on item 2: only
24 percent of the fourth-grade students found the correct value of 30. Given that
the items were related, it is reasonable to think that students would use the rule
they chose in the first item in order to answer the second item, but NAEP results
show that this was not the case. For example, of the students who selected the
correct rule in item 1, only about 50 percent also answered item 2 correctly, with
about another 40 percent giving numerical values that were incorrect and the
OCR for page 9
PATRICIA ANN KENNEY
9
OCR for page 10
10
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
remaining 10 percent leaving the item blank. Unfortunately, NAEP does not
provide additional information on the kinds of incorrect answers students pro-
duced for item 2. Thus, for example, there is no information on ways in which
students could have implemented the correct rule from item 1 to get an incorrect
answer in item 2. We can only speculate on some of these ways. For example,
did students make an error in dividing 120 by 4 and give an answer of 3 instead
of 30? Did students arbitrarily change the operation from division to multiplica-
tion, giving an answer of 480 (120 x 4~? Did students completely ignore their
correct choice of the rule in item 1 and instead perform an arbitrary operation
using an arbitrary divisor, such as dividing 120 by 10 for an answer of 12? Based
on the way NAEP results were reported for the item pair, we have no answers to
these and other questions about the kinds of errors students made.
The two kinds of related items just discussed, however, by no means match
the definition of families of items in NAEP. And perhaps there is no reason to
expect that they would, given that those related items appeared on the 1992
NAEP mathematics assessment and given that the directive about the assessment
including item families was first prescribed in the 1996 version of the mathematics
framework. Yet even in the 1996 NAEP there were few, if any, true families of
items that match the horizontal or vertical description in the mathematics frame-
work document. As was the case in 1992, the 1996 assessment included related
sets of items, but the majority of these sets included only two or three items that
were related by the common stimulus such as a graph or table or that were
scaffolded in limited ways. The exception to this was the sets of items related by
context and referred to as "theme blocks" (Hawkins et al., 1999~.
In the 1996 mathematics assessment the theme blocks were the operationali-
zation of the framework recommendation concerning contextually related sets of
tasks. According to a recent NAEP report, the questions in each theme block
"related to some aspect of a rich problem setting that served as a unifying theme
for the entire block" (Reese et al., 1997:79~. The theme blocks were designed as
a special study at the national level in NAEP and as such the results were reported
separately from the main NAEP assessment. For the 1996 assessment there were
five different theme blocks, two at each grade level with one block common to
grades 8 and 12. Each block, containing 6 to 10 items, was administered to a
special sample of students at each grade level. The item formats included
multiple-choice questions and short and extended constructed-response questions
developed around important mathematical ideas set in a real-world context. Each
student was allotted 30 minutes to complete the questions in the theme block.
(To complete the 45-minute testing time allocated to the cognitive items, a student
took another block of items consisting entirely of multiple-choice questions that
had a 15-minute time limit.) While working on the items in the theme block,
OCR for page 11
PATRICIA ANN KENNEY
11
students were permitted to use calculators and were provided with other materials
such as rulers, protractors, and models.
A released grade 4 theme block built around the context of a science fair
project on butterflies appears in Appendix 2A.1 While working on the six
constructed-response items in this block, students had access to calculators and
other materials (butterfly information sheet, butterfly cutouts, ruler). A close
look at the six constructed-response items that make up the block reveals that,
although all items involved the assessment of important mathematical concepts
and all were set in the context of the butterfly display, the items themselves do not
satisfy the definition of either a vertical or a horizontal family of items. Instead,
the items individually assess mathematical topics that ranged from geometry to
measurement to number concepts to proportions to patterns, with few if any
connections between the items. The only obvious connection between items was
that students had to use the wingspan measurements from the second item in
order to answer the sixth item in the block.
The performance results for the six theme block items, shown
in Table 2-1,
cannot be interpreted in any connected way that has to do with the mathematics in
the items. Moreover, the difficulty in making connections between performance
on the items is exacerbated by differences in the number of score levels. With
respect to the only two items that were connected, the results suggest that,
although students could measure wingspan (40 percent correct on item 2), they
had difficulty using these measurements as part of a complex problem (1 percent
correct on item 6~. However, these results were not presented so such direct
comparisons can be made between students' performance on those two items.
That is, we do not know the percentage of students who, having obtained the
correct wingspan measurement in item 2, provided correct answers accompanied
by reasonable work in item 6.
In recent NAEP mathematics assessments, then, there has been little imple-
mentation of the notion of families of items, although the 1996 framework docu-
ment makes a case for their inclusion. What might a family of items look like?
What kind of information would likely be generated about students' understanding
of a particular concept presented at varying levels of complexity in a mathematical
content area or their understanding of a concept presented across mathematical
content areas? The next section of this paper investigates these questions in light
of a suggested family of items structured around number patterns, an informal
algebra topic for grade 4.
1A discussion of the grade 4 released theme block (as well as the released theme blocks from the
grade 8 and grade 12 NAEP mathematics assessment in 1996) appears in Kenney and Lindquist (in
press).
OCR for page 12
2
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
TABLE 2-1 Performance on Items in the Butterflies Theme Block: Grade 4
Item Description
% Scoring
at Highest
Levela
1. Draw four missing markings on pictures of two butterflies to make each
butterfly symmetrical.
2. Obtain two correct measurements in centimeters for the wingspans of two
butterfly models.
28
40
3. Determine the greatest number of butterflies that can be stored in a case 5
and the number of cases needed to hold 28 butterflies; show how your
answer was obtained.
4. Determine the maximum number of butterfly models that can be made from 3
a given number of parts (wings, bodies, antennae); show or explain how
your answer was obtained.
5. Given that two caterpillars eat five leaves per day, determine the number
of leaves needed each day to feed 12 caterpillars; show how your answer
was obtained.
6. Find the number of each type of butterfly needed to create a repeating
pattern on a banner that is 130 centimeters long; show how your answer
was obtained. (Note: measurements obtained in item 2 are needed in
this problem.)
1
Note: All items in this block were either short or extended constructed-response items.
aBecause of differences in the number of levels in scoring guides, the highest score level
varied among the items from three to five levels. The highest score levels are as follows:
"satisfactory" for items 1, 4, and 6 (based on four score levels); "extended" for items 2 and 3
(based on five score levels); and "complete" for item 5 (based on three score levels).
A Sample Family of Items Based on Number Patterns at Grade 4
This section of the paper is divided into three parts. The first part contains a
discussion of the importance of number patterns in the elementary mathematics
curriculum and of the appropriateness of this topic as the basis for an item family
in NAEP. The purpose of the next part is to justify the use of released NAEP
mathematics items as the basis for the family of items and the limitations inherent
in this method. The concluding part contains the sample item family, an explana-
tion of its structure, and the kinds of information that might be obtained from
looking at students' performance across the items.
OCR for page 13
PATRICIA ANN KENNEY
Number Patterns in Elementary Mathematics and in NAEP
13
The topic of patterns and relationships, particularly number patterns in
elementary school mathematics, is an appropriate and important content topic
around which to create a family of items. Exploring patterns helps students in the
early grades develop the ability to think algebraically (Armstrong, 1995; National
Council of Teachers of Mathematics [NCTMi, 1989; Reys et al., 1995~. In fact,
the NCTM Algebra Working Group realized that "children can develop algebraic
concepts at an early age" (NCTM, 1994:5) and suggested that working with
patterns of shapes and numbers helps to build the foundation for algebraic think-
ing needed in later grades.
In addition to information about the importance of patterns and relationships
in the elementary mathematics curriculum, there exists a body of research that
examines elementary students' performance on various types of numerical series
and pattern items (e.g., Holzman et al., 1982, 1983; Pellegrino and Glaser,1982~.
Included in these studies is important information on the nature of pattern items
and possible factors that affect their ease or difficulty. For example, Holzman et
al. (1982) report that the degree of difficulty of a numerical pattern item depends
on such factors as the operations used to generate the pattern (i.e., incrementing
operations [addition, multiplication] are easier than decrementing operations [sub-
traction and divisional; the number of operations used to generate subsequent
numbers in the pattern (i.e., patterns based on one operation [e.g., add 41 are
easier than those based on two operations [e.g., first multiply by 2 and then add
11~; and the magnitude of the numbers used (i.e., patterns based on increments or
decrements of small numbers [5 or less] are easier than those based on larger
numbers [11 or greater]~.
The importance of patterns and relationships in the elementary mathematics
curriculum is further supported by the fact that the NAEP mathematics frame-
work included this topic at grade 4. There is evidence that patterns, particularly
numerical patterns, were a topic assessed in NAEP. Within the Algebra and
Functions content strand at grade 4, recent NAEP mathematics assessments have
included items that assess informal algebraic thinking through patterns and rela-
tionships. In 1992 about 10 percent of the items on the grade 4 assessment dealt
with informal algebra, and most of those items involved patterns of figures,
symbols, or numbers (Kenney and Silver, 1997~. The 1996 assessment had about
the same percentage of items at grade 4, and most of them also involved a variety
of patterns, including number patterns. Thus, given the importance of patterns in
both the elementary mathematics curriculum and recent NAEP mathematics
assessments as well as a research base that speaks to characteristics of pattern
items that can affect performance, selecting this as the topic for a family of items
was both reasonable and appropriate.
OCR for page 14
4
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
Released NAEP Pattern Items as the Basis for an Item Family
Not only did numerical pattern items appear on recent NAEP mathematics
assessments, but also some of those items were released to the public. These
released pattern items were not part of an item family in the assessment; instead,
they appeared as single items in various parts of the assessment. However,
because released pattern items and related performance data on those items were
available from NAEP, it seemed reasonable to use these single items along with
appropriate supplemental items to form a sample family of items. The advantage
of this method of constructing a sample family of items is that the sample family
uses items that have already appeared on a NAEP assessment, and we know how
students performed on them. In addition, using existing items enabled the item
family to be created with minimal time devoted to the development of original
items. Finally, the existing items could be evaluated not only according to
student performance but also using findings from published research studies
(e.g., Holzman et al., 1982) and then organized into a family in a somewhat
hierarchical fashion reflecting the level of cognitive demand.
However, this method of creating a sample item family using existing NAEP
items carries with it potentially serious limitations. In particular, taking items
developed individually and putting them together as a set post hoc has a degree of
artificiality. The ideal method that should be used to create an item family is to
begin with a particular topic and information based on research about students'
understanding of that topic, build the family of items, and then validate the
structure of the set by administering it to students and examining performance
results. Obviously, this method was not used in this paper. As a consequence,
the family of items presented herein should be considered as an illustrative but
very modest example of what such a family might look like, with the understand-
ing that better families of items will be created for future NAEP assessments. It
is hoped that the simple example in this paper will be used as the basis for further
thought about and discussion of important features of families of items in NAEP.
A Proposed Item Family Based on Numerical Patterns
The six items in Appendix 2B constitute a proposed family of items built
around the topic of numerical patterns. The set was developed according to the
following guidelines:
· Each item in the set involves an increasing pattern of numbers based on a
particular rule that governs the growth. In the elementary mathematics curricu-
lum, these kinds of patterns are often referred to as "growing patterns" (Reys et
al., 1995; NCTM, 1992~. In some items the pattern is based on constant increases
between consecutive terms, and in others the pattern is based on nonconstant
Increases.
OCR for page 15
PATRICIA ANN KENNEY
15
· The set represents an attempt to organize the items from the easiest to the
most difficult. In the case of released NAEP items, performance data were used
to determine the level of difficulty (e.g., an item for which performance was 75
percent correct was "easier" than an item for which performance was 53 percent
correct). For items created especially for the set, the degree of difficulty was
based either on a rational analysis of mathematical concepts or on factors found
to influence the difficulty of number pattern item, such as kind of operation,
number of operations, and magnitude of numbers, as presented in published
research articles (e.g., Holzman et al., 1982~.
.
In some cases the items are presented in two formats: multiple choice and
constructed response. Given that the NAEP has always advocated a judicious
blend of multiple-choice and constructed-response items, presenting an alterna-
tive format for items (especially those developed specifically for the sample item
family) seemed to be appropriate. However, because of the performance differ-
ences in NAEP concerning lower percent-correct results on constructed-response
items, this could affect the hierarchy of items (easiest to more difficult) in the
sample set.
The source is given for each item in Appendix 2B (e.g., a released NAEP
item; an item created for the set). Following each item is a rationale for why the
item was included in the item family and the kind of information that could be
obtained from performance results. Figure 2-1 summarizes the concepts and
progression of items in the sample family. Performance on these related items
could provide insights into students' understanding of numerical patterns and
where that understanding falters. For example, performance results could show
that most fourth-grade students can work with patterns involving constant
increases between the terms (items 1, 2, and 3), but performance levels could be
lower for items involving patterns based on nonconstant increases (items 4 and
5), especially for complex problems (item 6~. Performance results could also
provide information on misunderstandings that students have about number pat-
terns, with the same misunderstandings possibly occurring across items in the
family. For example, some students may expect a number pattern always to have
a constant increase between contiguous numbers. In this case, when faced with a
pattern containing nonconstant increases such as the number pattern in item 4
(i.e., 8, 9, 12, 17, 24, 33, 44, . . . [increase based on the set of odd numbers]),
those students could reason that, because the increase between the last two
numbers shown in the pattern is 11, 55 (44 + 11) is the next number in the pattern.
Because the next two items in the family (items 5 and 6) also involve nonconstant
increases, results from those items can provide additional evidence about this
misunderstanding.
Some might argue that such information about students' understanding and
misunderstandings of numerical patterns is already available from the NAEP
mathematics assessment results. All that would be needed would be to analyze
OCR for page 33
PATRICIA ANN KENNEY
33
Rationale for Item 2
A version of the 1992 NAEP item would appear next in the item family.
Although the pattern of numbers is still constantly increasing and although the
operation is still addition, the increase itself is a double-digit number greater than
11, which adds to the difficulty of the item for elementary school students
(Holzman et al., 1982~. Despite the fact that the increase is a multiple of both 5
and 10 and that the increase is given in the problem, this item is considered as a
"step up" from the first problem because of its constructed-response format and
the need to work with a pattern involving a two-digit number increase.
The NAEP version, however, should be modified so that more information
can be obtained from student responses. In particular, the original NAEP item
asked for three numbers in the pattern based on a given constant increase of 50
students. The results showed that just over half the fourth-grade students gave
completely correct responses. However, the results did not reveal which of the
three numbers was the most difficult to obtain. The version proposed for the item
family could remedy this situation by providing information on whether the
students understood that the enrollment increases in the first year by 50 students,
and then by that same number in each of the next two years.
OCR for page 34
34
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
OCR for page 35
PATRICIA ANN KENNEY
35
OCR for page 36
36
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
Rationale for Item 3
The third item in the family represents a transition from patterns based on
addition of a constant to patterns based on multiplicative models. This item
would reveal whether students understand that patterns can be based on arith-
metic operations other than addition, for example, multiplication or division.
The original NAEP item was discussed in an earlier section of this paper, and
it was noted there that the results did not completely reveal the degree of consis-
tency between the rule selected by students and whether they used that rule to
answer the second question. Using one of the revised versions, both of which are
constructed-response questions, perhaps we can better relate the students' de-
scription of the rule in part 1 and their use (or misuse) of that rule in part 2. For
example, in version 1 for students who answered "Divide the number in Column
A by 4," but who wrote "3" in Column B in the second part of the problem, we
could more accurately attribute this incorrect answer to a place-value error or
perhaps to carelessness. For other students who wrote the correct rule, but who
answered "480" in the second part, it is likely that their error involved multiply-
ing instead of dividing.
With respect to the two versions suggested for the family, one version might
be preferable over the other depending on whether the multiplicative model (Ver-
sion 2) or the division model (Version 1) is more easily recognized by students.
Perhaps both versions could be pilot-tested to answer this question, with only one
version included in the item family.
OCR for page 37
PATRICIA ANN KENNEY
Rationale for Item 4
The fourth item, presented in two versions (multiple choice and constructed
response), serves as a transition between numerical patterns based on constant
increases to those based on nonconstant increases. In an important way,
nonconstant increases are in themselves a pattern within a pattern. For example,
the pattern in the item (8, 9, 12, 17, 24, 33, 44, . . . ~ also has a pattern of increases
(1, 3, 5, 7, 9, . . . ~ the set of odd numbers. Because the notion of nonconstant
increases is likely to be difficulty for some fourth-grade students, basing the
nonconstant increases on the set of odd numbers could make the item more
accessible. Also, the operation used to create the pattern is again simple addition.
As noted earlier in the paper, this item and the ones that follow could provide
evidence about an important misunderstanding about patterns; that is, the notion
that all patterns (even those that are based on nonconstant differences) contain
pairs of numbers that have a constant difference. For item 4 in the family, it is
likely that some students could choose B (55) for the multiple-choice version or
write 31 and 38 as the next two numbers in the pattern for the constructed-
response version. In both cases, such responses show evidence of changing the
nonconstant increase to a constant increase based on the difference between the
last two numbers shown in the pattern.
OCR for page 38
38
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
OCR for page 39
PATRICIA ANN KENNEY
39
OCR for page 40
40
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
Rationale for Item 5
This item within the family has the potential to be the most difficult question
to this point. Results from the original NAEP version of the item showed that
about the same percent of students selected choice C (25 pounds) as selected the
correct choice D (24 pounds). This error pattern shows that some students may
expect a number pattern to have a constant difference between some contiguous
numbers: that is, in the puppy problem, students retained the 3-pound weight
gain between the third and fourth months and used it as a constant to calculate the
weight at 5 months (22 + 3 = 25~. Also, the high omitted and not-reached rates
suggest that some fourth-grade students thought that this problem was so difficult
that they did not even try to answer it.
The version proposed for the item family attempts to make the question more
accessible to students. It is scaffolded so that students must identify the first two
nonconstant differences between the weights, in the hope that students will more
easily recognize that the weight gains are decreasing between consecutive months.
The final question involves a transition from the nonconstant differences to the
actual weight of the puppy.
As for Item 4 in the family, this item has the potential to provide additional
evidence of the misunderstanding about nonconstant increases. Despite the at-
tempt at scaffolding, students could still change to a constant increase and answer
25 pounds or some other number based on a constant increase in weight.
OCR for page 41
PATRICIA ANN KENNEY
4
OCR for page 42
42
FAMILIES OF ITEMS IN THE NAEP MATHEMATICS ASSESSMENT
OCR for page 43
PATRICIA ANN KENNEY
43
Rationale for Item 6
The original NAEP item, called Marcy's Dot Pattern in NAEP reports (e.g.,
Dossey et al., 1993), was administered to students in the 1992 eighth-grade
sample as an extended constructed-response question in the Algebra and Func-
tions content strand. As shown by the performance results, this question was
difficult for the eighth-grade students: only 6 percent produced a response that
was scored as satisfactory or extended. However, the fact that the item was last
in an item block with previous items having little or no connection to number
patterns could have affected performance levels. How would students have
performed if this question, or an appropriate version thereof, appeared in a family
of items devoted to number patterns?
Given the structure of the family of items describe thus far, it seemed reason-
able to think about including an adaptation of the Marcy's Dot Pattern as the
culminating item in the family. As the culminating item, it has characteristics
based on work done on the previous items. For example, one way to view the
pattern in this task is that the pattern involves nonconstant increases between the
number of dots in each step. Solving the problem requires students to identify the
rule that underlies the pattern of nonconstant increases. The version for the item
family begins with an introductory question about the number of dots in the
fourth step as a way to introduce students to the problem. Here, it would be
reasonable for students to draw the fourth figure so that they can better under-
stand the pattern. The next part of the problem is similar to that given to students
in the eighth-grade sample, but the steps are reduced from the 20th step to the
10th step. This last decision needs careful thought, however, because drawing 7
more sets of dots is more accessible than drawing 17 more sets.
Representative terms from entire chapter:
mathematics assessment
