Two significant challenges faced NAEP’s sponsors as they revised their policies and procedures. First, the policies and procedures used by states, districts, and schools vary with respect to which students are classified as having disabilities and being English language learners. These variations in policies and procedures affect decisions about (1) who is included in the assessment, (2) who receives accommodations, (3) what accommodations are allowed and provided, and (4) which students’ scores are included in reports. The second major challenge lay in the lack of clear guidance from the available research base regarding the effects of accommodations on test performance. While a considerable body of research exists, findings from these studies are both inconsistent and generally inconclusive (Sireci et al., 2003). The available research is discussed in Chapter 5.
The variation in state policies for handling the assessment of students with disabilities and English language learners is particularly relevant for NAEP. NAEP officials identify the sample of students to be included in the assessment at each participating school, but they must rely on school-level staff to make the decisions about which of the selected students can meaningfully participate and which cannot. That is, selected students whom the local education agency has classified as students with disabilities or English language learners may be excluded from NAEP if school-level staff judge that they cannot meaningfully participate or if they require testing accommodations that NAEP does not permit.1 It is therefore the local education agency that makes the ultimate decisions about which students will participate in NAEP and which accommodations they may be given, using the guidelines provided by NAEP officials combined with their knowledge of the students.
Students are selected to participate in NAEP on the basis of a complex sampling scheme designed to ensure that a nationally representative subset of students is assessed. Variability in state, district, and school policies and procedures for determining which students are considered to have disabilities or to be English language learners, which of these students can meaningfully be assessed, and which accommodations they will receive, all affect the outcome of the sampling.
This variability has implications for the interpretation of NAEP results. First, local decisions about which students will be included will affect the specific samples that are obtained. Second, the accommodations with which students are