. "4 Factors That Affect the Accuracy of NAEP's Estimates of Achievement." Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessment. Washington, DC: The National Academies Press, 2004.
The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments
FINDING 4-2: The target population for NAEP assessments is not clearly defined. It is not clear to whom the results are intended to generalize.
FINDING 4-3: The extent to which the demographic estimates in NAEP reports compare with the actual proportions of students with disabilities and English language learners is not known; in part, this is the result of deficiencies in the national and state-level demographic data that are available.
Our review of these circumstances leads us to make the following recommendations:
RECOMMENDATION 4-1: NAEP officials should:
review the criteria for inclusion and accommodation of students with disabilities and English language learners in NAEP in light of federal guidelines;
clarify, elaborate, and revise their criteria as needed; and
standardize the implementation of these criteria at the school level.
RECOMMENDATION 4-2: NAEP officials should work with state assessment directors to review the policies regarding inclusion and accommodation in NAEP assessments and work toward greater consistency between NAEP and state assessment procedures.
RECOMMENDATION 4-3: NAEP should more clearly define the characteristics of the population of students to whom the results are intended to generalize. This definition should serve as a guide for decision making and the formulation of regulations regarding inclusion, exclusion, and reporting.
RECOMMENDATION 4-4: NAEP officials should evaluate the extent to which their estimates of the percentages of students with disabilities and English language learners in a state are comparable to similar data collected and reported by states to the extent feasible given the data that are available. Differences should be investigated to determine the causes.
In addition to those four recommendations to NAEP officials, we also recommend that:
RECOMMENDATION 4-5: Efforts should be made to improve the availability of data about students with disabilities and English language learners. State-level data are needed that report the total number of English language learners and students with disabilities by grade level in the state. This information should be compiled in a way that allows comparisons to be made across states and should be made readily accessible.