The most readily available first impressions of student achievement are provided by test scores. There is a long history of relying on student test data as a measure of the effectiveness of public education, and it is tempting to simply rely on those readily available data for judgments about student achievement and about causes and effects. However, student test scores alone provide useful but limited information about the causes of improvements or variability in student performance.
The results of achievement tests provide only estimates about students’ skills and knowledge in selected areas—usually, what they know and can do in mathematics and reading and sometimes other subjects. Aggregate year-to-year comparisons of test scores in the District’s schools are confounded by changes in student populations that result from student moves in and out of the city and between DC Public Schools (DCPS) and charter schools, dropout and reentry, and also from variations in testing practices that may exclude or include particular groups of students.1 For these and other reasons, therefore, it is important to remember that the consensus of measurement and testing experts has long been to use test scores cautiously.
For this discussion, it is perhaps most important to underscore that most tests are not designed to support inferences about related questions, such as how well students were taught, what effects their teachers had on their learning, why students in some schools or classrooms succeed while those in similar schools and classrooms do not, whether conditions in the schools have improved as a result of a policy change, or what policy makers should do to solidify gains or reverse declines. Answering those sorts of questions requires other kinds of evidence besides test scores. Looking at test scores should be only a first step—not an end point—in considering questions about student achievement, or even more broadly, about student learning.
Nevertheless, changes in student test scores since 2007 provide one set of impressions regarding progress in DC schools. We offer here an overview of publicly available data from both the District of Columbia Comprehensive Assessment System (DC CAS) and the U.S. Department of Education’s National Assessment of Educational Progress (NAEP). We first discuss these data sources, then look at the trend data, and end the chapter with a discussion of how to interpret the data. But we note again that a systematic and comprehensive analysis of achievement data for DC was beyond the scope of this report; the readily available information provides only a useful first
1Test scores also come with measurement issues that have to be considered if they are to provide an accurate picture of even those areas they do measure (Koretz, 2008; National Research Council, 1999; Office of Technology Assessment, 1992).