year at the school and district levels (see http://dc.gov/DCPS/About+DCPS/Satisfaction+Stakeholder+Surveys [accessed March 2011]).
We were not able to review all of these data systems, but have a few comments. A report recently released by the Council of the Great City Schools (2010),1 for example, listed as its first finding about DC that there were “significant challenges to data quality” and “there was a lack of universal practice and oversight by the district in creating data comparable across DCPS schools and ensuring accurate information within the system. For example, there was no central control over student ID creation and no validations (automatic or hand-checked) to the system to guard against duplication.”
The DC data and accountability chief made a presentation to the Committee, in which she acknowledged that, although the office had made significant progress in improving data collection efforts, much more needs to be done. She cited as an example of problems she found on taking office the formerly standard practice in DC’s student tracking system of counting students as present unless otherwise noted by the school, which led to greatly overstated attendance rates.
Quality issues have also been raised with other DCPS databases. For example, in 2007, independent monitors of DC’s special education system said of the special education data that “Most [case analyses] require tracking down the student at a school that differs from the one listed as the attending school in [the data system]…[the system] does not meet standard system requirements of…data quality control[.]… There are several hundred ‘lost students.’…No one is really sure where they are at any one time.” In 2009, the District also terminated its contractor on the building of their State Longitudinal Data System for default.
These preliminary findings do not in any way suggest that all of the district data are of poor quality or unsuitable for use in a thorough evaluation. They do suggest that, as would be done at the beginning of any research study, the evaluation begin with careful consideration of the quality of the data available to support investigation of the specific research questions and methods envisioned.
Table C-1, below, is the list of data sources related to education that DCPS staff provided to the committee. These include data collected by each of the relevant city agencies.
DCPS also provided information about data being collected by the National Center for the Analysis of Longitudinal Data in Education
1Smerdon, B., and Evan, A. (2010). The Senior Urban Education Research Series, Volume 1: Lessons for Establishing a Foundation for Data Use in DC Public Schools. Washington, DC: Council of the Great City Schools. See http://www.cgcs.org/publications/DC_FellowReport2010.pdf [accessed March 2011].