Education Data for the District of Columbia
The data available for use in an evaluation of District of Columbia (DC) Public Schools include both that collected by the city itself (by various offices, including DC Public Schools [DCPS], the Office of the State Superintendent of Education [OSSE], and the Office of the Chief Financial Officer [OCFO]), and data collected by National Center for Education Statistics (NCES) of the U.S. Department of Education.
DISTRICT OF COLUMBIA DATA SOURCES
The District of Columbia has a number of data systems related to schools. DCPS provided the committee with a list of current databases housed within DCPS, OSSE, and the OCFO, and a list of evaluations or studies currently under way or recently completed (see tables below, as of March 2011). The DCPS systems include all of the basic student attendance, achievement, attainment, and tracking systems (e.g., DC STARS, ThinkLink Online), reading interventions (e.g., Read 180), human capital management systems (e.g., IMPACT), and management, operations, and finances systems (e.g., Transportation Management System). OSSE’s sources of data include the State Longitudinal Education Data (SLED) warehouse (not yet operational) and a tracking system for individualized education plans (IEPs) required under IDEA. The OCFO data systems include the procurement and accounting systems. The evaluations include on-time studies, such as the City Year Evaluation, as well as ongoing assessments, such as the stakeholder surveys for which DC reports data every
year at the school and district levels (see http://dc.gov/DCPS/About+DCPS/Satisfaction+Stakeholder+Surveys [accessed March 2011]).
We were not able to review all of these data systems, but have a few comments. A report recently released by the Council of the Great City Schools (2010),1 for example, listed as its first finding about DC that there were “significant challenges to data quality” and “there was a lack of universal practice and oversight by the district in creating data comparable across DCPS schools and ensuring accurate information within the system. For example, there was no central control over student ID creation and no validations (automatic or hand-checked) to the system to guard against duplication.”
The DC data and accountability chief made a presentation to the Committee, in which she acknowledged that, although the office had made significant progress in improving data collection efforts, much more needs to be done. She cited as an example of problems she found on taking office the formerly standard practice in DC’s student tracking system of counting students as present unless otherwise noted by the school, which led to greatly overstated attendance rates.
Quality issues have also been raised with other DCPS databases. For example, in 2007, independent monitors of DC’s special education system said of the special education data that “Most [case analyses] require tracking down the student at a school that differs from the one listed as the attending school in [the data system]…[the system] does not meet standard system requirements of…data quality control[.]… There are several hundred ‘lost students.’…No one is really sure where they are at any one time.” In 2009, the District also terminated its contractor on the building of their State Longitudinal Data System for default.
These preliminary findings do not in any way suggest that all of the district data are of poor quality or unsuitable for use in a thorough evaluation. They do suggest that, as would be done at the beginning of any research study, the evaluation begin with careful consideration of the quality of the data available to support investigation of the specific research questions and methods envisioned.
Table C-1, below, is the list of data sources related to education that DCPS staff provided to the committee. These include data collected by each of the relevant city agencies.
DCPS also provided information about data being collected by the National Center for the Analysis of Longitudinal Data in Education
_______________
1Smerdon, B., and Evan, A. (2010). The Senior Urban Education Research Series, Volume 1: Lessons for Establishing a Foundation for Data Use in DC Public Schools. Washington, DC: Council of the Great City Schools. See http://www.cgcs.org/publications/DC_FellowReport2010.pdf [accessed March 2011].
Research (CALDER), through a memorandum of understanding with DC and supported by the U.S. Department of Education. This project links data from the multiyear enrollment automated database (MEAD), assessment data, student residential information, and school files. Tables C-2 and C-3 list the sorts of information being collected through this project.
DATA FROM THE NATIONAL CENTER FOR EDUCATION STATISTICS
The federal data collections within NCES of the U.S. Department of Education that include information about DC’s schools and students are the Common Core of Data (CCD), the Schools and Staffing Survey (SASS), the National Assessment of Educational Progress (NAEP), and the Trial Urban District Assessment (TUDA). Below is a summary of the measures included in each.
Common Core of Data (CCD)
The CCD survey annually collects data about all public elementary and secondary schools, all local education agencies, and all state education agencies throughout the United States. CCD contains three categories of information: general descriptive information on schools and school districts; data on students and staff; and fiscal data. The general descriptive information includes name, address, phone number, and type of locale; the data on students and staff include selected demographic characteristics; and the fiscal data cover revenues and current expenditures. Most of the data are compiled by state education agencies and sent to the Department of Education. The CCD data are comparable across all states. Data are also collected for DCPS, and, since 2004, for charter schools operating in DC.
Specific data include the number of students by grade level; fulltime staff by major employment category; high school graduates and completers in the previous year; average daily attendance; school district revenues by source (local, state, federal); and expenditures by function (instruction, support services, and noninstruction) and subfunction (school administration, etc).
Then Acting Commissioner of NCES, Stuart Kerachsky, made a presentation to the committee about these data. He noted that collecting data for DC is more challenging than doing so for other jurisdictions for several reasons. Both the quality and the timeliness of the DC data have not been comparable to those of other states. For example, the District’s data exhibit a high percentage of missing data, especially for staff categories. In addition, the average tenure of the fiscal data coordinator in DC is markedly shorter than the average for states, which hampers continuity. Finally, new
TABLE C-1 List of Education-Related Data Systems Provided by DCPS
DCPS Data Systems Inventory | ||||
# | Title | Owner (Agency) | Description | |
1 | Genesis Earth | DCPS | Head Start workflow management | |
2 | Work Sampling Online | DCPS | Head Start comprehensive child assessment tool | |
3 | DCPS Administrator | DCPS | Used to process principal and assistant principal applications | |
4 | Gov Delivery | OCTO | Used to send bulk emails to parents, etc. | |
5 | Imagine Learning | DCPS | Reading intervention program for certain English language learner 3rd through 5th grade students. Used as learning aid directly by students | |
6 | Early Steps and Stages Tracker | OSSE | Early Stages IDEA Part C tracking. Tracks students identified as possibly having special needs (birth - 2yr) | |
7 | Early Stages Tracking, Monitoring, and Reporting | DCPS | Early Stages IDEA Part B tracking. Tracks students identified as possibly having special needs (2yr - 4yr 9mo) | |
8 | DC STARS | DCPS | Student Information System (SIS) | |
9 | Blackman-Jones Database | DCPS | Tracks Blackman-Jones statistics for consent decree reporting and case management | |
10 | CAASS | DCPS | Student access control system. Tracks students as they enter school buildings, currently used for school security | |
11 | DCPS Public Website | DCPS | Public website for DCPS | |
12 | SEDS (Easy IEP) | OSSE | Individualized Education Program (IEP) management system | |
13 | FileNet | OCTO | Document imaging system | |
14 | PD Planner | DCPS | Online catalog and activity management system for professional development offerings for DCPS educators and employees | |
15 | PASS | OCTO | Procurement management system | |
16 | SOAR | OCFO | System of accounting and reporting (financial mgmt system) - General ledger, accounts payable, budget, fixed assets, accounts receivable, cash management, inventory management |
17 | PeopleSoft | OCTO | Human resources management system |
18 | WinSnap/WebSmart | DCPS | Food services management and point of sale system |
19 | FSS (Full Service Schools) Dashboard (beta) | DCPS | Allows principals to view current state of different measures of their school, combining information from different data sources in one place |
20 | IQ | OCTO | Districtwide correspondence management |
21 | Out-of-Boundary Lottery | DCPS | Lottery system to randomly select enrollment for out-of-boundary students |
22 | DCPS PS/PK and Out-of-Boundary Database | DCPS | Manages postlottery activities (results, managing waitlist, etc.) for the DCPS PreSchool / Pre-K / Head Start and Out-of-Boundary Lottery |
23 | Capital Gains | DCPS | Allows teachers to enter data on student performance for Capital Gains Program |
24 | DCPS CFO Budget V2 | DCPS | Allows principals to work with CFO analysts to develop coming year budget |
25 | DCPS CFO Budgeting | DCPS | Tracks actual vs budgeted spending for Central Office divisions |
26 | IMPACT | DCPS | Manages school-based staff assessments |
27 | DCPS UELIP Application | DCPS | Allows people (DCPS and non-DCPS) to apply for UELIP Internship Program |
28 | Comp Ed Database V2 | DCPS | Houses data about compensatory education providers, program, and services for use in comp ed determinations |
29 | Labor Management and Employee Relations | DCPS | Tracks grievances, adverse actions, etc. for DCPS employees |
30 | Performance Assessments | DCPS | Central Office employee performance evaluations |
31 | Probationary ET-15 Portal | OSSE | Tracks final disposition of teachers on probation |
32 | SST Tracker | DCPS | Pilot application used in a limited number of schools. Provides basic tracking of students referred to Student Support Team |
33 | Textbook Request System | DCPS | Allows principals to make textbook requests |
34 | Destiny | DCPS | Textbook tracking system |
35 | DCPS Summer School Application | DCPS | Allows people (DCPS and non-DCPS) to apply for summer school positions |
36 | TCTL Application | DCPS | Allows people to apply for TCTL Summer Program |
37 | Nonpublic and Charter School Assessments: School Year 2009-2010 | DCPS | Used by both nonpublic and charter schools to track the ordering and completion of special needs assessments |
38 | DCPS Provider Management Application | DCPS | Monitors, reports on, and updates related service provider information |
39 | DCPS Accommodations and IEP Changes | DCPS | Used to manage DCPS’ Read Aloud and DC CAS alt process for state testing (DC CAS) |
40 | Read 180 | DCPS | Online reading intervention program designed to accelerate the learning of students who are reading below grade level |
# | Title | Owner (Agency) | Description | |
41 | EBIS | OCTO | A web-based application that maps residential addresses to school boundaries | |
42 | Transportation Management System | DCPS | Used to submit transportation requests for students who have transportation as a related service in their IEP | |
43 | Trapeze | OSSE | Routes students for transportation | |
44 | Extrata | DCPS | Part of system and used to scan, classify, and index human resource documents that belong in an employee’s personnel folder | |
45 | IGP-Individual Graduation Plan | DCPS | Online system that allows students to view electronic portfolio of educational, career, and achievement information | |
46 | Connect Ed | DCPS | Automated calling system used to confirm student absences and for principals to send messages home to parents | |
47 | Dibels | DCPS | Package of services that includes handheld Palm devices installed with software that allows teachers to assess students on DIBELS (Dynamic Indicators of Basic Early Literacy Skills) and sync data with online monitoring system | |
48 | Burst | DCPS | Reading intervention system related to Dibels (see above) | |
49 | SLED | OSSE | State Longitudinal Education Data warehouse | |
50 | ELIS | DCPS | Serves as system of record for individuals who seek licensure in the District of Columbia. Tracks educator preparation (e.g., degrees earned, degree major, etc.), teaching testing info, teacher license info, etc. | |
51 | ThinkLink Online | DCPS | Discovery Education’s online data system for capturing DCPS Benchmark Assessment Data (DC BAS) student-level data. This system contains assessment data, teacher reports, sample items, and teacher resources | |
52 | Security Incident Tracker | DCPS | Tracks security incidents occurring at schools | |
53 | Schools DataLINK | DCPS | System of record for school-level data for which the Student Information System (DC STARS) is not the system of record. Provides aggregate-level information | |
54 | Nonpublic Unit Tracker | DCPS | Tracks information about students in nonpublic schools that DCPS oversees for special education purposes | |
SOURCE: Personal communication from DCPS, March 2011.
TABLE C-2 Information on DC Schools Being Organized by CALDER
MEAD | Assessment | Residential | School | |
Students | Traditional + Charter | Traditional + Charter | Traditional + Charter | Traditional + Charter |
Years | 2001/2–2008/9 | Spr2005-Spr2009 | 2003/4–2008/9 | 2003/4–2008/9 |
Data Elements | USIs Student characteristics: Gender Ethnicity English proficiency Special ed. Date of birth School attended Grade | Reading & math scores Student characteristics: Gender Ethnicity English proficiency Special ed. Date of birth School attended Grade Full name | Student address School attended Grade Full name Date of birth | School addresses Grades served |
SOURCE: Jane Hannaway presentation to committee, prepared by the National Center for Analysis of Longitudinal Data in Education Research (CALDER).
TABLE C-3 Links That Can Be Established Using the DC Data, By Year
School Years | |||||||||
01-02 | 02-03 | 03-04 | 04-05 | 05-06 | 06-07 | 07-08 | 08-09 | 09-10 (expected) | |
Test scores | N | N | N | Y | Y | Y | Y | Y | Y |
Gender | Y | N | N | Y | N | Y | Y | N | Y |
Ethnicity | N | N | N | N | N | Y | Y | Y | Y |
Special Ed. | N | Y | Y | N | Y | Y | Y | Y | Y |
English Prof. | N | Y | Y | N | Y | Y | Y | Y | Y |
Grade | Y | Y | Y | Y | Y | Y | Y | Y | Y |
School Att. | Y | Y | Y | Y | Y | Y | Y | Y | Y |
Address | N | N | Y | Y | Y | Y | Y | Y | Y |
SOURCE: Jane Hannaway presentation to committee, prepared by the National Center for Analysis of Longitudinal Data in Education Research (CALDER).
strategies have been needed each year to make the data comparable to those for other states and prior years, for example, to account for changing classifications of charter schools.
Schools and Staffing Survey (SASS)
The SASS is an integrated sample survey comprised of four questionnaires targeting public and private schools, school districts, and principals and teachers. Data collection at the state level began in 1987-1988 and again in 1990-1991, 1993-1994, 1999-2000, 2003-2004, and 2007-2008, resulting in public release and restricted-use datasets. The next SASS collection is scheduled for the 2011-2012 and 2015-2016 school years.
One year following each administration of the SASS survey, a follow-up questionnaire is administered to the initial group of teachers to determine the numbers of those who have left their positions or have moved on to other positions. For the first time in 2007-2008, this follow-up questionnaire, known as the Teacher Followup Survey (TFS), was also used to monitor the movement of first-year teachers. In addition, the year after SASS 2007-2008, a state-level Principal Follow-up Survey (PFS) was conducted on all principals interviewed in SASS.
The SASS 2007-2008 collection includes 104 DC traditional public schools, their principals and school libraries at the state level, as well as an additional sample of public charter schools from 16 districts (many DC charter schools are their own districts). Finally, a sample of three to eight teachers per school based on school enrollment is also included.
The SASS provides data on
- Districts, including enrollment, teachers, principals, count of newly-hired teachers, teacher schedules and salaries, types of benefits offered to teachers, number of newly-hired principals, principal salary schedule, number of contract days for teachers and principals per year, existence of a collective bargaining agreement or other type of agreement, school choice policies, high school graduation requirements, presence of incentives to recruit or retain teachers, poor performance dismissals or contract nonrenewals, high school graduation policy;
- Schools, including staffing counts, Title I teacher count, grade levels offered, student enrollment by race/ethnicity, IEP and LEP students, migrant students, school year length, programs/services offered, public school designation as charter or traditional;
- Library Media Centers, including services, policies, expenditures in previous year, types of holdings and equipment, assistive technology availability;
- Principals, including demographics, salary, hours worked per week, number of instructional hours students receive per week at grades 3 and 8 for core academic subjects, attitudes and school climate, policies on school safety; and
- Teachers, including demographics, salary, workload, preparation, certification, teaching assignment, grades taught, number of students taught (average class size can be calculated), professional development, attitudes on school climate.
National Assessment of Student Progress and Trial Urban District Assessment
The National Assessment of Educational Progress (NAEP) collects academic achievement data and related background information. Table C-4 shows the DC data available from NAEP.
The Trial Urban District Assessment (TUDA) is designed to explore the feasibility of using NAEP to report on the performance of public school students at the district level, for those districts selected to be a part of TUDA (see Table C-5). As authorized by federal law, NAEP has administered the mathematics, reading, science, and writing assessments to samples of students in selected urban districts public schools.
TABLE C-4 Available NAEP Data
Subject | National Only Results | National and State Results | Participating Urban District Results | |
Arts | — | — | ||
Civics | — | — | ||
Economics | — | — | ||
Geography | — | — | ||
Long-Term Trend | — | — | ||
Mathematics | — | |||
Reading | — | |||
Science | — | |||
U.S. History | — | — | ||
Writing | — | |||
NOTE: Unshaded rows are areas for which DC data are available.
SOURCE: Nation’s Report Card, available: http://nationsreportcard.gov/about.asp [accessed December 2010].
TABLE C-5 District Assessment Participation
2002 Reading and Writing | 2003 Reading and Mathematics | 2005 Reading, Science, and Mathematics | 2007 Reading, Mathematics, and Writing | 2009 Reading, Mathematics, and Science | |
Atlanta Public Schools | |||||
Austin Independent School District | |||||
Baltimore City Public Schools | |||||
Boston School District | |||||
Charlotte-Mecklenburg Schools | |||||
Chicago Public Schools | |||||
Cleveland Metropolitan School District | |||||
Detroit Public Schools | |||||
District of Columbia Public Schools | |||||
Fresno Unified School District | |||||
Houston Independent School District | |||||
Jefferson County Public Schools (KY) | |||||
Los Angeles Unified SchoolDistrict | |||||
Miami-Dade County Public Schools | |||||
Milwaukee Public Schools | |||||
New York City Public Schools | |||||
San Diego Unified SchoolDistrict | |||||
School District of Philadelphia | |||||
NOTES: Beginning in 2009, if the results for charter schools are not included in the school district’s Adequate Yearly Progress (AYP) report to the U.S. Department of Education under the Elementary and Secondary Education Act, they are excluded from that district’s TUDA results. See more information on the comparability of the 2009 NAEP design. Due to an insufficient sample size, the District of Columbia did not participate in the science assessment in 2005 and 2009 and the writing assessment in 2007.
SOURCES: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 2002-2009 assessments. Nation’s Report Card, TUDA, available: http://nationsreportcard.gov/tuda.asp [accessed December 2010].
NAEP and TUDA provide scale scores and achievement level data, along with background information and allow for trend analyses within states and districts, and comparisons with others. NAEP and TUDA also survey school administrators regarding information about the school and teachers regarding their educational background, experiences, and instructional practices. Every 4 years a high school transcript study is conducted.
In 2009, charter school results were included in the state-level NAEP assessment but were not included in the district-level TUDA results for DC. In that year, the math sample for DC 4th grade included approximately 1,900 students in NAEP and 1,400 in TUDA; for 8th grade approximately 1,800 students in NAEP and 900 in TUDA. State-level (NAEP) and district-level (TUDA) math and reading assessments will be next administered for DC in 2011, 2013, and 2015 (schedule subject to change, see table). The schedule for NAEP and TUDA administrations is shown in Table C-6.
TABLE C-6 Schedule of NAEP and TUDA Assessments, 2005-2017
Year | National | State/TUDA | Long-Term Trend | |
2005 | reading mathematics science high school transcript study | reading (4, 8) mathematics (4, 8) science (4, 8) | ||
2006 | U.S. history civics economics (12) | |||
2007 | reading (4, 8) mathematics (4, 8) writing (8, 12) | reading (4, 8) mathematics (4, 8) writing (8) | ||
2008 | arts (8) | reading mathematics | ||
2009 | readinga mathematicsb sciencea high school transcript study | reading (4, 8, 12)a,c mathematics (4, 8, 12)c science (4, 8)a | ||
2010 | U.S. history civics geography |
Year | National | State/TUDA | Long-Term Trend | |
2011 | reading (4, 8) mathematics (4, 8) science (8) writing (8,12)a | reading (4, 8) mathematics (4, 8) science (8, state only) | ||
2012 | economics (12) | reading mathematics | ||
2013 | reading mathematics science high school transcript study | reading (4, 8) mathematics (4, 8) science (4, 8) | ||
2014 | U.S. historya civicsa geography technology and engineering literacya (grades TBD) | |||
2015 | reading (4, 8) mathematics (4, 8) writing | reading (4, 8) mathematics (4, 8) writing (4, 8) | ||
2016 | arts (8) | reading mathematics | ||
2017 | reading mathematics science high school transcript study | reading (4, 8) mathematics (4, 8) science (4, 8) | ||
Last Updated March 10, 2010.
aUpdated or new framework is planned for implementation in this subject. In the case of subjects for which frameworks are already adopted, the Board will decide whether a new or updated framework is needed for this assessment year.
bNew framework for grade 12 mathematics only, in 2009.
cFor 2009, there is a pilot study of state-level results, for which 11 states volunteered.
NOTES: At the national level, grades tested are 4, 8, and 12 unless otherwise indicated, except that long-term trend assessments sample students at ages 9, 13, and 17. The Governing Board intends to conduct assessments at grade 12 in world history and foreign language during the assessment period 2018-2011.
SOURCE: See http://nces.ed.gov/nationsreportcard/about/assessmentsched.asp [accessed March 2010].