Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 169
Appendix C Education Data for the District of Columbia The data available for use in an evaluation of District of Columbia (DC) Public Schools include both that collected by the city itself (by various offices, including DC Public Schools [DCPS], the Office of the State Super- intendent of Education [OSSE], and the Office of the Chief Financial Officer [OCFO]), and data collected by National Center for Education Statistics (NCES) of the U.S. Department of Education. DISTRICT OF COLUMBIA DATA SOURCES The District of Columbia has a number of data systems related to schools. DCPS provided the committee with a list of current databases housed within DCPS, OSSE, and the OCFO, and a list of evaluations or studies currently under way or recently completed (see tables below, as of March 2011). The DCPS systems include all of the basic student atten- dance, achievement, attainment, and tracking systems (e.g., DC STARS, ThinkLink Online), reading interventions (e.g., Read 180), human capi- tal management systems (e.g., IMPACT), and management, operations, and finances systems (e.g., Transportation Management System). OSSE’s sources of data include the State Longitudinal Education Data (SLED) warehouse (not yet operational) and a tracking system for individualized education plans (IEPs) required under IDEA. The OCFO data systems include the procurement and accounting systems. The evaluations include on-time studies, such as the City Year Evaluation, as well as ongoing as- sessments, such as the stakeholder surveys for which DC reports data every 169
OCR for page 170
170 APPENDIX C year at the school and district levels (see http://dc.gov/DCPS/About+DCPS/ Satisfaction+Stakeholder+Surveys [accessed March 2011]). We were not able to review all of these data systems, but have a few comments. A report recently released by the Council of the Great City Schools (2010),1 for example, listed as its first finding about DC that there were “significant challenges to data quality” and “there was a lack of uni- versal practice and oversight by the district in creating data comparable across DCPS schools and ensuring accurate information within the system. For example, there was no central control over student ID creation and no validations (automatic or hand-checked) to the system to guard against duplication.” The DC data and accountability chief made a presentation to the Committee, in which she acknowledged that, although the office had made significant progress in improving data collection efforts, much more needs to be done. She cited as an example of problems she found on taking office the formerly standard practice in DC’s student tracking system of count- ing students as present unless otherwise noted by the school, which led to greatly overstated attendance rates. Quality issues have also been raised with other DCPS databases. For example, in 2007, independent monitors of DC’s special education system said of the special education data that “Most [case analyses] require track- ing down the student at a school that differs from the one listed as the at- tending school in [the data system] . . . [the system] does not meet standard system requirements of . . . data quality control[.] . . . There are several hundred ‘lost students.’ . . . No one is really sure where they are at any one time.” In 2009, the District also terminated its contractor on the building of their State Longitudinal Data System for default. These preliminary findings do not in any way suggest that all of the dis- trict data are of poor quality or unsuitable for use in a thorough evaluation. They do suggest that, as would be done at the beginning of any research study, the evaluation begin with careful consideration of the quality of the data available to support investigation of the specific research questions and methods envisioned. Table C-1, below, is the list of data sources related to education that DCPS staff provided to the committee. These include data collected by each of the relevant city agencies. DCPS also provided information about data being collected by the National Center for the Analysis of Longitudinal Data in Education 1 Smerdon, B., and Evan, A. (2010). The Senior Urban Education Research Series, Vol ume 1: Lessons for Establishing a Foundation for Data Use in DC Public Schools. Wash- ington, DC: Council of the Great City Schools. See http://www.cgcs.org/publications/ DC_FellowReport2010.pdf [accessed March 2011].
OCR for page 171
171 APPENDIX C Research (CALDER), through a memorandum of understanding with DC and supported by the U.S. Department of Education. This project links data from the multiyear enrollment automated database (MEAD), assessment data, student residential information, and school files. Tables C-2 and C-3 list the sorts of information being collected through this project. DATA FROM THE NATIONAL CENTER FOR EDUCATION STATISTICS The federal data collections within NCES of the U.S. Department of Education that include information about DC’s schools and students are the Common Core of Data (CCD), the Schools and Staffing Survey (SASS), the National Assessment of Educational Progress (NAEP), and the Trial Urban District Assessment (TUDA). Below is a summary of the measures included in each. Common Core of Data (CCD) The CCD survey annually collects data about all public elementary and secondary schools, all local education agencies, and all state education agencies throughout the United States. CCD contains three categories of information: general descriptive information on schools and school dis- tricts; data on students and staff; and fiscal data. The general descriptive information includes name, address, phone number, and type of locale; the data on students and staff include selected demographic characteristics; and the fiscal data cover revenues and current expenditures. Most of the data are compiled by state education agencies and sent to the Department of Education. The CCD data are comparable across all states. Data are also collected for DCPS, and, since 2004, for charter schools operating in DC. Specific data include the number of students by grade level; fulltime staff by major employment category; high school graduates and completers in the previous year; average daily attendance; school district revenues by source (local, state, federal); and expenditures by function (instruction, support services, and noninstruction) and subfunction (school administra- tion, etc). Then Acting Commissioner of NCES, Stuart Kerachsky, made a pre- sentation to the committee about these data. He noted that collecting data for DC is more challenging than doing so for other jurisdictions for several reasons. Both the quality and the timeliness of the DC data have not been comparable to those of other states. For example, the District’s data exhibit a high percentage of missing data, especially for staff categories. In addi- tion, the average tenure of the fiscal data coordinator in DC is markedly shorter than the average for states, which hampers continuity. Finally, new
OCR for page 172
TABLE C-1 List of Education-Related Data Systems Provided by DCPS 172 DCPS Data Systems Inventory Owner # Title (Agency) Description 1 Genesis Earth DCPS Head Start workflow management 2 Work Sampling Online DCPS Head Start comprehensive child assessment tool 3 DCPS Administrator DCPS Used to process principal and assistant principal applications 4 Gov Delivery OCTO Used to send bulk emails to parents, etc. 5 Imagine Learning DCPS Reading intervention program for certain English language learner 3rd through 5th grade students. Used as learning aid directly by students 6 Early Steps and Stages Tracker OSSE Early Stages IDEA Part C tracking. Tracks students identified as possibly having special needs (birth - 2yr) 7 Early Stages Tracking, Monitoring, and DCPS Early Stages IDEA Part B tracking. Tracks students identified as possibly having Reporting special needs (2yr - 4yr 9mo) 8 DC STARS DCPS Student Information System (SIS) 9 Blackman-Jones Database DCPS Tracks Blackman-Jones statistics for consent decree reporting and case management 10 CAASS DCPS Student access control system. Tracks students as they enter school buildings, currently used for school security 11 DCPS Public Website DCPS Public website for DCPS 12 SEDS (Easy IEP) OSSE Individualized Education Program (IEP) management system 13 FileNet OCTO Document imaging system 14 PD Planner DCPS Online catalog and activity management system for professional development offerings for DCPS educators and employees 15 PASS OCTO Procurement management system 16 SOAR OCFO System of accounting and reporting (financial mgmt system) - General ledger, accounts payable, budget, fixed assets, accounts receivable, cash management, inventory management
OCR for page 173
17 PeopleSoft OCTO Human resources management system 18 WinSnap/WebSmart DCPS Food services management and point of sale system 19 FSS (Full Service Schools) Dashboard DCPS Allows principals to view current state of different measures of their school, (beta) combining information from different data sources in one place 20 IQ OCTO Districtwide correspondence management 21 Out-of-Boundary Lottery DCPS Lottery system to randomly select enrollment for out-of-boundary students 22 DCPS PS/PK and Out-of-Boundary DCPS Manages postlottery activities (results, managing waitlist, etc.) for the DCPS Pre- Database School / Pre-K / Head Start and Out-of-Boundary Lottery 23 Capital Gains DCPS Allows teachers to enter data on student performance for Capital Gains Program 24 DCPS CFO Budget V2 DCPS Allows principals to work with CFO analysts to develop coming year budget 25 DCPS CFO Budgeting DCPS Tracks actual vs budgeted spending for Central Office divisions 26 IMPACT DCPS Manages school-based staff assessments 27 DCPS UELIP Application DCPS Allows people (DCPS and non-DCPS) to apply for UELIP Internship Program 28 Comp Ed Database V2 DCPS Houses data about compensatory education providers, program, and services for use in comp ed determinations 29 Labor Management and Employee Relations DCPS Tracks grievances, adverse actions, etc. for DCPS employees 30 Performance Assessments DCPS Central Office employee performance evaluations 31 Probationary ET-15 Portal OSSE Tracks final disposition of teachers on probation 32 SST Tracker DCPS Pilot application used in a limited number of schools. Provides basic tracking of students referred to Student Support Team 33 Textbook Request System DCPS Allows principals to make textbook requests 34 Destiny DCPS Textbook tracking system 35 DCPS Summer School Application DCPS Allows people (DCPS and non-DCPS) to apply for summer school positions 36 TCTL Application DCPS Allows people to apply for TCTL Summer Program 37 Nonpublic and Charter School DCPS Used by both nonpublic and charter schools to track the ordering and completion Assessments: School Year 2009-2010 of special needs assessments 38 DCPS Provider Management Application DCPS Monitors, reports on, and updates related service provider information 39 DCPS Accommodations and IEP Changes DCPS Used to manage DCPS’ Read Aloud and DC CAS alt process for state testing (DC CAS) 40 Read 180 DCPS Online reading intervention program designed to accelerate the learning of students who are reading below grade level 173 continued
OCR for page 174
TABLE C-1 Continued 174 Owner # Title (Agency) Description 41 EBIS OCTO A web-based application that maps residential addresses to school boundaries 42 Transportation Management System DCPS Used to submit transportation requests for students who have transportation as a related service in their IEP 43 Trapeze OSSE Routes students for transportation 44 Extrata DCPS Part of system and used to scan, classify, and index human resource documents that belong in an employee’s personnel folder 45 IGP-Individual Graduation Plan DCPS Online system that allows students to view electronic portfolio of educational, career, and achievement information 46 Connect Ed DCPS Automated calling system used to confirm student absences and for principals to send messages home to parents 47 Dibels DCPS Package of services that includes handheld Palm devices installed with software that allows teachers to assess students on DIBELS (Dynamic Indicators of Basic Early Literacy Skills) and sync data with online monitoring system 48 Burst DCPS Reading intervention system related to Dibels (see above) 49 SLED OSSE State Longitudinal Education Data warehouse 50 ELIS DCPS Serves as system of record for individuals who seek licensure in the District of Columbia. Tracks educator preparation (e.g., degrees earned, degree major, etc.), teaching testing info, teacher license info, etc. 51 ThinkLink Online DCPS Discovery Education’s online data system for capturing DCPS Benchmark Assessment Data (DC BAS) student-level data. This system contains assessment data, teacher reports, sample items, and teacher resources 52 Security Incident Tracker DCPS Tracks security incidents occurring at schools 53 Schools DataLINK DCPS System of record for school-level data for which the Student Information System (DC STARS) is not the system of record. Provides aggregate-level information 54 Nonpublic Unit Tracker DCPS Tracks information about students in nonpublic schools that DCPS oversees for special education purposes SOURCE: Personal communication from DCPS, March 2011.
OCR for page 175
175 APPENDIX C TABLE C-2 Information on DC Schools Being Organized by CALDER MEAD Assessment Residential School Students Traditional + Traditional + Traditional + Traditional + Charter Charter Charter Charter Years 2001/2–2008/9 Spr2005-Spr2009 2003/4–2008/9 2003/4–2008/9 Data USIs Reading & math Student address School addresses Elements Student scores School attended Grades served characteristics: Student Grade Gender characteristics: Full name Ethnicity Gender Date of birth English Ethnicity proficiency English Special ed. proficiency Date of birth Special ed. School attended Date of birth Grade School attended Grade Full name SOURCE: Jane Hannaway presentation to committee, prepared by the National Center for Analysis of Longitudinal Data in Education Research (CALDER). TABLE C-3 Links That Can Be Established Using the DC Data, By Year School Years 09-10 01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 (expected) Test scores N N N Y Y Y Y Y Y Gender Y N N Y N Y Y N Y Ethnicity N N N N N Y Y Y Y Special Ed. N Y Y N Y Y Y Y Y English Prof. N Y Y N Y Y Y Y Y Grade Y Y Y Y Y Y Y Y Y School Att. Y Y Y Y Y Y Y Y Y Address N N Y Y Y Y Y Y Y SOURCE: Jane Hannaway presentation to committee, prepared by the National Center for Analysis of Longitudinal Data in Education Research (CALDER).
OCR for page 176
176 APPENDIX C strategies have been needed each year to make the data comparable to those for other states and prior years, for example, to account for changing clas- sifications of charter schools. Schools and Staffing Survey (SASS) The SASS is an integrated sample survey comprised of four question- naires targeting public and private schools, school districts, and principals and teachers. Data collection at the state level began in 1987-1988 and again in 1990-1991, 1993-1994, 1999-2000, 2003-2004, and 2007-2008, resulting in public release and restricted-use datasets. The next SASS collec- tion is scheduled for the 2011-2012 and 2015-2016 school years. One year following each administration of the SASS survey, a follow-up questionnaire is administered to the initial group of teachers to determine the numbers of those who have left their positions or have moved on to other positions. For the first time in 2007-2008, this follow-up question- naire, known as the Teacher Followup Survey (TFS), was also used to moni- tor the movement of first-year teachers. In addition, the year after SASS 2007-2008, a state-level Principal Follow-up Survey (PFS) was conducted on all principals interviewed in SASS. The SASS 2007-2008 collection includes 104 DC traditional public schools, their principals and school libraries at the state level, as well as an additional sample of public charter schools from 16 districts (many DC charter schools are their own districts). Finally, a sample of three to eight teachers per school based on school enrollment is also included. The SASS provides data on • Districts, including enrollment, teachers, principals, count of newly- hired teachers, teacher schedules and salaries, types of benefits offered to teachers, number of newly-hired principals, principal sal- ary schedule, number of contract days for teachers and principals per year, existence of a collective bargaining agreement or other type of agreement, school choice policies, high school graduation requirements, presence of incentives to recruit or retain teachers, poor performance dismissals or contract nonrenewals, high school graduation policy; • Schools, including staffing counts, Title I teacher count, grade levels offered, student enrollment by race/ethnicity, IEP and LEP students, migrant students, school year length, programs/services offered, public school designation as charter or traditional; • Library Media Centers, including services, policies, expenditures in previous year, types of holdings and equipment, assistive tech- nology availability;
OCR for page 177
177 APPENDIX C • Principals, including demographics, salary, hours worked per week, number of instructional hours students receive per week at grades 3 and 8 for core academic subjects, attitudes and school climate, policies on school safety; and • Teachers, including demographics, salary, workload, preparation, certification, teaching assignment, grades taught, number of stu- dents taught (average class size can be calculated), professional development, attitudes on school climate. National Assessment of Student Progress and Trial Urban District Assessment The National Assessment of Educational Progress (NAEP) collects academic achievement data and related background information. Table C-4 shows the DC data available from NAEP. The Trial Urban District Assessment (TUDA) is designed to explore the feasibility of using NAEP to report on the performance of public school stu- dents at the district level, for those districts selected to be a part of TUDA (see Table C-5). As authorized by federal law, NAEP has administered the mathematics, reading, science, and writing assessments to samples of stu- dents in selected urban districts public schools. TABLE C-4 Available NAEP Data National National and Participating Subject Only Results State Results Urban District Results Arts — — ü Civics — — ü Economics — — ü Geography — — ü Long-Term Trend — — ü Mathematics — ü ü Reading — ü ü Science — ü ü U.S. History — — ü Writing — ü ü NOTE: Unshaded rows are areas for which DC data are available. SOURCE: Nation’s Report Card, available: http://nationsreportcard.gov/about.asp [accessed December 2010].
OCR for page 178
TABLE C-5 District Assessment Participation 178 2002 2003 2005 Reading, 2007 Reading, 2009 Reading, Reading Reading and Science, and Mathematics, Mathematics, and Writing Mathematics Mathematics and Writing and Science ü ü ü ü ü Atlanta Public Schools ü ü ü ü Austin Independent School District ü Baltimore City Public Schools ü ü ü ü Boston School District ü ü ü ü Charlotte-Mecklenburg Schools ü ü ü ü ü Chicago Public Schools ü ü ü ü Cleveland Metropolitan School District ü Detroit Public Schools ü ü ü ü ü District of Columbia Public Schools ü Fresno Unified School District ü ü ü ü ü Houston Independent School District ü Jefferson County Public Schools (KY) ü ü ü ü ü Los Angeles Unified SchoolDistrict ü Miami-Dade County Public Schools ü Milwaukee Public Schools ü ü ü ü ü New York City Public Schools ü ü ü ü San Diego Unified SchoolDistrict ü School District of Philadelphia
OCR for page 179
NOTES: Beginning in 2009, if the results for charter schools are not included in the school district’s Adequate Yearly Progress (AYP) report to the U.S. Department of Education under the Elementary and Secondary Education Act, they are excluded from that district’s TUDA results. See more information on the comparability of the 2009 NAEP design. Due to an insufficient sample size, the District of Columbia did not participate in the science assessment in 2005 and 2009 and the writing assessment in 2007. SOURCES: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Edu- cational Progress (NAEP), various years, 2002-2009 assessments. Nation’s Report Card, TUDA, available: http://nationsreportcard.gov/tuda.asp [accessed December 2010]. 179
OCR for page 180
180 APPENDIX C NAEP and TUDA provide scale scores and achievement level data, along with background information and allow for trend analyses within states and districts, and comparisons with others. NAEP and TUDA also survey school administrators regarding information about the school and teachers regarding their educational background, experiences, and instruc- tional practices. Every 4 years a high school transcript study is conducted. In 2009, charter school results were included in the state-level NAEP assessment but were not included in the district-level TUDA results for DC. In that year, the math sample for DC 4th grade included approxi- mately 1,900 students in NAEP and 1,400 in TUDA; for 8th grade approximately 1,800 students in NAEP and 900 in TUDA. State-level (NAEP) and district-level (TUDA) math and reading assessments will be next administered for DC in 2011, 2013, and 2015 (schedule subject to change, see table). The schedule for NAEP and TUDA administrations is shown in Table C-6. TABLE C-6 Schedule of NAEP and TUDA Assessments, 2005-2017 Long-Term Year National State/TUDA Trend 2005 reading reading (4, 8) mathematics mathematics (4, 8) science science (4, 8) high school transcript study 2006 U.S. history civics economics (12) 2007 reading (4, 8) reading (4, 8) mathematics (4, 8) mathematics (4, 8) writing (8, 12) writing (8) 2008 arts (8) reading mathematics reading (4, 8, 12)a,c readinga 2009 mathematicsb mathematics (4, 8, 12)c sciencea science (4, 8)a high school transcript study 2010 U.S. history civics geography
OCR for page 181
181 APPENDIX C TABLE C-6 Continued Long-Term Year National State/TUDA Trend 2011 reading (4, 8) reading (4, 8) mathematics (4, 8) mathematics (4, 8) science (8) science (8, state only) writing (8,12)a 2012 economics (12) reading mathematics 2013 reading reading (4, 8) mathematics mathematics (4, 8) science science (4, 8) high school transcript study U.S. historya 2014 civicsa geography technology and engineering literacya (grades TBD) 2015 reading (4, 8) reading (4, 8) mathematics (4, 8) mathematics (4, 8) writing writing (4, 8) 2016 arts (8) reading mathematics 2017 reading reading (4, 8) mathematics mathematics (4, 8) science science (4, 8) high school transcript study Last Updated March 10, 2010. aUpdated or new framework is planned for implementation in this subject. In the case of subjects for which frameworks are already adopted, the Board will decide whether a new or updated framework is needed for this assessment year. bNew framework for grade 12 mathematics only, in 2009. cFor 2009, there is a pilot study of state-level results, for which 11 states volunteered. NOTES: At the national level, grades tested are 4, 8, and 12 unless otherwise indicated, except that long-term trend assessments sample students at ages 9, 13, and 17. The Governing Board intends to conduct assessments at grade 12 in world history and foreign language during the assessment period 2018-2011. SOURCE: See http://nces.ed.gov/nationsreportcard/about/assessmentsched.asp [accessed March 2010].
OCR for page 182