APPENDIX A
Data Needs

The challenge to school finance policies to become more closely aligned with improved student learning presents a new challenge to data collection efforts, one final area the committee was asked to address. Traditionally, finance data collected by federal and state education statistics agencies have served primarily to answer questions about how education funds are distributed. Much less attention has been directed to collecting finance and related data needed to understand in a sophisticated way how financial resources are linked to particular educational programs and to the performance of schools and students. For example, we have come to understand how much revenue each school district receives from various local, state, federal, and philanthropic sources. We understand what proportions of these revenues are expended for such things as teacher salaries, fringe benefits, and textbooks and supplies. However, we have virtually no understanding of what these budget ''objects" are intended to accomplish. In short, what is missing is a linkage between the objects of expenditure and their educational purposes.

This situation is beginning to change as state and federal statistical office respond to increased public interest in school reform and accountability and to growing demands for information about what education dollars buy. The committee was not able to explore in depth the demands on the education statistics system. We did hold a one-day workshop to discuss with 25 invited experts what the new challenges for school finance might mean for the data collection efforts of the National Center for Education Statistics (NCES).

NCES, the statistical arm of the U.S. Department of Education, is in a unique position to serve as a catalyst in improving both national and state efforts to



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 315
Making Money Matter: Financing America's Schools APPENDIX A Data Needs The challenge to school finance policies to become more closely aligned with improved student learning presents a new challenge to data collection efforts, one final area the committee was asked to address. Traditionally, finance data collected by federal and state education statistics agencies have served primarily to answer questions about how education funds are distributed. Much less attention has been directed to collecting finance and related data needed to understand in a sophisticated way how financial resources are linked to particular educational programs and to the performance of schools and students. For example, we have come to understand how much revenue each school district receives from various local, state, federal, and philanthropic sources. We understand what proportions of these revenues are expended for such things as teacher salaries, fringe benefits, and textbooks and supplies. However, we have virtually no understanding of what these budget ''objects" are intended to accomplish. In short, what is missing is a linkage between the objects of expenditure and their educational purposes. This situation is beginning to change as state and federal statistical office respond to increased public interest in school reform and accountability and to growing demands for information about what education dollars buy. The committee was not able to explore in depth the demands on the education statistics system. We did hold a one-day workshop to discuss with 25 invited experts what the new challenges for school finance might mean for the data collection efforts of the National Center for Education Statistics (NCES). NCES, the statistical arm of the U.S. Department of Education, is in a unique position to serve as a catalyst in improving both national and state efforts to

OCR for page 315
Making Money Matter: Financing America's Schools obtain education finance data related to the important question of how funding and school improvement policies intersect, as well as continuing to improve understanding of the distributional consequences of school finance policies. Through its national surveys, procedural handbooks, national outreach and professional communication efforts, and technical assistance, the agency already plays an essential role in fostering the availability and comparability of education data on a nationwide basis, and it has even now begun a number of innovative activities designed to enhance the usefulness of its school finance information. For example, the NCES school-finance Internet web site is a remarkably advanced high-tech effort in data distribution. We divide our suggestions for additional improvements in school finance data into four topics: types of data collected, the level of education about which data are collected, methods of data collection, and the usefulness of data. We conclude with some thoughts about the resources NCES will need to follow through on these suggestions. TYPES OF DATA COLLECTED The core of the NCES current school finance data program is information on revenues and expenditures for every state and school district, collected annually from state administrative records. Although it is often assumed that NCES itself collects data from school districts, this is not the case. The agency has always depended on state-collected data, which in turn reflect the fact that state administrative records were developed mainly for the purpose of financially auditing school districts for the appropriate receipt and spending of education funds. The federal government has so far not chosen to require that state and district data be reported to Washington using a uniform accounting standard, nor is there any nationwide requirement that school districts follow generally accepted accounting principles. Instead, NCES permits flexibility and diversity within an overall standard, which each state can interpret and to which it can align its data for purposes of reporting to NCES. The standard involves a function and object accounting approach1 using common definitions, rather than an approach based on programmatic categories. State and district data from the fiscal surveys are linked in the database called the Common Core of Data (CCD) to nonfiscal education data on states, districts, and schools. Through development and refine- 1   In school finance accounting parlance, an “object code” refers to the subject of the expenditure, what was purchased with the money. This can be a teacher’s salary, fringe benefits, a computer, food for the cafeteria, or fuel for school buses, for example. “Function” codes refer to the broad purposes for which such objects of spending were made. The conventional function codes are so large and amorphous as to be of little use to researchers. The conventional function codes are broad areas, such as instruction, administration, pupil services, food service, maintenance, and transportation. They may assist fiscal auditors, but they do not build a link to instructional or school effectiveness.

OCR for page 315
Making Money Matter: Financing America's Schools ment of the CCD surveys, NCES has been instrumental in improving the availability and comparability of education revenue and expenditure data on a nationwide basis and in enabling these data to be linked to basic characteristics of states and districts as reported in the nonfiscal surveys. Until recently, virtually no finance data were included in other parts of the NCES data collection program, such as the longitudinal surveys and international comparative studies. NCES can help meet the new challenge of linking school finance more closely to educational performance by adding productivity concerns to its traditional emphasis on revenues and expenditures. (Revenue and expenditure data continue to be important as well, and we suggest some improvements in these statistics at the end of this section.) Giving more attention to data linking finances and performance requires attention to the level of education for which data are collected as well as to how data are collected, topics which are addressed in the next two sections. But interacting with those issues are questions about the types of finance data that will be needed. We were not able to undertake detailed analysis of the feasibility, trade-off, and cost considerations that NCES will need to address before making decisions about additional data to be sought; rather, we suggest some key areas that should be explored as instruments are developed to link finance data more closely to issues of student and school performance. Outcome Data Outcome data are clearly central in evaluating school performance and its connection with finance. It is also clear that developing appropriate outcome indicators will be a complex and expensive task with great potential for inappropriate use of available measures, for all the reasons discussed in this report. Not only in data reporting, but in state and local administrative structures as well, there has been a sharp separation between those who are concerned with fiscal information and those who are concerned with curricular and assessment issues, and so we have little experience with bringing these separate universes together in a common reporting framework. State-level policy makers, however, are beginning to demand that outcome data be included in routine fiscal reporting; and the Government Accounting Standards Board (GASB) has for several years been involved in research efforts that in the foreseeable future could result in requirements that performance measures be included in annual financial reports for all governmental entities, including those responsible for elementary and secondary education.2 NCES should be 2    Information about the board's government performance measures project can be found on the its web site: http://www.rutgers.edu/Accounting/raw/gasb/seagov/SEARR1.htm. Examples of input, output, outcome, and explanatory factors under consideration are shown at: http://www.rutgers.edu/Accounting/raw/gasb/seagov/E&SEducRR1.htm.

OCR for page 315
Making Money Matter: Financing America's Schools a major player in efforts to think about what performance measures might be appropriate to link with finance, given its knowledge about the availability and quality of outcome data and its experience in cooperating with state and district officials to develop standard and comparable reporting categories. The agency has already begun to respond to requests from states about how to meet new demands for performance measures; the demand for more and improved measures seems sure to grow in the foreseeable future. Programmatic and Service Data The traditional function and object categories that were developed to track revenues and expenditure data for fiscal auditing purposes do not represent a particularly useful lens on educational activity when the focus shifts to what schools strive to do instructionally and how they do it. Programmatic or service categories have the potential for revealing much more about how resource use relates to performance. (Program reporting, for example, might seek to allocate school resources across major areas of activity, such as regular education, special education, Title I or other remedial education, and so forth. Resource reporting attempts to identify the direct services being delivered and the resource mix being devoted to them.) Past efforts to develop program and resource reporting, however, indicate the dilemmas involved; for example, program reporting requires difficult decisions about allocating activities such as staff time across multiple programs; resource reporting, which tends to build from the micro-level of the classroom up, is extremely data-intensive and expensive. Such problems will need to be addressed and overcome, nonetheless, to find out what patterns of resource use are in educationally relevant ways. "Off Budget" Data To link school finance to educational performance, and not just track the appropriate use of public funds, requires new attention to counting all the financial resources that may be available to schools. Traditional school finance data collection instruments will not be sufficient for this task, as nontraditional streams of revenue become increasingly available to schools. For example, local foundations in California were established in the wake of the Serrano v. Priest, 487 P.2d 1241 (Cal. 1971), decision to funnel privately raised funds to schools as a way of circumventing state-imposed limits on public spending. Likewise, it appears that some districts in Wyoming are responding to that state's takeover of school finance in the wake of an adequacy-based court suit by moving some items conventionally funded through school budgets to municipal budgets. Furthermore, charter schools, which are public schools, may in some cases receive significant funds from parents or others in addition to their publicly provided revenues.

OCR for page 315
Making Money Matter: Financing America's Schools School Organization Data Many new forms of school organization are being tried in the name of improving school performance, including charter schools, outsourcing of both auxiliary and instructional activities, schools-within-schools, site-based management, and others. None of these distinctions are captured by traditional school finance reporting categories. Data collection programs must allocate attention to gathering new kinds of information about school structure so that questions regarding possible differences in patterns of resource use and their effects, and how these differences might be connected to organizational differences and instructional outcomes, can be addressed. Data on Quality of Inputs While NCES surveys have historically attempted to gather basic information on school inputs, we are beginning to understand that traditional input measures do not capture quality so much as quantity. For example, research is providing evidence that conventional measures of teacher inputs, such as teacher experience, tell us too little about the underlying trait of teacher quality (see Chapter 5). It is the quality of resources, not just their amount, that is important in terms of understanding how the way money is spent affects school and student performance. This new concern for educational productivity and its determinants poses an important challenge for NCES in developing input measures that address new rather than old policy concerns. Better measures of productivity will be dependent on improved understandings from research on how educational inputs affect outcomes. Equity-Related Data Issues While many of the demands for new school finance data arise from the need to link finance more closely to educational productivity, NCES statistics are also vital for continued monitoring of the distribution of education resources. One avenue for new data development is taxable wealth and tax rates; it is at present impossible on a national basis to link school spending to the ability and effort of state and local governments to provide educational revenues. Such data have not been collected in the past largely because of the variation across states and localities in how closely they adhere to true market value in the assessment of property. Any effort to collect information on the property tax base by school district would have to take into account the differing assessment-to-sales ratios across districts. We recognize that in some states, these adjustments would be difficult, if not impossible, for a federal agency to make. In other states, the task has already been completed by a state agency. Analysts seeking to evaluate the wealth neutrality of school finance policies are forced to use personal income

OCR for page 315
Making Money Matter: Financing America's Schools data as an imperfect proxy for taxable wealth. Another avenue for further data development relates to cost adjustments in school finance formulas (discussed in Chapter 7). LEVEL OF EDUCATION ABOUT WHICH DATA ARE COLLECTED In the past, school finance data, especially on a national basis, have been collected almost exclusively at the state and school district level. Growing concern about educational performance, however, has heightened the demand for all kinds of data, including finance data, to be collected at the school level, as efforts to decentralize decision making to principals, teachers, and parents move forward. Scholars interested in probing productivity questions would even like to study resource levels in individual classrooms and for individual students. Increasing awareness of intradistrict disparities in resource allocation has also spurred demand for better measures of school-to-school differences in spending. Finally, policy makers concerned about accountability and about how to determine proper funding levels for schools (like charter schools) that may operate outside district budgets need improved information on expenditures at the individual school level. School-level finance data development is, in most states, in its infancy, although there are notable exceptions. Florida, Ohio, Oregon, and Texas are among states that have or are developing school-based financial reporting systems; and New York City does school-based reporting for its approximately 1,000 schools. By all accounts these are massive and time-consuming undertakings, especially since policy makers may (as in Oregon) accompany requirements for new financial accounting with mandates that school-level financial data be combined with school-level information about staffing, student demographics, school processes, and student outcomes and be available to the public and to policy makers via the Internet. Such efforts require major investments in standardizing accounting procedures across districts and schools, developing new accounting categories to capture new data elements of policy interest, automating record-keeping and data transmission, and providing training for district and school personnel. While state policy makers, auditors, and business leaders are often behind the push for school-level information on performance, supporters of school-level accounting also hope that these new systems will encourage a more data-oriented climate at the local level, with principals, teachers, and parents beginning to ask new questions about their own schools and how they compare with others. In this way, better information might become an engine of school reform, increasing community pressures for improvement that may be more successful than top-down mandates. While the big investments that are necessary to develop school-based accounting systems may indeed pay off in data that are particularly useful and used for improving education, this outcome is by no means certain. The political

OCR for page 315
Making Money Matter: Financing America's Schools climate undoubtedly has become much more supportive of performance measurement in public agencies (witness such developments as the GASB performance measurement project mentioned above and the Government Performance and Results Act of 1993 mandating federal agencies to focus on measuring the results of their programs and services). Herrington (1996), however, provides a useful reminder that the mere existence of data on school expenditures and student performance does not ensure that they will be used to explore the relations between dollars and learning. Florida created a school-level fiscal system, along with a sophisticated and comprehensive assessment system, over two decades ago. Herrington (1996:246–248) found, however, that the additional data did not have the desired effects on school improvement efforts, for several reasons: State political interests do not necessarily support the development of a capacity for systematic inquiry that is necessary to convert school-level fiscal data into knowledge useful for policy. Educator professional norms do not support the development of a capacity for systematic inquiry that is necessary to convert school-based fiscal data into knowledge useful for management. The publication and dissemination of data on school-level finance do not necessarily stimulate public interest in or public pressure for school improvement. The primitive state of school-level data in many states, along with questions about how useful these data will eventually be, suggest that the time is not yet ripe for NCES to undertake nationwide school-level data collection. Instead, the appropriate agency role at this time is to catalyze and assist states in their school-level data development efforts. NCES can play an important role in helping states learn from one another's efforts, in developing standards for states to use if they wish, and in supporting the development of school-based data collection software, which is not cost-effective for individual states or private companies to create. At the same time, NCES can use data collection instruments other than its national census surveys to explore the feasibility and usefulness of gathering financial resource data at the school, classroom, and individual student level for use in illuminating educational productivity questions. One example of how the agency has already begun moving in this direction involves the new Early Childhood Longitudinal Survey. Designers are attempting to insert measures of the relative resources behind individual children from kindergarten through sixth grade, eventually enabling researchers to study the impact of these resources on their performance over time. Teacher resources are the focus: base salary, merit pay, benefit rate, and full-or part-time status of the specific teachers to which each child is assigned. Earlier NCES longitudinal studies have been important sources of data for scholars studying productivity in education, but until now they have lacked explicit information on the financial resources available to individual

OCR for page 315
Making Money Matter: Financing America's Schools students. The early childhood study is an important first step in overcoming that deficiency. METHODS OF DATA COLLECTION NCES finance data have in the past been collected through the agency's nationwide, census-type surveys of all states and school districts. While these surveys continue to be important for tracking revenue and expenditure data, they are not the only or even the best way of gathering the kind of statistics that will help illuminate productivity issues. We have also indicated how NCES is beginning to use the longitudinal studies for this purpose, and we support continued investment in the developmental work necessary to devise and test fiscal measures that can be included in those surveys. We also urge NCES along with the Office of Educational Research and Improvement (OERI, the Department of Education's research arm) to provide support for analysis and dissemination of the school-level information available from individual states. There is no reason to believe that a full 50-state census of schools is necessary to explore what school-level data can tell us about how instructional effectiveness might be improved. Fully exploiting state-generated data for productivity purposes, however, will require many years of sustained research and development effort, something states have no track record in supporting. The federal government rather than state government has generally been the locus of public support for research and development efforts. Herrington's (1996) history of Florida's experience with a school-based data system confirms the absence of political interest in the research necessary to learn about improved delivery of instruction; Florida's school-based data system eventually focused on holding individual students accountable for performance rather than on ways to improve schools and foster their accountability. NCES and OERI could augment the analysis states are likely to do of their own data and make the results generally available for others to learn from rather than directly undertaking national data collection. At the classroom level, NCES has already been experimenting with innovative techniques for illuminating instructional processes and developing methods for making data about these processes available for analysis. In particular, the classroom videotape studies and accompanying software programs utilized in the Third International Mathematics and Science Study provide fascinating new information (sometimes at odds in interesting ways with information gathered through traditional survey instruments) about what actually goes on inside the classroom. Video studies appear to be a promising addition to the traditional arsenal of data collection instruments; for example, more such studies might help illuminate what it is about how instruction proceeds in smaller classrooms that contributes to higher levels of student learning, and under what conditions. Finally, we suggest that NCES consider increased use of samples to augment

OCR for page 315
Making Money Matter: Financing America's Schools its nationwide census surveys. Many important questions can be addressed without obtaining information on the whole population by relying on the behavior that can be inferred from appropriately chosen samples. For example, with regard to the issue of how much money might be flowing from private sources to schools whose public revenues are capped, useful information on this subject could come from careful sample surveys much more quickly than from nationwide data collection efforts. NCES's existing quick-response survey program is a very appropriate mechanism for using samples to get information on such topics. We understand, however, that funding for quick-response surveys is limited and that school finance topics have to compete with the many other data demands on NCES. USEFULNESS OF DATA Data are valuable only to the extent that they match the needs of potential users. Timeliness is a perennial concern, particularly of policy makers. Researchers yearn for additional education and demographic variables that will let them test more refined and sophisticated hypotheses about student learning. Suppliers of data are apt to be unenthusiastic about providing information unless they can see some immediate benefits to themselves, yet in many cases they do not have the capacity for sophisticated data manipulation or analysis. All data users are concerned about comparability of education data across 50 states, 15,000 school districts, and 80,000 schools. Technological advances offer options for addressing many of these concerns. NCES has been innovative in exploiting these opportunities, and we urge that even more attention be given to them. Concerns over timeliness, for example, might be alleviated by making survey data available on the Internet as soon as it has been checked for accuracy, even before all relevant units have been heard from. The publication of data is frequently delayed because some jurisdictions are slow in returning surveys and because NCES does not want to publish incomplete or inaccurate statistics. An interim solution would be to make partial (but accurate) information, with appropriate caveats, available on line. Researchers' needs for more data could be addressed in several ways, even within the confines of the available survey instruments. NCES could invest more resources in merging its own data with related data from other agencies; demographic data from the Census Bureau is one key target of opportunity. Demographic data provide crucial mediating variables in the quest to understand how school finance is linked to school performance, but until recently we have been dependent on the decennial census for demographic data, which is thus outdated almost as soon as it becomes available. The Census Bureau is currently developing the American Community Survey (ACS) to identify rapid changes in population and provide annual estimates of housing, social, and economic characteristics

OCR for page 315
Making Money Matter: Financing America's Schools every year for all states, as well as for cities, counties, metropolitan areas, and population groups of 65,000 persons or more. This annual data collection should be fully implemented in 2003; estimates for smaller areas and population groups will also be made available on a more frequent (but not annual) basis beginning in 2008. As ACS data become available, NCES should position itself to create links to education data and to make merged databases available to researchers. NCES can also expand the universe of variables available to researchers by serving as a clearinghouse for information about merged datasets that scholars have developed to support their analyses. NCES should not disseminate these merged datasets itself; it is not in a position to exercise quality control over individual researchers' efforts and the reputation of its own data hinges on the high statistical standards to which it holds itself. NCES is currently working with Inter-University Consortium for Political and Social Research (ICPSR) at the University of Michigan to contribute its own data to the ICPSR Internet-accessible archive of international education statistics. The agency could expand this effort to include helping researchers lodge their own datasets there and disseminating information about the data variables that these datasets contain. Through its Internet web site on education finance (http://nces.ed.gov/edfin), NCES is already taking innovative steps to make education statistics more usable to practitioners. A good example is the peer search feature, which allows web users to select a school district and see how its finances compare with a group of peer districts, on the basis of characteristics such as enrollment, student/teacher ratio, median household income, district type, and metro status. Current per-pupil expenditures, core instructional expenditures, student/teacher ratio, administration ratio, and revenue sources can be compared with peer districts as well as with state and national averages. As states improve their own district and school-level databases, there will be increasing opportunities for NCES to develop web-based search functions providing valuable and instant information to policy makers and the public. STAFFING AND RESOURCES AT NCES For almost a decade, the Department of Education has been operating under a congressionally imposed personnel ceiling, which has had the effect of capping staff levels at NCES while the agency's overall budget was increasing roughly threefold. Data gathered by the committee 3 indicate that NCES now manages a $116 million budget with a staff of 115 (a ratio of roughly $1 million per staff member). Budget-to-staff ratios are noticeably lower in other statistical agencies 3    Estimates for FY 1999 funding come from the Office of Management and Budget (1998); staff estimates are based on raw data gathered from various statistical agencies and the Office of Management and Budget.

OCR for page 315
Making Money Matter: Financing America's Schools we examined: National Center for Health Statistics ($86 million budget; 528 staff; ratio of $163,000 per staff member); Bureau of Justice Statistics ($31 million budget; 65 staff; $477,000 per staff member); Bureau of Economic Analysis ($48 million budget; 520 staff; ratio of $92,000 per staff member); and Bureau of Labor Statistics ($399 million budget; 2,642 staff; ratio of $151,000 per staff member.) The only way in which NCES could begin to comply with the added expectations for education that have evolved over the last decade is through the creative construction of the Education Statistics Services Institute and by relying heavily on outside contract vendors. As a consequence, NCES personnel have been converted from statisticians and researchers into contract managers. Even so, their capacity to oversee the ever-enlarging world of education statistics accumulation, compilation, and distribution is seriously strained. Moreover, heavy reliance on outside contractors, while not necessarily detrimental to the quality of data collected, does detract from the agency's ability to develop in-house talent and expertise that can be helpful in analyzing data and using findings from current surveys to help plan future activities (see remarks by the executive director of the Council of Professional Associations on Federal Statistics, reported in Rothman, 1992). It will not be possible the for nation to undertake the data collection efforts required to address important questions of education equity, adequacy, and productivity unless NCES is accorded the necessary budget and professional staff. REFERENCES Herrington, C. 1996. The politics of school-level finance data and state policy making. Pp. 236–252 in Where Does the Money Go? Resource Allocation in Elementary and Secondary Schools, L.O. Picus and J.L. Wattenbarger, eds. Sixteenth Annual Yearbook of the American Education Finance Association, 1995. Thousand Oaks, CA: Corwin Press. Office of Management and Budget 1998. Statistical Programs of the United States Government, Fiscal Year 1999. Washington, DC: Office of Management and Budget. Rothman, R. 1992. Desire for better picture of schools ups NCES's standing. Education Week (September 23) [Online]. Available: http://www.edweek.org [1999, April 26 ].