Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 39
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series 5 Costs While saving money is not the primary goal cited by most advocates of common standards, questions about what it costs states to have standards and how those costs might be affected by a switch to common standards are nevertheless important. These questions raise a host of complexities. Calculating the cost of a policy that can be implemented in a seemingly infinite number of ways is a formidable challenge in itself, but economists also consider what they refer to as opportunity costs—the value of the potential benefits of policy options not chosen—when they evaluate the costs of different choices. Looking first at the question of what states must spend in order to have standards, the committee identified five framing questions to guide the discussion: What are the major activities states undertake to develop and maintain a standards-based K-12 education system? What is the nature of the costs to states associated with each of these major activities? What are the sources of variation in these costs by state? What are the costs associated with each major activity across select states? How much do state cost estimates vary for each activity? What are the conceptual and technical issues involved in developing empirical estimates of these costs?
OCR for page 40
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series Margaret Goertz presented a framework for considering the costs, providing an analysis of what states actually do in implementing a standards-based K-12 education system (Goertz, 2008). Douglas Harris and Lori Taylor followed with a detailed investigation of the challenges of estimating the costs of this kind of enterprise, as well as empirical estimates of the costs incurred in three states: Florida, North Dakota, and Texas (Harris and Taylor, 2008b). COST ESTIMATION FRAMEWORK Goertz organized her framework around what she identified as the six primary activities that constitute a standards-based reform system, although she noted that others might define the major activities differently (see Box 5-1). Goertz cautioned that the first four activities, which describe the mechanics of the standards themselves and the accountability system, generally receive most of the attention, but that the last two activities, which describe the ways in which the standards and accountability system may affect teaching and learning, are equally important. Goertz used these six activities as the basis for a discussion of ways in which variation in implementation may affect the costs to states. Looking at the first three activities, she identified several primary sources of variation with cost implications. The frequency with which a state reviews and revises its standards and updates its assessments (as well as the number of subjects for which there are standards and the number of assessments) is one factor. A second factor is the process used for setting and reviewing content and performance standards, which varies in complexity, in part because of the number of people and groups involved. States may rely primarily on their department of education staff and volunteers, for example, or hire a contractor, use paid experts, or perhaps do all three. With regard to rewards and sanctions (activity 4), there is a large range of approaches and of potential costs. Reponses to classifications of schools or districts as falling short of No Child Left Behind (NCLB) performance targets might begin with instructional audits or needs assessments. Interventions might include developing school improvement plans, measures to build capacity at the district level, or professional development in such areas as curriculum and instruction, data analysis, assessment, and leadership. For failing students, states vary in terms of how they determine eligibility and in how they structure and deliver remediation, as well as in how much funding may be available from the state for this purpose. The characteristics of a state also play a significant role. Readily apparent differences, such as the size of a state and the demographics of its student population, affect costs in predictable ways, but other factors are important as well. States vary in their mechanisms for funding
OCR for page 41
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series BOX 5-1 Major Standards-Based Reform Activities Standards-setting Developing and revising academic content standards Setting performance standards Disseminating standards and training State Assessment Aligning assessment with standards Item development Test construction Test administration Test scoring Score reporting Technical review, validation of the system State Accountability System Data system (student, school, district) Reporting (school, district) Identifying school status, monitoring progress Other accountability measures (process, etc.) Rewards and Sanctions Rewards to successful/improving schools Sanctions for underperforming/failing districts, schools, or students Intervention for failing schools, districts Intervention for failing students Alignment with Other Policies Teacher preparation and program approval Teacher certification and licensure Curriculum (also textbook adoption in some states) Instruction Delivery System Supporting teachers, schools, and districts in implementing new or revised standards (professional development, curriculum alignment, resources for expanded standards, adequacy studies to determine what would be required to give all students the opportunity to meet proficiency standards) Supporting students in meeting more challenging standards (programs and resources to address special education needs) SOURCE: Goertz (2008).
OCR for page 42
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series public education and in the relative share that is paid by districts. States that have had standards-based reform policies for a decade or more are likely to have more firmly established systems and streamlined processes, which reduce costs. States with newer policies may thus have somewhat higher costs. A state’s fiscal health also plays a role, perhaps because education budgets may remain more stable in the absence of economic downturns. A key component of the variation in costs is the number of person hours required to accomplish the tasks involved. Salaries for regular staff, fees for consultants or contractors, and stipends for teachers who take on extra responsibilities are the prime costs for most of the activities. States also incur meeting and travel costs, as well as the costs of providing grants to districts, schools, or regional consortia. These are all costs that can be estimated, but Goertz noted that it is inherently more difficult to estimate the cost of a policy idea, such as standards-based reform, than a specific program. Goertz returned to a point raised earlier, that standards-based reform is a term that can mean different things to different people. In order to estimate costs, one must determine which expenditures should be classed as standards-based reform costs and which are general K-12 education costs. One must also ask which costs would be incurred by states even without that conceptual approach and which are extra expenditures. Another complication is the task of distinguishing state costs from local costs. Since state education funding formulas vary, the extent to which costs incurred at the local level are covered by state K-12 education funding can vary significantly—which makes it more difficult to compare across states. APPLYING THE FRAMEWORK IN THREE STATES Harris and Taylor echoed the points that standards-based reform is a complex idea and that developing cost estimates for an idea is inherently difficult. Thus, in their analysis of the costs, they focused on three questions: What costs are now being incurred to create, update, and minimally comply with current state and federal laws and rules regarding standards, assessments, and accountability? What costs would the nation incur if the state-based system of standards, assessments, and accountability required by NCLB were implemented as a common system? What are the costs of some of the specific “add-ons” to these systems that are not required by state or federal law?
OCR for page 43
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series To answer these questions, they first developed estimates of the costs of standards-based reform activities for three states—Florida, North Dakota, and Texas—that were chosen to reflect a range in terms of size, education spending, and approaches to assessment and accountability.1 Their notion was that, since no one state could be viewed as representative or typical, they could use the data from these three states to compile a cost profile for a “typical state” and use that to calculate costs for the nation. They noted that their estimates are all based on the costs of simply complying with all requirements, rather than the costs of meeting goals, such as bringing all students to proficiency. The challenge of comparing the costs of the current system with those of a potential common system raises several economic concepts. One important distinction is that between fixed costs, which are the same regardless of the scale of the program, and variable costs, which vary depending on the scale of the effort. A significant fixed cost of standards-based reform is developing the content standards and setting the performance standards: this cost would be the same whether the standards were to apply to 100 or 1 million students. However, Harris explained, a common standard is likely to set a higher bar, which is likely to mean significant additional costs. Moreover, a substantial majority of the costs of standards-based reform are variable and depend on design decisions (including how high the bar is set). Other variables would be the number of subjects and grade levels to be included, the frequency of assessment, the degree of support provided to schools, and the incentives and sanctions to be implemented. Another distinction used by economists is that between opportunity costs and expenditures: reforms may switch resources from one activity to another, and it is important to include such resource costs in the cost estimate, even if they do not result in new expenditures. So, for example, the time that teachers need to spend on testing-related tasks does not require additional state or district expenditures because the teachers would otherwise have been engaged in another task and would be receiving their salaries. However, there is a real resource cost in the loss of whatever they might otherwise have accomplished during that time. Harris and Taylor focused on these costs (they used the term “real resource costs” to avoid technical jargon) in order to capture that value. Harris and Taylor highlighted some of the features of the three states’ systems to illustrate more specifically the kinds of variation that affect costs. For example, Florida and Texas have both gone consider- 1 Harris and Taylor conferred with committee members in finalizing the list of states for analysis so that their results could be considered together with those of Massell’s study discussed in Chapter 1.
OCR for page 44
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series ably beyond NCLB requirements (by testing at more grades and in more subjects), whereas North Dakota has not. Florida has a bonus program so that districts can give substantial monetary rewards to teachers for improvements made by their students, and Texas has a program to identify and accelerate certain students. Assigning the costs of these kinds of policies is complex, Harris explained, because one could argue that they are part of the cost of meeting the requirement—to push students toward proficiency—or one could argue that they are not required elements of a standards-based reform system. Moreover, a state government might reallocate resources in a way that poses a conundrum for cost estimators, for example, by switching instruction time from music to mathematics. If making this change is not treated as a cost, the implication is that music instruction has no value. Assigning a dollar value to what is lost is not straightforward, but treating it as an additional cost may unreasonably inflate the overall estimate. Having described some of the issues and assumptions that guided their work, Harris and Taylor presented their cost data (their paper provides further detail about the costs they considered; see Harris and Taylor, 2008b). Their principal sources were national databases, data made publicly available by the three states, and interviews with local officials. Looking first at the costs per pupil and per standard (e.g., grade 3 mathematics) of developing standards, the authors included both per-pupil and per-standard costs in order to capture both one-time (initial development) and annual costs (such as those for administering assessments). Those results are shown in Table 5-1. Harris and Taylor also calculated the overall costs per student of assessment and accountability programs in each state, shown in Table 5-2, and some of the additional fixed costs states incur when they go beyond legal requirements, in this case the cost of annual reviews of the standards. These figures are shown in Table 5-3. For context, annual expenditures TABLE 5-1 Estimated Cost of Developing Academic Standards in Three States State Cost per Subject and Grade Level Standard Florida $45,000 North Dakota $18,000 Texas $32,000 SOURCE: Harris and Taylor (2008a).
OCR for page 45
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series TABLE 5-2 Estimated Costs of Assessment and Accountability Per Student Florida North Dakota Texas State Test contracts $15 $34 $20 Other assessment $10 $10 $3 Accountability $8 $1 $1 Local Test administration $35 $24 $24 Professional development $53 $64 $65 Data management $15 $6 $11 Total $136 $139 $123 NOTE: Estimates are the lower bound estimates for real resource costs. SOURCE: Harris and Taylor (2008a). TABLE 5-3 Additional Fixed Annual Costs of Standards, Assessment, and Accountability Cost of Annual Test Reviews Florida N/A North Dakota $61,000 Texas $842,000 NOTE: N/A = Not available. SOURCE: Harris and Taylor (2008a). for preK-12 education are $15.5 billion for Florida, which has 2,539,929 students; $711.0 million for North Dakota, which has 104,225 students; and $28.2 billion for Texas, which has 4,259,823 students (http://www.edweek.org/rc/states/ [April 2008]). Harris and Taylor cautioned that their figures are estimates, because of the many factors that complicated the analysis. For example, in some cases data were available in one state but not another, so they developed estimated figures. Nevertheless, despite significant differences among the three states, the per-student estimates were remarkably similar and provided a reasonable basis for national estimates. Adjusting for comparatively low cost-of-living averages in the three states studied, Harris and Taylor came up with an estimate of $124 to $173 as a range of per-student cost across the nation. Multiplying that by the nation’s student population, they came up with a range of $6.1 to $8.5 billion as the total national expenditure on standards, assessment, and accountability.
OCR for page 46
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series TABLE 5-4 Total Fixed and Variable Costs for Three States ($ millions) Florida North Dakota Texas Total fixed costs $27 $1.4 $14.3 Total variable costs $328.0 $12.9 $528.0 Fixed costs as a percentage of annual per pupil costs 7.6% 9.8% 2.6% SOURCE: Harris and Taylor (2008a). The next step was to estimate the cost impact of switching to a common system. For this analysis, Harris and Taylor broke down for the three states the fixed costs (e.g., the standards themselves, annual test reviews, and state assessment divisions), which stay the same regardless of the number of students served by the program, and the variable costs (e.g., testing contracts, professional development, local data management), which vary with the numbers of students involved. Estimates for these two types of costs are shown in Table 5-4. From these numbers, Harris and Taylor were able to estimate the total cost savings of converting to a system of common standards: the range was 2.6 to 8 percent, or $183 to $850 million per year. Characterizing the saving as “pretty modest,” Harris suggested that “we can conclude that the cost-savings issue is not going to be the main reason for going to a common system.” However, he and Taylor also developed cost estimates for some of the activities that states undertake as part of standards-based accountability that are not required, such as conducting quarterly benchmarking assessments, developing higher quality (not machine scored) assessments, initiatives designed to support low-performing schools, and teacher performance incentives. These costs significantly increase the total cost of states’ programs and may exceed the cost complying with current policies. Harris pointed out that their estimates are significantly higher than others that have been suggested—about six times higher. He attributed this in part to the fact that theirs is the first to be developed following enactment of NCLB and thus includes the expanded requirements of that law. He and Taylor also included a broader range of state and local costs. Harris and Taylor’s portrait of the complexity of estimating costs and identifying potential savings that could be achieved with a model based on common standards stimulated a range of reactions. Discus-
OCR for page 47
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series sant Thomas Toch was struck by how small the investment has been. He argued that standardized tests have become, by default, the central driver of standards-based reform, the mechanism that largely determines what is taught and when and how it is taught. Yet of the average $8,000 spent per pupil annually, only one-half of 1 percent goes toward building the tests that are to measure progress toward the high standards that are widely supported. From his perspective, a move to common standards and tests would, by reducing the significant financial burden to states of developing their own standards and assessment programs, free resources for other targeted investments in improving teaching and learning. Discussant Susan Traiman also voiced concerns about the quality of the existing system. She questioned the usefulness of estimating the cost of producing systems that have received relatively low marks for quality, referring not just to the three states that were analyzed, but to all 50. She was also concerned that the approach Harris and Taylor used did not account for relative quality among states’ standards, pointing out that a state with a smaller number of very focused, thoughtfully developed standards might have a higher per-standard cost than a state with a multitude of standards. She argued that a more critical question was what kinds of resources the nation and the states are willing to invest to make sure that all students meet rigorous standards. Other participants added to this point, noting that of most interest would be a sense of the cost of developing common standards that had the effect of helping more students meet high standards. To calculate the cost in that way would mean including the costs for such interventions as professional development courses and new instructional materials. Discussant Dave Driscoll pointed out additional costs that could be considered in light of the earlier discussion about resources and capacity. One example is the cost to Massachusetts of releasing every single test item that is used so that the process is completely transparent to students, parents, and the public: doing so increases the cost substantially (other states bank many items for reuse), but it has important political and practical benefits. Driscoll also noted that “at the beginning we had plenty of money … the legislature provided whatever we needed.” Over time, however, budgets have been squeezed. The Massachusetts legislature had initially provided extra funds to support students who were in danger of failing the high school graduation test, but as first-time pass rates passed 80 percent, those funds were discontinued. Discussant Mark Harris considered the capacity issue from a qualitative perspective. He observed that although states prize their autonomy and flexibility in developing systems that will best serve their students, many nevertheless base much of their instruction on commercially available programs that have very little connection to state standards. These
OCR for page 48
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series programs are often designed to provide so-called teacher-proof curricula and instructional plans and thus do very little to develop the capacities of teachers who use them or to push the state-specific education goals forward. Participants proposed many potential costs that had not been considered in the analysis, while acknowledging both the complexity involved in developing the estimates and their value as a starting point for discussion. Laurie Wise summed up the message that many drew from the consideration of costs with a reminder of the fundamental question: “Is what we are investing in actually helping students to meet these standards, not just in defining and measuring them?” ESTIMATING THE COST OF EDUCATIONAL ADEQUACY The estimates provided by Harris and Taylor were for the costs of developing and maintaining a standards-based system, but the ultimate purpose of these tasks is to provide every student with a so-called adequate education. Educational adequacy is a concept that has gained currency in the context of lawsuits that have been brought in many states to force their governments to address persistent inequalities in facilities and instruction. The question of what implications a move to common standards might have on adequacy lawsuits is addressed below. Lori Taylor explored the question of how one might measure educational adequacy and its costs.2 Although there is no firm consensus about how one should measure adequacy in an education context, there are two approaches that researchers use, she explained. One is a bottom-up analysis, in which the researchers try to define a model, or prototypical, school and then calculate what it would cost to replicate that school in all of the settings found in a particular state. This can be done either by asking experts to describe a school that will meet a set of performance standards (professional judgment method) or by considering what resources would be necessary to replicate the schools designed by whole-school reform models that have been proven effective by some agreed-upon metric (evidence-based method). These approaches have been used in just over half the states over the past decade. Alternatively, top-down analyses examine the existing relationship between student performance, school expenditures, and other cost factors in a state and use those findings to predict the cost of meeting specified performance standards. One method is to use a variety of data on student performance to identify a set of schools that are successful and then 2 Taylor’s remarks were based on Baker, Taylor, and Vedlitz (2008).
OCR for page 49
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series predict that the cost of providing an adequate education is the average cost of providing education in those successful schools (successful school method). Another is to use cost and performance data from schools to estimate the relationship between expenditures and the prices of resources, student outcomes, and student need characteristics (such as the fraction of the study body that is receiving compensatory education for low socioeconomic status, school-district size, and any other variables that may affect the cost of providing education). The estimate of adequacy is the predicted cost of achieving designated outcomes in settings with particular local characteristics (cost function method). Fewer than 20 states have used one of these methods, and some have used both top-down and bottom-up approaches. These various approaches yield somewhat different results, with the minimum cost of adequacy, in 1999 dollars, ranging from $3,000 to $9,000 per pupil. The highest estimates tend to come from the professional judgment studies, the lowest estimates tend to come from cost function studies, and results can differ even within states. For example, in Missouri, three types of studies have been done, with different results: the professional judgment method yielded an estimate of approximately $7,500, and the successful school and cost function methods both came in near $5,000 per pupil. Taylor suggested several reasons for these differing results. First, there are differences in what the cost estimates actually represent. For example, researchers might have reported only an average of the total cost per district or per pupil, or they might have reported the base cost of regular education programs and treated separately the additional costs for special, bilingual, and compensatory education. The successful school method may be based on the average costs at schools that have relatively advantaged student populations, thus understating the actual costs of replication. Another factor is differences in definitions of adequacy. As discussed earlier, states have widely divergent standards of proficiency, and definitions of adequacy may be pegged to those descriptions. Moreover, top-down estimates are based on a quantifiable standard that is already being achieved, while professional judgment studies develop estimates of performance standards that are not necessarily being achieved and are difficult to quantify. Differences in school district size and student needs can also yield substantial differences in per-pupil costs, so differences in the ways these studies adjust for scale and need are another important source of variance. For example, Taylor pointed out, in Texas there are districts with fewer than 40 students. Thus the superintendent’s salary and other fixed costs are divided by a very small number, which would contribute to a very
OCR for page 50
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series high per-pupil estimate. Districts with high proportions of disadvantaged students, who require various supplementary services and resources, will also have higher costs, but these costs are typically not factored into successful school and evidence-based studies, whereas they are in cost function and professional judgment studies. Even when adjustments are made, there are several methods of estimating the necessary adjustment, particularly with respect to the costs of educating disadvantaged students, which further contributes to the variation. A further complication is the intersection of the extra costs of educating underserved students with the proficiency level that is used as a bar. The differential cost of getting low-income students to a relatively low cut-point standard might be modest, but the differential cost of getting low-income students up to the proficiency standard might increase in a disproportionate way with higher and higher performance standards. Moreover, Taylor added, “we are using the same household income thresholds or poverty thresholds in New York City as we use in Dalhart, Texas.” Thus, the instrument’s ability to discriminate between truly needy children and moderately needy children is much stronger in some states than in others, which will lead to much stronger estimates of the differential costs of serving those students in some states than in others. Finally, policy differences among states and districts affect the estimates as well. Policies regarding class size, length of the school day and year, teacher compensation, and other factors may mean large differences in the cost of providing services from one district to the next, which will in turn yield differences in the cost of adequacy. There are strengths and weaknesses to each of these approaches to measuring adequacy, and, in Taylor’s view, no single one of them is clearly superior either theoretically or practically. What is clear, she argued, is that increasing student performance will require either more resources, a reallocation of resources from some other purpose, or an increase in school efficiency. As a result, estimating the cost of meeting common standards would be a massive undertaking. Existing studies are not well designed to support an estimate of the cost of meeting common national standards. Ideally, this analysis would be done on a state-by-state basis, so that it could examine the ways existing state performance levels differ from the intended common performance standards, and factor in local prices, local union regulations, and the numerous other factors that affect the cost of education. Regardless of whether the analysis is done state by state, the choice of methodology is likely to affect the results, so multiple estimates would be more reliable than a single one. However, Taylor concluded, “all the estimates are likely to indicate a substantial increase in costs … potentially orders of magnitude greater than the actual costs of standards, assessments, and accountability.”
OCR for page 51
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series Participants responded by discussing the way the question is framed politically—with most recognizing that saving money is not likely to be a persuasive reason for adopting common standards. One noted that the costs are really the costs of providing a good education, and that parceling out the elements as if some are optional may disguise the obligation. Another noted the complexity of making a political case for the benefits of increased spending for a policy approach. There are too many cases, such as the push to reduce class size, for which it was never possible to muster sufficient evidence to prove that the expenditure was worthwhile—and achievement gaps remained. The challenge is to sell the whole package—to make the case for a holistic approach to thinking about what constitutes a good education and side-stepping prolonged arguments about the costs and benefits of isolated strategies that are unlikely to be sufficient in themselves to have meaningful impact. The two presentations dealt separately with the costs of complying with specific legal requirements and the costs of actually meeting standards. The fact that the analyses are so different highlights the equity implications of the relationship between funding, notions of what should be considered “good enough” for students, and how much both vary from state to state.
OCR for page 52
Common Standards for K-12 Education?: Considering the Evidence - Summary of a Workshop Series This page intentionally left blank.