Improving the Productivity of Schools
The 1980s and 1990s saw public attention shift from the paramount concern for educational equity that characterized the late 1960s and 1970s to a growing concern for how well schools are performing and how effectively the nation's huge education budget is being spent. Skepticism about the quality of schools was reinforced by a series of critical reports from blue-ribbon committees and commissions—15 of them in the single year of 1983 when A Nation at Risk came out (Guthrie et al., 1988:151). Questions about whether the public was getting value for money from its education spending were reinforced by a highly visible debate among scholars about whether money matters in determining the quality of education provided to the nation's schoolchildren. These developments pushed the question of the productivity of education to the forefront of the education policy agenda and highlighted the importance of learning how to spend education dollars wisely.
Sorting out what is known about educational productivity is crucial for its own sake, but also because of its implications for achieving educational equity. How readily this concept can be translated into practical school finance systems depends on how well financial resources are used to produce the desired educational outcomes. Likewise, the fate of efforts to align school finance systems with efforts to accomplish key education goals—raising achievement levels for all students and breaking the nexus between student background characteristics and student performance—hinges on the ability to make finance decisions that lead to improved productivity of schools.
While interest in obtaining education as inexpensively as possible has long been a concern of those who finance schools, it took a back seat following World
War II to the need to expand the school system quickly to cope with rapidly rising enrollments stemming from the baby boom. Scholars and reformers returned to the issue of how best to use education dollars with new vigor in the years after publication of the Coleman report (Coleman et al., 1966), with its finding that resource differences apparently had little effect on the outcomes of schooling. To the traditional efforts of psychologists and educational psychologists to understand learning and how to enhance it were added new efforts by economists and others interested in understanding how the resources purchased by schools were linked to educational outcomes (Monk, 1990:312).
Much has been learned from the investigations of these researchers. The issue of educational productivity remains a complex one, however. For a number of reasons, the concept itself is elusive and difficult to measure. There is as yet no generally accepted theory to guide finance reform efforts; rather there are multiple theories, each of which is incomplete. The various theories do not generate consistent strategies for action. In addition, empirical studies seeking to determine the best ways to direct resources to improve school performance have often not produced consistent findings. This is not surprising, given the conceptual difficulties and data limitations.
All of this is not to argue that there is no useful theory or evidence about promising avenues to pursue to increase educational productivity: there is a great deal. Shortcomings in current scientific methods for studying educational productivity, however, mean that much of existing knowledge is best viewed as tentative and contingent. The chief implication of this fact for school finance is that good policy will reflect both the best knowledge available to date and the need to continue experimenting and evolving as new knowledge becomes available.
DEFINING AND MEASURING PRODUCTIVITY
"Educational productivity" is a term with a variety of possible meanings.
When Americans say that they want their schools to perform better, they are saying that they want their schools to be more "productive." This usage of the term corresponds to the dictionary definition of productivity as the ability of an entity to produce abundantly or to yield favorable or useful results.
Productivity also has a narrower definition, however, one drawn primarily from economics. The economic perspective on productivity emphasizes the relationship between outputs and inputs of a firm or organization or economic sector. At its most familiar, perhaps, productivity is a measure of output per unit of labor. Statistics on productivity measured in this way are routinely gathered, especially for private-sector firms, and widely reported. They have also been the subject of much public discussion over the past 30 years, especially because private-sector productivity growth in the United States had suffered a slowdown beginning in 1973 and lasting until the 1990s, reversing the economic conditions that had prevailed and undergirded American prosperity since World War II.
In education, productivity is often taken to mean using the inputs and processes of schooling in ways that increase desired outcomes. The most common measures of outcomes have been students' academic achievement while they are in school (often measured by scores on standardized tests) and student performance upon entering the labor market (generally measured by wages) (Burtless, 1996:3).
This comparatively narrow view of the desired outputs of education, and therefore of the meaning of productivity in education, does not take into account the variety of goals that Americans typically hold for their schools. Improvements in character, citizenship, and physical and mental health are just some of the nonacademic outcomes that schools have been expected to foster. Moreover, schools (like churches, for example) are often valued for the quality of the experience itself. For example, many people would argue that an important aspect of school performance is the kind of environment children experience during their many years of enforced school attendance. In other words, the process of schooling may be a valued aspect of school performance in and of itself, distinguishable from the value placed on the outcomes of education.
The variety of goals Americans hold for their schools is a key reason why educational productivity is hard to define and measure. To make operational the concept of productivity first requires the specification of which educational outcomes are of primary interest. While applying the concept of productivity in education is often frustratingly hard, it is worth observing that contrary to popular wisdom, productivity is also a complex topic in business. The committee found that the difficulties in grappling with the concept echoed those of an earlier National Research Council panel, which was established to think about how improved organizational linkages might contribute to productivity growth in American business. That panel found itself similarly torn over the appropriate concept and definition of productivity to use. It noted (National Research Council, 1994:8):
Perhaps the panel is not alone in being unable to arrive at a consensus. In a review of the literature on productivity, Pritchard (1991) found that the term productivity was used to encompass constructs as diverse as efficiency, output, motivation, individual performance, organizational effectiveness, production profitability, cost-effectiveness, competitiveness, and work quality . . . .
Panel members held one or the other of two positions regarding the concept of productivity. Some wanted to define productivity as the ratio of outputs to inputs, in line with the original definition of the term by labor economists. They believe that this is the only definition that is unique to the concept. Others argued that this definition is too restrictive. They believe that productivity must encompass concepts such as quality and effectiveness to be meaningful.
That panel resolved its dilemma by adopting a systems model of organizational performance and recognizing productivity as but one of seven interrelated
and interdependent criteria of organizational performance. For that panel, the seven criteria were effectiveness, efficiency, quality, productivity, quality of work life, innovation, and profitability (or, for cost-center organization systems, budgetability—Sink and Smith, 1994:134–7). In this view, productivity provides just one part of the performance picture, and the different criteria of performance in addition to productivity might each require several different measures.
In the committee's judgment, it is essential to recognize that the question of improving school performance similarly requires consideration of a variety of criteria in addition to the input-output ratio orientation of the economic definition of productivity. Likewise, the multiplicity of process and outcome objectives of education must be reflected in any evaluation of whether American schools are accomplishing what the nation demands of them.
For the purposes of this report, nevertheless, we deliberately adopt a narrower focus for productivity analysis, one in keeping with our charge: What do we know about how to improve school performance in terms of improving the academic achievement of students? Academic achievement is widely (although perhaps not universally) recognized as a key objective of education. It is the most frequently measured outcome. It is generally viewed as a springboard to the "good life," defined not only as economic opportunity via participation in the labor market but also as opportunity actively to engage in and benefit from the social, cultural, and political aspects of society.
Settling on academic achievement as the focus of interest in school performance, however, reduces but by no means eliminates the difficulties in defining and measuring educational productivity. Several additional problems deserve mention.
First, as discussed in Chapter 4, there are serious disagreements over how well existing measures capture the most significant aspects of academic achievement.
Second, weighing outcomes is a problem even if one narrows the discussion to something as seemingly simple as academic achievement measured by test scores in various subject areas or as graduation from high school. This situation differs from the private sector, where outcomes can be weighted by prices. For example, consider a firm that produces cars and trucks. Total output can be measured as the sum of the cars and trucks with each weighted by their price. The prices are appropriate weights because they reflect both the consumers' valuation of the two types of products and the relative costs of producing the two products. With respect to education, the appropriate weights for the various outcomes is not obvious and, because different people or groups of people are likely to have different values, is ultimately a political issue.
Resources—that is, the inputs used in the education process—are also multidimensional. However, because resources are purchased through the private market, it is not unreasonable to add them together using their prices as the weights. One major complication arises to the extent that different levels of
resources are used for different students. For example, if some of the teachers in a school are used to teach special education students and those students are in smaller classes than regular students, it would be inappropriate to treat the average number of teachers per student (based on all students) as the resource level for educating regular students. This issue is especially salient to the extent that special education or other students are not tested under a state's regular system for assessing student performance. That is, when some groups of students are excluded from the state's measure of student performance, the resource costs of educating them should not be included in the measure of inputs.
A further complication arises because a child's family background and certain characteristics of the community in which he or she lives (for example, the incidence of poverty) contribute significantly to his or her educational success. By highlighting these types of factors, the 1966 Coleman report (Coleman et al., 1966) and recent work by Miller (1995) and Steinberg (1996) suggest that educational outcomes are not produced by schools alone.
Actually, the situation is even more complicated because human interactions and motivations are at the heart of the educational process and these interactions and motivations are constantly in flux and are difficult to observe (Murnane and Nelson, 1984:368–9). Good teachers continuously adjust their "production techniques" to the particular situation of their classroom and the individual students in it. Students, who are the presumed objects of the educational process, are also subjects: active, thinking, feeling, inconsistent human beings whose attitudes and moods and values and varying degrees of willingness to commit themselves to academic pursuits influence productivity mightily. And many of them, in addition, do not think of themselves foremost as students. They are young people trying to work out their needs and longings and insecurities and identities in this vast arena called the school because that is where society tells them they have to go, whether or not they take the formal learning part seriously. Moreover, the way in which these students engage or decline to engage with their studies in the school, and the values they acquire and the behavior they exhibit, are shaped as much (or more) by the peer culture and the youth media environment as by school inputs.
For all of these reasons, defining and measuring the productivity of American schools is far from an easy task. Over the years many have expressed skepticism about how accurately or fully production relationships in education can ever be captured (e.g., Levin, 1974; Guthrie, 1976; Murnane and Nelson, 1984; Monk, 1990; Hanushek, 1997b). This caveat must be kept in mind in reviewing the research on improving the educational performance of schools and the academic achievement of their students.
UNDERSTANDING EDUCATIONAL PRODUCTIVITY
Scholars from a number of disciplines have conducted studies aimed at understanding how educational resources are linked to the academic achievement
of schoolchildren, resulting in what Monk (1990:315) has characterized as a "large and unwieldy body of research." The growing breadth of academic perspectives being brought to bear on educational productivity is a good sign for the ultimate prospects of understanding the complex relationships involved. The downside of all of this activity, however, is the difficulty of trying to ascertain what is known about educational productivity and sorting out the diverse and sometimes conflicting messages emerging from studies that ask different questions, draw on different theoretical models, utilize different research approaches (in methods of both selecting study subjects and collecting evidence), and weigh evidence differently.
Standards of evidence are particularly problematic when multiple research traditions are involved. Studies carried out utilizing quantitative research methods and random samples of schools or students may be easier to evaluate because "the canon of positivist, quantitative research is well established" and "there is a relatively high degree of agreement about what constitutes a good quantitative research design" (Kroesen et al., 1998:2). Yet such studies often suffer from serious limitations in the details they can capture about the important determinants of outcomes in an activity like education. Qualitative research methods (interviews, observations, ethnographies) can incorporate information about how schools work that extends far beyond the statistics gathered in surveys or administrative records and that probes not only which but how resources make a difference in learning. However, the methods used for choosing who will be studied in these qualitative ways sometimes lead to serious questions about the generalizability of the results. Moreover, the ''canon for qualitative research is less clear" and, according to one thoughtful guide to the literature, the profound philosophical differences among those who use these methods may mean "there may never be agreement on the 'standard' or 'good' ways of using qualitative research" (Kroesen et al., 1998:2).
We deal with these difficulties by making transparent the different research approaches underlying the findings and by pointing out the strengths and shortcomings of the incomplete theories and imperfect methods and data that characterize virtually all educational productivity studies.
The large and unwieldy research base seems to provide at least three alternative lenses for viewing the important relationship between educational resources and academic achievement. The first two, input-output studies and studies of effective educational practices, while differing significantly in method, both "rest on the notion that there is an imitable 'technology' of education, in that it is presumed that if one system or school or class can do something within certain effects, so can others" (Murnane and Nelson, 1984:356). Researchers in these two traditions focus on finding the resources or the educational practices that are associated with good performance and can enhance school effectiveness. They approach this task in dramatically different ways, however, with input-output researchers treating schools as a black box and focusing on identifying the statistical relationships between resources and achievement, while students of effec-
tive practice concentrating on specifying the exact nature of what goes on inside the classroom and the school that seems to matter for student performance. The third lens offers an institutional perspective on schools and educational productivity, suggesting that the environments in which schools function exercise a crucial influence on the educational choices schools make (or have thrust upon them). Like the first lens, the third lens is not specifically concerned with how schools use resources to improve effectiveness; instead, lens 3 focuses on creating conditions that will encourage school personnel to use their resources well.
We adopted these lenses as a heuristic device, not intending to suggest that research neatly falls into these categories. The lenses have proven helpful to us, however, in sifting a large amount of evidence and in understanding why the messages from research about educational productivity are so difficult to bring into sharp relief.
Lens 1: Input-Output Studies
Input-output studies (also known as studies of the education production function) have fueled the fires of debate over whether money matters in education since the Coleman report of the mid-1960s produced its surprising and counter-intuitive finding that school resources (at least those it measured) did not have much effect on the academic achievement of students.
The Coleman report became the progenitor of literally hundreds of additional studies, not only because of the controversial nature of its conclusions, but also because it represented one of the most massive and complex social science efforts that had been mounted up to that time. It involved a huge data collection effort (the Equality of Educational Opportunity Survey) that gathered information on 570,000 students and 60,000 teachers from a sample of 4,000 schools across the country. It entailed special testing of students in five grades on their verbal, reading, and mathematics comprehension. Students also filled out extensive surveys regarding their family background and other education-related factors (such as the amount of time they spent on homework, their classroom experiences, and their classmates). Teachers provided information on their educational backgrounds, professional activities, working conditions, and attitudes about school. In addition, extensive information on school facilities was also collected. Coleman and his colleagues utilized the most sophisticated statistical techniques available to look at the effects of school resources on student achievement while taking account of differences in the background characteristics of students and other likely influences on educational outcomes.
The Coleman report finding that drew the most attention and controversy and that has been studied and debated ever since was:
Taking all these results together, one implication stands out above all: that schools bring little influence to bear on a child's achievement that is independent of his background and general social context; and that this very lack of an
independent effect means that the inequalities imposed on children by their home, neighborhood, and peer environment are carried along to become the inequalities with which they confront adult life at the end of school. For equality of educational opportunity through the schools must imply a strong effect of schools that is independent of the child's immediate social environment, and that strong independent effect is not present in American schools (Coleman et al., 1966:325).
The research techniques and theoretical underpinnings on which the Coleman team and subsequent analysts have built draw on a production function framework first developed in economics to describe the relationship between the output of a firm and the inputs used in the production process. This framework assumes "that a systematic process governs the transformation of inputs into outcomes" (Monk, 1990:342). It translates this relationship into a mathematical function that can be applied to data based on the actual experiences of firms. The results express the amount of additional output that can be obtained from additional quantities of input.
Education researchers in the input-output study tradition typically (though not always) apply statistical multiple regression techniques to survey data on schools to measure the education production function. They attempt to isolate the effects of educational inputs that can be manipulated (typically those that can be purchased or otherwise directly shaped by policy) while controlling for influences not readily controllable by educational authorities. The quasi-experimental research design draws on the natural variation in school resources and other practices found in schools and school systems, rather than trying to manipulate these variations deliberately via true experimental research, such as that frequently conducted in the medical sciences. Only a handful of education experiments have been conducted, in which random assignment of subjects to treatment groups allows for direct measurement of treatment effects (thereby avoiding the potential for biased or misleading findings that can result from the need in quasi-experiments to control for important nonschool differences among subjects via statistical techniques). The most significant of the education experiments for this discussion of productivity involves a Tennessee study of the effects of class size on student achievement in the early grades, discussed below.
In the 30-plus years since the Coleman report was published, literally hundreds of input-output studies have been conducted seeking to confirm or deny the Coleman findings and extend the research to include additional possible explanatory variables and apply new or different research techniques.
Hanushek (1986, 1996, 1997a) undertook influential summaries of 377 educational production function studies, which led him to conclude that "there is no strong or consistent relationship between variations in school resources and student performance" (Hanushek, 1997a:141). Hanushek focused on what the input-output studies revealed about the impact of inputs frequently assumed to matter to educational achievement and therefore frequently studied: teacher-
pupil ratio, teacher education, teacher experience, teacher salary, per-pupil expenditure, administrative inputs, and school facilities. He conducted his summary using a synthesis method known as "vote counting." This method essentially involves examining the regression coefficients for the same resource from different studies; categorizing each result according to whether it indicated a positive or negative or zero effect on academic achievement; and tabulating the results to summarize the overall conclusion about the effect of the resource based on all the studies that examined it.
Hanushek's summaries have probably had an impact second only to the original Coleman report itself in persuading people that money (or school) doesn't matter in efforts to improve education. Therefore, it is important to emphasize his point that he cannot find systematic relationships between variations in school resources and student performance. This is quite different from saying that schools and their attributes never matter. As Hanushek has expressed his view in recent work: "This finding of a lack of any general resource relationship is, however, very different from finding that schools have no differential impact. A number of subsequent studies [i.e., subsequent to the Coleman report] document rather conclusively that schools have significantly different effects on student achievement, even if the good schools are not necessarily those rich in traditionally measured inputs" (Hanushek, 1997b:302).
One possible objection to Hanushek's literature review is that the variables he examined did not involve the full array of school-related input measures that might possibly influence student achievement. Monk (1990) reviewed studies focusing on an additional set of school inputs suggested by educational production theory: learning technologies, the uses of time, public versus private organization of schools, and the size of districts and schools. Like Hanushek, he found no conclusive evidence in the literature that these resources systematically contribute to higher student achievement.
For a quarter of a century after Coleman, the dominant view of scholars in the input-output tradition was skepticism about the possibility of finding reliable relationships that could guide policy makers in their decisions about how to allocate resources and organize schools in ways that would lead to improved student academic outcomes. Comparatively recently, the prevailing view that school inputs cannot be unambiguously linked to student achievement has begun to be challenged, although a new consensus about the nature of the input-output relationship has not yet emerged.
A major challenge to Hanushek's work has come on methodological grounds. Hedges, Laine, and Greenwald (1994a) argued that Hanushek's vote-counting method for synthesizing studies results in a bias against finding positive effects of resources on student outcomes. They point, for example, to the fact that while vote counting can be used to summarize the direction and significance of the effects of resource variables on outcomes, it cannot determine the magnitude of any statistical effect.
Hedges et al. therefore applied a statistical technique called "meta-analysis to the same studies that Hanushek examined. Meta-analysis is designed to combine statistical significance values from different studies that test the same hypothesis, thus testing the combined statistical significance of the relationship between school inputs and educational outcomes across studies (rather than counting each study separately). Hedges et al. concluded from their summary that the relationship between resources and student achievement is significant and generally large.
The debate among the synthesizers over what the literature says about whether money and schools matter has not yet been won by either side. Ongoing exchanges continue between Hanushek and the Hedges team; see, for example, Hanushek (1994) and Hedges et al. (1994b). Serious shortcomings in many studies of school productivity give both sides ammunition to question the conclusions drawn by the other about the meaning and significance of the results.1 A recent Brookings Institution effort to answer the question "Does money matter?" (Burtless, 1996) reflected the range of scientific disagreement on the influence of school spending without resolving the differences that have emerged from over 30 years of research on the subject.
Meanwhile, scholars continue to undertake new studies using improved datasets and statistical approaches designed to overcome methodological criticism of prior input-output research. Many of these studies are reporting positive findings about the relationship between school resources and student academic performance. Our sense is that the deep skepticism of the first 25 years after the Coleman report has given way in the academic community to a more cautious optimism that some regularities in the relationship may be identifiable through input-output research. Nevertheless, caution remains the operative word, since positive findings that schools make a difference (assuming that they hold up in repeated studies) may not easily translate into specific policies for improving the connection between resource use and results.
This dilemma can be seen in considering the implications of recent research suggesting that factors relating to teachers (especially teacher quality and class size) are significant in explaining differences in student academic achievement.
A much-cited study by Ferguson (1991) dealt with one major criticism of
input-output studies (the limited number of explanatory variables they typically include) by drawing on a large and unusually complete set of data assembled by school districts for the state of Texas. The dataset included comparatively rich information on teachers (such as their scores on a statewide recertification exam as well as their experience, education, and salary levels), average school characteristics (such as school size), as well as district characteristics such as total enrollment and pupil-teacher ratios, measures of school spending by major category, the size of the district's property tax base per student, and characteristics of surrounding districts as well as of each district and its students itself.
Ferguson found in Texas that the quality of schooling explained between a quarter and a third of the variation among Texas school districts in students' scores on statewide reading exams and that most of this effect was due to a single measure of teacher quality: teachers' performance on the recertification exam. Teaching experience and educational level also mattered, as did class size.2 Reducing class size appeared to be very important in the primary grades. Ferguson and Ladd (1996) followed up his Texas work with an analysis of data for Alabama, making some methodological improvements (e.g., measuring class size directly rather than rely on a proxy measure and disaggregating data not just to the district but also to the student level). The analysis largely confirmed the earlier Texas findings and provided "strong support for the hypothesis that measurable school inputs affect student learning. . . . The primary unresolved issue is the level of the class size threshold below which further reductions would lead to no additional systematic gains in student learning" (Ferguson and Ladd, 1996:288).
An issue that arises in production studies in education is the direction of causality. It has long been recognized in production function research that biased estimates of coefficients can arise if there is two-way causality (see Berndt, 1991:457, for a brief discussion and references to the literature). Production function studies in education presume that the direction of causality is from inputs to achievement. This is eminently plausible. However, it is also quite plausible that causality also runs in the opposite direction. For example, the work of Hanushek et al. (1999) suggests that teachers prefer certain types of students over others. If more able teachers seek out schools with higher-achieving students, then a classic case of simultaneous equation bias is present. Likewise, if districts spend more for education when achievement is higher, then again reverse causality may be present. The neglect of potential simultaneous equations
bias in much education research is arguably an important reason for inconsistent findings across different studies.
The difficulty of sorting out causality in quasi-experimental studies helps explain why truly experimental studies, though few in education, are often quite influential. The most significant example is a major class size experiment launched in 1985 in Tennessee.
Project STAR (summarized in Finn, 1998) ran from 1985 to 1989 in 79 elementary schools in Tennessee. Entering kindergarten students were randomly assigned to one of three class types: small (enrollment of 13–17), regular (enrollment of 22–26), or regular with a full-time teaching aid in addition to the regular teacher. Classes remained the same type for 4 years, through 3rd grade, while a new teacher was assigned at random to each class each year. About 7,500 pupils in more than 300 classrooms participated. After the original STAR project ended, Tennessee authorized a follow-up study (the lasting benefits study) to see how long the original benefits of small classes would persist.
Differences in the three class types were highly statistically significant, thanks to achievement gains in the small classes and not in the regular classes with aides. The benefits of small classes were found to be greater for minority students (most of whom were black) and for students attending inner-city schools. After kindergarten, the effects on reading and mathematics achievement were typically twice as large for blacks as for whites (Nye et al., 1993) and even larger for blacks in inner cities (Krueger, 1997). The effects were robust even after sensitivity analysis examined several limitations in the study design and implementation (Krueger, 1997), although researchers have questioned the extent to which meaningful gains occurred after the first year of enrollment in a small class (Hanushek, 1998).
Because of the experimental nature of the class-size study (and, no doubt, because its results correspond to the belief of many parents and teachers that smaller classes are better than larger ones), the Tennessee results have spurred efforts around the country to reduce class sizes, especially in the early grades. While not necessarily disputing the Tennessee findings, however, scholars have questioned whether reducing class sizes is the more effective use of resources.
Hanushek et al. (1998), for example, have drawn on the Texas database mentioned earlier, now augmented by longitudinal data on academic test scores for several cohorts of students at different grade levels, to examine whether there are significant differences among schools in their ability to raise academic achievement, what characteristics of schools seem to account for any differences in impact, and whether any such differences are systematically related to school resources or to measurable aspects of schools and teachers. While they found that schools vary greatly in the impact they have on student achievement and that the differences centered on the differential impact of teachers, they also found that differences among teachers are not readily measured by simple characteristics of the teachers and classrooms. In other words, the study provides strong
support for the idea that teacher quality matters but indicates that the limited set of teacher attributes usually captured in input-output studies (those that directly affect school costs, like how much experience a teacher has or whether he or she has a master's degree) do not explain much of the variation in teacher quality. Class size had some impact on children from low-income families (especially younger children), but the effect of class size was swamped, according to the authors, by differences in teacher quality. The authors thus implicitly question the current weight being given in policy circles to class size reduction. They also point out a series of questions about class sizes that are highly relevant for policy purposes but about which there is as yet little or no knowledge.3
Using a comprehensive, longitudinal database and statistical mixed-model methodology developed to support the Tennessee Value-Added Assessment System, Sanders and his colleagues also found that teacher effects were the dominant factors affecting student achievement gains. Their results indicated that classroom context variables (including class size) were comparatively unimportant (Sanders and Rivers, 1996; Wright et al., 1997).
The input-output lens continues to generate provocative insights into educational productivity, and improved data and research methods may result in fewer conflicting findings in the future than in the past. Monk (1992), in his update on educational productivity research, cautions against assuming too quickly that new optimistic findings will be sustained upon further examination. Past history explains part of his caution, but so does the continuing fact of
serious conceptual inadequacies in the underlying productivity model. Despite these studies' growing econometric sophistication, they remain fundamentally primitive black-box formulations where analysts have made little progress toward modeling what makes education distinct from other types of . . . production function techniques. In particular, scant attention has been paid to the nested nature of educational production wherein schools themselves produce inputs that are subsequently (or even simultaneously) used in the production of final outcomes. Neither has much progress been made toward modeling dynamic aspects of educational productivity. Instructional realities are not static and do not reproduce themselves in simple ways. The failure to model the changeable nature of education production processes is a serious limitation on this line of research (Monk, 1992:309–10).
Research that examines school performance through the lens of effective practice, by contrast, does attempt to get into the black box and offers additional perspectives on how resources and student academic performance might relate.
Lens 2: Studies of Effective Educational Practice
While input-output studies largely trace their parentage to the 1966 Coleman report, scholars in schools of education and in fields such as psychology, sociology, and anthropology have been seeking to identify and understand the factors influencing school performance since the turn of the century.
Research in the effective practice tradition, unlike the economic research described above, is not guided explicitly by the idea of a production function in education, although much of it, too, appears to have been a search for a "technology" of education that could be identified and replicated. The search for this technology has been carried out in part through statistical studies, but more commonly via other methods such as case studies and program evaluations. These more qualitative approaches have enabled researchers to examine aspects of schools not readily captured in large-scale surveys or in administrative record-keeping systems, but at the price of raising serious questions about the reliability of the results and their generalizability to schools other than those specifically studied. Monk illustrates the difference in describing a subset of effective practice research, called effective school studies, of the 1970s and 1980s, but his point applies to the larger research tradition as well: "Instead of focusing on inputs and attributes, the alternative strategy is to focus first on outcomes. In particular, the idea is to identify a school believed to be unusually successful according to some criteria. Once identified, the school becomes the subject of intensive study, often making use of observation and interview data. The goal is to understand why and how the school attains its success. The underlying hope is that through the accumulation of such studies of successful . . . schools, insights can be gained into how to improve education offered elsewhere" (Monk, 1990:413).
Such an approach raises concerns among statisticians because it is likely to generate upward-biased estimates of the explanatory variables. In addition, by focusing on successful schools alone, researchers are unable to determine which factors are responsible for the superior measured outcomes rather than simply being correlated with them, and they are unable to determine which factors are of secondary importance. While the effective practice approach yields the benefit of intensive observation of a small sample, it has traditionally foregone the larger sample benefits of the less fine-grained production function approach.
Increasingly, though, scholars are drawing on insights from qualitative studies as well as developments in statistics (such as hierarchical linear modeling, which can help disentangle individual from organizational or group effects) to
search for evidence of stable relationships using quantitative research designs that can address selection bias and other methodological concerns.4
Many studies of effective practice concentrate on teaching techniques, curricula, and organizational design, hoping to identify effective approaches that can be transferred to other educational settings. Unlike the input-output studies, which stemmed explicitly from a desire to explain the influences on academic performance, studies of effective practice frequently refer to measures other than or in addition to academic achievement. In fact, specific links to improvements in academic achievement have often been assumed or implied rather than explicit.
Thus, another important difference between input-output studies and traditional studies of effective practice is in the questions and evidence they consider. Smith et al. (1996) point out that in most of the former, "there is no measure of whether the teachers and schools in the surveys were focusing their instruction on bringing all students to achieve to high standards. There are not even any measures of the curriculum coverage or the depth to which material was taught. There are no adequate measures of teachers' quality, their knowledge of curriculum, or their ability to engage students. There are no measures of the degree to which schools have the autonomy and responsibility they need to design effective strategies or of the degree to which the overall district and state systems support the efforts of the schools . . . The school survey data used by most researchers generally do not include these measures. Very little of what has been found to influence achievement by psychologists, sociologists, and political scientists who actually get into and study classrooms and the educational system is ever evaluated as an 'input' in surveys" (Smith et al., 1996:21–2).
Researchers seeking to identify effective practice also reach outside education for evidence about improving performance in other kinds of organizations that might be applied to schools. Odden and Busch (1998:26–7) list a series of strategies used by organizations seeking significant improvements in performance: "set clear performance goals at the top, flatten the organizational structure, decentralize power and authority to work teams, involve employees in making key decisions about how to organize and conduct their work, invest heavily in capacity development, and hold teams accountable for results."5
While the research base of studies in the effective practice tradition is so large that it defies a short summary, Smith et al. (1996:15–18) identified eight key ideas growing out of the past 25 years of research on educational effectiveness that have been important in influencing current thinking about education reform:
All students can learn to far higher levels than we ever imagined in the past.
What a student is taught matters.
The quality of teaching matters.
Teachers are more likely to teach well things that they understand well and that they have been taught to teach.
Schools, and the teaching and learning that occur in them, are more likely to change when the staff of the school has ownership and some control over the nature of the change.
Teachers and the public do not have a common conception of what is meant by high and internationally competitive academic standards.
Individual school reform has a long, complex, and unhappy history in the United States.
The education system often does little to support change or to sustain schools that appear to be effective.
Unlike the findings from input-output studies, which have tended to be inconclusive about how to change schools to improve performance, research on effective educational practice has been used to develop logically coherent and research-based designs for improving education.
A central theme in current reform efforts is that improvement requires changing what happens in classrooms between teachers and students. Most previous attempts to reform education have not really had much effect on teaching and learning (Elmore and McLaughlin, 1988; Tyack and Cuban, 1995). School reforms thus appear to be a constant in American education, but the intensity and visibility of reform efforts tends to come in cycles, the latest of which was spurred by the raft of critical reports that emerged in the early 1980s.
Some scholars (e.g., Smith and O'Day, 1991) argued that the first wave of school reform in response to A Nation at Risk (National Commission on Excellence in Education, 1983) and other criticisms of public schooling failed to produce meaningful gains in learning because it followed a conventional, top-down, ''more of the same" strategy to educational change: expanding and improving educational inputs (longer school day, increased graduation requirements, better teachers) and ensuring competency in basic skills (graduation tests, lock-step curricula, promotional criteria). It "did little to change the content of instruction, to directly involve teachers in the reform process, or to alter the reigning notions of teaching and learning" (pp. 233–234).
A second wave of reform that began building in the late 1980s addressed
these shortcomings by calling for fundamental rethinking and restructuring of the process of schooling. Second-wave thinking was distinguished by the argument that schools, as the basic unit of productivity in education, ought to be the unit of improvement. "Upgrading classroom life is best done on a school-by-school basis. Teachers assist each other. Principals help create the setting and secure additional help. The action and rewards for in-service education and school improvement shift from where they have been traditionally—with the superintendent's office and districtwide activities—to the principal's office and the school as the key unit. Research increasingly supports such a process" (Goodlad, 1984:129).
The focus on improving practice at the individual school overcame objections that traditional reform strategies ignored (1) the idiosyncratic and context-dependent nature of education production due to the human relationships involving teachers and students that determine the effectiveness of teaching methods (Murnane and Nelson, 1984; Elmore and McLaughlin, 1988) and (2) the historical evidence from a century of public school reform that schools change reforms as much as reforms change schools (Tyack and Cuban, 1995).
School-based reform strategies focus on both classroom instructional programs and on school (and to a much lesser extent, district) restructuring. Smith and O'Day (1991:234) reported that the second wave of reform produced in short order an "avalanche of ideas, strategies, and structures." Evidence of this avalanche can be found in a recent report (Northwest Regional Educational Laboratory, 1998), prepared to help implement the Comprehensive School Reform Demonstration (CSRD) program enacted by Congress in 1997. CSRD provides financial incentives for schools, particularly Title I schools, to implement school reform programs based on reliable research and effective practice and including an emphasis on basic academics and parental involvement. The Northwest Regional Education Laboratory made an initial listing of 26 whole-school reform models and 18 skill-and content-based reform models for schools to consider in developing proposals for CSRD funding, while emphasizing that the list was not comprehensive and was not meant to limit the array of reform ideas being considered by schools, districts, and states.
School-based change efforts typically emphasize capacity building (especially the professional development of teachers) and decentralized decision making (including parental and community involvement). Despite these general similarities, reform models differ widely when it comes to specific practices in such areas as instructional strategies, types and uses of assessment, and features of school organization (e.g., school-based management) and climate. Wang et al. (1997) reviewed 12 widely implemented reform programs (both whole-school and skill and content); they identified 54 prevalent program practices in these programs of which they characterized 25 as being "firmly grounded in research on what influences student learning."
One of the most potentially significant developments of the second-wave
awareness of the importance of the individual school and the inefficacy of many earlier, piecemeal reforms has been the development of designs for change that call for restructuring the whole school, not just individual elements of it. Whole-school designs (the best known of which were listed in Chapter 4) provide schools with frameworks for restructuring virtually all aspects of the school, including curriculum, classroom organization, assessment, the allocation of decision-making authority over resources, and more. Frequently they use skill-and content-based models of reform in various curricular areas as building blocks in the comprehensive reform design.
Interest in implementing whole-school designs overlaps with other school-based reform proposals to alter educational management systems at the school and district level. The target of change here is the traditional centralized district management strategy, in which the central office has made all key decisions for the schools in its district: when and where to build schools and how large to make them, what the curriculum program would be (including selection of textbooks and instructional materials), how schools would be staffed (how many teachers, how many specialized personnel), what kind and how much professional development would be provided, and so forth.
One restructuring proposal to free schools from the constraints of traditional district control has been school-based management (SBM). SBM is not a new reform strategy: it emerged initially in the 1960s in response most particularly to the desire of inner-city residents to have more influence (community control) over their schools (Murphy and Beck, 1995; Tyack, 1993; Wohlstetter and Odden, 1992). Management changes that devolved some authority to the local level could be found in many large U.S. cities in the 1970s and 1980s, including New York, Chicago, Washington, DC, and Dade County, Florida. As concerns about educational productivity increasingly took center stage, SBM gained new adherents who drew lessons for schools from new organization patterns emerging in U.S. industries (Odden and Busch, 1998). Trying to overcome a long productivity slump, the latter were attempting to increase their competitive advantage in world markets by transforming themselves into high-performance workplaces that rejected the 20th century mass production factory model of organization for a new model emphasizing flexible decentralization, participative management, and greatly increased attention to development of the firm's human resources (U.S. Office of Technology Assessment, 1990:115).
SBM has been practiced in many ways, but various approaches are built on one key premise: "that the school site becomes the central locus of control in decision making. The rationale behind SBM is that those who are closest to the primary business of schools will make the best-informed decisions. . . . The term 'school-based management' has many variations—school-site management, school-site autonomy, shared decision making, shared governance, school improvement program (or project or process), school-based budgeting, and administrative decentralization" (Summers and Johnson, 1996:76–77).
Three main models of SBM have emerged (Bimber, 1993; Wohlstetter and Odden, 1992). The first model places principals in control, with this individual serving as a chief executive with broad-reaching powers over budget, staffing, and program design. There may be some kind of school-site council, but it tends to serve in an advisory rather than decision making capacity. The second model focuses on administrative decentralization, delegating decision making to teachers and giving them broad discretion over the professional judgments needed each day as they encounter students in classroom learning situations. School-site councils under this model tend to give the greatest representation to teachers and other school-site educators. The third model is characterized by a shift of power from educators to community-based control. Most visibly exemplified in the 1988 Chicago school reforms, this approach shifts power from the school board and professional educators to parent and community representatives by giving them majority representation on school-site councils with significant control over budgets, personnel, management, and program design.
Evaluation of SBM has been hampered by the multiplicity of objectives and practices that have been pursued, by the fact that real devolution of decision making to the school level has been far less than the rhetoric around SBM would imply, and by the failure of most of the research on SBM (as revealed in a literature review by Summers and Johnson, 1996) even to address the question of effects on student achievement. Assessments based largely on SBM as it was implemented in the 1970s and 1980s suggested that it seldom had the positive benefits predicted for it, either in terms of improving student achievement or changing the behavior of schools and school participants (Malen et al., 1990; Murphy and Beck, 1995; Newmann and Wehlage, 1995; Summers and Johnson, 1996; Wohlstetter and Odden, 1992). More recent research (Odden and Busch, 1998; see Chapter 6) suggests that older strategies of SBM lacked the necessary organizational conditions for it to lead to improved student achievement.
While much restructuring has focused on the school level, there have also been efforts to reform district operations, to reorient them toward improvements in teaching and learning while retaining a significant measure of central control. One prominent example is Community School District #2 in New York City, which has based a comprehensive and sustained district-wide reform agenda on a strategy of instructional improvement through professional development. While more and more budget and administrative responsibility has been lodged at the school level, largely in the hands of principals, central control has remained strong in areas key to the success of the strategy, such as personnel decisions, the hiring of professional development consultants with expertise consistent with the strategy, decisions about which instructional areas will receive priority attention, and policies and practices that keep school-site decisions focused on district-wide priorities (Elmore, 1997a).
Changes in school and district practice, based to a greater or lesser extent on the research base developed using the lens of effective practice, have clearly had an impact on American schools. What is much less clear at this point is whether
these changes are making a difference in the academic performance of students. This is not to be critical, but rather to acknowledge two important facts about the status of knowledge on the effects of recent reforms. First, evidence of effectiveness will be complicated to find for some of the same reasons that bedevil input-output research: methods of analysis are contested; it is difficult to isolate the effects of particular changes because numerous things tend to change at the same time; and context may matter importantly in how reforms play out in different places, complicating the explanation of effects and limiting generalizability.
Moreover, new approaches to practice such as whole-school restructuring have not yet been fully implemented, yet alone independently evaluated. Implementing whole-school designs such as New American Schools and the Edison Project has proven more time-consuming and complex than designers originally anticipated (Bodilly, 1998; Chubb, 1998; Glennan, 1998). RAND looked at the progress of 40 schools in the first two years (1995–97) of the scale-up phase of New American Schools and found "significant variation among the schools in the levels of implementation obtained, which ranged from no implementation through the stages of planning, piloting, implementing, and fulfilling" (Bodilly, 1998:xiv). Nearly 45 percent were not yet clearly implementing the core elements of a design across the school. The Edison Project, which began its research and development phase in 1991, only began operating schools in 1995 and had only 25 in operation in 1998 (Chubb, 1998).
Reviews of what is known about school and curriculum designs are beginning to appear (Fashola and Slavin, 1997; Kentucky Department of Education, n.d; Northwest Regional Education Laboratory, 1998; Slavin and Fashola, 1998; American Institutes for Research, 1999), but at this point most of the currently popular designs lack evidence about the research base of the program, effects on student achievement, effective implementation, and replicability. Designs have achieved popularity in spite rather than because of strong evidence of effectiveness and replicability. "Typically they have been advanced by supporters because the model is associated with a well-known educator or theorist, because they have worked well in pilot sites, because they are based on a plausible theory of school reform, or some combination of these factors" (Consortium for Policy Research in Education, 1998:2).
Finally, theories about the science of learning are evolving. Research developments in the cognitive sciences are challenging prior understandings how humans learn and suggest that ideas about effective educational practice may undergo substantial revision as scholars and practitioners learn how to bring the insights of research into the classroom (National Research Council, 1999).
Lens 3: The Institutional Perspective
While school-based reform efforts and changes in district practices aimed at building school capacity show promise of improving school performance, it remains an open question whether they will be implemented or sustained in any
comprehensive way. Skepticism about the potential of school-by-school change stems both from the magnitude of the task and from evidence that the institutional environment within which schools operate is too fragmented and oriented toward short-term results to support meaningful improvement throughout the system. Opinion is divided on whether systemic reform can be accomplished by deliberate policy action within the current institutional framework or whether parents and students should be allowed to opt out of the public school system as it is currently structured because there is something inherent in that structure that "systematically creates and nurtures the kinds of schools that no one really wants" (Chubb and Moe, 1990:25).
Those who argue that improvements in educational productivity require close attention to institutional influences do not disagree with the second-wave reformers about the importance of focusing on teaching and learning at the school level and giving individual schools the autonomy to adopt effective practices and adapt them to their local contexts. Rather, they argue that these ideas by themselves are insufficient, because they ignore the key question: not "what works?" but how the desirable characteristics of schools can be developed and nurtured. In this view, central features of the way American schools are governed, especially the dispersion of control over educational policy among many actors and the political pluralism of a constitutional system that encourages "conflict as an antidote to the concentration of power" (Elmore, 1997b:41), inhibit the emergence of effective organizations. In other words, what Chubb and Moe call the "institutions of direct democratic control" undermine school autonomy, fail to support successful schools, and put insufficient pressure on schools whose performance in unsatisfactory (Chubb and Moe, 1990; Smith and O'Day, 1991; Elmore, 1997b; Brandl, 1998). Viewing schools in terms of the environment in which they operate gives yet another perspective on the productivity problem in education and its solution.
Smith and O'Day (1991:237) graphically describe "the fragmented, complex, multi-layered educational policy system," which they identify as a "fundamental barrier to developing and sustaining successful schools in the USA":
This system consists of overlapping and often conflicting formal and informal policy components on the one hand and, on the other, of a myriad of contending pressures for immediate results that serve only to further disperse and drain the already fragmented energies of dedicated and well meaning school personnel. On the formal policy side, school personnel are daily confronted with mandates, guidelines, incentives, sanctions, and programs constructed by a half-dozen different federal congressional committees, at least that many federal departments and independent agencies, and the federal courts; state school administrators, legislative committees, boards, commissions and courts; regional or county offices in most states; district level administrators and school boards in 14,000 school districts (with multiple boards and administrative structures in large systems); and local school building administrators, teachers and committees of interested parents. Every level and many different agencies within levels at-
tempt to influence the curriculum and curricular materials, teacher in-service and pre-service professional development, assessment, student policies such as attendance and promotion, and the special services that schools provide to handicapped, limited English-proficient and low-achieving students.
The complexity of this institutional environment has been building for a century, but the pace of change intensified after World War II and especially after the mid-1960s (Kirst, 1995).
Since the "common school" of the nineteenth century, in which local control was paramount, more and more layers have been added to the organization of the public schools. Many of these layers were added during the Progressive Era of the late 19th and early 20th centuries, in which reformers argued that schools should reflect the highly bureaucratized nature of business organizations if they were to be efficient (Tyack, 1993). The new institutional form in education was designed to rid education of problems stemming from the existence of large lay boards and the active involvement of politicians. The new "modern" system strove to centralize control, standardize practice, put control in the hands of professionals, and run the system by scientific principles. Community values were to be represented by elites on locally elected boards, but managing the schools was left to professional administrators.
By the 1960s, this system was under attack by groups whose interests the new system did not serve well. Teachers were dissatisfied at being treated like low-level functionaries in a bureaucratic system (Tyack, 1974), and minority groups protested their relegation to low academic tracks in a system that then credentialed and certified their low standing (Berg, 1970; Weeres and Kerchner, 1996; Tyack, 1974, 1993). The system responded with a place at the table for teachers, recognition of disenfranchised groups, and a governance system run by bargaining and interest group pressures, rather than by central enlightenment (Kirst et al., 1980). Responsiveness to demands, often in the form of new specialized bureaucratic units and new programs, became the basis of accountability. Researchers used the term "fragmented centralization" to characterize the altered system (Meyer, 1991). Fragmentation was exacerbated by the growing involvement of state and federal officials in funding education as well as in handing down programmatic and accountability mandates (Kirst, 1995).
Observers disagree on whether reforms powerful enough to overcome the shortcomings of direct democratic control can be instituted within the current system of publicly controlled schools or whether control must be taken out of the hands of bureaucrats and political partisans via the creation of an education marketplace. This debate reflects the cardinal economic choice societies face when deciding how to allocate scarce goods and services: whether to use the market or government as the predominant regulator (Wolf, 1993:1). With regard to the public schools, the argument is currently one between those who believe in the possibilities of reforming the bureaucratic, professional model of public
schooling that emerged at the beginning of the 20th century and those who believe that this model is bankrupt.
Those who would address the problem within the framework of the existing school system "compare the existing public school system to a business firm that has been poorly managed and needs to rationalize and integrate its parts. Its advocates say there is nothing inherently wrong with a public education system that is controlled by policy-making boards and administered by a traditional bureaucracy; today's problems can be solved by re-engineering the system so that its parts are correctly aligned" (Hill et al., 1997:105).
Smith and O'Day (1991) noted that school-by-school reform engaged primarily those who already had a history of reform experience and interest, not necessarily those whose practices were most in need of change. They also pointed out the changes in content and pedagogy being called for by second-wave reformers implied rethinking the knowledge and skills that children are expected to learn and the nature of the teaching and learning process itself. They judged that "[s]uch a reorientation is not likely to happen on a widespread school-by-school basis among educators who have themselves been schooled in a philosophy and settings that embody fact-based conceptions of knowledge, hierarchical approaches to skill development, and a near total reliance on teacher-initiated and teacher-directed instruction." They called for "a coherent systemic strategy that can combine the energy and professional involvement of the second wave reforms with a new and challenging state structure to generalize the reforms to all schools within the state" (p. 234).
This systemic strategy operates alongside the second wave of school reform; it emphasizes state (and to a lesser degree federal) actions to complement school and district restructuring by creating a more coherent environment within which successful schools can thrive and by creating external pressure for change when it does not emerge spontaneously. The linchpin in the system is the development of content standards expressing shared understandings about what students need to know and be able to do, with which other elements of the educational system (school curricula, assessments, teacher education and professional development, and accountability) can be aligned.6 With content standards and performance
standards (i.e., examples and definitions of what proficiency in the content standards would look like) together defining what students should know and how well they should be expected to perform, authority for determining how these goals should be met can be decentralized down to the district or (preferably, in the eyes of many) the school level, in keeping with the tradition of local control of education and consistent with both economic and "good practice" research, which suggests that those closest to the "production site" are in the best position to make efficient and effective decisions about meeting the needs of students in specific classrooms and schools.
The standards-based systemic strategy has dominated state education reform efforts for a decade and has become remarkably pervasive. Model subject-matter standards have been developed by organizations of experts in a number of academic fields (e.g., the pioneering mathematics standards issued by the National Council of Teachers of Mathematics and science standards developed by the National Academy of Sciences, drawing on materials and ideas developed by the American Association for the Advancement of Science and the National Science Foundation). A 1998 survey (American Federation of Teachers, 1998) revealed that every state except Iowa has set or is in the process of setting common academic standards for students: 47 states have or are planning to have assessments aligned with learning standards; 24 have or will have high school exit exams based on the standards; and 20 have or are developing incentives to motivate students to achieve a higher standard than that required of all students (American Federation of Teachers, 1998). There is evidence that systemic reform may be beginning to pay off. For example, a study of the two states (North Carolina and Texas) that have made positive gains on the greatest number of indicators being tracked by the National Education Goals Panel confirmed that the gains were both significant and sustained. Moreover, the study concluded that the most plausible explanation for relatively large gains in both states on NAEP tests could be found in the policy environment: "an aligned system of standards, curriculum, and assessments; holding schools accountable for improvement for all students; and critical support from business in developing, implementing, and sustaining these changes over time" (Grissmer and Flanagan, 1998:i).
Despite this progress, states and districts have found that systemic reform presents many challenges. For example, although many states developed coherent education policies around standards, few have removed the previous policies that conflict with the new direction of reform. In addition, political pressures have caused some to change strategies during a short time. Other states have found it difficult to achieve consensus around standards for student achievement (Massell, 1994; Massell and Fuhrman, 1994; Cohen, 1995). Differing views of reform among local staffs and other priorities have made change at the local level difficult as well (Cohen, 1996; Hertert, 1994; Massell et al., 1997; Spillane et al., 1995). Finally, it is worth noting that some of the challenges arise because not
everyone accepts the premises of standards-based reform. For example, some people may believe that the control over knowledge involved in national or statewide standards is inconsistent with personal liberty or pluralism. As an earlier National Research Council report noted, "though standards-based reform was conceived as a way to compensate for the fragmented system that governs education in the United States, the institutional arrangements it espouses still reflect that fragmentation. All three levels of government are involved, with the federal government essentially serving as a 'bully pulpit,' exhorting states and localities to move in a new direction, states choosing to play roles that range from strict regulator of local behavior to cheerleader for reform, and local communities responding to federal and state initiatives while still trying to maintain their own agendas" (National Research Council, 1997:31).
Skeptics about the possibilities of improving educational performance within the existing institutional environment question how significantly institutional arrangements can be changed so long as direct democratic control of schools exists. They argue that real school improvement cannot happen under anything like the current arrangements (e.g., Chubb and Moe, 1990; Hill et al., 1997). Drawing on organizational theory, in particular the "new institutionalism" (March and Olsen, 1989), they maintain that the educational choices of actors in the system are shaped by the institutional context in which they move. "Different institutions constrain and aggregate individual choices in very different ways, and this, in the end, is why different kinds of organizations emerge, prosper, or fail within them." In this view, "familiar arrangements for direct democratic control do indeed impose a distinctive structure on the educational choices of all the various participants—and . . . this structure tends to promote organizational characteristics that are ill-suited to the effective performance of American public schools" (Chubb and Moe, 1990:21).
One way in which the institutions of direct democratic control influence the organizational structure of schools is by distributing authority broadly. Chubb and Moe (1990:35) argue that "under a system of democratic control, the public schools are governed by an enormous, far-flung constituency in which the interests of parents and students carry no special status or weight." This distribution of authority is problematic for Chubb and Moe (while it might not be for others) because they seem to assume that the preferences of parents and students should have greater standing in deciding what schools should be like.
Even if one accepts in principle the idea that authority over education ought to be widely dispersed in a democratic society, there are still grounds to be concerned that existing arrangements for exercising that authority may interfere with the development of effective school practices and the ability of professionals to adapt these practices to local circumstances. The problem arises from the particular institutional arrangements through which the United States has chosen to bring into balance the variety of interests and values contending for influence over public education. There are three fundamental approaches to mediation and
control: bureaucracy, markets, and ''clans" or self-governing communities of interest and value (Ouchi, 1980). Bureaucracy, which is appropriate when the quality of performance is not transparent and when potential conflicts of interest are present, has been the preferred method for managing superior-subordinate relationships in light of stakeholder interests and values. And, as noted earlier, the organization of school systems has become increasingly bureaucratic over the years as interest and values have become more complex and as more and more layers of control have been added.
While bureaucracy can be a force to promote fairness and quality, it also can interfere with the efforts to improve educational productivity in a number of ways. Bureaucratization can whittle away discretion and autonomy at the school level (Chubb and Moe, 1990; Hill et al., 1997; Brandl, 1998). Policy makers increasingly act to reduce the discretion permitted at the school level. They do this (1) to reduce compliance problems that can result from school personnel who may or may not be inclined to act in accordance with policies determined at higher levels of the system; (2) to reduce the possibilities of other actors in this system of multiple authorities using the existence of discretion left in local hands to impose their own (possibly competing) interests; and (3) to insulate their decisions from change by future policy makers. Bureaucrats have incentives to expand their budgets, programmatic authority, and administrative controls; and so their increasing presence, too, serves to reduce the discretion and autonomy left to school personnel. Teachers unionize to gain influence in an environment increasingly characterized by powerful, organized interests outside the school; collective bargaining results in detailed contracts that further formalize public education and reduce or eliminate managerial discretion.
Bureaucratization can also draw administrators away from duties concerned with instruction. As the environment within which schools operate becomes more complex, key aspects of education become institutionalized. Administrators become increasingly involved in complying with the decisions made at higher levels about teachers, students, and curriculum and spend less time providing instructional leadership (Meyer and Rowan, 1978; Rowan, 1981).
Bureaucratization can constrain teachers in their efforts to teach as well (Chubb and Moe, 1990:58–59). Regulations "produce bureaucratic rather than professorial controls over the content and structure of the work . . . [These] controls [are] aimed at standardizing procedures rather than building knowledge that can be applied differently, depending on the given needs of the child." While teaching requires a great deal of flexibility and creativity, such controls often "place teachers in the unprofessional position of having to treat diverse students uniformly" (Darling-Hammond and Cobb, 1996:20). Thus, bureaucratization encourages depersonalized and standardized instruction and keeps pedagogical strategies simple and routing as possible (Darling-Hammond, 1996).
Bureaucratization can also lead to a lack of accountability for education outcomes. Because educational bureaucracy is not closely related to the work of
instruction, schools may not exercise meaningful control over their instructional activities or outputs, despite shifts toward "accountability." Meyer and Rowan (1978:80) suggest two reasons for this. First, "close supervision of instructional activity can uncover inconsistencies and create more uncertainty than unenforced demands for conformity to bureaucratic rules." Second, centralized governmental and professional controls in education are weak and schools depend heavily on local funding and support. Thus, Meyer and Rowan argue that administrators may actually avoid developing formal controls over instruction so that inconsistencies between local practices and instructional rules are not uncovered. The lack of accountability coupled with the heavy hand of bureaucracy over the formal aspects of education may help explain why the weight of tradition operates so powerfully. "Over long periods of time schools have remained similar in their core operation, so much so that these regularities have imprinted themselves on students, educators, and the public as the essential features of a 'real school'" (Tyack and Cuban, 1995:7).
Finally, bureaucratization can deny schools and school systems free resources to invest in school improvement. "Competition for resources has created an overconstrained system in which every dollar is allocated to teacher salaries or to existing programs. New funds, e.g., from tax levy increases, are spoken for before they arrive, usually to fund deferred maintenance or roll back increases in average class size. Even the supposedly flexible categories of funds, such as staff development, are committed in advance to separate categorical programs or to programs selected by central office administrators" (Hill et al., 1997:29).
How important the institutional environment is as a barrier to educational productivity depends on the extent to which bureaucracy and direct democratic control affect school systems as the foregoing arguments suggest they do. Chubb and Moe (1990:64) argue that environments that are relatively homogeneous and problem-free are likely to be the least bureaucratic, while urban environments ("teeming with diverse, conflicting interests of political salience" and home to deeply troubled schools) are more likely than suburbs or rural areas to suffer from the negative consequences of bureaucratic controls and political pluralism. These are, of course, the places where productivity problems are the worst and where the need for exceptionally effective schools is therefore the greatest.
Alternative institutional arrangements for providing public education are often grouped under the broad label of (1) "choice," giving parents more control over which schools their children will attend, including traditional public schools or more recently charter schools, which are freed from much public regulation or (2) "privatization," referring either to the use of public funds at private schools, particularly through vouchers, or to greater involvement of private or nonprofit firms in the provision of education (through contracts from school districts, for example). These alternative institutional arrangements and what is known about their effects on educational achievement are discussed in detail in Chapters 6 and 7. Although choice and privatization options have been growing, they are not
unique to the current period of reform. Families, especially relatively well-off families, have been able to exercise school choice by choosing where to live. Within-district and cross-district choice plans (including magnet school options) were a common remedy selected by or imposed by courts on school districts as they struggled in the latter half of the 20th century to accomplish desegregation goals, but the existence of special public schools for students with particular academic or vocational talents and interests (like the Bronx High School of Science and the Aviation High School in New York) goes back much further.
Here, we simply note several distinguishing features of the current debate over choice and privatization options: the attention being paid to their potential for raising student achievement as well as for serving other educational purposes valued by Americans; the growing willingness of states and districts to try new institutional arrangements for providing publicly funded education; and the intensity of the controversy over the desirability of making these comparatively radical shifts in familiar patterns of educational governance.
USING FINANCE-RELATED STRATEGIES TO IMPROVE SCHOOL PERFORMANCE
A quarter of a century ago, a book entitled Indeterminacy in Education reviewed social science research generated in the first decade after the Coleman report and came to the conclusion that "educational policymaking is now in a state of indeterminacy. No satisfactory criteria exist by which to make important decisions regarding [among other things] school finance" (McDermott, 1976:1).
We are not yet certain about how to make schools better or how to deploy resources effectively. The hope that productivity studies might provide ready answers to public officials about how much money to allocate, under what circumstances, to whom to obtain specified academic outcomes is as yet impossible to fulfill. Still, our assessment of the last several decades of research and policy development on educational productivity makes us more optimistic than our mid-1970s predecessors about the prospects for making informed school finance choices.
The past 25 years worth of insights have generated a host of ideas about how to use school finance to improve school performance. Input-output research has heightened interest in policies affecting key variables that appear linked to student achievement, such as teacher quality and class size. The renewed involvement of economists in educational productivity studies has brought a long-absent economic perspective to school finance and has drawn attention to the lack of performance incentives and financial accountability measures in traditional school finance systems. Studies of effective practice have spawned a vigorous effort to make school reform actually affect what happens between teachers and students in classrooms, raising interesting questions about how school finance reforms, such as school-based management and teacher development, can influence the
processes of teaching and learning. The focus on classrooms has also raised awareness of the need to ensure that students come to school ready and able to learn. The institutional perspective on school performance has highlighted fundamental disagreements about the way Americans have conceived of their public schools for the better part of a century and a half, by calling into question the traditional reliance on government both to fund and to supply public education and by raising the possibility of giving greater attention to market options for supply (such as public or private school choice for parents and contracting-out to private providers for services).
These advances in the understanding of educational productivity will be further enhanced as researchers work on developing more accurate and comprehensive outcome measures and as policy makers systematically try and evaluate instructional and policy options.
No matter how much progress is made in understanding productivity through improved research and practice, there is an important sense in which education continues to be and probably always will be indeterminate. In the foreword to the 1976 volume on indeterminancy in education, Arthur Wise noted that indeterminacy was used in several senses in the book: as lack of a consensus on the aims of education as well as a state of being "not fixed," "not clear," "not established," or "not settled" (McDermott, 1976:x). He also noted, however, that in mathematics ''an equation is said to be 'indeterminate' when it can be satisfied by more than one value for each unknown" and he asked "Is the search to reduce technical indeterminacy in education a search for equations which can be satisfied by only one value for each unknown?"
We suggest that indeterminacy will always characterize educational production because of the impossibility (and undesirability) of standardizing the characteristics and behavior of the key factors of production in the education productivity equation: teachers and students. In other words, the search for answers to improving school performance and student achievement will never yield just one value—that is, solutions that will work for all schools and students in all times and places.
In the face of this indeterminacy, there are no simple answers about how to use school finance to make schools better. Instead, we seek to identify major strategies for change and to synthesize and evaluate the evidence on how well and under what conditions such strategies might contribute to meeting our goals for an education finance system. We will return to this task in Part III.