Research-Based School Reform: The Clinton Administration's Agenda
MARSHALL S. SMITH,1 BRETT W. SCOLL, AND JEFFREY LINK
U.S. Department of Education
The Clinton Education Agenda
The Clinton administration's legislative accomplishments in education in the 103rd Congress span preschool, elementary, and secondary education through higher education and job training (See Figure 2.1). Let us consider the key pieces of legislation in roughly the chronological order that they became law.
The first Clinton education bill, passed early in the 103rd Congress, created a program designed to facilitate student access to postsecondary education while reducing the overall cost of loan programs to taxpayers and students. The Direct Loan program provides an attractive alternative to the existing Guaranteed Student Loan (GSL) program. The new program borrows money from the Treasury to lend to college students, rather than paying for the use of private money from banks as in the GSL program. Eliminating the middlemen reduces costs, which creates savings that are shared with students through lower interest rates and with taxpayers through returns to the Treasury. By 1996, the total savings to taxpayers
FIGURE 2.1 Lifelong Learning: The Clinton Agenda.
All children healthy and ready to learn
Head Start and parent education
A reauthorized Head Start upgrades quality and opportunity; encourages intergenerational learning
WIC and Immunizations
Nutrition and immunizations for all children through increased appropriations
High standards and opportunity to achieve for all students
Coherent strategy and framework to support state and local school reform based on high standards
In context of common high standards for all students:
Support state and local reforms to develop workplace-based and classroom learning to earn nationally recognized credentials in high-skilled occupations
College student aid reform
Workers have the skills to compete
A simplified, more efficient loan program; new payment options; reduced overhead costs; savings for students and taxpayers
Improved education and training opportunities
Consolidated and simplified programs; increased information and accountability; easier and earlier access to training
will be roughly $1 billion a year. Between 1995 and 2000 we estimate the total scored savings to be $12 billion for taxpayers and at least $2 billion for students.2
Established with the Direct Loan program is a set of repayment provisions, including the Income-Contingent or "Pay as You Go" Loan Repayment Provision. Income-contingent loans allow graduates who do not make a high salary in a given year to pay back only a small amount that year. Others with the same loan responsibility but higher earnings would make larger loan payments and thus pay off their loans faster. Income-contingent loans will give young people a chance to "get going" after college or graduate school without having a heavy debt repayment burden while taking entry-level or public service jobs or by starting their own businesses. A variety of other payback strategies are included in the legislation to provide flexibility and choice for students. These include the option for a student to consolidate his or her new or existing loans with standard repayment provisions into a single income-contingent loan.
Improving Occupational Opportunities for All Students
A second major piece of legislation passed by Congress is the School-to-Work Act. The School-to-Work program is designed to serve students who do not take a high school course of study that would lead them directly to a four-year college. It promotes the kind of rigorous alternative to traditional academic training that exists in a number of European countries. What we envision, and what is now being implemented in various states, is something like an occupational major that students could choose at the end of the tenth grade. An occupational major might be in health care, finance, or 25 to 30 other areas.
State standards for occupational majors that set out content and performance expectations would be developed or adopted by representatives of industry and vocational educators. Eventually, the standards would be common across the states. Students would be assessed after a couple of years in high school and one or two years in a community college. If students passed the assessment, they would receive a certificate that would carry weight when they sought employment anywhere in the country. Widely recognized, portable certification is not only valuable to job seekers but also helps employers identify qualified workers, thus saving hiring and training costs.
As part of their education programs, students would receive on-the-job training in their selected occupational areas. For example, if a student majored in health, he or she might hold a job or multiple jobs in a hospital, health maintenance organization (HMO), or public health clinic. At work, students would see what the occupational ladder looks like and understand what it means to work in
a job. In school, students would learn academic content in the context of their occupational major. Health majors, for example, would learn something about the physics, biology, and chemistry of health care. They would also learn about the economics and politics of health care and might study how health affects the lives and stability of families.
All 50 states have received planning grants for the School-to-Work program. State implementation grants are being awarded on a competitive basis. Twenty-two states applied for implementation grants in 1994 and eight won. In 1995 another nineteen states won implementation grants. States are using different strategies. Some, such as Maine, focus more on the community college postsecondary system. Others, including Oregon, integrate academics and occupational programs to reform high schools for all students. What we have, in effect, is a large variety of different, rigorous, standards-based experiments going on around the country in the area of school-to-work education.
School Reform Focused on Challenging Academic Standards
The third major Clinton legislative victory was the passage of the Goals 2000: Educate America Act, the cornerstone of the administration's K-12 reform initiative. Goals 2000 codifies into law the National Education Goals established in 1990 by President Bush and then head of the National Governors Association, Governor Clinton. More importantly, the Goals 2000 legislation creates a stimulus for states to initiate reforms that are focused on all students meeting challenging academic standards established by states and local education agencies—standards that set out in clear prose what all students are expected to be able to know and do in a subject (content standards, illustrated in Figure 2.2) and what level of performance they should be expected to achieve (performance standards).
The aims of Goals 2000 are 1) to encourage states to establish their own challenging standards for reform; 2) to assist states in developing support systems, such as better teacher training; and 3) to help ensure that the resources, flexibility, and authority necessary to bring reform to all students are pushed down to the local level, so that teachers, principals, and parents have the primary voice in how to achieve the high-quality teaching and learning necessary to help all children learn to the challenging standards.
Revamping All Major Federal K-12 Education Programs
Reauthorization of the Elementary and Secondary Education Act (ESEA) of 1965, signed into law in October 1994, was the last major piece of the Clinton education agenda to be passed by the 103rd Congress. Called the Improving America's Schools Act (IASA) of 1994, the legislation restructures ESEA programs. In the past these programs supplemented the existing school system.
FIGURE 2.2 Life Science Content Standards. SOURCE: National Research Council (1994), Table 5-2, p. v-5.
The new ESEA is designed to help change the overall system by supporting state and local reforms for all students—the same reforms reinforced by Goals 2000. For the first time, the U.S. Department of Education will coordinate its major elementary and secondary programs so that they are in tune with each other and with state and local education reform efforts. Instead of having Title I, the massive compensatory education program, heading in one direction, Goals 2000 in another direction, and a professional development program in yet another direction, we are letting states pull these programs together to focus them on the same ends. Those ends are to support state and local reforms to bring all students up to demanding state and local academic standards. In addition to the $100 million allotted for Goals 2000 in 1994 and the $400 million in 1995, ESEA will leverage about $9 billion in support of those reforms.
Title I of ESEA, the largest of the ESEA programs, reformulates ''compensatory aid" by focusing on academic standards as a means to promote both quality and equality for children in high-poverty areas. Under the new law, Title I students, like all other students in a school or a state, are expected to have an
opportunity to achieve to the same standards set by states and local districts under Goals 2000. No longer can we encourage two different sets of expectations for what students should know and be able to do—one for children in high-poverty rural and inner-city schools and another, often more challenging set of expectations, for children in the suburbs (Knapp, 1992). Both groups of children deserve challenging, worldclass standards.
Other changes in ESEA include new programs designed to support widespread reforms in teachers' professional development, in technical assistance, and in promoting safe schools. In addition, the federal government will support, for the first time, a major technology effort in education. Finally, the Secretary of Education has a new waiver authority so he or she can respond to state and school district requests for relief from statutory or regulatory requirements that the states argue get in the way of local reforms. States will simply need to make the case that they or their local schools need extra flexibility to carry out reforms in the most efficient way.3
This has been a brief overview of four of the major education initiatives of the Clinton administration. Let us now consider some of the theory and detail of the K-12 reforms.
Systemic School Reform
The Department of Education hopes to stimulate state and local reforms through Goals 2000 and then to reinforce those efforts using the added resources of the newly reauthorized ESEA. A central idea behind this "systemic" school reform strategy is to establish at the state and local levels challenging content and performance standards that serve as clear academic expectations for the system and for all students. A second part of the strategy calls for states to provide maximum resources and flexibility to the local education agency, so that teachers, parents, and schools have the wherewithal and responsibility to decide how best to educate their students to those standards (Smith and O'Day, 1990). Toward those principles, 90 percent of Goals 2000 resources are pushed down to the local level, and in IASA's Title I program the figure is well over 95 percent.
Standards and local flexibility are two parts of systemic reform. The third challenge is to align the system to support the reforms. A fundamental step is to change the teacher training process. Institutions that train teachers must make sure that the teachers they graduate understand the content and skills set out in the academic standards and that they are able to teach to those standards. Furthermore, state and local assessment must be aligned with the curriculum standards
so that tests will measure achievement of the content and skills of those curricula. As it is, many standardized, norm-referenced tests created by profit-making organizations are strikingly independent of what is taught in schools, particularly in areas such as science. A rich performance assessment system that is aligned with student content standards and consequently with schools' curricula would serve to reinforce both student and teacher efforts and would operate as a much more legitimate accountability mechanism (O'Day and Smith, 1993).
To create an environment of support for teaching and learning to the challenging new state academic standards, school and state administrators should ask themselves: Are our strategies for resource allocation and other key administrative decisions focused on supporting all students to learn to higher standards? Are existing resources being used in the most efficient way to bring all students to high standards? If the answer is "no," then the strategies and decisions should be reconsidered.
The implementation of systemic reform will happen differently in school after school, district after district, and state after state. The fundamental idea is that if a state takes advantage of the Goals 2000 money it should set very challenging standards for all of its students and work diligently to help them achieve to those standards. But, just as the federal government should avoid prescribing how states should structure their reforms, the states should provide school districts with the flexibility and resources they need to improve teaching and learning based on their standards.
The administration's K-12 education legislation embodies a marked change in the federal role in public education. In the past the federal role focused almost exclusively on categorical programs for defined populations, such as the poor, migrants, or students with limited English proficiency or on specific subjects such as reading, science, or drug prevention. Goals 2000 and the School-to-Work program are attempts to help strengthen the functioning of the entire system rather than only a particular aspect of it. This is a new way of thinking about the role of the federal government. It means getting the federal government to support, rather then operate independently of, the goals of state and local education agencies.
A Research Base for Reform
These reforms were not invented out of whole cloth. They grew out of a generation of research, much of which has implications for how we go about changing our education system as well as how we conceptualize and measure educational effectiveness. Research on education and educational effectiveness of the past 25 years has taught us a great deal, especially in eight key areas.
1. All students can learn to far higher levels than we ever imagined in the past. Jim Greeno, Lauren Resnick, Howard Gardner, and others have carried out
research in cognitive science, anthropology, and other areas that has begun to provide real support for this notion in careful studies spanning a broad range of content (Gardner, 1991; Glaser, 1984; and Resnick, 1988). In addition, there are an extraordinary number of "existence proofs," of examples of teachers such as Jaime Escalante, who inspire "average" or ''below-average" students to intellectual heights others would not have thought possible. It was not just Jaime Escalante, it was the teacher next door to him, and the one next door to her, and the people and schools throughout the nation who emulated Escalante, who help to expand our understanding of the learning potential of all kinds of students. Nationally, we have the beginnings of strong evidence from surveys of advanced placement (AP) course taking. The number of AP courses taken and succeeded in by all types of kids has skyrocketed in the last eight or 10 years.4
We also have existence proofs from other sources such as studies of international achievement. Many Americans have been amazed and appalled that so many Japanese students score well on algebra tests in eighth grade while Americans fail miserably (Crosswhite et al., 1985; Stevenson and Baker, 1991; and UNESCO, 1983). It is not so amazing at all. In Japan most students take algebra in seventh grade, so they score pretty well in eighth grade. This makes a certain amount of sense. In most cities in the United States, however, we do not let our students take algebra in seventh grade. In fact, we greatly limit the percentage of students taking algebra in eighth grade because we—"we" being the general public and the people in the schools—do not believe that American students can learn it.
2. What you are taught matters. Again, we can look to the example of algebra, among other subject areas. If you are not taught a foreign language, you are probably not going to learn one. It is not just a matter of teaching a particular subject area, however; what is taught in a given subject area is equally important. If a student is taught only a tiny bit of mathematics or science in the first three or four years of elementary school, he or she is not going to score very well on a mathematics or science test. If a student is taught a great deal of math, in terms of both breadth and depth, he or she is likely to know more and to demonstrate that achievement on appropriate assessments. Despite the seeming simplicity of this logic, our schools have been slow to either deepen their courses or make other changes that would connect what is taught to what is tested (Madaus, 1991; UNESCO, 1983; and Resnick and Resnick, 1985).
3. The quality of teaching matters. Ann Brown, Lee Shulman, David Cohen, Maggie Lampert, and many others have provided a great deal of recent data on how children are taught complex skills (Brown, 1985a and 1985b; Shulman, 1986 and 1987; Cohen, 1989 and 1990; Cohen et al., 1993; and Lampert, 1988). These studies and others clearly indicate that teachers with mastery of the content and knowledge of how to engage students in hard intellectual work can have a dramatic effect on student achievement. Magically, studies have shown that children who are interested in what and how they are learning and who have some control over the nature of the learning environment actually learn more. Students' motivation and commitment, like teachers', are heightened when they have ownership and are engaged in complex tasks for sustained periods of time. Depth of content and a potential understanding and relevance of experience help promote engagement. Combining challenging content and engaging instruction almost certainly has a multiplier effect on the quality and quantity of student learning (Sizer, 1992; Tomlinson, 1990c and 1991; and Newman, 1989).
4. Teachers are more likely to teach well things that they understand well and that they have been taught to teach. Today, in many American elementary schools few teachers feel comfortable teaching mathematics or science beyond a very basic level, in large part because they believe they do not understand the content well enough. Consequently, in many American elementary schools students are taught little more than routine arithmetic and almost no science. If teachers are going to be expected to teach to the kinds of science standards under development by the National Academy of Sciences, or the mathematics set out in the National Council of Teachers of Mathematics standards, they will need ongoing and sustained support, beginning with preservice training and continuing through a lasting system of professional development opportunities and technical assistance (Cohen, 1989; Cuban, 1984; Darling-Hammond and Berry, 1988; Darling-Hammond and Green, 1990; Lieberman and Miller, 1991; National Council of Teachers of Mathematics, 1982 and 1989; and National Research Council, 1994).
5. Schools, and the teaching and learning that occur in them, are more likely to change when the staff of the school has ownership and some control over the nature of the change (Conley, 1991). Social psychologists have long known, and organizational theorists have discovered over the past decade, that control over one's environment can increase investment in it and thus lead to greater participation in shaping and improving that environment (Mosteller and Moynihan, 1972). The lessons we learn from the private sector about pushing responsibility for implementation decisions to the lowest levels applies in schools as well (Brown, 1993).
6. Teachers and the public do not have a common conception of what is meant by high and internationally competitive academic standards . Some data suggest that the nation does not have a common understanding about performance standards, despite widespread support for such standards. Many teachers
have little current understanding of challenging content and skills or effective methods of teaching to them, in part owing to a national history of low expectations and low-level curricula, particularly for minority and low-income students (Louis Harris and Associates, Inc., 1992). Disparities in grading procedures have contributed to a lack of comparability of grades, especially between schools in high-poverty areas and those that are more affluent. For example, achievement test score surveys have shown that students who receive "A" grades in lower-income schools score at about the same level on independent tests as "C" students in higher-income schools (U.S. Department of Education, 1993). Thus, while students in lower-income schools believe they are succeeding to some high level, as do their parents and presumably their teachers, in fact they are scoring at what would be an average to low level in middle-income schools (Tyson-Bernstein, 1988, and U.S. Department of Education, 1993).
7. Individual school reform has a long, complex, and unhappy history in the United States. School reform has looked much like a field at twilight with some 85,000 fireflies in it—one for every public school in the country. For each effort at "reform" or "restructuring," a light blinks on, only to blink off again in a relatively short time. What does the metaphor suggest? To us it suggests that schools blossom into change under the right circumstances, under the charismatic leadership of the right principal, or under the guidance of a change agent, such as Hank Levin or Ted Sizer or Jim Comer (Sizer, 1992; Levin, 1987; and Comer et al., 1987–1988). Over time, however, most such schools lose the support system that helped them begin the change process. Principals change jobs, supportive teachers move away, school district policies change, financial support dwindles, schools lose their momentum, and the light goes out. Study after study indicates that schools in which quality and effectiveness increase over three to four years have trouble maintaining those gains long-term (Cuban, 1990; David, 1990; Elmore and McLaughlin, 1988; and Fuhrman and Elmore, 1990). We believe this pattern is not simply one of regression but a phenomenon of the lack of organizational structure and of consistent goals and purpose that is endemic in our fragmented education system—thus, the last general finding.
8. The education system often does little to support change or to sustain schools that appear to be effective. Our education system is highly fragmented, dominated by belief in "magic bullets." New reforms constantly replace one another as new governors, secretaries of education, district superintendents, and school principals take office. Simultaneously, leaders at the federal, state, and locals levels may each adopt a different reform strategy, creating further confusion throughout the system. Sometimes these changes are made for thoughtful substantive reasons, sometimes for entirely political reasons. It is little wonder that many teachers are skeptical of reforms and prefer to close their doors and continue business as usual (Smith and O'Day, 1990).
These eight findings and others helped shape the rationale behind Goals 2000 and the new ESEA. Both pieces of legislation were conceived to support
state- and locally-based reforms that are built on high expectations. Operationalizing these high expectations requires challenging content standards that describe what all students should know and be able to do and performance standards that measure the level students have achieved in each subject area. The legislation also emphasizes local resources, autonomy, and responsibility and promotes strong professional development and technical assistance to prepare teachers to help students meet the standards.
Building on a Decade of Education Reform
Goals 2000 and the new ESEA also build on the knowledge and experience gained in a variety of states and local districts over the past two decades. Vermont, Colorado, Delaware, Kentucky, California, Oregon, Ohio, and South Carolina, for example, have begun to set standards for what all students should know and be able to do and to reshape their education systems in support of those standards. The shape of reform varies greatly among states, as it should.
The federal government has supported these efforts in a variety of ways. The Statewide Systemic Initiative, a program operated by the National Science Foundation that addresses math and science, is helping 24 states develop math and science standards and provide professional development for teachers to teach to those standards. The U.S. Department of Education has provided 18 states with grants for generating state standards, and Goals 2000 planning and implementation grants are helping 46 states and the District of Columbia devise strategies for developing state improvement plans, including standards. The reauthorized ESEA will put the weight of $9 billion in federal programs behind these reforms, by linking ESEA to state standards rather than erecting separate federal requirements.
Simultaneously, in many communities across the nation local schools are undergoing privately initiated reforms to improve teaching and learning and to engage students. Sponsored by the New American Schools Development Corporation, the Annenberg grants, the New Standards Project, and many other organizations, local teachers, principals, and superintendents are changing the quality of teaching and learning. A great challenge of the middle and late 1990s will be to use the energy of all of these groups to multiply rather than divide their productive effects of reform on the U.S. school system.
Some Persistent Issues in Education Reform
Let us shift now to a few of the many issues that constantly challenge education reform. First, time and persistence are clear requirements, especially as a constantly changing political landscape brings new ideas and new methods with each new leader. Real reform takes years of hard focused work that does not
capture headlines. The lure of promising "magic bullets" is very difficult for many politicians to resist.
Second, to make standards-based reform come to life, we need clear examples of high-quality standards and of the caliber of student work that we hope to see in the future. For example, while the Goals 2000 and ESEA legislation make it clear that states will develop or select their own content standards, exemplary standards can provide ideas about the content and structure of standards. The science standards that are being developed by the National Academy of Sciences will become a model for the states, as have those produced by the National Council of Teachers of Mathematics. Even if they are not directly adopted by states, they will have a substantial impact on how states think about and develop their own standards for science.
Useful standards must describe not only good content but also good performance. This should take the form of showing good student work. Too many parents, students, and even teachers in our country have little information about students' potential or what good school work looks like. To understand what good student work looks like and how it can be generated, experts have spent a great deal of time in different schools talking with many students, teachers, and parents. Unfortunately, most teachers and principals do not have that luxury. So often, they do not have a good conception of top-level student work. Parents do not either. Standards laced with examples of high-quality student work can help provide and disseminate information about what good work actually looks like.
Third, we need to address the professionalism of teachers through many strategies. Support for learning the content and strategies for how to teach students to achieve to the new standards are critical. As a by-product, mastery of the standards will mean that teachers know more in a particular subject than most of the population; it can provide teachers with the prestige that goes along with special knowledge, the kind of prestige that builds and reinforces professionalism.
Finally, we need to address both quality and equality. Concern about equality can tempt a state or local district or even a nation to lower its standards to reduce the likelihood of clear disparities between groups, especially between those that might have different kinds of advantages than others. Creating rigorous standards may, in the short run, exacerbate present disparities in student achievement. In the long run, however, unless we challenge all students to meet higher standards, it is likely that low-achieving students will continue to be taught less and learn less, both in terms of quality and quantity, than their higher-achieving peers. We need to create a system based on high standards and then provide the support that all students need to reach those standards, so that we can move at the same time and as quickly as possible toward both quality and equality.
Observations on School Effectiveness Research
Now we turn to the issue of measuring school effects. In particular, we will consider the work of Anita Summers and Amy Johnson on school-based management (chapter 5, this volume) and Eric Hanushek on a broader set of school reforms. The principal conclusion of Summers and Johnson's work is that the invention or implementation of any new management system or organizational structure in schools does not of itself necessarily lead to any improvement in education outcomes. Without organization around content-defined objectives for student performance, neither improved student achievement nor efficiency will follow willy-nilly from a governance change. In accord with the findings of the Panel on the Economics of Education Reform, an international study conducted through the Organization for Economic Cooperation and Development (OECD) reports that there are no clearly demonstrated links between increased school autonomy and student learning (OECD, 1994). School-based management may improve certain complementary aspects of schooling, not the least of which may be teacher morale and enthusiasm for the reform effort; however, little empirical evidence can be mustered to support the assertion that greater stakeholder participation directly improves student performance.
The OECD report goes on to suggest, however, that to the degree that a reorganization effort is conducted with a clarity of purpose to improve classroom teaching and learning, positive outcomes may accrue. In other words, to improve student learning, the content and instruction delivered to students must change as well as the organizational structure of the school. They complement each other.
This is not rocket science. Included in the administration's reform agenda discussed earlier are features intended to facilitate school responsibility in resource management by pushing resources down to schools, by granting waivers, and by other strategies. At the same time, incentives are provided so that schools will focus on challenging content standards and aligned performance assessments that together raise expectations for student learning, hopefully leading to increased efficiency and improved student achievement.
Eric Hanushek, too, finds no positive relationship between a variety of measures and school performance.5 How does one reconcile this with the eight different sets of findings we described earlier that, taken collectively, suggest we know a lot about how to change the opportunities for children to achieve to higher levels. How do we balance these different perspectives?
A close look at the survey/production function studies Hanushek examines shows that those studies by and large neglected to consider any of these eight research findings. There is no measure of whether the teachers and schools in the surveys were focusing their instruction on bringing all students to achieve to high
standards. There are not even any measures of the curriculum coverage or the depth to which material was taught. There are no adequate measures of teachers' quality, their knowledge of curriculum, or their ability to engage students. There are no measures of the degree to which schools have the autonomy and responsibility they need to design effective strategies or of the degree to which the overall district and state systems support the efforts of the schools.
These lapses are not unexpected. The school survey data used by most researchers generally do not include these measures. Very little of what has been found to influence achievement by psychologists, sociologists, and political scientists who actually get into and study classrooms and the education system is ever evaluated as an "input" in surveys.
Note that the variables that make up the backbone of the classic production function—number of years of experience of teachers, class size, proportion of teachers with master's degrees, or even school expenditures—are not among the research findings we listed as important for improving the quality of learning in U.S. schools. The number of years of experience a teacher has does not necessarily tell us anything about her or him as an educator. More relevant questions might probe: What does this teacher know about the content of science or math? Does this person know how to teach children who are difficult to teach? Can this teacher handle a child with Down syndrome who is mainstreamed into the classroom and make it a rich experience for everyone in the class?
Class size or teacher-student ratios are not always informative predictors of student achievement either. Depending on the climate of the school, discipline issues, a teacher's skills in making learning interesting, or even the student-tutoring assistance potentially available in heterogeneous classrooms, the impact of class size can vary greatly. No matter what the size of the class, if the curriculum is watered down, the teacher is not competent to teach the material, and the students are not engaged, learning will be minimal.6 The central evidence for the effect of class size comes from studies of extremely small classes where intensive tutoring can take place. Much carefully collected data suggest only marginal benefits to smaller class sizes (Robinson, 1990; Tomlinson, 1990a and 1990b; and Mitchell et al., 1989).
By the same token, the number of teachers with master's degrees is not a predictor of student achievement. A very large percentage of U.S. teachers now have master's degrees. Many teachers, acting rationally, pick up their degrees at night school to boost their salaries. Unfortunately, because the courses for a master's degree are typically not aligned with the classroom activities of teachers, higher degrees do not necessarily increase teachers' capacity to teach, or their understanding of the curriculum they are to teach, or their way of dealing with children.
Per-student expenditures are an extremely poor measure of education resources applied to students' learning. How money is spent is far more important than how much is spent. Moreover, the accounting of expenditures has become so complicated in recent years that it is difficult to focus on its classroom applications. The many services provided by schools in response to the demands of a changing society require increasing percentages of school budgets without any concomitant return in student achievement. In particular, a large percentage of the increases in real expenditures over the past 20 years has gone into special education and much of that into special procedures for health and other related services that special education students are entitled to by law (Chaikind et al., 1993). Often, the more difficult the teaching conditions, the larger the expenditures; in many production function analyses this may have led to the erroneous inference that expenditures have a negligible or even a negative effect on achievement. 7
In our view the independent variables presently in use are deficient. Instead of abandoning production function work altogether, however, we need to look at schooling inputs differently. In particular, we need to think about how to measure variation in inputs that we expect from other research to have some plausible relationship to student performance.
On the other side of the equation, the dependent variable in most production function research is also problematic. The measure most commonly used is a score on a national standardized exam. These tests are purchased from private publishers who devise them to be as widely applicable as possible so as to sell the most copies. The generic quality of the tests tends to make them insensitive to variations in the quality of any particular curriculum used by a school or school system. In other words, the most common dependent measure in education production function research is largely independent of the instructional and curriculum content and quality in any particular school, district, or state. Tests that are developed to suit many different curricula and instructional approaches will be inappropriate for any single approach. In the language of systemic school reform, if the dependent variable is not aligned with the teaching and learning going on in the school, it is no wonder that it does not pick up variations in school resources.
Yet the standardized norm-referenced tests have to be sensitive to variation in student performance, for they are required to be psychometrically reliable instruments. What variation do they pick up if they do not systemically assess variation in curricula or teaching quality or engagement of students? There is a clue in many studies (Coleman et al., 1966). These studies show far larger variations in achievement within classrooms and schools than across schools,
systems, and states. This suggests that much of the variation in these standardized test scores is caused by experiences outside schools, especially experiences that vary with social class and environmental opportunities. This in turn is not surprising since the tests are designed to be independent of particular curriculum experiences.
So, when Eric Hanushek reports there is no systematic evidence that resource differences among schools have a large effect on student achievement, we should not be shocked given his outcome measures. We should, however, carefully scrutinize the policy implications that might be drawn from these results.
We might also think about improving the current model by changing the dependent variable as well as the independent variables. It seems much more reasonable to have an assessment device that is designed to measure what is being taught. That is what happens in many countries. That is what happens in college classes. Examinations are designed to assess student learning in the course taught, not in some generic course.
Imagine, then, a new scenario for a K-12 education production function. We carry out our new study in a state that has challenging content and performance standards. For our dependent variable we use student performance on a state assessment that is aligned with challenging state standards. For our independent variables we use measures of the resources that our theory of schooling indicates are critical to providing students with the opportunity to learn to the challenging new standards. Our study could analyze, for example, the relationship between student achievement and variations in teachers' knowledge and the quality of their teaching of the substance and skills in the content standards. It could explore the impact not of having computers in a classroom but of using computers and software to support students in their efforts to achieve the state's standards. And it could examine the relationship between students' control over their learning and their actual achievement.
Our objective in examining production functions would be subtly different from the old version. We would no longer be interested in the global question "do variations in school resources and practices influence student achievement?" Of course, they do! We know that from many other studies. Few students learn French or calculus or plate tectonics unless they are taught it in school. Few students learn much science in their elementary years if their teachers lack the expertise to teach it effectively. The new production function should clearly show such effects.
Our interest would be in understanding how, to what extent, and under what circumstances the variations in specific circumstances and resources in classrooms relate to student achievement. One of the great advantages of an aligned system should be the efficiency that follows from having all of one's ducks in a row. The incentive would finally be right—hard work by students and well trained teachers would result in higher assessment scores. In addition to improving the quality of teaching and enhancing students' opportunities for learning, an
aligned system will make it easier to determine the performance of students, teachers, and schools. It will provide them with incentives based on this performance and more effectively support their needs. Our purpose in using the new production function would be to help understand how best to target the resources necessary to make the education system more productive for all children.
Brown, Ann L., Bolt, Beranek and Newman, Inc. 1985a. Reciprocal Teaching of Comprehension Strategies: A Natural History of One Program for Enhancing Learning. Technical Report No. 334. Illinois University Urbana. Center for the Study of Reading, Cambridge, Mass.
Brown, Ann L., Bolt, Beranek and Newman, Inc. 1985b. Teaching Students to Think as They Read: Implications for Curriculum Reform. Reading Education Report No. 58. Illinois University Urbana. Center for the Study of Reading, Cambridge, Mass.
Brown, Clair. 1993. "Employee involvement in industrial decision making: lessons for public schools." Pp. 202–231 in Decentralization and School Improvement, J. Hannaway and M. Carnoy, eds. San Francisco, Calif.: Jossey-Bass.
Carter, Launor F. 1983. A Study of Compensatory Education: The Sustaining Effects Study, Final Report. Systems Development Corporation, Falls Church, Va.
Chaikind, Stephen, Louis C. Danielson, and Marshal Brauen. 1993. "What do we know about the costs of special education? A selected review." The Journal of Special Education 26(4):344–370.
Cohen, D. K. 1989. "Teaching Practice: plus Ca change…" Pp. 27–84 in P.W. Jackson (ed.), Contributing to Educational Change: Perspectives on Research and Practice. Berkeley, Calif.: McCutchan.
Cohen, D. K. 1990. "Revolution in One Classroom." Pp. 103–123 in S.H. Fuhrman and B. Malen (eds.), The Politics of Curriculum and Testing. Philadelphia, Pa.: Falmer Press.
Cohen, D. K., Milbrey W. McLaughlin, and Joan E. Talbert. 1993, Teaching for Understanding: Challenges for Policy and Practice. San Francisco, Calif.: Jossey-Bass.
Coleman, James S., et al. 1966. Equality of Educational Opportunity . Washington, D.C.: U.S. Government Printing Office.
College Entrance Examination Board, Advanced Placement Program. AP National Summary Report, 1989–1994.
Comer, James P., et al. 1987–1988. "School power: a model for improving black student achievement." Urban League Review 11(1–2):187–200.
Conley, Sharon. 1991. "Review of research on teacher participation in school decision making," Pp. 225–266 in Review of Research in Education, Vol. 17, Gerald Grant, ed. Washington, D.C.: American Educational Research Association.
Crosswhite, F. J., et al. 1985. Second International Mathematics Study Summary Report for the United States. Champaign, Ill.: Stipes.
Cuban, L. 1990. "Reforming again, again, and again." Educational Researcher 19:3–13.
Cuban, L. 1984. How Teachers Taught: Constancy and Change in the American Classroom, 1890–1980, New York, N.Y.: Longman.
Darling-Hammond, Linda, and B. Berry. 1988. Evolution of Teacher Policy. Report of the Center for Policy Research in Education, Eagleton Institute of Politics, Rutgers University, and the Rand Corporation, Washington, D.C.
Darling-Hammond, Linda, and J. Green. 1990. "Teacher quality and equality." Pp. 237–258 in Access to Knowledge: An Agenda for our Nation's Schools. J. I. Goodlad and P. Keating, eds. New York: College Entrance Examination Board.
David, J. 1990. ''Restructuring in progress: lessons from pioneering districts." Pp. 209–250 in Restructuring Schools: The Next Generation of Educational Reform, R. Elmore and Associates, eds. San Francisco, Calif.: Jossey-Bass.
Elmore, R. F., and M. W. McLaughlin. 1988. Steady Work: Policy, Practice and the Reform of American Education. Santa Monica, Calif.: Rand Corp.
Fuhrman, S. H., and R. F. Elmore. 1990. "Understanding local control in the wake of state education reform." Educational Evaluation and Policy Analysis 12(1):82–96.
Gardner, Howard. 1991. The Unschooled Mind: How Children Think and How Schools Should Teach. New York: Basic Books.
Glaser, R. 1984. "Education and thinking: the role of knowledge." American Psychologist 39:93–104.
Hanushek, Eric A. 1979. "Conceptual and empirical issues in the estimation of educational production functions." Journal of Human Resources XIV(3):351–388.
Hanushek, Eric A. 1981a. "Education policy research—an industry perspective." Economics of Education Review 1(2):193–223.
Hanushek, Eric A. 1981b. "Throwing money at schools." Journal of Policy Analysis and Management 1(1):19–41.
Hanushek, Eric A. 1981c. "The continuing hope: a rejoinder." Journal of Policy Analysis and Management 1(1):53–54.
Hanushek, Eric A. 1986. "The economics of schooling: production and efficiency in public schools." Journal of Economic Literature. XXIV(Sept.):1141–1177.
Hanushek, Eric A. 1989. "The impact of differential expenditures on school performance." Educational Researcher 18(4):45–51.
Hanushek, Eric A. 1994. "Money might matter somewhere: a response to Hedges, Laine and Greenwald." Educational Researcher 23(4):5–8.
Hedges, Larry V., Richard D. Laine, and Rob Greenwald. 1994a. "Does money matter? a meta-analysis of studies of the effects of differential school inputs on student outcomes." Educational Researcher 23(3):5–14.
Hedges, Larry V., Richard D. Laine, and Rob Greenwald. 1994b. "Money does matter somewhere: a reply to Hanushek." Educational Researcher (23)4:5–8.
Knapp, Michael S., ed. 1992. Better Schooling for the Children of Poverty: Alternatives to Conventional Wisdom. A Study of Academic Instruction for Disadvantaged Students. Volume I: Summary. SRI International, Menlo Park, Calif.: Policy Studies Associates, Inc., Washington, D.C.
Lampert, M. 1988. "What Can Research on Teacher Education Tell Us About Improving Quality in Mathematics Education?" Teaching and Teacher Education 4(2):157–170.
Levin, Henry M. 1987. "New schools for the disadvantaged." Teacher Education Quarterly. 14(4):60–83.
Lieberman, Ann, and Lynne Miller, eds. 1991. Staff Development for Education in the '90s: New Demands, New Realities, New Perspectives , New York, N.Y.: Teachers College Press.
Louis Harris and Associates, Inc., for the Metropolitan Life Insurance Company. 1992. Reaching for New Standards: The Teachers' Perspective . Washington, D.C.: American Association for Higher Education.
Madaus, G. 1991. "The Effects of Important Tests on Students." Phi Delta Kappan 73(3):226–231.
Mitchell, E. Douglas, et al. 1989. "Modeling the relationship between achievement and class size: a re-analysis of the Tennessee project STAR data." Peabody Journal of Education 67(1):34–74.
Mosteller, Frederick, and Daniel P. Moynihan, eds. 1972. On Equality of Educational Opportunity. New York, N.Y.: Random House.
National Council of Teachers of Mathematics (NCTM). 1982. Road to Reform: A Mathematics Education: How Far Have We Traveled? Results of a pilot study conducted for the NCTM. Reston, Va.: NCTM.
National Council of Teachers of Mathematics (NCTM). 1989. Curriculum and Evaluation Standards for School Mathematics. Reston, Va.: NCTM.
National Research Council. 1994. National Science Educations Standards Table 5-2, p. v-5. http://www.nas.edu/nap/bookstore.
Newman, Fred M. 1989, "Student engagement and high school reform," Educational Leadership 46(5):34–36.
OECD, Directorate for Education, Employment, Labour and Social Affairs, Education Committee. 1994. Effectiveness of Schooling and of Educational Resource Management: Synthesis of Country Studies. Points 22 and 33. Paris: OECD.
O'Day, Jennifer A., and Marshall S. Smith, 1993. "Systemic reform and educational opportunity." In Designing Coherent Policy: Improving the System, S. Fuhrman, ed. San Francisco: Jossey-Bass.
Resnick, L. B. 1988. Education and Learning to Think. Washington, D.C.: National Academy Press.
Resnick, D. P., and L.B. Resnick. 1985. "Standards, Curriculum and Performance: A Historical and Comparative Perspective." Educational Researcher 14:5–20.
Robinson, Glen E. 1990. "Synthesis of research on the effects of class size." Educational Leadership 47(7):80–90.
Shulman, Lee S. 1986. "Those who understand: knowledge growth in teaching." Educational Researcher 15(2):4–14.
Shulman, Lee S. 1987. "Knowledge and teaching: foundations of the new reform." Harvard Educational Review 57(1):1–22.
Sizer, Theodore R. 1992. Horace's Compromise: The Dilemma of the American High School. Boston, Mass.: Houghton Mifflin Co.
Smith, Marshall S., and Jennifer A. O'Day, 1990. "Systemic school reform." Politics of Education Association Yearbook 1990. pp. 233–267.
Spencer, Bruce D., and David E. Wiley. 1981. "The sense and the nonsense of school effectiveness." Journal of Policy Analysis and Management 1(1):43–52.
Stevenson, D., and D. Baker. 1991. "State control of the curriculum and classroom instruction." Sociology of Education 64(1):1–10.
Tomlinson, Tommy M. 1990a. Class Size and Public Policy: Politics and Panaceas. Washington, D.C.: Office of Educational Research and Improvement.
Tomlinson, Tommy M. 1990b. "Class size and public policy: the plot thickens." Contemporary Education 62(1):17–23.
Tomlinson, Tommy M. 1990c. Hard Work and High Expectations: Motivating Students to Learn. Issues in Education. Office of Educational Research and Improvement, Programs for the Improvement of Practice. Washington, D.C.
Tomlinson, Tommy M. 1991. "Student effort: the key to higher standards." Educational Leadership 49(1):69–73.
Tyson-Bernstein, H. 1988. "The Academy's contribution to the impoverishment of America's textbooks." Phi Delta Kappan 70(3):193–198.
UNESCO. 1983. Course of Study for Lower Secondary Schools in Japan . Tokyo: UNESCO, Education and Cultural Exchange Division.
U.S. Department of Education. 1993. Reinventing Chapter 1: The Current Chapter 1 Program and New Directions. The Final Report of the National Assessment of the Chapter 1 Program. Washington, D.C.: U.S. Department of Education.