National Academies Press: OpenBook
« Previous: CONTENTS
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

Anticipating Goals 2000: Standards, Assessment, and Public Policy

Encouraged by the Goals 2000: Educate America Act and other federal and state legislation, a movement is under way to reform education by establishing ambitious standards at the national and state levels to guide the content of learning in core subjects, the performance expectations for all students, and the opportunities to learn afforded all children. Important components of this strategy are assessments aimed at measuring the progress of students, schools, districts, and states toward the achievement of the content standards. This shift toward voluntary national goals, standards, and assessments is a watershed in American education history and will influence the course of public schooling for years to come.

◦ ◦ ◦ ◦

“For the first time in the nation's history, we have codified in federal law a set of national educational goals, along with the concept of voluntary national standards for all students.”

Constance B. Newman

◦ ◦ ◦ ◦

Because standards-based reform could have enormous consequences for the ways millions of American schoolchildren are taught and assessed, as well as for the ways in which millions of young Americans are prepared and selected for productive employment, it is critical to explore the many educational, social, technological, and political dimensions of the reform strategy. The Board on Testing and Assessment (BOTA) believes that among its principal roles are to elucidate the underlying assumptions and expected effects of reforms that rely heavily on standards and assessment and to provide objective and scientifically rigorous information to policy makers charged with its implementation.

Toward this end, BOTA convened a day-long workshop on March 9, 1994, at the National Academy of Sciences in Washington, D.C. The goals of the workshop were to help policy makers and others better understand the complex issues emerging from the standards-based reform movement, to elevate the level of discourse about standards and assessments beyond conventional wisdom and common generalities, and to highlight areas in which further research and exploration are needed.

The workshop format included presentations on critical national and state issues con-

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

cerning standards and assessments, responses from specific board members, and a frank, free-ranging exchange among the entire board and invited participants. Observers included federal agency officials, congressional staff, representatives of professional associations and standards-setting bodies, state and local educators, education researchers, scientists, and others. This bulletin, the first of several publications intended to acquaint a wide audience with BOTA activities and deliberations, synthesizes the proceedings of the March 9 workshop. It is important to note that, as a workshop summary, this document is limited in its scope by the discussions that actually took place. At the same time, we have attempted to draw attention to certain issues that the board considers important to its current and future work. The bulletin is organized around four major themes that emerged from the presentations and that were discussed throughout the day:

  • the implications of using standards as accountability tools;

  • the challenges of designing assessments related to standards;

  • the implications of building the new form of education federalism implied by standards-based reform; and

  • the challenges of strengthening the state and local capacity to implement standards and linked assessments.

Following are four sections exploring each of these themes. Within each section is a brief review of the main issue, a synthesis of views raised during the workshop discussion, and a list of questions for further analysis.

USING STANDARDS AS ACCOUNTABILITY TOOLS

THE ISSUE IN BRIEF

Advocates of standards-based reform argue that methods traditionally used by states and the federal government to instill accountability —namely, regulating “inputs” into schools, such as minimum resource and process requirements—have not worked very well to ensure educational quality. It is time, they say, to loosen the federal and state input requirements that have locked up the system, in exchange for greater attention to outcome standards specifying the content knowledge and skills to be taught and learned and the levels of performance to be attained. In this way, accountability for tax dollars would be enforced by assessing, monitoring, publicizing outcomes, and possibly attaching sanctions and rewards to performance.

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

Some contend that it is unfair to hold students, teachers, or school authorities accountable to content and performance standards without also defining the conditions that must exist in schools to afford students the opportunities to meet performance expectations—hence the emphasis on “opportunity-to-learn” standards.

How accurate are conventional notions about the effects of input and outcome requirements? Are there other ways to analyze or predict the effects of input and outcome requirements? What challenges must be addressed to make standards effective tools for institutional accountability and student motivation? What are the legal ramifications of new accountability approaches? These questions dominated much of the workshop discussion.

VIEWS FROM THE WORKSHOP

Inputs Versus Outcomes

An important theme in the discussion was that, in designing a system of accountability based on standards, it helps to move away from oversimplified notions such as (1) that more inputs necessarily lead to better results or (2) that results can be improved without consideration of the possible need for increased inputs. Instead, the workshop discussion focused on relationships between inputs and outcomes. One misleading notion is that input and outcome requirements are polar opposites. Rather, some fundamental beliefs appear to be shared by those who favor an emphasis on outcomes and those who advocate regulation of inputs. For example, those who support fiscal incentives to schools with outstanding performance must implicitly assume that extra monetary inputs make a difference, else they would have little value as rewards. And those who explicitly urge continued attention to the inputs side must presume that inputs will ultimately produce tangible outcomes for students.

◦ ◦ ◦ ◦

“People who believe in the efficacy of performance incentives are no different in their assumptions from people who believe in the efficacy of inputs.”

Ewart Thomas

◦ ◦ ◦ ◦

Another common metaphor views input requirements and outcome standards as substitutable tools to enhance performance: performance can be enhanced by either raising (or setting) higher outcome standards or by raising (or setting) input levels. Yet, it was argued, the trade-offs are not so clean. Input requirements play an important and

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

necessary role, even in an outcome-oriented governance system. Some desirable results of schooling cannot be captured very well by outcome measures. Moreover, there will probably always be school districts that will not provide the necessary inputs and equity guarantees unless directed to do so. In fact, it was noted, there is a perverse logic in rewarding with deregulation those schools that have been successful under the current system. These observations suggest that, rather than asking how to balance input and outcome requirements, it may be more useful to ask which combinations of input and outcome policies are likely to ensure higher performance and equitable access to learning.

If opportunity-to-learn standards are going to be more than a replay of past experience or another layer of regulation, it may be advisable for states to look beyond the strategies used in the past—primarily centrally imposed mandates and incentives— toward more participatory strategies. Several options for doing so are described below in the discussion of strengthening state and local capacity.

◦ ◦ ◦ ◦

“We need to review existing regulation, not with the notion that we 're going to eliminate it, but with the notion that we're going to ration it, streamline it, . . . and focus on regulation that makes enforcement and compliance likely.”

Susan Fuhrman

◦ ◦ ◦ ◦

“Educators are saying that we don't know what the production function is: there is a great deal of uncertainty about the relationship between inputs and outputs. We might devote some intellectual resources to looking at how variations in inputs lead to very different worlds.”

Laurie Bassi

◦ ◦ ◦ ◦

The workshop discussion of inputs and outcomes turned also to the debate over the the usefulness of trying to determine a production function for education: to identify, or even quantify, which kinds of inputs produce particular kinds of student outcomes and then build those characteristics into standards and linked assessments. Is it possible, for instance, to identify how much training in specific content a teacher needs in order to teach students to a particular performance level?

Some observers assert that classrooms are too idiosyncratic and education too much of a human enterprise to be quantified in this way. Yet for standards-based reform to work, it was noted, we must reach some conclusions about what kinds of instructional strategies, professional development, and organizational policies lead to higher outcomes —whether or not we call this a production function.1

1  

One participant offered this suggestion after the workshop: “The metaphor of a recipe may be better than the black box of the production function. Not only do we need ingredients (books, curricula, teachers) but we also need to know how to cook the dish, i.e., the process variables” (Stephen Baldwin, personal communication).

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Challenges of Developing Effective Standards

Several challenges must be addressed in developing an effective accountability system based on standards. The reform movement, as articulated in Goals 2000 and elsewhere, rests on these basic tenets about standards and tests:

  • standards should be clear but not oversimplified;

  • assessments should come in multiple forms and be more closely aligned with the knowledge and skills sought than conventional tests;

  • standards and assessments should be understandable, acceptable, and motivating to students, teachers, and parents; and

  • the focal point should be at the state and local level, guided by voluntary models developed nationally.

◦ ◦ ◦ ◦

“Certainly within English studies, the standards movement is trying to think through what this discipline is all about at a particular time and place in history. . . .I feel sometimes that the documents have suggested that this job is a much simpler one than it actually is.”

Miles Myers

◦ ◦ ◦ ◦

What criteria should standards meet to be considered worthy of certification? One set of recommendations has been published by the Goals 3 and 4 Standards Review Technical Planning Group:2 For national subject-specific content standards, the criterion descriptors identified by the Technical Planning Group are: world-class, important and focused, useful, reflective of broad consensus-building, balanced, accurate and sound, clear and usable, assessable, adaptable, and developmentally appropriate. For state content standards, the criterion descriptors are: as rigorous as national subject-specific standards, feasible, cumulatively adequate, encouraging of students' ability to integrate and apply knowledge and skills from various subjects, and reflective of broad state consensus-building.

Although board members generally viewed these criteria as a good starting point, several areas were felt to be in need of further refinement: What does it mean to be “world class”? What is the middle ground between “specific” and “flexible”? How finely grained are the skills and knowledge being sought? To what extent should disciplinary content standards embody the skills valued in the workplace?

2

See “Promises to Keep: Creating High Standards for American Students,” report to the National Education Goals Panel, November 1993, pp. iii-iv.

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

One challenge receiving scant attention in the popular discussion is the need to integrate standards and assessments across the various disciplines into a feasible whole and to control the proliferation of standards. There may be a temptation for professional associations to produce standards to gain visibility, resulting in multiple standards that do not mesh and, once established, are difficult to revise. It was suggested that political mechanisms be designed to address this potential problem.

Another perplexing question is how to ensure that standards are genuine motivators for improved teaching and learning. The prevailing wisdom is that content and performance standards will motivate higher performance by providing a clearer direction to schools about instructional changes needed, a clearer message to students, teachers, and parents about the performance expected, and a clearer yardstick for the public and policy makers about the progress made. If still greater motivation is desired, then higher stakes can be attached to performance in the form of sanctions and rewards.3

◦ ◦ ◦ ◦

“Standards sound very progressive. But in their fully formed state, standards are an incredibly conservative policy instrument. What we'll be facing in a decade or so are standards that are really the congealed residue of interest-based politics around disciplines— which are going to be incredibly hard to change and incredibly difficult to ration, unless we have some mechanisms in place for questioning. ”

Richard Elmore

◦ ◦ ◦ ◦

This dynamic may be more complex than prevailing wisdom assumes, however, as explained below in the section on developing aligned assessments. Some board members submitted that genuine motivation occurs only when standards are “hard currency”: reflective of something meaningful in the real world, such as skills and performances valued in the international marketplace. Others felt that the evidence was fuzzy about what really motivates students.

Ensuring equity for special groups of students within a standards-based framework is another major challenge. Many feel that applying performance standards to the current system could make fiscal and other inequities more glaring; when sanctions

3

High stakes has become a general way of describing the use of test results to make decisions or allocate resources in ways that can have significant consequences. But the question is often “High stakes for whom?” Depending on the test and its uses, the answer can be (a) the student or test-taker, as in the case of grade retention decisions or college admissions; (b) the teacher, as in the case of using student test results as a basis for teachers' promotions or salary determinations; (c) schools or districts, as in the case of test results being reported in the newspaper or publicized in real estate advertisements; (d) states, as in the case of test results being used to rank state educational performance; (e) the nation, as in the case of national educational progress being ranked alongside performance in other countries; or (f) all (or some combination) of the above.

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

and rewards are attached, existing inequities could be exacerbated.

Concern was voiced that many states are making only token attempts to address key equity questions, especially in terms of fiscal equalization on the input side. A counterargument was that resources (virtually of any amount) can be used in widely different ways and that there is no assurance that new input requirements will promote greater equity or more effective use of resources.

◦ ◦ ◦ ◦

“Standards are being seen as the rabbit at a greyhound track—if only we put standards out there people will chase after them, and if we make the stakes really high, people will chase even faster. What we're really trying to build is the fattest, lushest looking rabbit that can be zipped down the track to get students and teachers chasing after it.”

Alan Lesgold

◦ ◦ ◦ ◦

“What will be of [particular] interest to the courts in Goals 2000 and in Chapter 1 is not so much the opportunity-to-learn standards—[although] they will be of [some] interest—but rather the content standards, the performance standards, and the assessment system designed to measure them.”

David Tatel

◦ ◦ ◦ ◦

Legal Ramifications of Standards

Will opportunity-to-learn standards generate a spate of law-suits by parents and others dissatisfied with schools, as some have suggested? David Tatel's presentation, and the discussion that ensued, shed light on a legal aspect of reform that is often overlooked: opportunity-to-learn standards may be a less effective tool for courts to order change than content and performance standards. Courts have focused for decades on whether schools are providing inputs, Tatel explained, particularly in school finance and school desegregation cases. Opportunity-to-learn standards do not represent a departure from this approach and therefore may not significantly increase the amount of school litigation.

The adoption of state content and performance standards, however, may hasten a new trend among courts to examine student outcomes and order outcome-based remedies. Tatel noted that content and performance standards present courts with refined, ready-made tools for assessing the quality of school systems by the state's own definitions. This does not mean that courts will abandon interest in inputs and resources altogether. Rather, the typical court order in a school finance case may include outcome and input factors.

Court challenges are also like to arise from the application of new standards-based assessments, especially if the assess-

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×

ments have high stakes or produce adverse effects for particular racial or ethnic groups. Whether courts will have confidence in the assessments—or for that matter in the standards—may depend on whether educators have confidence in them, since judges often rely on expert witnesses to illuminate complex technical issues. If the experts disagree deeply, then courts will be less likely to embrace standards and assessments.

◦ ◦ ◦ ◦

“The meaningfulness of content and performance standards is questionable if improved learning does not occur among the traditionally underserved. ”

Sylvia Johnson

◦ ◦ ◦ ◦

FOR FURTHER ANALYSIS

  • Which applications of content and performance standards, opportunity-to-learn standards, and other governance strategies or requirements can ensure both high performance and equitable resources for learning?

  • Which kinds of classroom inputs translate into desirable student outcomes?

  • How can input measures be employed as part of opportunity-to-learn standards?

  • Under what conditions can standards become effective motivators for students, teachers, and others?

  • What should be done to ensure fair and accurate portrayals of districts, schools, and students?

  • What should be done about districts, schools, and students who do not meet expected levels of progress or performance?

Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 1
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 2
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 3
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 4
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 5
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 6
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 7
Suggested Citation:"USING STANDARDS AS ACCOUNTABILITY TOOLS." National Research Council. 1995. Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/9291.
×
Page 8
Next: DEVELOPING ASSESSMENTS ALIGNED WITH STANDARDS »
Anticipating Goals 2000: Standards, Assessment, and Public Policy: Summary of a Workshop Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!