NATIONAL ACADEMY PRESS
2101 Constitution Avenue, NW Washington, DC 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract/Grant No. ESI-9618770 between the National Academy of Sciences and the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
Library of Congress Cataloging-in-Publication Data
Investigating the influence of standards: a framework for research in mathematics, science, and technology education/Iris R.Weiss…[et al.].
Includes bibliographical references and index.
ISBN 0-309-07276-X (pbk.)
1. Mathematics—Study and teaching—Standards—United States—Evaluation. 2. Science—Study and teaching—Standards—United States—Evaluation. 3. Technical education—Standards—United States—Evaluation. I. Weiss, Iris R.
QA13 .I55 2001
Additional copies of this report are available from
National Academy Press,
2101 Constitution Avenue, NW, Lockbox 285, Washington, DC 20055; (800) 624–6242 or (202) 334–3313 (in the Washington metropolitan area); Internet, http://www.nap.edu
Printed in the United States of America
Copyright 2002 by the National Academy of Sciences. All rights reserved.
Suggested citation: National Research Council. (2002). Investigating the influence of standards: A framework for research in mathematics, science, and technology education. I.R.Weiss, M.S.Knapp, K.S.Hollweg, and G.Burrill (Eds.), Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
THE NATIONAL ACADEMIES
National Academy of Sciences
National Academy of Engineering
Institute of Medicine
National Research Council
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M.Alberts is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm.A.Wulf is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M.Alberts and Dr. Wm.A.Wulf are chairman and vice chairman, respectively, of the National Research Council.
COMMITTEE ON UNDERSTANDING THE INFLUENCE OF STANDARDS IN K-12 SCIENCE, MATHEMATICS, AND TECHNOLOGY EDUCATION
Iris R.Weiss (Chair),
Horizon Research, Inc.
Maria Alicia Lopez Freeman,
California Science Project, University of California, Los Angeles
Ontario Institute for Studies in Education, University of Toronto
Consortium for Policy Research in Education, University of Pennsylvania
Center for the Study of Teaching and Policy, University of Washington
Hawaii Department of Education
Connecticut Department of Education
Department of Mathematical and Computer Science, San Diego State University
Office of the President, University of California
International Technology Education Association
The Charles A.Dana Center for Mathematics and Science Education, University of Texas at Austin
The National Center on Education and the Economy
Staff, Center for Education
Karen S.Hollweg, Project Director
Gail Burrill, Senior Program Officer
Tina Winters, Senior Project Assistant*
Viola Horek, Administrative Associate**
Henry Heikkinen, Consultant Writer/Editor
COMMITTEE ON SCIENCE EDUCATION K-12
J.Myron Atkin (Chair),
School of Education, Stanford University
Division of Biological Sciences, University of Montana
Caryl Edward Buchwald,*
Chittenden Central School District, Vermont
Keystone Science School, Colorado
Beatrice Chu Clewell,***
First Hand Learning, Inc.
William E.Dugger, Jr.,*
International Technology Education Association
City College (City University of New York)
Wade Ellis, Jr.,****
West Valley College
Maria Alicia Lopez Freeman,
California Science Project
The Robert A.Welch Foundation
San Francisco Unified School District
Carnegie Institute of Washington
Institute for Systems Biology
Jane Butler Kahle,****
Miami University, Ohio
Maricopa Community College
Massachusetts Institute of Technology
University of California, Riverside
Flynn Park Elementary School, Missouri
Urban Systemic Initiative, Euclid, Ohio
Boston Museum of Science
Los Angeles County Office of Education
Fresno Unified School District
Delaware State Department of Education
School of Education, Baylor University
MATHEMATICAL SCIENCES EDUCATION BOARD
Joan R.Leitzel (Chair),
University of New Hampshire
Jere Confrey (Vice Chair),
University of Texas at Austin
University of Wisconsin—Madison
Prescott High School, Arizona
Deborah Loewenberg Ball,
University of Michigan
University of Michigan
St. Paul Academy and Summit School
Lakeland Community College
University of Maryland, College Park
Jan de Lange,
St. Mary’s College of California
Michigan State University
Enterprise School District, California
The Baylor School, Tennessee
Michigan State University
Flathead High School, Montana
TQM Services Group
Hornedo Middle School, Texas
The National Center for the Improvement of Educational Assessment
Bronxville High School, New York
Chicago Public Schools
University of Michigan
GE Aircraft Engines
North Carolina State University
University of California, Los Angeles
University of Wisconsin—Madison
University of Illinois at Urbana—Champaign
J. Sargeant Reynolds Community College
This report could not have happened without support from a number of people, and I am grateful for their contributions. On behalf of the Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education, I want first of all to acknowledge our sponsor, the National Science Foundation, and in particular Janice Earle and Jane Butler Kahle for making this project possible.
A number of former staff members from the Center for Science, Mathematics, and Engineering Education (the precursor to the current Center for Education) were instrumental in getting this project off the ground. Susan Loucks-Horsley was the initial project director and saw the Committee through the first year of its work. The former executive and associate executive directors, Rodger Bybee and Joan Ferrini-Mundy, conceptualized the project and provided ongoing support. Their presentations on the nationally developed science and mathematics standards were invaluable to the Committee. Harold Pratt, former director of the Center’s Division on K-12 Policy and Practice, gave ongoing advice throughout the first year of the project. Kristance Coates, the former senior project assistant, provided essential support during the early months of the Committee’s work. We are also grateful to Suzanne Woolsey, the Center’s acting director in fall 1999, for her vigilance in seeing that this project received the support it needed during the Center’s transition.
The Committee was aided greatly by individuals who partici-
pated in our meetings and helped us understand the complex issues involved in examining how nationally developed mathematics, science, and technology standards have influenced K-12 education in the United States. The participants in the May 2000 workshop, listed in Appendix A, also very generously gave two days of their time to discuss the Committee’s work and provided critical input for the final stages of the Committee’s work following the workshop.
Many others have supported this project. The Mathematical Sciences Education Board and the Committee on Science Education K-12 provided oversight and advice to the Committee throughout the project. Several individuals provided advice along the way, including Richard Askey, Audrey Champagne, James J.Gallagher, Daniel Heck, Thomas Romberg, and Mark St. John. Joan Ferrini-Mundy and Harold Pratt, who were instrumental in the early stages of the project, rejoined our efforts in the final stages of the project, providing in-depth comments and advice on drafts of the report. Tina Winters and Viola Horek provided valuable support, coordinated many aspects of the project, and were important in seeing that the project moved forward.
The Committee was resolute in working through the many difficult issues that arose until we achieved consensus on the document. Special thanks are due to Mike Knapp, whose substantial written contributions throughout the process led us to include him as an editor, and to a subset of the Committee that participated in numerous conference calls and attended informal meetings, e.g., at the American Educational Research Association Annual Meeting, as we continued to hammer out the details: Peg Goertz, Mike Knapp, Maria Alicia Lopez Freeman, Patrick Shields, Judy Sowder, and Elizabeth Stage. And it is certainly no exaggeration to say that without the ceaseless support from National Research Council (NRC) staff members Karen Hollweg and Gail Burrill, this document would never have seen the light of day.
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
Henry L.Alder, University of California, Davis
Lynn Cornett, Southern Regional Education Board, Atlanta, GA
Karen Economopolous, TERC, Cambridge, MA
Manuel Gomez, University of Puerto Rico
Daniel Koretz, RAND, Arlington, VA
Ronald M.Latanision, Massachusetts Institute of Technology
Bill Linder-Scholer, SciMathMN, Roseville, MN
Roland J.Otto, E.O.Lawrence Berkeley National Laboratory, Berkeley, CA
Richard Satchwell, Illinois State University
Scott Stowell, Spokane Public Schools, WA
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Michael W.Kirst, Stanford University, and R. Stephen Berry, University of Chicago. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments
Standards have served as a basis of educational reform across the nation as educators and policy makers respond to the call for a clear definition of desired outcomes of schooling and a way to measure student success in terms of these outcomes. Public policies at the federal, state, and local levels have focused on standards as a way to raise achievement levels for all students. At the national level, content standards have been developed in nearly every academic discipline.
As the standards movement has gained momentum, there has been increasing interest among educators and the public concerning the impact of standards on what happens in classrooms. In response to this interest, and as part of its mission of working for improved mathematics and science teaching and learning across the nation, the National Research Council’s Center for Science, Mathematics, and Engineering Education (which became the Center for Education in 2000)—with funding from the National Science Foundation—undertook the task of designing a framework that would aid in the design, conduct, and interpretation of research regarding the influence on student learning of the standards created at the national level in mathematics, science, and technology. In 1999, the Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education began its work. The charge of the Committee was to “develop a framework that can be used to understand the influence of science, mathematics, and technology education standards on programs, policies, and
practices.” This Framework was to provide guidance for the design, conduct, and interpretation of research regarding the influences of nationally developed standards on student learning in mathematics, science, and technology education. Hence, the Committee’s primary concern has been to develop a structure and process for understanding the influence of these educational standards on practices, programs, and policies in K-12 education. The Committee was responsible to two standing boards in the Center, the Committee on K-12 Science Education (COSE K-12) and the Mathematical Sciences Education Board (MSEB), and the Committee’s work reflects the oversight of these two boards.
An overarching problem faced by the Committee was how to structure thinking about the impact of standards in a way that would consider the complex system in which the standards operate—a system that consists of several layers of policy makers, various educators involved in multiple ways, and an array of players with different responsibilities for what takes place in classrooms. It was clear from the beginning that we were not to answer the question of what influence the standards have had on educational reform. It was a constant struggle, however, to keep the focus on a framework and not an analysis of where and how the standards had taken root in education systems and whether they were affecting student learning.
As the Committee worked, we came to a common understanding of some of the issues in the daunting task of laying out a way to think about the influence of the nationally developed standards on what happens within this complex system and eventually on what happens to student learning. We agreed, for example, to focus our thinking on separate but interacting channels within the education system—curriculum, teacher development, and assessment and accountability—and to consider how reform ideas might traverse these channels to reach the classroom. At the same time, we recognized that contextual forces outside the education system, in the public and political world, directly and indirectly affect what
happens in the classroom, and we wanted to be sure the Framework considered those as well. We agreed to concentrate on the standards created by the National Council of Teachers of Mathematics, the National Research Council, and the International Technology Education Association that focus on teaching and learning in mathematics, science, and technology. We also agreed that the focus of our work was not on the quality of the standards documents themselves but rather on their influence. We did not analyze the nationally developed documents nor make judgments about their focus and content, but concentrated on developing ways to think about whether and how these standards were being used in the system and with what results.
As our work progressed, we came to see even more clearly the importance of having the document be neutral, neither advocating nor denigrating the standards and their impact. Our work was to lay out questions that should be answered in order to come to understand the influence of standards, regardless of whether or not the answers supported the thinking in the national documents. The Framework presented in this document is the result of our discussions.
We eventually came to agree that the Framework would describe the system in which the standards operate, then suggest places where the effect of the standards might be observed in this system, which became the organizing structure of the document. We have drawn on a broad body of research related to various aspects of the task, and have also relied heavily on the practical experience of the Committee members and outside experts in designing the Framework. The Committee produced a background paper for its work, and in spring 2000, convened a group of educators and researchers in a workshop to discuss and critique the paper. The Committee used the insights and input from workshop participants to help define the Framework and to stimulate ideas regarding its future uses.
Investigating the Influence of Standards is aimed at researchers
and those who use research in their work, including federal, state, and local policy makers. It is intended to provide a lens for considering the national standards and the role such standards play in improving mathematics, science, and technology education. This book is not intended to be a simple template prescribing a solution but rather lays out the major issues involved in considering the effects of standards in improving student learning, offering questions that should be asked when considering research about the standards. The document can be used to understand particular studies, to identify what the studies do and do not address, and to help consumers interpret their claims. The document is also designed to spur additional research in areas where answers to the questions are not well-formed. While the focus of this document is the nationally developed standards in mathematics, science, and technology, we believe the Framework is general enough to serve as a guide for those at state and district levels and in other disciplines to analyze the impact of their own standards.
Iris R.Weiss, Chair
Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education
FIGURES, BOXES, AND TABLE
The Black Box,
The Layers of Education Governance and Channels Through Which Reform Might Flow,
A Conceptual Map for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education,
A Set of Guiding Questions for Investigating the Influences of Nationally Developed Standards,
A Framework for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education,
Parts of the Framework Addressed in the Briars and Resnick Study,
Basic Principles and Features of Principles and Standards for School Mathematics,
Basic Principles and Features of National Science Education Standards,
Basic Principles and Features of Standards for Technological Literacy: Content for the Study of Technology,
Hypothetical Studies That Address One or More of the Framework Questions,