Investigating the INFLUENCE OF STANDARDS

A Framework for Research in Mathematics, Science, and Technology Education

Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education

Iris R.Weiss, Michael S.Knapp, Karen S.Hollweg, and Gail Burrill, editors

Center for Education

Division of Behavioral and Social Sciences and Education

National Research Council

NATIONAL ACADEMY PRESS
Washington, DC



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education Investigating the INFLUENCE OF STANDARDS A Framework for Research in Mathematics, Science, and Technology Education Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education Iris R.Weiss, Michael S.Knapp, Karen S.Hollweg, and Gail Burrill, editors Center for Education Division of Behavioral and Social Sciences and Education National Research Council NATIONAL ACADEMY PRESS Washington, DC

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education NATIONAL ACADEMY PRESS 2101 Constitution Avenue, NW Washington, DC 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract/Grant No. ESI-9618770 between the National Academy of Sciences and the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. Library of Congress Cataloging-in-Publication Data Investigating the influence of standards: a framework for research in mathematics, science, and technology education/Iris R.Weiss…[et al.]. p. cm. Includes bibliographical references and index. ISBN 0-309-07276-X (pbk.) 1. Mathematics—Study and teaching—Standards—United States—Evaluation. 2. Science—Study and teaching—Standards—United States—Evaluation. 3. Technical education—Standards—United States—Evaluation. I. Weiss, Iris R. QA13 .I55 2001 507.1'073–dc21 2001006094 Additional copies of this report are available from National Academy Press, 2101 Constitution Avenue, NW, Lockbox 285, Washington, DC 20055; (800) 624–6242 or (202) 334–3313 (in the Washington metropolitan area); Internet, http://www.nap.edu Printed in the United States of America Copyright 2002 by the National Academy of Sciences. All rights reserved. Suggested citation: National Research Council. (2002). Investigating the influence of standards: A framework for research in mathematics, science, and technology education. I.R.Weiss, M.S.Knapp, K.S.Hollweg, and G.Burrill (Eds.), Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education THE NATIONAL ACADEMIES National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M.Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm.A.Wulf is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M.Alberts and Dr. Wm.A.Wulf are chairman and vice chairman, respectively, of the National Research Council.

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education This page in the original is blank.

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education COMMITTEE ON UNDERSTANDING THE INFLUENCE OF STANDARDS IN K-12 SCIENCE, MATHEMATICS, AND TECHNOLOGY EDUCATION Iris R.Weiss (Chair), Horizon Research, Inc. William Browder, Princeton University Maria Alicia Lopez Freeman, California Science Project, University of California, Los Angeles Michael Fullan, Ontario Institute for Studies in Education, University of Toronto Margaret Goertz, Consortium for Policy Research in Education, University of Pennsylvania Michael S.Knapp, Center for the Study of Teaching and Policy, University of Washington Paul LeMahieu, Hawaii Department of Education Mari Muri, Connecticut Department of Education Patrick M.Shields, SRI International Judith Sowder, Department of Mathematical and Computer Science, San Diego State University Elizabeth K.Stage, Office of the President, University of California Kendall Starkweather, International Technology Education Association P.Uri Treisman, The Charles A.Dana Center for Mathematics and Science Education, University of Texas at Austin Marc Tucker, The National Center on Education and the Economy Staff, Center for Education Karen S.Hollweg, Project Director Gail Burrill, Senior Program Officer Tina Winters, Senior Project Assistant* Viola Horek, Administrative Associate** Henry Heikkinen, Consultant Writer/Editor *   Through March 2001 **   Since March 2001

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education COMMITTEE ON SCIENCE EDUCATION K-12 J.Myron Atkin (Chair), School of Education, Stanford University Carol Brewer, Division of Biological Sciences, University of Montana Caryl Edward Buchwald,* Carleton College George Bugliarello,** Polytechnic University Elizabeth A.Carvellas,* Chittenden Central School District, Vermont Christine Chopyak-Minor,**** Keystone Science School, Colorado Beatrice Chu Clewell,*** Urban Institute Peter B.Dow,**** First Hand Learning, Inc. William E.Dugger, Jr.,* International Technology Education Association Hubert Dyasi, City College (City University of New York) Wade Ellis, Jr.,**** West Valley College Maria Alicia Lopez Freeman, California Science Project Norman Hackerman,* The Robert A.Welch Foundation Patty Harmon, San Francisco Unified School District Robert M.Hazen,**** Carnegie Institute of Washington Leroy Hood,** Institute for Systems Biology Anne Jolly, SERVE Jane Butler Kahle,**** Miami University, Ohio Michael G.Lang,**** Maricopa Community College Ron Latanision, Massachusetts Institute of Technology William Linder-Scholer, SciMathMN Jim Minstrell, Talaria Inc. John A.Moore,** University of California, Riverside Darlene Norfleet,** Flynn Park Elementary School, Missouri Carolyn L.Ray,* Urban Systemic Initiative, Euclid, Ohio Cary I.Sneider, Boston Museum of Science William E.Spooner,*** CBSG Judith Sydner-Gordon,**** Los Angeles County Office of Education Jerry Valadez, Fresno Unified School District Rachel Wood,* Delaware State Department of Education Robert Yinger, School of Education, Baylor University *   Term ended January 2001 **   Term ended January 2000 ***   Term ended July 1999 ****   Term ended January 1999

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education MATHEMATICAL SCIENCES EDUCATION BOARD Joan R.Leitzel (Chair), University of New Hampshire Jere Confrey (Vice Chair), University of Texas at Austin Judy Ackerman, Montgomery College Richard A.Askey,* University of Wisconsin—Madison Sherry Baca,** Prescott High School, Arizona Deborah Loewenberg Ball, University of Michigan Hyman Bass,** University of Michigan Benjamin Blackhawk,*** St. Paul Academy and Summit School Richelle Blair,* Lakeland Community College Patricia Campbell,*** University of Maryland, College Park Ingrid Daubechies,* Princeton University Jan de Lange, Freudenthal Institute Keith Devlin, St. Mary’s College of California Karen Economopoulos,* TERC Susan Eyestone,* National PTA Joan Ferrini-Mundy, Michigan State University Arthur M.Jaffe, Harvard University Lee Jenkins,*** Enterprise School District, California Dan Kennedy, The Baylor School, Tennessee Glenda T.Lappan,** Michigan State University Karen Longhart, Flathead High School, Montana Miriam Masullo,* IBM Corporation David S.Moore,*** Purdue University Thomas L.Moore, Grinnell College Mari Muri,** Independent Consultant Richard S.Normington,** TQM Services Group Debra Paulson, Hornedo Middle School, Texas Marge M.Petit, The National Center for the Improvement of Educational Assessment Mark Saul,** Bronxville High School, New York Anthony Scott, Chicago Public Schools Edward A.Silver,** University of Michigan William Steenken, GE Aircraft Engines Lee V.Stiff,* North Carolina State University James W.Stigler,* University of California, Los Angeles William F.Tate,** University of Wisconsin—Madison Jerry Uhl,* University of Illinois at Urbana—Champaign Susan S.Wood,** J. Sargeant Reynolds Community College *   Term ended June 2001 **   Term ended June 2000 ***   Term ended June 1999

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education This page in the original is blank.

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education ACKNOWLEDGMENTS This report could not have happened without support from a number of people, and I am grateful for their contributions. On behalf of the Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education, I want first of all to acknowledge our sponsor, the National Science Foundation, and in particular Janice Earle and Jane Butler Kahle for making this project possible. A number of former staff members from the Center for Science, Mathematics, and Engineering Education (the precursor to the current Center for Education) were instrumental in getting this project off the ground. Susan Loucks-Horsley was the initial project director and saw the Committee through the first year of its work. The former executive and associate executive directors, Rodger Bybee and Joan Ferrini-Mundy, conceptualized the project and provided ongoing support. Their presentations on the nationally developed science and mathematics standards were invaluable to the Committee. Harold Pratt, former director of the Center’s Division on K-12 Policy and Practice, gave ongoing advice throughout the first year of the project. Kristance Coates, the former senior project assistant, provided essential support during the early months of the Committee’s work. We are also grateful to Suzanne Woolsey, the Center’s acting director in fall 1999, for her vigilance in seeing that this project received the support it needed during the Center’s transition. The Committee was aided greatly by individuals who partici-

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education pated in our meetings and helped us understand the complex issues involved in examining how nationally developed mathematics, science, and technology standards have influenced K-12 education in the United States. The participants in the May 2000 workshop, listed in Appendix A, also very generously gave two days of their time to discuss the Committee’s work and provided critical input for the final stages of the Committee’s work following the workshop. Many others have supported this project. The Mathematical Sciences Education Board and the Committee on Science Education K-12 provided oversight and advice to the Committee throughout the project. Several individuals provided advice along the way, including Richard Askey, Audrey Champagne, James J.Gallagher, Daniel Heck, Thomas Romberg, and Mark St. John. Joan Ferrini-Mundy and Harold Pratt, who were instrumental in the early stages of the project, rejoined our efforts in the final stages of the project, providing in-depth comments and advice on drafts of the report. Tina Winters and Viola Horek provided valuable support, coordinated many aspects of the project, and were important in seeing that the project moved forward. The Committee was resolute in working through the many difficult issues that arose until we achieved consensus on the document. Special thanks are due to Mike Knapp, whose substantial written contributions throughout the process led us to include him as an editor, and to a subset of the Committee that participated in numerous conference calls and attended informal meetings, e.g., at the American Educational Research Association Annual Meeting, as we continued to hammer out the details: Peg Goertz, Mike Knapp, Maria Alicia Lopez Freeman, Patrick Shields, Judy Sowder, and Elizabeth Stage. And it is certainly no exaggeration to say that without the ceaseless support from National Research Council (NRC) staff members Karen Hollweg and Gail Burrill, this document would never have seen the light of day.

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Henry L.Alder, University of California, Davis Lynn Cornett, Southern Regional Education Board, Atlanta, GA Karen Economopolous, TERC, Cambridge, MA Manuel Gomez, University of Puerto Rico Daniel Koretz, RAND, Arlington, VA Ronald M.Latanision, Massachusetts Institute of Technology Bill Linder-Scholer, SciMathMN, Roseville, MN Roland J.Otto, E.O.Lawrence Berkeley National Laboratory, Berkeley, CA Richard Satchwell, Illinois State University Scott Stowell, Spokane Public Schools, WA Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Michael W.Kirst, Stanford University, and R. Stephen Berry, University of Chicago. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. Iris R.Weiss, Chair Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education PREFACE Standards have served as a basis of educational reform across the nation as educators and policy makers respond to the call for a clear definition of desired outcomes of schooling and a way to measure student success in terms of these outcomes. Public policies at the federal, state, and local levels have focused on standards as a way to raise achievement levels for all students. At the national level, content standards have been developed in nearly every academic discipline. As the standards movement has gained momentum, there has been increasing interest among educators and the public concerning the impact of standards on what happens in classrooms. In response to this interest, and as part of its mission of working for improved mathematics and science teaching and learning across the nation, the National Research Council’s Center for Science, Mathematics, and Engineering Education (which became the Center for Education in 2000)—with funding from the National Science Foundation—undertook the task of designing a framework that would aid in the design, conduct, and interpretation of research regarding the influence on student learning of the standards created at the national level in mathematics, science, and technology. In 1999, the Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education began its work. The charge of the Committee was to “develop a framework that can be used to understand the influence of science, mathematics, and technology education standards on programs, policies, and

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education practices.” This Framework was to provide guidance for the design, conduct, and interpretation of research regarding the influences of nationally developed standards on student learning in mathematics, science, and technology education. Hence, the Committee’s primary concern has been to develop a structure and process for understanding the influence of these educational standards on practices, programs, and policies in K-12 education. The Committee was responsible to two standing boards in the Center, the Committee on K-12 Science Education (COSE K-12) and the Mathematical Sciences Education Board (MSEB), and the Committee’s work reflects the oversight of these two boards. An overarching problem faced by the Committee was how to structure thinking about the impact of standards in a way that would consider the complex system in which the standards operate—a system that consists of several layers of policy makers, various educators involved in multiple ways, and an array of players with different responsibilities for what takes place in classrooms. It was clear from the beginning that we were not to answer the question of what influence the standards have had on educational reform. It was a constant struggle, however, to keep the focus on a framework and not an analysis of where and how the standards had taken root in education systems and whether they were affecting student learning. As the Committee worked, we came to a common understanding of some of the issues in the daunting task of laying out a way to think about the influence of the nationally developed standards on what happens within this complex system and eventually on what happens to student learning. We agreed, for example, to focus our thinking on separate but interacting channels within the education system—curriculum, teacher development, and assessment and accountability—and to consider how reform ideas might traverse these channels to reach the classroom. At the same time, we recognized that contextual forces outside the education system, in the public and political world, directly and indirectly affect what

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education happens in the classroom, and we wanted to be sure the Framework considered those as well. We agreed to concentrate on the standards created by the National Council of Teachers of Mathematics, the National Research Council, and the International Technology Education Association that focus on teaching and learning in mathematics, science, and technology. We also agreed that the focus of our work was not on the quality of the standards documents themselves but rather on their influence. We did not analyze the nationally developed documents nor make judgments about their focus and content, but concentrated on developing ways to think about whether and how these standards were being used in the system and with what results. As our work progressed, we came to see even more clearly the importance of having the document be neutral, neither advocating nor denigrating the standards and their impact. Our work was to lay out questions that should be answered in order to come to understand the influence of standards, regardless of whether or not the answers supported the thinking in the national documents. The Framework presented in this document is the result of our discussions. We eventually came to agree that the Framework would describe the system in which the standards operate, then suggest places where the effect of the standards might be observed in this system, which became the organizing structure of the document. We have drawn on a broad body of research related to various aspects of the task, and have also relied heavily on the practical experience of the Committee members and outside experts in designing the Framework. The Committee produced a background paper for its work, and in spring 2000, convened a group of educators and researchers in a workshop to discuss and critique the paper. The Committee used the insights and input from workshop participants to help define the Framework and to stimulate ideas regarding its future uses. Investigating the Influence of Standards is aimed at researchers

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education and those who use research in their work, including federal, state, and local policy makers. It is intended to provide a lens for considering the national standards and the role such standards play in improving mathematics, science, and technology education. This book is not intended to be a simple template prescribing a solution but rather lays out the major issues involved in considering the effects of standards in improving student learning, offering questions that should be asked when considering research about the standards. The document can be used to understand particular studies, to identify what the studies do and do not address, and to help consumers interpret their claims. The document is also designed to spur additional research in areas where answers to the questions are not well-formed. While the focus of this document is the nationally developed standards in mathematics, science, and technology, we believe the Framework is general enough to serve as a guide for those at state and district levels and in other disciplines to analyze the impact of their own standards. Iris R.Weiss, Chair Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education CONTENTS     Executive Summary   1 1   Introduction   11 2   Standards for Mathematics, Science, and Technology Education   17     The Context in Which Standards Evolved,   18     Developing National Standards in Mathematics,   20     Developing National Standards in Science,   22     Developing Standards for Technological Literacy,   24     Commonalities Across the Mathematics, Science, and Technology Standards,   24 3   A Framework for Investigating the Influence of Education Standards   29     An Overview of the U.S. Education System,   29     One View of the Education System,   31     The Elements of the Framework,   33

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education 4   Curriculum as a Channel of Influence: What Shapes What Is Taught to Whom?   39     Curriculum in the Education System,   39     Implications of Policy Decisions,   39     Development of Instructional Materials and Programs,   41     Selection Mechanisms,   42     How Standards Might Influence Curriculum,   44     The Curriculum Channel and Nationally Developed Standards,   46 5   Teacher Development as a Channel of Influence: How Do Teachers Learn What and How to Teach?   49     Teacher Development in the Education System,   50     Teacher Preparation,   50     Certification and Licensure,   52     Ongoing Professional Development,   53     How Standards Might Influence Teacher Development,   55     The Teacher Development Channel and Nationally Developed Standards,   57 6   Assessment and Accountability: What Kinds of Assessment Are Used and for What Purposes?   59     Assessment and Accountability in the Education System,   60     Accountability,   60     Classroom Assessment,   62     State and District Assessment and Accountability Policies,   63     College Entrance and Placement Practices,   64     Impact and Unintended Consequences of Assessment,   65

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education     How Standards Might Influence Assessment and Accountability,   67     The Assessment and Accountability Channel and Nationally Developed Standards,   68 7   Contextual Forces that Influence the Education System   71     Who Affects the Education System from “Outside”?,   71     Outside Forces Affecting Decision Making,   73     Outside Forces Affecting Components Within the Education System,   74     How Can Nationally Developed Standards Influence Those “Outside” the System? How Might They, in Turn, Influence the Education System?,   75     Contextual Forces and Nationally Developed Standards,   76 8   Using the Framework   79     The Framework in Review,   79     How the Framework Can Be Used,   83     Situating Current Studies,   86     Examining Claims and Inferences Reported in Current Studies,   88     Generating Questions and Hypotheses for Future Investigations,   92     Aspirations for Framework-Driven Research on Nationally Developed Standards,   94     References   97

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education     Appendixes     A   Participants at the Workshop on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education   109 B   A Framework for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education   113 C   Biographical Sketches   115     Index   125 FIGURES, BOXES, AND TABLE Figures 1–1   The Black Box,   12 3–1   The Layers of Education Governance and Channels Through Which Reform Might Flow,   32 3–2   A Conceptual Map for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education,   34 3–3   A Set of Guiding Questions for Investigating the Influences of Nationally Developed Standards,   35 8–1   A Framework for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education,   80 8–2   Parts of the Framework Addressed in the Briars and Resnick Study,   87

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education Boxes 2–1   Basic Principles and Features of Principles and Standards for School Mathematics,   21 2–2   Basic Principles and Features of National Science Education Standards,   23 2–3   Basic Principles and Features of Standards for Technological Literacy: Content for the Study of Technology,   25 Table 8–1   Hypothetical Studies That Address One or More of the Framework Questions,   84

OCR for page R1
Investigating the Influence of Standards: A Framework for Research in Mathematics, Science, and Technology Education This page in the original is blank.