*What Can We Learn?*

People often look at the results of international studies to find out their country’s ranking. But knowing that one country’s 9 year olds scored higher in science than another’s tells us little about what made their achievement possible. Why do students in one country do better than students in another? What explains international differences in students’ achievement?

A recently completed landmark study of mathematics and science education in more than 40 countries gathered information that can help address questions such as these. Results from the Third International Mathematics and Science Study (called TIMSS) will be released beginning in the fall of 1996 and continuing through the spring of 1998. This brochure outlines what such international studies can help us learn, how this one was conducted, and what the United States and other countries might be able to glean from TIMSS. Included are a time line for the release of TIMSS data and directions about whom to contact for further information.

**A PUBLICATION OF THE CENTER FOR SCIENCE MATHEMATICS, AND ENGINEERING EDUCATION OF THE NATIONAL RESEARCH COUNCIL**

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.

Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1

MATHEMATICS AND SCIENCE EDUCATION AROUND THE WORLD
What Can We Learn?
People often look at the results of international studies to find out their country’s ranking. But knowing that one country’s 9 year olds scored higher in science than another’s tells us little about what made their achievement possible. Why do students in one country do better than students in another? What explains international differences in students’ achievement?
A recently completed landmark study of mathematics and science education in more than 40 countries gathered information that can help address questions such as these. Results from the Third International Mathematics and Science Study (called TIMSS) will be released beginning in the fall of 1996 and continuing through the spring of 1998. This brochure outlines what such international studies can help us learn, how this one was conducted, and what the United States and other countries might be able to glean from TIMSS. Included are a time line for the release of TIMSS data and directions about whom to contact for further information.
A PUBLICATION OF THE CENTER FOR SCIENCE MATHEMATICS, AND ENGINEERING EDUCATION OF THE NATIONAL RESEARCH COUNCIL

OCR for page 1

What is TIMSS and what can we learn from it?
Will the achievement of students from the U.S. exceed that of other competitors? Will we “do better” than we did in the last such international study? Many Americans will ask these questions as they look at the parts of TIMSS that involve student achievement data. These data alone cannot help us know what we are doing well or how we can improve, however. For this we need a great deal of additional information about what is taught and how it is taught here and in other countries. TIMSS will give us a start, in part by showing us what teachers in many different schools, districts, and countries do. With TIMSS data on what students in different countries learn, and on what and how they are taught in those countries, researchers will be able to pursue questions about what makes a difference in those countries for students’ learning of mathematics and science.
TIMSS is the most sweeping study of mathematics and science curriculum, instruction, and learning ever undertaken. Conducted between 1991 and 1995, TIMSS assessed more than 500,000 students, at ages 9, 13, and 18, in more than 40 countries. To learn about curriculum, more than 1,600 curriculum frameworks and textbooks were analyzed. Teachers were surveyed about what they teach, how they teach, and why. In three countries, classrooms were videotaped, and case studies of educational context were conducted. TIMSS is a project of the International Association for the Evaluation of Educational Achievement, and is funded by the National Science Foundation, the National Center for Education Statistics, and the Canadian government.
WITH TIMSS DATA ON WHAT STUDENTS IN DIFFERENT COUNTRIES LEARN, AND ON WHAT AND HOW THEY ARE TAUGHT IN THOSE COUNTRIES, RESEARCHERS WILL BE ABLE TO PURSUE QUESTIONS ABOUT WHAT MAKES A DIFFERENCE IN THOSE COUNTRIES FOR STUDENTS' LEARNING OF MATHEMATICS AND SCIENCE.
In sum, unlike previous studies of its kind, TIMSS will provide detailed descriptions about what is intended to be taught in mathematics and science, as seen in curriculum goals documents and in textbooks, and what is actually taught in classrooms. It will also report on teaching approaches. All this information from TIMSS about students’ opportunities to learn will provide context for the TIMSS student achievement data. When a group of students achieves high scores in a certain area, we will be able to investigate factors that influenced their success.

OCR for page 1

Preliminary observations and possible questions
The Study of Mathematics and Science Opportunities (called SMSO) is a small exploratory study of six TIMSS countries, in which the researchers developed methods to be used in the TIMSS research. These methods were intended to help take into account the fact that schools and cultures are different. They also were intended to help researchers compare data and observations across the schools and cultures studied. The countries involved—France, Japan, Norway, Spain, Switzerland, and the United States—were chosen because the differences in their schools and cultures were expected to be similar to the range of differences represented in the full TIMSS study.
Although the researchers expected to find differences across the schools and cultures they studied, they were surprised by what they learned. For example, they found that there can be great variation simply in the common activities of school. Practices that Americans assume or take for granted may be unfamiliar or even unknown elsewhere, and vice versa. Take “seatwork,” for example. In the United States, seatwork means independent written practice work. It is a common part of most American mathematics classes. Yet in France SMSO researchers observed students’ independent work as an integral part of whole group work and very different from what is meant by seatwork in the U.S. What the teachers in France and the teachers in the U.S. were asking students to do, and their reasons for asking students to do it, differed significantly.
THESE TYPES OF OBSERVATIONS FROM THE SMSO STUDY CANNOT BE USED TO GAUGE WHICH METHODS OF INSTRUCTION ARE MOST EFFECTIVE OR WHICH CURRICULA ARE BEST, BUT THEY CAN POINT TO QUESTIONS FOR FURTHER INVESTIGATION ABOUT EDUCATIONAL PRACTICES AND WHAT THEY MAY IMPLY FOR STUDENTS’ LEARNING.
These types of observations from the SMSO study cannot be used to gauge which methods of instruction are most effective or which curricula are best, but they can point to questions for further investigation about educational practices and what they may imply for students’ learning. In short, these early observations can help set the stage for analysis and discussion of the TIMSS data.
What is the “intended curriculum” like in the six SMSO countries?
When TIMSS researchers gathered information about curriculum from different countries, they examined materials, such as frameworks and textbooks. Analyses of these data will reveal what is in the curriculum, what students are expected to do, and how decisions about curriculum are made. Researchers call this view of the curriculum the intended curriculum because it is what educators intend students to learn.
What did the SMSO study find out about the intended curriculum? The following section provides highlights and then raises questions for further analysis.

OCR for page 1

In some of the six countries studied, particular topics in both science and mathematics are emphasized at certain grade levels and are then sustained over several years. In contrast, those same topics can appear in other countries’ curricula over and over, without strong emphasis at any point. Curricula in Japan and Spain concentrate on a small number of mathematics topics each year. Curricula in Norway, France, and the U.S. tend to cover many more. In science a similar pattern is seen, although not with the same countries: in France, Japan, and Norway, fewer topics are taught each year but with more depth.
Observation
THE NUMBER OF MATHEMATICS AND SCIENCE TOPICS COVERED EACH YEAR VARIES WIDELY BY COUNTRY. THERE IS NO UNIVERSAL SET OF MATHEMATICS OR SCIENCE TOPICS TAUGHT AT EACH GRADE LEVEL.
There are interesting differences among the six countries in terms of when topics first appear in the curriculum and when they are dropped. For example, the Japanese mathematics curriculum introduces some algebra topics to students as young as 8, while in Spain these topics are not introduced to students until later. Also, the Japanese curriculum emphasizes “equations and formulas” almost every year of schooling, while in Norway and Spain this topic is not emphasized in any grade.
Observation
CURRICULUM MATERIALS FROM DIFFERENT COUNTRIES VARY IN TERMS OF THE ORDER OF TOPICS, EMPHASIS ON PARTICULAR TOPICS, AND EXPECTATIONS FOR STUDENTS’ PERFORMANCE.
In the U.S. there is a tendency to address a given topic repeatedly over a period of several grades, while in Japan topics are often started and finished over a two-year period. In France the topic of “chemical properties of matter” is introduced to students at age 13 and emphasized at ages 15, 16, and 17, while in Norway the same topic is introduced to students at age 10 and addressed each year thereafter but never emphasized.
Textbooks differ, too, in how much seems to be expected of students. In Japan, and to a lesser extent in France and Spain, the material in science textbooks for 13 year olds is more complex and more organized around important scientific themes than in other SMSO countries. In mathematics the relatively complex topic of “properties and meanings of whole numbers” takes up 20 percent of the space in French textbooks but only 8 percent of the space in Norwegian texts and 2 percent in Swiss texts.
Some of the current reforms in mathematics and science education are based on the idea that students will develop a deeper understanding if they study fewer topics in greater depth: this idea is often referred to as “less is more.”
A question to ask
IS LESS ACTUALLY MORE?
For example, a fourth-grade mathematics curriculum might devote much more time to fractions and less to geometry, measurement, and long division. The point would be to help students develop deeper understandings of fractions.
In science, the objectives for a given grade might be organized around a small set of related topics in biology, offering students the

OCR for page 1

possibility of delving into an area of the life sciences with greater rigor and depth. An alternative would be to include a variety of topics in order to provide more exposure to a wider range of scientific study.
SMSO shows that countries make different choices about how much time to devote to topics, about how many topics to present at each grade level, and about the level of complexity to emphasize. This international variation provides an opportunity to find out what students learn under many kinds of curricular approaches.
In France, Japan, Norway, and Spain, centralized government agencies control the development of national curriculum frameworks. These frameworks list curriculum goals that regions and schools must implement. On the other hand, each of the Swiss cantons has its own curriculum framework. Curriculum is less centralized in the U.S. than in any other SMSO country studied. Here, curriculum goals are established by states and individual school districts.
Observation
DECISIONS ABOUT WHAT SHOULD BE TAUGHT ARE MADE IN VERY DIFFERENT WAYS IN DIFFERENT COUNTRIES.
Curriculum frameworks, guides, and textbooks are developed and used in a wide variety of ways throughout the world. Countries differ in whether or how national curricula are specified, who has the authority to establish national goals, and how teachers are obliged to work with the curricula. Do these different arrangements affect what teachers actually teach? Do differences in the control of curriculum affect student learning? Does a national curriculum provide coherence in local curriculum, instruction, and assessment?
A question to ask
HOW DOES NATIONAL CURRICULUM WORK?
What is teaching like in the six SMSO countries?
Researchers define what teachers do as the implemented curriculum. This is the curriculum as it is experienced by students. SMSO researchers conducted a few observations in mathematics and science classes in each of the six countries—too few to permit us to draw general conclusions about national trends or practices in these countries. However, the results of these observations do raise important questions about the connections between teaching practices and students’ learning. These questions can help guide interpretation of the TIMSS achievement data.

OCR for page 1

Some professionals believe that the teaching of mathematics and science is universally similar. However, the SMSO observations suggest that the teaching of these subjects may be strongly influenced by country and culture. For example, researchers from outside the U.S. were unfamiliar with the common American practice of having students trade papers to check each other’s homework. Other differences found from country to country included class size, teacher roles, and the typical homework and in-class assignments that teachers give students.
Observation
TEACHING PRACTICES APPEAR TO VARY ACROSS THE SIX COUNTRIES.
The TIMSS teacher questionnaires asked teachers about their preparation, instructional practices, use of textbooks, and beliefs about educational issues, which will enable educators to learn more about the relationship of country and teaching practice from the TIMSS data. In some countries videotapes were made, which will provide glimpses of how teachers interact with students, present the content, and teach the curriculum.
How teachers think about and conduct their practice is key to understanding mathematics and science education in a given country. Items on the TIMSS teacher questionnaires asked teachers about their practices. Teachers were asked, for example, how often they ask students in science classes to explain the reasoning behind an idea or to write out explanations about what was observed or how it happened. Teachers were also asked how often they ask students to represent and analyze relationships using tables, charts, or graphs and why. From the answers provided, educators can examine the international variations in common teacher beliefs and practices and how these variations are related to students’ learning.
A question to ask
WHAT ABOUT TEACHERS?
SMSO researchers thought that there might be recurrent patterns of instruction and classroom activity that are rooted in the culture of each country studied. They found some evidence to back up their ideas. For example, in the mathematics lessons observed, French teachers tended to make formal theoretical presentations of complex subject matter, as did many of the Spanish teachers. Mathematics lessons in Norway seemed to be less teacher centered and more focused on active engagement with the ideas at hand. The small sample of mathematics and science lessons observed in the U.S. tended to be directed by the teacher and to emphasize definitions and vocabulary.
Observation
A CHARACTERISTIC STYLE AND APPROACH TO TEACHING SEEMS PERVASIVE IN EACH COUNTRY.
If characteristic patterns of mathematics and science teaching practice exist a in country, how are they tied to student achievement? Can particular patterns of teaching be linked with student performance and achievement? Are the differences more striking between countries than they are within countries? What would this imply for efforts to “transport” educational practices and ideas across contexts? Can practices be exported and imported?
A question to ask
WHAT ROLE DO CULTURAL FACTORS PLAY?

OCR for page 1

TIMSS: An opportunity to learn
Take a moment to review the time line for release of TIMSS-related data and observations (see last page). The SMSO preliminary findings are a sample of what is to come. The TIMSS data will tell us far more about mathematics and science education across the 40-plus countries studied.
From the TIMSS curriculum analysis of textbooks and frameworks we will know more about the intended curriculum. From the TIMSS videotape, case study, and teacher survey data we will have more information about instructional practice and classroom activities—the implemented curriculum. When we put this together with the TIMSS achievement data, which researchers refer to as the attained curriculum, we will be able to ask important questions about what impact any of these cross-national differences have on what students actually learn.
FOR EACH OF THE COUNTRIES INVOLVED, THE TIMSS DATA WILL TELL A STORY THAT BEGINS WITH NATIONAL GOALS AND GOES ON TO PROVIDE SNAPSHOTS OF CLASSROOM LIFE.
It is too soon to tell how the TIMSS data will help countries with their educational policies and practices. Some countries’ policies and practices will seem attractive and worthy of imitation. However, they will not necessarily be easy to export. Their cultural roots will prove to be key elements in their effectiveness.
The past few years have brought a wave of reforms in mathematics and science education in the U.S. and elsewhere. The TIMSS data will provide a portrait of American mathematics and science education quite early in the reform efforts. In addition, we will have access to information about other educational systems and practices that can inform and inspire our own ongoing efforts at improvement.
For each of the countries involved, the TIMSS data will tell a story that begins with national (and sometimes state and local) goals and goes on to provide snapshots of classroom life. All of this will add up to complex information about the participating countries’ education systems. Analyses of this information will be useful in making informed judgments about ways to improve education in the U.S. and around the world.
The SMSO report highlights the fact that we must maintain a serious respect for the complexity of ideas such as curriculum, teaching, and culture and not settle for the drawing of simplistic relationships among these complicated elements. Examining variations in intended and instructional practices is interesting. Much can be learned about the different forms that education can take across countries. However, the most important questions focus on how these different approaches to curriculum and instruction affect educational outcomes: What do students actually learn in different educational systems and what influences their learning?

OCR for page 1

What’s next?
THE TIMSS REPORTING PLANS
OCTOBER 1996
Characterizing Pedagogical Flow (SMSO report)
Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in School Mathematics
Many Visions, Many Aims: A Cross-National Investigation of Curricular Intentions in School Science
A Splintered Vision: An Investigation of U.S. Science and Mathematics Education
NOVEMBER 1196
Eighth Grade Mathematics and Science Performance in International Perspective
Findings from the U.S. Assessments and Questionnaires
Technical Report on the U.S. Findings and Questionnaires
TIMSS Classroom Videotape Studies: Preliminary Findings and Methodology
Findings from Ethnographic Case Studies in Germany, Japan, and the U.S
Databases (U.S. assessments and questionnaires, mathematics classroom instruction videotapes, case study interview and field notes)
SUMMER 1997
Fourth-Grade Achievement and Questionnaire Results
WINTER 1998
Twelfth-Grade Achievement and Questionnaire Results
Additional reports, papers, and analyses will continue for several years.
References
National Research Council (1996). Mathematics and Science Education Around the World: What Can We Learn from the Survey of Mathematics and Science Opportunities (SMSO) and the Third International Mathematics and Science Study (TIMSS)? Washington DC: National Academy Press.
Schmidt et al. (in press). Characterizing Pedagogical Flow: An Investigation of Mathematics and Science in Six Countries. Dordrecht, The Netherlands: Kluwer Academic Publishers.
FOR ADDITIONAL INFORMATION ABOUT SMSO AND TIMSS, CONTACT:
Dr William H.Schmidt. U.S. National Research Coordinator, TIMSS Curriculum Analysis Project, Michigan State University College of Education, 457 Erickson Hall, East Lansing, MI 48824:517–353–7755: bschmidt@pilot.msu.edu: http://ustimss.msu.edu
Dr. Larry Suter TIMSS Program Officer, National Science Foundation, Division of Research Evaluation and Communication: 4201 Wilson Boulevard, Arlington, VA 22230 703–306–l650: lsuter@nsf.gov
Dr. Albert Beaton, Director, TIMSS International Study Center, Campion Hall 323 Boston College, Chestnut Hill, MA 02167, 617–552–4521: timss@hermes.bc.edu
Dr. Lois Peak, TIMSS Project Officer, U.S Department of Education, National Center for Education Statistics, 555 New Jersey Avenue, NW Room 402A, Washington, DC 20208:202–219–1804: loispeak@ed.gov: http//www.ed.gov/NCES/TIMSS.html
Development of this brochure was overseen by a committee of the National Research Council The From SMSO to TIMSS project is a collaborative effort of the Mathematical Sciences Education Board and the Committee on Science Education K-12 of the NRC’S Center for Science Mathematics and Engineering Education The project is supported wtih funds from the National Science Foundation the opinions expressed here are those of the author and do not necessarily reflect those of the National Science Foundation.
FOR ADDITIONAL COPIES OF THIS PUBLICATION, WRITE:
Mathematical Sciences Education Board or Committee on Science Education K 12
Center for Science, Mathematics and Engineering Education timss@nas.edu
National Research Council
210l Constitution Ave NW (HA 450)
Washington, DC 20418