endorses nor critiques this research. As these data about curriculum, teaching, and student achievement emerge, the methods of the work will be criticized, questioned, and lauded. Written before the release of the TIMSS data and hence too early for a careful review of the study or its products, this report aims instead to inform the reader about TIMSS and to foreshadow the kinds of questions, information, and interpretations that may follow. The scope of the study is large; the challenge to make sense of the information great. The utility, reliability, and validity of the information and interpretations that TIMSS will produce remains to be seen.

This report is intended to set the stage for discussions and constructive utilization of an undeniably extensive data set about mathematics and science education. We do this in two ways. First, we present a summary of the TIMSS research framework and methods, then we highlight preliminary findings from SMSO about curriculum and teaching and point to questions raised by these findings.

What is the Third International Mathematics and Science Study (TIMSS)?

Beginning in 1996-97, data from the Third International Mathematics and Science Study (TIMSS), will be released. This ambitious cross-national study measures student achievement in 44 countries (see Appendix 1). In addition, extensive data have been gathered about conditions of schooling, curriculum, and instructional practice. The ultimate potential of TIMSS is to be able to place each country's student achievement data within a sufficiently complex portrait of that country's educational system to enable identification, examination, and analysis of important relations among factors. Such analyses are essential to informed judgments about ways to improve education.

TIMSS will report on the mathematics and science achievement of students enrolled in the two adjacent grades that contain the largest proportion of 9-year-olds, the two adjacent grades that contain the largest proportion of 13-year-olds, and students in their final year of secondary education. All countries participated in testing 13-year-olds; testing at other grade levels was optional. The achievement data were gathered primarily in the 1994-95 academic year. See Appendix 4 for data release plans.

How is Opportunity to Learn Viewed in TIMSS?

The TIMSS design reflects the importance of analyzing relationships between opportunities to learn and educational outcomes. The three basic elements of TIMSS are the intended curriculum (the educational system's aims and goals); the implemented curriculum (the actual strategies, practices, and activities found in classrooms); and the attained curriculum (student learning). Consistent with this basic structure, the TIMSS perspective on educational experience includes school variables and processes and a view of student learning influenced by psychological and sociological perspectives. The TIMSS conceptual model (Appendix 2) provided a theoretical foundation for collecting data and an initial approach to examining relationships among findings. Four questions are at the heart of the design. One central question, ''What are students expected to learn?", corresponds to the intended curriculum. A second question adds a focus on teachers—"Who delivers the instruction?" The third



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 3
--> endorses nor critiques this research. As these data about curriculum, teaching, and student achievement emerge, the methods of the work will be criticized, questioned, and lauded. Written before the release of the TIMSS data and hence too early for a careful review of the study or its products, this report aims instead to inform the reader about TIMSS and to foreshadow the kinds of questions, information, and interpretations that may follow. The scope of the study is large; the challenge to make sense of the information great. The utility, reliability, and validity of the information and interpretations that TIMSS will produce remains to be seen. This report is intended to set the stage for discussions and constructive utilization of an undeniably extensive data set about mathematics and science education. We do this in two ways. First, we present a summary of the TIMSS research framework and methods, then we highlight preliminary findings from SMSO about curriculum and teaching and point to questions raised by these findings. What is the Third International Mathematics and Science Study (TIMSS)? Beginning in 1996-97, data from the Third International Mathematics and Science Study (TIMSS), will be released. This ambitious cross-national study measures student achievement in 44 countries (see Appendix 1). In addition, extensive data have been gathered about conditions of schooling, curriculum, and instructional practice. The ultimate potential of TIMSS is to be able to place each country's student achievement data within a sufficiently complex portrait of that country's educational system to enable identification, examination, and analysis of important relations among factors. Such analyses are essential to informed judgments about ways to improve education. TIMSS will report on the mathematics and science achievement of students enrolled in the two adjacent grades that contain the largest proportion of 9-year-olds, the two adjacent grades that contain the largest proportion of 13-year-olds, and students in their final year of secondary education. All countries participated in testing 13-year-olds; testing at other grade levels was optional. The achievement data were gathered primarily in the 1994-95 academic year. See Appendix 4 for data release plans. How is Opportunity to Learn Viewed in TIMSS? The TIMSS design reflects the importance of analyzing relationships between opportunities to learn and educational outcomes. The three basic elements of TIMSS are the intended curriculum (the educational system's aims and goals); the implemented curriculum (the actual strategies, practices, and activities found in classrooms); and the attained curriculum (student learning). Consistent with this basic structure, the TIMSS perspective on educational experience includes school variables and processes and a view of student learning influenced by psychological and sociological perspectives. The TIMSS conceptual model (Appendix 2) provided a theoretical foundation for collecting data and an initial approach to examining relationships among findings. Four questions are at the heart of the design. One central question, ''What are students expected to learn?", corresponds to the intended curriculum. A second question adds a focus on teachers—"Who delivers the instruction?" The third

OCR for page 3
--> question represents the implemented curriculum, "How is instruction organized?", while the fourth, "What have students learned?", corresponds to the attained curriculum. What Kinds of Information Have the TIMSS Researchers Collected? In order to make sense of the TIMSS data and analyses, it is important to know how the researchers conceptualized—and then investigated—several elements of educational systems. Analyses that explore connections among these elements are likely to be most productive in enabling valid interpretations and conclusions. For instance, information gathered on the content and use of textbooks provides a view of the curriculum that is complementary to data gathered on curriculum frameworks and tests. Knowing what is in textbooks and how widely they are used or whether and how they are even required offers insight into the potential influence of curriculum material on what students have opportunities to learn. Yet such information does not necessarily provide insight into the curriculum that students encounter in the classroom. Teachers' decisions about the organization and presentation of material also shape the curriculum. And knowing what students are taught still would not tell us what any particular student experiences, as mediated by his or her own interpretations. All these differences in meaning of curriculum highlight the interface between intended and implemented curriculum. There are many different ways to interpret what is meant by curriculum. No one is somehow "right"; instead, the interpretation depends on the questions one is trying to answer about curriculum. In examining data, it is important to understand how curriculum is being defined. What did the TIMSS mean by, and how did researchers gather data on, documents and texts — the intended curriculum? On instructional practices—the implemented curriculum? On student learning—the attained curriculum? Understanding their meanings is crucial to interpreting the TIMSS findings. The intended curriculum: TIMSS focused on three elements of the intended curriculum. First, researchers gathered and examined a variety of documents from each country that offered clues to the curriculum, as it was publicly defined.5 In many countries, the intended curriculum is articulated nationally, through framework documents. In some countries, such as the United States and Switzerland, the curriculum is for the most part specified at the state, district, or canton6—not the national—level. In all cases, the researchers read and analyzed curriculum guides and lists of objectives. Textbooks comprised a second element of the curriculum examined by TIMSS researchers. Researchers assembled a collection of mathematics and science textbooks from each country. Approximately 1,600 teachers' guides and textbooks were analyzed across participating countries. Wherever possible, the materials were selected to represent those used by a majority of a given nation's students. 5   The publication dates of these materials ranged from 1976 to 1994. The set of U.S. materials was dated between 1978 and 1994. 6   A canton in Switzerland corresponds roughly to a state in the U.S.

OCR for page 3
--> TIMSS researchers developed analytic frameworks7 for examining the intended curriculum according to the aspects of content, performance expectations, and perspectives. In Appendix 3, categories within each of these three aspects are listed. Separate frameworks were developed for mathematics and science. Researchers coded each sample of material according to its categorization in each of the three aspects of the framework. The TIMSS analysis of curriculum documents and textbooks will offer a multidimensional portrait of the intended curriculum in participating countries. The implemented curriculum: The implemented curriculum was studied in several different ways in TIMSS. Questionnaires were used to survey teachers about their instructional practices. Teachers were asked how they use textbooks and technology, the activities they employ, what they cover during the year, what they believe about students and about learning, and what they think in many other areas of their teaching.8 In addition, they were asked about staffing and other administrative matters. In another major component of TIMSS, researchers videotaped a sample of eighth grade mathematics lessons in Germany, Japan, and the U. S. and conducted close analyses of the different pedagogical styles and the implemented curricula.9 Finally, detailed case studies, examining issues of education context more generally, were conducted in the same three countries.10 As part of the case study project, school administrators were interviewed for school-level information about the cultural and contextual factors that influence students' academic achievement. The attained curriculum: More than a half million students were assessed in TIMSS. Students in three age groups were studied: 9-year-olds (Population 1), 13-year-olds (Population 2), and a broad range of students in the final year of secondary school (Population 3). The assessment measures included multiple-choice and free-response items. With a random subsample of students in Populations 1 and 2, performance assessment tasks also were administered.11 Many different mathematics and science topics were included across the assessment instruments, and some tasks involved understandings of more than one topic.12 7   Robitaille, D., Schmidt, W., Raizen, S., McKnight, C., Britton, E., Nicol, C. (1993). Curriculum frameworks for mathematics and science. (TIMSS Monograph No. 1). Vancouver: Pacific Educational Press. 8   To be reported in U.S. Survey Report, International Questionnaire Results. 9   Stigler, et al. (in press, 1996). TIMSS classroom videotape studies: Preliminary findings and methodology. (National Center for Education Statistics of the U.S. Department of Education). Washington, DC: Government Printing Office. 10   To be reported in Stevenson, H. Findings from ethnographic case studies in Germany, Japan, and the U.S. 11   Westat, Inc., U.S. TIMSS Bulletin. No. 1. [Brochure]. Rockville, MD. 12   Garden, R.A. (1996). Development of the TIMSS achievement items. In D.F. Robitaille & R.A. Garden, (Eds.), Research questions & study design. (TIMSS Monograph No. 2). Vancouver: Pacific International Press., and Schmidt, W.H. & McKnight, C.C. (1995). Surveying educational opportunity in mathematics and science: An international perspective. Educational evaluation and policy analysis, 17(3), 337-353.

OCR for page 3
--> What are Some of the Challenges and Opportunities of Cross-National Research? International studies are sometimes interpreted as a competition among the participating countries, a "horse race" in which countries seek to rank high on some finish list. However, such comparisons are not the most productive way to use and learn from complex multinational studies. Studying curriculum, teaching, and learning is a complicated endeavor. When such research is conducted in multiple educational contexts, a wider set of interesting contrasts and possible relationships exists. It becomes possible to delve deeply into how curriculum and teaching interact to affect students' experiences and various outcomes of schooling. Moreover, the wider variation in educational practice also offers views of what is possible in curriculum and teaching. Such images can be a crucial resource for the design of educational improvement.13 Some issues that can be examined with cross-national studies include the following: The identification of factors that affect learning: There is considerable variation in educational practices across countries. In international studies, variables such as class size, age of school entry, who attends what schools, and the role and nature of assessment can be examined more comprehensively and described within different countries' contexts. The identification and examination of pedagogical approaches: Just as cultural and political parameters that may affect learning vary across countries, so, too, do teaching practices. Different approaches to teaching can be identified, described, and analyzed for their underlying assumptions about learning, children, and subject matter as well as about the purposes of schooling. In international studies, a wide array of alternatives is offered, which can contribute to expanding our images of teaching. The challenges of educational innovation also can be examined in looking across the various efforts in different countries. Analysis of how curriculum and pedagogy are embedded in broader educational and social contexts: Across countries there not only are differences in elements such as structures of schooling, curriculum, and pedagogical approach, but there are salient cultural, social, and political differences as well. International studies offer opportunities to probe factors that seem to shape the interplay of content, pedagogy, and learning. Where systems produce consistently high or low student achievement, comparative studies make it possible to examine factors that may influence such outcomes. Although the naturally occurring variation among countries makes international studies rich opportunities for learning, TIMSS also has the potential to provide a current portrait of practice and achievement within each country. TIMSS will give Americans unprecedented access to recent, systematically gathered information about curriculum, teaching, and learning in the United States. 13   Board on International Comparative Studies in Education. (1993). A collaborative agenda for improving international comparative studies in education. Washington, DC: National Research Council.