SURVEY OF MATHEMATICS AND SCIENCE OPPORTUNITIES (SMSO)

International Organization

Michigan State University

Years of Data Collection

1991-1992

1992-1993

Purpose The purpose of the Survey of Mathematics and Science Opportunities is to develop better opportunity-to-learn methodology for the IEA Third International Mathematics and Science Study and to characterize the educational opportunities for learning mathematics and science in SMSO participating countries. This survey is a research and development effort studying intended curriculum, opportunity to learn, classroom processes in mathematics and science instruction, and teachers' knowledge of mathematics and science; it will provide conceptual, empirical, and instrumentational input for TIMSS. The SMSO survey will enable TIMSS to provide better characterizations of achievement level differences among and within countries and to explain the curricular context from which these differences arise.

Organization and Management The SMSO project and project director are located at the Michigan State University College of Education. The project director works with a team of specialists and an advisory board of internationally known experts.

Design To develop measurement of intended curriculum, opportunity to learn, classroom processes, and teachers' knowledge of mathematics and science, and to provide a characterization of mathematical and science opportunities in each participating SMSO country, the following were used:

  • Analysis of curricular materials and development of a curriculum framework onto which the intended curriculum of a country could be mapped. These are multi-aspect frameworks that include a content domain, performance expectations, and context. The curriculum framework was then applied to a set of curriculum syllabi and revised as necessary.

  • Classroom-based study in each of the SMSO participating countries. As separate aspects of the classroom studies, the development of improved measures of subject matter-specific pedagogy, teacher knowledge, and opportunity to learn were combined.

Participants

France, Japan, Norway, Spain, Switzerland, and the United States

Sample The materials prepared to measure subject matter specific pedagogy, teacher knowledge, and opportunity to learn were used in a nationally generated sample of 20-30 classrooms within each of the participating countries for a full school year.

Procedures and Summary of Content Logs, case studies, and other instruments were used in the selected classrooms. Trained interviewers visited the schools every other week to collect logs and other materials responded to by teachers. Teachers were also interviewed and classroom observations of mathematics and science lessons were conducted.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 101
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies SURVEY OF MATHEMATICS AND SCIENCE OPPORTUNITIES (SMSO) International Organization Michigan State University Years of Data Collection 1991-1992 1992-1993 Purpose The purpose of the Survey of Mathematics and Science Opportunities is to develop better opportunity-to-learn methodology for the IEA Third International Mathematics and Science Study and to characterize the educational opportunities for learning mathematics and science in SMSO participating countries. This survey is a research and development effort studying intended curriculum, opportunity to learn, classroom processes in mathematics and science instruction, and teachers' knowledge of mathematics and science; it will provide conceptual, empirical, and instrumentational input for TIMSS. The SMSO survey will enable TIMSS to provide better characterizations of achievement level differences among and within countries and to explain the curricular context from which these differences arise. Organization and Management The SMSO project and project director are located at the Michigan State University College of Education. The project director works with a team of specialists and an advisory board of internationally known experts. Design To develop measurement of intended curriculum, opportunity to learn, classroom processes, and teachers' knowledge of mathematics and science, and to provide a characterization of mathematical and science opportunities in each participating SMSO country, the following were used: Analysis of curricular materials and development of a curriculum framework onto which the intended curriculum of a country could be mapped. These are multi-aspect frameworks that include a content domain, performance expectations, and context. The curriculum framework was then applied to a set of curriculum syllabi and revised as necessary. Classroom-based study in each of the SMSO participating countries. As separate aspects of the classroom studies, the development of improved measures of subject matter-specific pedagogy, teacher knowledge, and opportunity to learn were combined. Participants France, Japan, Norway, Spain, Switzerland, and the United States Sample The materials prepared to measure subject matter specific pedagogy, teacher knowledge, and opportunity to learn were used in a nationally generated sample of 20-30 classrooms within each of the participating countries for a full school year. Procedures and Summary of Content Logs, case studies, and other instruments were used in the selected classrooms. Trained interviewers visited the schools every other week to collect logs and other materials responded to by teachers. Teachers were also interviewed and classroom observations of mathematics and science lessons were conducted.

OCR for page 101
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Data Collection and Analyses During 1992-93 the focus was on self report measures validated in the case studies. Analysis of the data was initiated with a meeting of representatives from each participating country. Representatives compiled the data, documented any irregularities in the data collection procedures, and established data analysis plans. SMSO has developed curriculum frameworks for mathematics and science. The frameworks are systems for coding and describing materials such as curriculum guides, syllabi, textbooks, and examinations along each of several dimensions and with regard to multiple attributes. SMSO has contributed to TIMSS by developing various teacher and school questionnaires, the TIMSS Conceptual Framework for Contextual Questions, the TIMSS Mathematics and Science Curriculum Frameworks, the TIMSS Document Analysis Manual, and the TIMSS Topic Trace Mapping Instructions and Forms. Six countries have participated in the development of a conceptual model of educational opportunity and in the development and piloting of related questionnaires and procedures in preparation for TIMSS. During 1991-92 observations were made in multiple classrooms in France, Japan, Norway, Spain, Switzerland, and the United States. Discussions of these observations served to inform questionnaire development for TIMSS. The Opportunity to Learn pilot was conducted in over 75 classrooms in France, Japan, Spain, Switzerland, and the United States. It gathered information from teachers and students on students' opportunity to learn various topics as represented in the TIMSS curriculum frameworks. This pilot was designed to address issues of validation of teachers ' indications of students' opportunity to learn, which can provide a rich and valuable context for understanding students' performance on achievement assessment measures. A report on what SMSO research has learned was presented at a symposium at the American Educational Research Association annual meeting in Atlanta. In January 1994 representatives from the six SMSO countries held an international working meeting, focusing on classroom observations of teachers' instructional practices. Participants read, discussed, and analyzed summaries of over 100 classroom observations that had been conducted in the SMSO countries. Analyses of these observational data summaries will be presented in a book on instructional practices. Timetable 1991 Development of curriculum frameworks began. A taxonomy of frameworks was drafted and used in trial coding. Development of methodology and instruments for classroom studies began. Data was reviewed and taxonomy revised. A protocol manual was developed. 1992 The full-scale study was conducted. Curricular materials data were analyzed. Curricular materials report was written. A trial of test items was conducted using students in classroom study. U.S. data was coded. Data from SMSO countries was banked, cleaned, and analyzed. A final instrumentation recommendation was made to TIMSS.

OCR for page 101
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies 1992-1993 Classroom observations were made in the 6 SMSO countries. 1992 Three international meetings were held to discuss classroom observation information. 1993 A report on what SMSO research has learned was presented at a symposium at the AERA annual meeting in Atlanta. (April) A meeting was held to produce versions of the student, school, and teacher questionnaires for international piloting. (May) Final pilot forms were submitted to the TIMSS international coordinating center. Final pilot forms were used in the October teacher and school questionnaire pilot. International meeting was held, to discuss plans for further data analyses and publications. (fall) Data collection for the Opportunity to Learn pilot was completed. 1994 Representatives from the six SMSO countries held an international working meeting, focusing on classroom observations of teachers' instructional practices. Paris. (January) A revised version of the student questionnaire was used in the March international field trial. Two major SMSO papers were published by the TIMSS international coordinating center Representatives from the six SMSO countries held an international working meeting, to complete analyses for the Opportunity to Learn pilot and discuss drafts of the Instructional Practices book. (summer) The Instructional Practices book will be completed by the end of 1994. Two SMSO papers were published by the TIMSS international coordinating center. Publications Two major SMSO papers have been issued as IEA Third International Mathematics and Science Study international coordinating center publications: Test Blueprints: A Description of the TIMSS Achievement Test Content Design. University of British Columbia, Vancouver, Canada. (1994) TIMSS Educational Opportunity Model: Detailed Instrumentation And Indices Development. University of British Columbia, Vancouver, Canada. (1994) The SMSO Research Report Series includes 57 draft and final documents on topics such as mathematics, science, and opportunity-to-learn curriculum framework, and topic trace mapping. It also includes questionnaires and manuals. Three proposed SMSO survey reports are: Instructional practices: An international report of differences across 6 countries (Japan, France, Norway, Switzerland, Spain, United States) in the delivery of classroom instruction Opportunity-to-learn achievement study: The report of a study of opportunity to learn -- how it is measured, its validity, and its reliability

OCR for page 101
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Qualitative textbook analyses: A qualitative description of mathematics textbooks in the six SMSO countries Database At this time the information that has been gathered for the SMSO/Third International Mathematics and Science Study is not public information. It will be a year or two before this information can be released. Funding Funding is provided by the National Science Foundation in conjunction with the National Center for Education Statistics. Information Sources William H. Schmidt, Project Director 463 Erickson Hall Michigan State University College of Education East Lansing, Michigan 48824-1034 telephone: 517/ 353-7755 facsimile: 517/ 432-1727 e-mail: wschmidt@msu.edu McKnight, Curtis, William H. Schmidt, and Senta Raizen 1994 Test Blueprints: A Description of the TIMSS Achievement Test Content Design. University of British Columbia, Vancouver, Canada. Schmidt, William H. 1991 Survey of Mathematics and Science Opportunity and U.S. TIMSS - status of activities. Presentation to the Board on International Comparative Studies in Education. September. 1992 Survey of Mathematics and Science Opportunity and U.S. TIMSS - status of U.S. participation. Presentation to the Board on International Comparative Studies in Education. February. 1992 Survey of Mathematics and Science Opportunity and U.S. TIMSS - status of activities. Presentation to the Board on International Comparative Studies in Education. October. 1993 Survey of Mathematics and Science Opportunity: Results of the curriculum analysis. Presentation to the Board on International Comparative Studies in Education. June. 1994 TIMSS Educational Opportunity Model: Detailed Instrumentation And Indices Development. University of British Columbia, Vancouver, Canada. 1994 TIMSS curriculum analysis plans. Presentation to the Board on International Comparative Studies in Education. May. Schmidt, William H., and Senta Raizen 1991 Survey of Mathematics and Science Opportunity - update. Presentation to the Board on International Comparative Studies in Education. May. Survey of Mathematics and Science Opportunities 1992 TIMSS Mathematics Curriculum Framework. March. SMSO, Michigan State University, East Lansing, Michigan. 1992 TIMSS Document Analysis Manual. March. SMSO, Michigan State University, East Lansing, Michigan. 1992 TIMSS Topic Trace Mapping Instructions and Forms. SMSO, Michigan State University, East Lansing, Michigan.

OCR for page 101
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies 1993 TIMSS: Curriculum Analysis: A Content Analytic Approach. Research Report Series No. 57. March. SMSO, Michigan State University, East Lansing, Michigan. ****** NOTE: This study summary was reviewed and edited by William Schmidt at Michigan State University in East Lansing on July 10, 1994; database information was provided by Schmidt on November 29, 1994.