THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY (TIMSS)

International Organization

International Association for the Evaluation of Educational Achievement

Years of Data Collection

1993-1995

1997-2000

Purpose The Third International Mathematics and Science Study will be a major source of information for discussion of the quality of education around the world. It will not only measure student achievement in mathematics and science internationally, but will also investigate differences in curriculum and instruction, and include alternative assessment options.

Policy makers are increasingly called upon to make decisions affecting the quality of education in their nations. The quality of the educational systems of a nation is seen to be linked to the ability of a nation to compete in the international marketplace. Therefore, increasingly policy decisions for education are being examined in light of the economic status of a nation and policy makers are looking to the results of international studies as a source of information. During this time of rapid and pervasive technological advancement, their interest is particularly focused on the areas of mathematics and science achievement that are seen to be strongly linked with economic development.

TIMSS will simultaneously investigate science and mathematics curricula and achievement. This design will enable researchers to collect data on two different subjects concurrently and will enable educators and policy makers to identify relationships between the two disciplines as they are intended for learning, as they are taught, and as they are learned.

Previous IEA studies in the areas of mathematics and science have produced valuable insights into a number of aspects of the teaching and learning process. In addition, they have provided a number of important lessons with respect to the design and conduct of large-scale international research projects in education. TIMSS will build on this foundation, and will provide further information about several important matters, such as:

  • The need to provide current national and international information that educational systems can use to compare and contrast their curricula, teaching practices, and student outcomes with the educational systems from other countries of interest.

  • An assessment of the potential impact that alternative curricular offerings, teaching strategies, and administrative arrangements have on learning.

  • An identification of what is possible in the teaching of mathematics and science. For example, results from the Second International Mathematics Study showed extremely high growth rates in two countries, and that finding indicates that more significant growth rates might be possible in other countries. Identification of the underlying causes of those high growth rates would have profound educational implications.

  • A greater understanding of how and why student attitudes change, and what relationship the development of positive attitudes bears to classroom practices. Such an understanding is crucial to the development of a more complete picture of how science and mathematics learning takes place.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies THIRD INTERNATIONAL MATHEMATICS AND SCIENCE STUDY (TIMSS) International Organization International Association for the Evaluation of Educational Achievement Years of Data Collection 1993-1995 1997-2000 Purpose The Third International Mathematics and Science Study will be a major source of information for discussion of the quality of education around the world. It will not only measure student achievement in mathematics and science internationally, but will also investigate differences in curriculum and instruction, and include alternative assessment options. Policy makers are increasingly called upon to make decisions affecting the quality of education in their nations. The quality of the educational systems of a nation is seen to be linked to the ability of a nation to compete in the international marketplace. Therefore, increasingly policy decisions for education are being examined in light of the economic status of a nation and policy makers are looking to the results of international studies as a source of information. During this time of rapid and pervasive technological advancement, their interest is particularly focused on the areas of mathematics and science achievement that are seen to be strongly linked with economic development. TIMSS will simultaneously investigate science and mathematics curricula and achievement. This design will enable researchers to collect data on two different subjects concurrently and will enable educators and policy makers to identify relationships between the two disciplines as they are intended for learning, as they are taught, and as they are learned. Previous IEA studies in the areas of mathematics and science have produced valuable insights into a number of aspects of the teaching and learning process. In addition, they have provided a number of important lessons with respect to the design and conduct of large-scale international research projects in education. TIMSS will build on this foundation, and will provide further information about several important matters, such as: The need to provide current national and international information that educational systems can use to compare and contrast their curricula, teaching practices, and student outcomes with the educational systems from other countries of interest. An assessment of the potential impact that alternative curricular offerings, teaching strategies, and administrative arrangements have on learning. An identification of what is possible in the teaching of mathematics and science. For example, results from the Second International Mathematics Study showed extremely high growth rates in two countries, and that finding indicates that more significant growth rates might be possible in other countries. Identification of the underlying causes of those high growth rates would have profound educational implications. A greater understanding of how and why student attitudes change, and what relationship the development of positive attitudes bears to classroom practices. Such an understanding is crucial to the development of a more complete picture of how science and mathematics learning takes place.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Educational inputs, processes, and outputs constitute three basic components of the educational system, and each of these is related to the others in a complex, integrated whole. An important objective of TIMSS is to examine the comparative influence of a variety of constituent variables within each component on the success of a given educational system as indicated by measures of student achievement, participation, attitudes, and aspirations. IEA has established a cycle of studies covering the major areas of the school curriculum: the 1990s will be the TIMSS decade. It has been more than 10 years since the IEA Second International Mathematics Study and Second International Science Study were conducted (although publication of most of the international reports from those studies was delayed because of lack of funding needed to carry out the analyses of the international data). In the interim, interest in cross-national comparisons generally, and in the areas of science and mathematics specifically, has increased dramatically. Mathematics and science are important components of the school curriculum in every country. They are seen as integral components of every student 's educational program. As the role and impact of technology continue to increase in society and in the workplace, the concepts, processes, and skills of science and mathematics are likely to become even more highly valued and the importance of these subjects in school curricula to become even greater. Many countries around the world are currently involved in major reforms of their curricula in mathematics and science, and many of them will look to IEA and to TIMSS for the kind of information they need to guide that process. The findings from TIMSS will be of great interest internationally to educators, curriculum developers, researchers, policy makers, and politicians. Some of the major topics on which TIMSS will report are: Mathematics and science curricula. The results of the curriculum analysis will document international variation in mathematics and science curricula. Opportunity to learn. Opportunity-to-learn data, in conjunction with the curriculum analysis, will illustrate what is possible in the teaching of science and mathematics. Students' achievement. TIMSS will give considerable emphasis to students' achievement, including students' ability to apply their knowledge and skills in non-routine settings. Use of technology. The role of technology in the teaching and learning of mathematics and science, particularly as regards the use of calculators and computers for instructional purposes, is a topic of significant interest for TIMSS. Participation rates. The participation of students in pre-university courses in mathematics and science will be examined, particularly with regard to gender-based differences. Tracking and streaming. There is a considerable amount of international interest in the kinds of practices employed by schools and school systems to direct students' course selection, including tracking and streaming. Information about such practices will be collected at several levels within each participating system. The role of textbooks. An investigation of the nature, role, and influence of officially prescribed textbooks on the teaching of mathematics and science is one of the main goals of the curriculum analysis component of TIMSS. Instructional practices. A comparison of instructional practices, based on teacher self-report data, will be a major aspect of the study.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies The data from this rich array of cross-national comparisons of a large number of widely different countries should contribute significantly to the unfolding research agenda and to the needs of policy makers, researchers, and educators in many countries. Organization and ManagementThe International Association for the Evaluation of Educational Achievement (IEA) has a chair and a board of directors. The IEA Secretariat coordinates all ongoing studies. Individual projects have their own international coordination centers responsible for carrying out a given study, and these are backed by an international steering committee and the IEA international headquarters. The international steering committee chair is located at the Ontario Institute for Studies in Education in Toronto, Canada. The International Association for the Evaluation of Educational Achievement (IEA) headquarters is located in The Hague in The Netherlands. The International Coordinating Center for TIMSS is located in the University of British Columbia Department of Mathematics and Science Education in Vancouver, Canada. The International Coordinator is now assisted by a Deputy International Coordinator who has a major administrative responsibility for all aspects of the study. As TIMSS coordination has decentralized, a range of study activities has been contracted to institutions in several countries around the world. The 1993 IEA General Assembly confirmed the new position of Study Director for TIMSS. The Study Director and Study Center are located at Boston College. The study director has overall direction of all study activities, including those that the International Coordinating Center continues to manage. A National Research Coordinator in each participating country is responsible for the conduct of the TIMSS study in that country. (The IEA General Assembly approved a motion to discontinue the use of the term National Project Coordinator, and to replace it with National Research Coordinator, which has been used in previous IEA studies.) Data processing for the pilot study is being conducted at the Institute of Comparative Education at The University of Hamburg in Germany; data analysis for the pilot study is being conducted in Australia at the Australian Council for Educational Research; questionnaires are being developed at Michigan State University in East Lansing, Michigan; performance assessment tasks and scoring rubrics are being developed at the National Foundation for Educational Research in England. The study will draw upon the expertise accumulated across the IEA community during the past thirty years of IEA's existence. The roles of the International Coordinating Center and Study Director are to coordinate the development of timelines, instruments, sampling procedures, data entry procedures, analysis methods, translation verification systems, and reports from the data supplied by the participating educational systems. To insure the integrity of the study, national centers for the participating educational systems will use standardized procedures and systems developed by the International Coordinating Center and Study Director. The U.S. National Research Center and National Research Coordinator are located at Michigan State University College of Education in East Lansing, Michigan. U.S. TIMSS data collection is being conducted by Westat as contractor to the National Center for Education Statistics. The Westat contract also involves several subcontractors: Educational Testing Service, National Computer Systems, Harold W. Stevenson of the University of Michigan, and James W. Stigler of the University of California at Los Angeles. Each subcontractor operates a specific portion of the contract under the overall direction of Westat. A TIMSS Project Officer at the National Center for Education Statistics is the government project officer for all contractor activities.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Westat is responsible for U.S. TIMSS production of questionnaires and assessments and for receipt and processing of completed instruments for the spring 1994 field test and spring 1995 main data collections. Westat will receive support from National Computer Systems in these activities. Educational Testing Service will also assist by reviewing instruments and materials and collaborating with Westat in data analysis activities. U.S. states who desire to compare the achievement of their students to that of other nations participating in TIMSS have an opportunity to contract with Westat to participate as a “mini-nation” in the TIMSS data collection. U.S. TIMSS case study activities will be directed by Harold Stevenson; videotape observation project will be directed by James Stigler. Design The design of TIMSS includes a basic set of investigations, and participating educational systems will be free to select one or more of the investigations and a number of international options. TIMSS will focus on the teaching and learning of mathematics and science at three levels of the school system: the grade or level at which most students attain the ages of nine and thirteen and the grade or level at which most students are completing the last year of secondary education. One of the studies for the latter population will concentrate on an investigation of the scientific and mathematical attainments of students completing secondary school with some degree of specialization in either mathematics or science. The conceptual model for TIMSS was derived in large part from the models used in earlier IEA studies (especially the Second International Mathematics Study and Second International Science Study), modified and updated to meet the particular demands of the present study. Fundamental to the design of TIMSS is the centrality of curriculum as a variable. Also fundamental to the design of the study is the relationship between science and mathematics as separate, yet linked, components of that curriculum. The TIMSS curriculum frameworks are designed to provide rich descriptions of the intended curriculum, ones that can also be used in the development of achievement items. The TIMSS frameworks are powerful organizing structures that are both flexible and sophisticated. The multi-aspect, multi-category nature of the frameworks enables the description of traditional as well as reform-oriented curricula, providing dynamic ways of characterizing curricula internationally. The development of the TIMSS curriculum frameworks in analyzing curricula will help provide new insights into curriculum development internationally. Participants About 50 educational systems from all regions of the world and at all levels of economic development will participate in TIMSS. Participation from developing nations is encouraged by several agencies, IEA headquarters, and the TIMSS International Coordinating Center. Efforts are being made to provide appropriate opportunities for meaningful involvement by the educational systems in these nations. Sample Achievement test booklets for Populations 1 and 2 were piloted to students in grades three, four, seven, and eight. Fifty-two countries piloted the instruments. In the United States the pilot study involved twenty-seven schools and 3,000 elementary and middle school students from California and Michigan. A concerted effort was made to identify school systems in rural, suburban, and urban settings.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies In the United States the teacher questionnaires were piloted to a convenience sample of 50 Population 1 teachers and 50 Population 2 teachers. The appropriate version of the school questionnaire was piloted among 25 principals representing all the populations at which a country is participating (in the U.S. -- Populations 1,2, and 3). Countries provide sampling information to the international study director for review and approval. Statistics Canada continually advises national research coordinators; it has prepared a third version of the sampling manual. Procedures and Summary of Content The major source of data on student achievement in mathematics and science for TIMSS will be from achievement surveys containing multiple-choice items selected on the basis of their relationship to the mathematics and science curriculum frameworks developed for the study. The item pool for the achievement surveys is constructed so as to include items designed to measure cognitive behaviors at all levels, and also contains some free-response items, including short answer and extended response, as well as performance tasks. Questionnaires designed to elicit information about student and teacher backgrounds will be included for descriptive comparisons and to act as controls for multivariate analyses. School and classroom variables, including school climate, opportunity to learn, and time on task, will be investigated in a variety of relational analyses to be conducted. In addition to the main study, two international options have been proposed. One international option for participating educational systems will be concerned with an in-depth investigation of students ' problem-solving strategies. A second option will examine the linkage between teaching practices and changes in student achievement and attitudes. U.S. TIMSS field test and main study data collections: Westat is responsible for production of questionnaires and assessments, receipt and processing of completed instruments for the spring 1994 field test and spring 1995 main data collections. Westat will receive support from National Computer Systems in these activities. Educational Testing Service will also assist by reviewing instruments and materials and collaborating with Westat in data analysis activities. U.S. TIMSS state-level: States who desire to compare the achievement of their students to that of other nations participating in TIMSS have an opportunity to contract with Westat to participate as a “mini-nation” in the TIMSS data collection. Such states will administer the full TIMSS assessments and background questionnaires to a random sample of students in their states. Key features of U.S. State TIMSS are: Assessment scores collected through State TIMSS can be compared reliably to international TIMSS scores. Data will include both mathematics and science scores. States will have the choice of assessing their 3rd-4th, 7th-8th, and/or 12th-grade students. States will receive internationally comparable student, teacher, and school data from TIMSS background questionnaires. Reports are scheduled to be available in December 1996. States will be responsible for funding their own participation in State TIMSS.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies The State TIMSS program is now available for participation. Details of the design options and costs can be obtained from Westat. U.S. TIMSS case studies of education policy issues in the United States, Japan, and Germany: Year-long case studies in ministries, schools, and homes in three countries will supplement the data from the main study about four topics relating to opportunity-to-learn issues: teacher working conditions, the implementation of standards, how ability differences are dealt with, and the role of school in adolescents' lives. U.S. TIMSS videotapes of classroom instruction in the United States, Japan, and Germany: Videotapes will be used to provide observations of mathematics instruction in a subsample of 100 eighth grade classrooms participating in TIMSS. These observations will provide both quantitative data and qualitative descriptions of how instruction occurs in these countries. Data Collection and Analyses In May 1993 the United States participated in the TIMSS Achievement Item Pilot by piloting student test booklets in over 25 schools. More than 3400 students from California and Michigan completed booklets (twelve different booklets [six mathematics and six science at Population 1] and twenty booklets [ten mathematics and ten science booklets at Population 2]. The U.S. National Center coordinated all aspects of the U.S. pilot including recruiting and communicating with school district personnel, publishing and distributing pilot booklets, and entering and cleaning pilot data. Data was sent to the IEA data processing center; preliminary data summaries were forwarded to national research coordinators. The U.S. National TIMSS/Survey of Mathematics and Science Opportunity Center is directing development and production of the context questionnaires. The final pilot forms were submitted to the International Coordinating Center for use in the October-December 1993 Teacher and School Questionnaire pilot in participating countries; the revised version of the Student Questionnaire was used in the March 1994 International Field Trial. The Center was also responsible for writing the Pilot Manual, Administration Manual, and Data Code books for the international pilot. The U.S. National Center is coordinating all aspects of the pilot and is coordinating revision of the questionnaires following the pilot. The Technical Advisory Committee met in July 1993 in Melbourne, Australia to discuss the continued development of the data analysis plan for TIMSS. National research coordinators met in Frascati, Italy in October. Their agenda included discussions and presentations on the Populations 1 and 2 item pilot results and plans for the field trial operations. The national research coordinators reviewed Subject Matter Advisory Committee suggestions for Populations 1 and 2 item improvement. Educational Testing Service modified the items and assembled the items into booklets for the Field Trial. The TIMSS field test was conducted February -March 1994; in the United States Westat and its subcontractors were responsible for this activity. The data collection materials used included 8 two-part, pencil-and-paper assessment booklets for each population (provided by the IEA international coordinating center) and the U.S. versions of the student and teacher background questionnaires (provided by the U.S. National Research Center). The Australian Council for Educational Research, with the aid of Stanford Research Institute International and many individuals, worked to develop appropriate tests for Population 3. There will be a mathematics test, a science test, and a computer literacy test for the general part of Population 3 and advanced tests for mathematics and physics specialists.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies A special working group was established to assist the National Foundation for Educational Research in developing performance assessment tasks and scoring rubrics. They met in London in February to select the items. Performance assessments were conducted March - April 1994: in the United States groups of eight students in each of eight schools for Populations 1 and 2. Field test data collection for twelfth-grade students was then conducted April - May. It involved pencil-and-paper assessment of 50-60 twelfth grade students in each of 20 schools. Statistics Canada will review and evaluate field trial operational procedures. The procedural forms completed by participants in the field trial activities will be reviewed with the objective to determine each participant's ability to provide the necessary information and to determine how well current procedures work. Data from the teacher and school questionnaire pilot was sent from participating countries and is being analyzed. In addition to this quantitative data, large data bases have been created containing the free responses of questionnaire respondents and the evaluative comments made by the national research coordinators regarding specific items on the questionnaires and the pilot in general. Selected National research coordinators met in June 1994 to recommend and implement revisions of the school and teacher questionnaires. TIMSS assessments and questionnaires and administration procedures will be revised in preparation for the main study in spring 1995. As of April 1994, the TIMSS curriculum analysis had entered its analysis phase. As of March, document analysis data from over 830 documents from 45 countries had been entered into the data base. A comprehensive series of meetings with curriculum specialists and research methodologists resulted in the design of a variety of innovative analytical techniques for this component of TIMSS. This represents a major breakthrough in the comparative analysis of curricula and represents the first in what are expected to be many innovations arising from this ground-breaking component of TIMSS. Publication of the analysis in international reports is scheduled for late 1994-early 1995. The focus of the first report will be on the provision of educational opportunity through the intended curriculum and will closely examine commonalities and differences in curricula, with particular attention given to the empirical description of a world core. Field Trial data cleaning and preliminary analysis will be done in Hamburg; scaling will be done at the Australian Council for Educational Research. An international working group on context questionnaires met in Washington, D.C. in May 1994 to refine the school, teacher, and student questionnaires. At this working meeting, they reviewed and discussed the questionnaires that had been rearranged and reformatted to improve their clarity at the Australian Council for Educational Research. Final recommendations were then made regarding the questions that would be used in the piloting of the school and teacher questionnaires and revisions were made in the student questionnaire. In October 1994 an international meeting was held at the U.S. National Research Center to examine initial data analysis results and to develop strategies for data analyses and reporting as well as additional data collection for the TIMSS curriculum analysis. The Technical Advisory Committee met in May 1994 to discuss Population 3 definition and sampling, student-teacher linkages, “below the line ” definitions, field trial data analysis, management issues, the

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies handling of missing responses during analysis, operations issues, strategy for main survey item selection and performance assessment data analysis. Members of the recently established international Free Response Item Coding Committee met in April-May for preliminary discussions on the development of a coding strategy for extended response items and performance assessment tasks. In response to concerns about the definition and sampling of Population 3, the United States and France co-sponsored a meeting of selected individuals to address these issues. A group of selected national research coordinators met in early June to revise the teacher and school questionnaires. The Subject Matter Advisory Committee will meet in July to select candidate items for the main survey test booklets. National research coordinators will meet in August 1994 for selection of the Populations 1 and 2 main survey instruments. In 1993 and 1994 the U.S. national research coordinator, staff, and team of experts made presentations at meetings of the American Educational Research Association, the American Association for the Advancement of Science, National Association for Research in Science Teaching, Comparative and International Education Society, and National Science Teachers Association, and at the Council of Chief State School Officers Conference on Large Scale Assessments on: international curriculum analysis design issues for TIMSS measuring learning opportunities and instructional practices in mathematics and science in large-scale surveys development of teacher questionnaires what states can learn from TIMSS development of a model for the international study of learning opportunities use of observational research in the design of items for cross-national measurement of lesson structure development and validation of instruments for assessing opportunities to learn empirical analysis of intended curricula for a cross-national study of mathematics and science preliminary results from TIMSS international mathematics and science curricula international study of opportunities to learn cross-national measurement of opportunities to learn in mathematics and science world-class standards In 1994 the Board on International Comparative Studies in Education organized a session on TIMSS for the American Association for the Advancement of Science meeting. Papers were presented and discussed on evaluation of international comparative studies, TIMSS design, what is expected to be learned from TIMSS, and international mathematics and science curricula. Also in 1994 two books produced by the Survey of Mathematics and Science Opportunity at Michigan State University were published by the TIMSS International Coordinating Center at the University of British Columbia. Test Blueprints: A Description of the TIMSS Achievement Test Content Design presents the specifications for the TIMSS student test. It details the testing time and number of items to be allocated for specific framework categories and the various question types that will be used. TIMSS Educational Opportunity Model: Detailed Instrumentation and Indices Development describes how the data from the context questionnaires will be used to address specific research questions. It also details some proposed analyses and the development of key indices and typologies.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Timetable 1991 The Sampling and Methodology Committee met in Vancouver. (June) National research coordinators met in Vancouver. (August) The international steering committee met in Washington, D.C. (May) The working group on alternative assessment met at the University of British Columbia. (May) The Australian Council for Educational Research hosted a conference on assessment in mathematics at the elementary, secondary, and pre-tertiary levels. (November) Mathematics and science curriculum frameworks were developed. Preliminary topic trace and document analysis data of mathematics and science curricula were obtained from 30 countries. Methodology for the study was developed. 1992 The Subject Matter Advisory Committee met in Vancouver. (January) National research coordinators met. (March) Curriculum frameworks were distributed to participating educational systems. Student assessment items were collected. Document analysis and topic trace coding were conducted. 1993 The Subject Matter Advisory Committee met in Vancouver. (January) The International Steering Committee met in Vancouver. (February) National research coordinators met in Vancouver. (March) The U.S. National Center/SMSO directed development and production of context questionnaires. Final pilot forms were used in the October international Teacher and School Questionnaire Pilot. The field test was conducted. (February-May) An international working group on context questionnaires met in Washington, D.C. (May) The Technical Advisory Committee met in Melbourne, Australia. (July) The U.S. Steering Committee met twice in Washington, D.C. to discuss performance testing, achievement test, sampling, and preliminary curriculum analysis data. (May and November) National research coordinators met in Frascati, Italy. (October) Field trials of instruments for all populations were conducted in the southern hemisphere. Presentations were made by the U.S. national research coordinator and others at AERA, CIES, NARST meetings and a CCSSO conference. 1994 A special working group established to assist NFER in developing performance assessment tasks and scoring rubrics met in London. (February) A Field Trial data analysis plan was drafted. (March) The revised version of the Student Questionnaire was used in the April International Field Trial in the northern hemisphere. Data will be submitted to the University of Hamburg in June. Three books were published. Document analysis data was collected and analyzed. U.S. Phase 1 quality control activities were completed: data was cleaned and assembled for the international curriculum analysis. An international meeting was held at the U.S. National Research Center. (October) The Technical Advisory Committee met in Boston. (May)

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies   The Free Response Item Coding Committee met in Boston. (April-May) The United States and France co-sponsored a meeting in Washington, D.C., to discuss Population 3 definition and sampling. (May) Selected national research coordinators met in Hamburg to revise teacher and school questionnaires. (June) The Subject Matter Advisory Committee will meet to select candidate items for main survey test booklets. (July) National research coordinators will meet in Boston for selection of Populations 1 and 2 main survey instruments. (August) U.S. State TIMSS is available for participation. Presentations were made by the U.S. national research coordinator and others at AERA, AAAS, NSTA, and NARST meetings and at a CCSSO conference. The Board on International Comparative Studies in Education organized a session on TIMSS for the AAAS meeting. (February) Phase 1 main data will be collected for all populations in the southern hemisphere. The TIMSS Encyclopedia (1st edition) will be published, to serve as a reference text for all aspects of the study. The document analysis international report will be published. 1994-1995 International reports will be published on curriculum analysis. 1995 Phase 1 main data collection will be conducted in the northern hemisphere. (spring) The TIMSS Encyclopedia (2nd edition with updated information) will be published. 1996 Phase 1 main data will be cleaned; data banks will be created. Reports of U.S. State TIMSS are scheduled to be available. (December) Phase 1 main data collection international report will be published. Phase 2 will be designed and developed. 1997-2000 Phase 2 main data will be collected. Phase 2 main data will be analyzed and reported. Publications TIMSS curriculum analysis: Topic trace mapping. In Prospects. By William H. Schmidt. UNESCO. Curriculum Frameworks for Mathematics and Science. TIMSS Monograph No. 1. Pacific Educational Press, Vancouver, Canada. (1993) Test Blueprints: A Description of the TIMSS Achievement Test Content Design. University of British Columbia, Vancouver, Canada. (1994) TIMSS Educational Opportunity Model: Detailed Instrumentation And Indices Development. University of British Columbia, Vancouver, Canada. (1994) TIMSS Encyclopedia (1st edition, and 2nd edition with updated information) will be published.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies A document analysis international report will be published. The first TIMSS reports dealing with international comparisons of student outcomes are scheduled for publication in late 1996. International and national reports are proposed on U.S. and international pre results, curriculum, Survey of Mathematics and Science Opportunity, and survey results. A Phase 1 main data collection international report will be published. Database At this time, the information that has been gathered for TIMSS is not public information. It will be a year or two before this information can be released. Funding Funding is provided by the National Science Foundation, in cooperation with the National Center for Education Statistics, for the U.S. National Research Center at Michigan State University. U.S. states that participate in state TIMSS will be responsible funding their own participation. Information Sources Albert Beaton, Study Director Center for the Study of Testing, Boston College Campion Hall, Room 320 140 Commonwealth Ave. Chestnut Hill, Massachusetts 02167-3807 telephone: 617/ 552-4521 facsimile: 617/ 552-8419 e-mail: beatonal@hermes.bc.edu David F. Robitaille, International Coordinator University of British Columbia Department of Mathematics and Science Education 2125 Main Mall Vancouver, British Columbia, CANADA V6T 1Z4 telephone: 604/ 822-5337 facsimile: 604/ 822-4714 e-mail: David_Robitaille@mtsg.ubc.ca Robert Garden, Deputy International Coordinator University of British Columbia Faculty of Education 2125 Main Mall Vancouver, British Columbia CANADA V6T 1Z4 telephone: 604/ 822-9136 facsimile: 604/ 822-8571 e-mail: npc@timss.ubc.ca International Coordinating Center University of British Columbia Faculty of Education Vancouver, British Columbia, CANADA V6T 1Z4 telephone: 604/ 822-9136 facsimile: 604/ 822-8571 e-mail: TILMSS_ICC@mtsg.ubc.ca

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies William H. Schmidt, National Research Coordinator 463 Erickson Hall Michigan State University College of Education East Lansing, Michigan 48824-1034 telephone: 517/ 353-7755 facsimile: 517 432-1727 e-mail: wschmidt@msu.edu Lois Peak, U.S. TIMSS Project Officer (contract activities) National Center for Education Statistics 555 New Jersey Avenue, Room 304-C Washington, D.C. 20208 telephone: 202/ 219-2195 facsimile: 202/ 219-1736 Sandra Reider Westat, Inc. 1650 Research Blvd. Rockville, Maryland 20850 telephone: 301/ 738-3616 facsimile: 301/ 294-2038 (details of U.S. State TIMSS design options and costs; mailing list; periodic updates on the status of State TIMSS) International Association for the Evaluation of Educational Achievement (IEA) Secretariat c/o SVO Sweelinckplein 14 2517 GK The Hague THE NETHERLANDS telephone: 31-70-346-96-79 facsimile: 31-70-360-99-51 e-mail: iea@svo.nl Gordon M. Ambach, U.S. Liaison to IEA Council of Chief State School Officers One Massachusetts Ave., N.W., Suite 700 Washington, D.C. 20001-1431 telephone: 202/ 408-5505 facsimile: 202/ 408-8076 e-mail: gambach@nas.edu Ambach, Gordon, Senta Raizen, William Schmidt, Jeanne Griffith, and Larry Suter 1994 What Can States Learn from the Third International Mathematics and Science Study? A panel chaired by Gordon Ambach at the Council of Chief State School Officers National Conference on Large Scale Assessment. June. Anderson, Barry 1993 IEA Third International Mathematics and Science Study: Issues related to the sample design. Presentation to the Board on International Comparative Studies in Education.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Beaton, Albert 1993 IEA Third International Mathematics and Science Study. Presentation to the Board on International Comparative Studies in Education. November. 1994 IEA Third International Mathematics and Science Study: Current issues. Progress report and presentation to the Board on International Comparative Studies in Education. January; February. Beaton, Albert, and Maryellen Harmon 1994 IEA Third International Mathematics and Science Study: Issues (plans for test development; Population 3 meeting; dissemination plans; performance assessment). Status report and presentation to the Board on International Comparative Studies in Education. May. Beaton, Albert, and Andreas Schleicher 1993 IEA Third International Mathematics and Science Study: Meeting on TIMSS sample design. Presentation to the Board on International Comparative Studies in Education. January. Caldwell, Nancy 1994 IEA Third International Mathematics and Science Study: U.S. TIMSS studies - case studies. Presentation to the Board on International Comparative Studies in Education. February. Cogan, Leland 1993 Development and validation of instruments for assessing opportunities to learn. Presentation to the Comparative and International Education Society regional meeting. October. Gilford, Dorothy M., Daniel G. Horvitz, Ronald K. Hambleton, Albert E. Beaton, David E. Wiley, Richard Wolfe, William H. Schmidt, and Donald B. Rubin. 1994 The Third International Mathematics and Science Study. Session organized by Gilford and Horvitz for the Board on International Comparative Studies in Education, with presentations by Hambleton (Evaluation of International Comparative Studies), Beaton (TIMSS Design), Wiley and Wolfe (What is expected to be learned from TIMSS), and Schmidt (International Mathematics and Science Curricula), and discussion by Rubin at the American Association for the Advancement of Science meeting. February. Griffith, Jeanne 1991 IEA Third International Mathematics and Science Study: Report of the National Center for Education Statistics International Conference on TIMSS. September. Jeanne Griffith, Eugene Owen, and Larry Suter 1993 IEA Third International Mathematics and Science Study: Issues. Presentation to the Board on International Comparative Studies in Education. November. Horvitz, Daniel 1993 IEA Third International Mathematics and Science Study: Report of the board working group on TIMSS. Presentations to the Board on International Comparative Studies in Education. January and June. Houang, Richard, Gilbert Valverde, and Virginia Keen 1993 An empirical analysis of intended curricula for a cross-national study of mathematics and science. A forum for discussion of preliminary results of the curriculum analysis component of TIMSS presented at the Comparative and International Education Society regional meeting. October. International Association for the Evaluation of Educational Achievement (IEA) 1993 Activities, Institutions, and People: IEA Guidebook 1993-1994. IEA, The Hague, The Netherlands.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies IEA TIMSS International Coordinating Center 1991 TIMSS UPDATE. June 1 and July 1. IEA TIMSS International Coordinating Center, University of British Columbia, Vancouver, Canada. 1991 TIMSS Quarterly. Summer and Fall. IEA TIMSS International Coordinating Center, University of British Columbia, Vancouver, Canada. 1992 TIMSS UPDATE. December 8. IEA TIMSS International Coordinating Center, University of British Columbia, Vancouver, Canada. 1993 TIMSS Quarterly. October, February 3 and 19,, March 19, April 8 and 23, May 7 and 24, July 2 and 23. IEA TIMSS International Coordinating Center, University of British Columbia, Vancouver, Canada. 1994 TIMSS UPDATE. January 12 and 28. IEA TIMSS International Coordinating Center, University of British Columbia, Vancouver, Canada. IEA TIMSS International Study Director 1994 TIMSS UPDATE. April 22. IEA TIMSS International Study Director, Boston College, Chestnut Hill, Massachusetts. IEA TIMSS U.S. National Research Center 1993 IEA TIMSS U.S. National Research Center Report No. 2. August. TIMSS U.S. National Research Center, Michigan State University College of Education, East Lansing, Michigan. 1994 IEA TIMSS U.S. National Research Center Report No. 3. March. TIMSS U.S. National Research Center, Michigan State University College of Education, East Lansing, Michigan. Levine, Daniel 1992 IEA Third International Mathematics and Science Study: International Coordinating Center management of TIMSS operations. Presentation to the Board on International Comparative Studies in Education. May. Levine, Daniel, Eugene Owen, and Trevor Williams 1992 IEA Third International Mathematics and Science Study: Report of management review of TIMSS. Presentation to the Board on International Comparative Studies in Education. February. McKnight, Curtis, William H. Schmidt, and Senta Raizen 1994 Test Blueprints: A Description of the TIMSS Achievement Test Content Design. University of British Columbia, Vancouver, Canada. McLean, Leslie 1991 IEA Third International Mathematics and Science Study: Report from the International Steering Committee. Presentation to the Board on International Comparative Studies in Education. May. Owen, Eugene, and Lois Peak 1992 IEA Third International Mathematics and Science Study: Comments on National Project Coordinators meeting. Presentation to the Board on International Comparative Studies in Education. May. Owen, Eugene, Lois Peak, and Larry Suter 1993 IEA Third International Mathematics and Science Study: Meeting reports (Steering Committee, Subject Matter Advisory Committee, Technical Advisory Committee, National Research Coordinators). Presentation to the Board on International Comparative Studies in Education. November.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Peak, Lois 1991 IEA Third International Mathematics and Science Study: Plans for case studies and finance data collection in connection with TIMSS. Presentation to the Board on International Comparative Studies in Education. September. 1994 IEA Third International Mathematics and Science Study: U.S. TIMSS Studies - videotape studies; field test status report; state participation. Presentations to the Board on International Comparative Studies in Education. February and May. Plomp, Tjeerd 1993 IEA Third International Mathematics and Science Study: Status. Presentation to the Board on International Comparative Studies in Education. January. Raizen, Senta 1992 IEA Third International Mathematics and Science Study: Report of the Subject Matter Advisory Committee. Presentation to the Board on International Comparative Studies in Education. October. Raizen, Senta, Lee Burstein, William Schmidt, Smithson, Rolf Blank, and Gilbert Valverde 1993 Measuring Learning Opportunities and Instructional Practices in Mathematics and Science in Large-Scale Surveys: Examining the State of the Art. Session moderated by Raizen at the American Educational Research Association annual meeting. April. Raizen, Senta, William Schmidt, Ted Britton, Curtis McKnight, Gilbert Valverde, and David Robitaille 1993 International Curriculum Analysis: A New Approach. Session chaired by Raizen at the American Association of Educational Research annual meeting. April. Robitaille, David F. 1992 IEA Third International Mathematics and Science Study: Response to report of management review of TIMSS; Characteristics of national samples and sampling frames; TIMSS objectives and their priorities; Report on March National Project Coordinators meeting; International dissemination plans. Presentations to the Board on International Comparative Studies in Education. February and May. 1993 IEA Third International Mathematics and Science Study: Overview of status (staffing at the international coordinating center; procedures for scheduling work flow; sampling issues; definition of Population 3 for TIMSS; achievement tests; analysis plan). Presentation to the Board on International Comparative Studies in Education. June. Robitaille, David F., General Editor 1993 Curriculum Frameworks for Mathematics and Science. By David F. Robitaille, William H. Schmidt, Senta Raizen, Curtis McKnight, Edward Britton, and Cynthia Nicol. TIMSS Monograph No. 1. Pacific Educational Press: University of British Columbia, Vancouver, British Columbia, Canada. Robitaille, David, and Gordon Ambach 1992 IEA Third International Mathematics and Science Study: Status. Presentation to the Board on International Comparative Studies in Education. October. Robitaille, David, Pierre Foy, William Schmidt, David Wiley, Richard Wolfe, Albert Beaton, and Daniel Horvitz 1993 Design Issues For the Third International Mathematics and Science Study. Session chaired by Robitaille at the American Association of Educational Research annual meeting. April. Schmidt, William H. 1990 Proposal for U.S. National Center for IEA Third Mathematics Study. Presentation to the Board on International Comparative Studies in Education. February.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies 1991 IEA Third International Mathematics and Science Study: Report from the U.S. National Research Coordinator. Presentation to the Board on International Comparative Studies in Education. May. 1992 IEA Third International Mathematics and Science Study: Status of U.S. participation in TIMSS and SMSO; Use of item “signatures” in data analyses; Status of SMSO and U.S. TIMSS activities. Presentations to the Board on International Comparative Studies in Education. May and October. 1992 TIMSS curriculum analysis: Topic trace mapping. Prospects. XXII(3)83. UNESCO. 1993 IEA Third International Mathematics and Science Study: Report of the U.S. National Steering Committee. Presentation to the Board on International Comparative Studies in Education. June. 1993 IEA Third International Mathematics and Science Study: U.S. National Center Update (spring item pilot; fall context questionnaires pilot); SMSO activities. Prepared for the Board on International Comparative Studies in Education. October. 1993 The development of a model for the international study of learning opportunities. Presentation at the Comparative and International Education Society regional meeting. October. 1994 International Mathematics and Science Curricula. Presented on the panel on The Third International Mathematics and Science Study at the American Association for the Advancement of Science meeting. February. 1994 International Mathematics and Science Curricula. Presented on a panel on National and International Tests: How Good is the Science at the American Association for the Advancement of Science meeting. February. 1994 IEA Third International Mathematics and Science Study: Curriculum analysis plans. Status report and presentation to the Board on International Comparative Studies in Education. May. 1994 TIMSS Educational Opportunity Model: Detailed Instrumentation And Indices Development. University of British Columbia, Vancouver, Canada. Schmidt, William, [?] Jorde, and Ted Britton 1993 TIMSS: A Report on the Development of Teacher Questionnaires from Five Countries. A panel presenting Survey of Mathematics and Science Opportunity and opportunity-to-learn work at the National Association for Research in Science Teaching meeting. April. Schmidt, William, Gilbert Valverde, Curtis McKnight, and Ted Britton 1993 New Trends in the Cross National Comparison of Curricula. Panel with papers presented. March. Kingston, Jamaica. Schmidt, William, Senta Raizen, David Wiley, Eugene Owen, and Larry Suter 1991 IEA Third International Mathematics and Science Study: Report of TIMSS National Project Coordinators meeting. September. Selden, Ramsay, Lois Peak, and Eugene Owen 1992 IEA Third International Mathematics and Science Study: Report of National Project Coordinators meeting. Presentation to the Board on International Comparative Studies in Education. October. Travers, Kenneth 1994 IEA Third International Mathematics and Science Study: Status report - Population 3. Presentation to the Board on International Comparative Studies in Education. February. Travers, Kenneth, and Jeanne Griffith 1993 IEA Third International Mathematics and Science Study: Issues of concern to the National Science Foundation and the National Center for Education Statistics. Presentation to the Board on International Comparative Studies in Education.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies U.S. TIMSS National Research Center 1994 International Study of Opportunities to Learn, Including Textbooks and Instructional Practice. Special session at the National Association for Research in Science Teaching. March. 1994 Analysis of Science Curricula from Over Forty Countries: Launching the Third International Mathematics and Science Study. Presentation at the NSTA meeting. March. 1994 Cross-National Measurement of Opportunities to Learn in Math and Science. Presentation at the American Educational Research Association meeting. April. 1994 What in the World Are World Class Standards? Presentation at the American Educational Research Association meeting. April. 1994 International Curriculum Analysis: Results from the TIMSS. Presentation at the American Educational Research Association meeting. April. 1994 Measurement of Opportunity to Learn Elementary and Secondary Science and Mathematics: Recent Experience with Methods, Validity, and Interpretation. Presentation at the American Educational Research Association meeting. April. Valverde, Gilbert 1993 Development and validation of instruments for assessing opportunities to learn. Presentation at the Comparative and International Education Society regional meeting. October. 1994 International Curriculum Analysis: Preliminary Results from the Third International Mathematics and Science Study. Presentation at the Comparative and International Education Society annual conference. March. 1994 United States of America: System of education. International Encyclopedia of Education. Second edition. Wiley, David 1992 IEA Third International Mathematics and Science Study: Report of the Sampling and Methodology Committee. Presentation to the Board on International Comparative Studies in Education. October. Wolfe, Richard 1992 IEA Third International Mathematics and Science Study: Sampling plans. Presentation to the Board on International Comparative Studies in Education. May. ****** NOTE: This study summary was reviewed and edited by William Schmidt at the U.S. National Research Center for TIMSS at Michigan State University on July 10, 1994; Schmidt provided database information on November 29, 1994.

OCR for page 106
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies This page in the original is blank.