READING LITERACY STUDY

International Organization

International Association for the Evaluation of Educational Achievement

Years of Data Collection

1990 - 1991

Purpose Because policy issues are of paramount importance for many of the educational leaders who support the IEA international studies, the Reading Literacy Study was designed to provide results that could serve as a basis for decisionmaking concerning the improvement of reading in primary and secondary schools. The study coincided with the UNESCO World Literacy Year and set 1990 as the year in which the first large-scale baseline data were established against which future achievements in literacy would be measured. This international study was designed to develop instruments for measuring literacy in each of the 32 participating country school systems. It was expected to establish a clear unified definition of literacy and to measure the comparative ability of educational systems to teach literacy skills.

For this study, reading literacy refers to the linguistic and cognitive processing of written language forms required by society and/or valued by the individual. As this implies, literacy is regarded as both a cultural practice and an intellectual achievement. For purposes of the study, literacy is defined as the “ability to understand and use those written language forms required by society and/or valued by the individual.” This definition has been operationalized in terms of reading test items along three scales (narrative, expository, and document) that correspond to text types.

The purposes of the IEA Reading Literacy Study are to:

  • Describe the achievement levels in reading literacy of comparative samples of students in education systems

  • Describe the voluntary reading activities of 9 and 14 year-old-students

  • Identify differences in policies and instructional practices in reading, and study the ways in which they relate to students' achievement and voluntary reading

  • Produce valid international tests and questionnaires that could be used to investigate reading literacy development in other countries

  • Provide national baseline data suitable for monitoring changes in literacy levels and patterns over time.

The products of the international study are:

  • valid measures suitable for measuring literacy across countries

  • estimates of the percentage of each population reaching specified levels of literacy

  • comparative data across countries of literacy achievement in three domains (narrative passages, expository passages, documents) on an international scale

  • identification of which school, teacher, and societal factors influence literacy and to what extent

  • comparative data across countries on school and teacher practices in the teaching of reading

  • establishment of a 1990 baseline database in 30 countries

These products provide each education system with data on literacy levels that can be related to economic, health, and other educational indicators. (The study identified variables that, if changed,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies READING LITERACY STUDY International Organization International Association for the Evaluation of Educational Achievement Years of Data Collection 1990 - 1991 Purpose Because policy issues are of paramount importance for many of the educational leaders who support the IEA international studies, the Reading Literacy Study was designed to provide results that could serve as a basis for decisionmaking concerning the improvement of reading in primary and secondary schools. The study coincided with the UNESCO World Literacy Year and set 1990 as the year in which the first large-scale baseline data were established against which future achievements in literacy would be measured. This international study was designed to develop instruments for measuring literacy in each of the 32 participating country school systems. It was expected to establish a clear unified definition of literacy and to measure the comparative ability of educational systems to teach literacy skills. For this study, reading literacy refers to the linguistic and cognitive processing of written language forms required by society and/or valued by the individual. As this implies, literacy is regarded as both a cultural practice and an intellectual achievement. For purposes of the study, literacy is defined as the “ability to understand and use those written language forms required by society and/or valued by the individual.” This definition has been operationalized in terms of reading test items along three scales (narrative, expository, and document) that correspond to text types. The purposes of the IEA Reading Literacy Study are to: Describe the achievement levels in reading literacy of comparative samples of students in education systems Describe the voluntary reading activities of 9 and 14 year-old-students Identify differences in policies and instructional practices in reading, and study the ways in which they relate to students' achievement and voluntary reading Produce valid international tests and questionnaires that could be used to investigate reading literacy development in other countries Provide national baseline data suitable for monitoring changes in literacy levels and patterns over time. The products of the international study are: valid measures suitable for measuring literacy across countries estimates of the percentage of each population reaching specified levels of literacy comparative data across countries of literacy achievement in three domains (narrative passages, expository passages, documents) on an international scale identification of which school, teacher, and societal factors influence literacy and to what extent comparative data across countries on school and teacher practices in the teaching of reading establishment of a 1990 baseline database in 30 countries These products provide each education system with data on literacy levels that can be related to economic, health, and other educational indicators. (The study identified variables that, if changed,

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies could help to improve literacy rates.) On an international level, the large 1990 database will be useful in the future for assessing changes in literacy achievement over time. Because the United States has large national and state data collection efforts underway, the U.S. national steering committee for the IEA Reading Literacy Study considers the primary purpose of participating in this study to be the gathering of comparative international data and information that will not only compare student competence but provide insights to help explain differences in performance by U.S. students and other students. The IEA Reading Literacy Study provides an opportunity to investigate how reading is taught in a large number of education systems with wide variations in traditions, economic development, school organization, classroom conditions, teacher characteristics, and orthography, and to contribute to a more context-sensitive perspective on successful means of teaching reading and successful reading acquisition. Because there is much unity in the culture of reading instruction around the world, texts have many universal features in their functions, contents, structures, and textual characteristics. This unity makes it possible to translate texts. The Reading Literacy Study assumes that there is a common basis that makes international comparison possible. Organization and Management IEA selected an international coordinator, established an international coordinating center and data processing center at the University of Hamburg in Germany, and appointed a steering committee and technical advisor. The members of the steering committee, the international coordinator, and the data manager worked together with the national research coordinators who were responsible for the conduct of the project in their respective countries. The national center in each participating country appointed a national research coordinator to assume overall responsibility for the project within that country and a national committee composed primarily of reading literacy experts. National centers that wished to attach additional research to the project appointed appropriate additional members to their national committees. National centers appointed staff as required. All conceptual and operational decisions were made cooperatively by the steering committee and national research coordinators. The international sampling referee approved and checked sampling plans. A data processing team at the University of Hamburg created the computer-stored working files. The U.S. national center and national research coordinator are both located at the National Center for Education Statistics. As subcontractors, Westat, Data Recognition Corporation, and the Council of Chief State School Officers have worked with the NCES to conduct the testing, analysis, and reporting for the United States. The U.S. national center (NCES) has provided technical assistance to other participating country national centers, through two seminars. In the seminars national research coordinators, international coordinating center staff, NCES representatives, and Westat staff worked together to generate appropriate descriptive tables reflecting the core concerns of the international Reading Literacy Study. At the conclusion of each seminar, participants had the basis for a national report in the form of a set of tabulations around which such a report could be built.

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Design Participants Research institutes from thirty-two systems of education participated in the study. Twenty-seven countries participated in testing populations A and B: Belgium (French), Canada (British Columbia), Cyprus, Denmark, Finland, France, Germany (East), Germany (West), Greece, Hong Kong, Hungary, Iceland, Ireland, Italy, The Netherlands, New Zealand, Norway, Portugal, Singapore, Slovenia, Spain, Sweden, Switzerland, Thailand, Trinidad |andsymbol| Tobago, the United States, and Venezuela. One country participated only in Population A: Indonesia; four countries participated only in Population B: Botswana, Nigeria, the Philippines, and Zimbabwe. Population A was defined as: all students attending mainstream schools on a full-time basis at the grade level in which most students aged 9:00-9:11 years were enrolled during the first week of the eighth month of the school year. Population B was defined as: all students attending mainstream schools on a full-time basis at the grade level in which most students aged 14:00-14:11 years were enrolled during the first week of the eighth months of the school year. Sample The formal survey was conducted on scientifically selected national samples of 9-and 14-year-olds, typically 1,500 to 3,000 pupils per country and their teachers. To obtain comparable samples of students, multi-stage sampling was used in each country and schools or classes were typically drawn with a probability proportional to the size of the school or class. Where schools were drawn, an intact class was selected at random within each school, but in Population B some national research coordinators selected students at random from all classes in the grade level in the school. To overcome fluctuations in the execution of the sampling, weighting was used to adjust for any variations in the probability of selecting students. These sampling weights were used in all data analyses. The U.S. sample included 332 participating schools. The target population is similar to that of the National Assessment of Educational Progress in that it uses the most inclusive population possible. However, unlike NAEP, IEA Reading Literacy targeted all students in the modal grades for 9- and 14-year-olds. This differs from the NAEP target, because IEA Reading Literacy tested intact classes and 14- rather than 13-year-olds. Procedures and Summary of Content Reading Test: The Reading Literacy Study reading test included passages and items representing narrative text, expository text, and documents. The tasks required an ability to comprehend specific vocabulary and continuous text and to locate and use document information. Background Questionnaires: The student questionnaire was designed to obtain information on the students' home and school circumstances, and included questions concerning parents' education and interaction with the student and his schooling, the students' reading and leisure time activities, and personal descriptive data. The teacher questionnaire was designed to obtain information on each teacher's background, instructional practices, and beliefs, and included questions concerning training and experience, personal descriptive data, classroom materials and environment, and teaching strategies and activities. The school questionnaire was designed to obtain information about the school circumstances and policies as viewed by the principal, and included questions concerning the principals' training and experience, personal descriptive data, and school descriptive data.

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies National Case Study Questionnaire: The national case study questionnaires were designed to obtain information on national policy, enrollment patterns, and economic conditions and included questions on curriculum, age of entry to school, teachers' salaries, hours of instruction, percentage of an age group enrolled in formal schooling, and teacher training emphasis. National research coordinators completed these questionnaires. Data Collection and Analyses All students took reading tests and responded to a background questionnaire about their reading at home and at school. Teachers and school principals responded to questionnaires about themselves, their teaching, and the school organization. Each national center completed a national case study questionnaire. Constructs were identified as the basis of the international data analysis. The major analysis was performed for Population A. In December 1991 the U.S. national steering committee presented a symposium at the National Reading Conference meeting in Palm Springs, California. This was the first presentation of the U.S. data with regard to reading test design, family characteristics, differences in scaling, teacher characteristics, and principals as instructional leaders. Four reports of preliminary international results were published in 1992-1993; the main international research report was published in 1994. The IEA Reading Literacy Study international database was established in 1993. A full study archive for the study was established in 1994. The archive includes the conceptual framework; test items as submitted and revised; test item selection criteria, national test instruments and case study questionnaires; national sample designs; final versions of reading tests and questionnaires, manuals, codebooks, and datasets; and national and international reports and publications. Three U.S. national reports will be published in 1994. The first is more technical in nature and covers the procedures used in gathering and processing the data for the U.S. portion of the international study; it also explores issues of validity, comparisons to the National Assessment of Educational Progress, the relationship of the Reading Literacy Study to the research literature, and modeling techniques used in developing the U.S. analyses. The second U.S. national report is intended for a more broad general audience. The third consists of papers that deal with issues in comparative studies. A fourth report will focus on the United States in international perspective and include special analyses of modeling, bilingual instructional programs, quality of school life, and instructional practices. Timetable 1986 The IEA General Assembly approved the Reading Literacy Study and formed a steering committee and technical advisory group. 1988 The international coordinating center was established and the international coordinator was selected, both located at the University of Hamburg. Participating country research institutes appointed national research coordinators. National research coordinators met. (November)

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies 1989 The international steering committee met in San Francisco to review and revise test items. (April) National research coordinators met in Washington to agree on field test items and procedures and to approve sampling plans. (June) The international steering committee met with national research coordinators. (November) The U.S. national advisory panel was formed; it met in December. 1988-1990 Items for the tests and questionnaires were generated, translated, and pilot tested. (November 1988-July 1990) 1990 The U.S. national steering committee met in Boston. The international steering committee met in Hamburg. (May) The international steering committee and national research coordinators met in Frascati, Italy to reach agreement on final instruments and study design. Final tests were selected. (July) A baseline database was established in 30 countries. 1990-1991 The formal survey was conducted. (October 1990-April 1991, depending on the school year in each country) 1991 The international steering committee held a technical advisory committee meeting. (April) The U.S. national steering committee met to review the U.S. national data and plan for analysis and reporting. (June) The international steering committee held a national research coordinators meeting in Denmark. (October) The U.S. national steering committee presented a symposium at the National Reading Conference in Palm Springs, California. (December) 1992 National research coordinators met in Madrid. (October) 1992-1994 Preliminary results were published in four books. 1993 The international coordinating center completed its work and transferred data tapes and other materials to IEA headquarters. An international database was established. 1994 A full study archive was established. The main research report was published. Four U.S. national reports will be published. Publications How in the World Do Students Read? Preliminary results. IEA, The Hague, The Netherlands. (1992) Effective Schools in Reading: Implications for Educational Planners (An Exploratory Study). Preliminary results. IEA, The Hague, The Netherlands. (1992)

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Teaching Reading Around the World. Preliminary results. IEA, The Hague, The Netherlands. (1993) Gender Differences in Reading. Preliminary results. IEA, The Hague, The Netherlands. (1994) International Report: The IEA Study of Literature: Achievement and Instruction in Thirty-two School Systems. Oxford: Pergamon Press. (1993) The IEA Reading Literacy Study: Technical Report. IEA, The Hague, The Netherlands. (1993) Main research report. Oxford: Pergamon Press. (1994) U.S. national reports will be published in 1994. Database The IEA Reading Literacy Study international database was established in 1993. Funding International funding was provided by The MacArthur Foundation, The Maxwell Family Foundation, The Melon Foundation, the National Center for Education Statistics (through the National Academy of Sciences), Commission of the European Communities, UNESCO, and annual contributions from participating countries. Participating country research institutions also provided funds for their national costs of conducting the study and for travel and subsistence costs for their national research coordinators to attend national research coordinator meetings. Information Sources Marilyn Binkley, National Research Coordinator National Center for Education Statistics 555 New Jersey Ave., N.W. Washington, D.C. 20208 telephone: 202/ 219-2195 facsimile: 202/ 219-1736 e-mail: bmt@cu.nih.gov International Association for the Evaluation of Educational Achievement Secretariat c/o SVO Sweelinckplein 14 2517 GK The Hague THE NETHERLANDS telephone: 31-70-346-96-79 facsimile: 31-70-360-99-51 e-mail: iea@svo.nl Gordon M. Ambach, U.S. Liaison to IEA Council of Chief State School Officers One Massachusetts Ave., N.W., Suite 700 Washington, D.C. 20001-1431 telephone: 202/ 408 = 5505 facsimile: 202/ 408-8076 e-mail: gambach@nas.edu

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies Binkley, Marilyn R. 1989 IEA Reading Literacy Study. Status reports prepared for the Board on International Comparative Studies in Education. January and August. Presentations to the board. April and September. 1990 IEA Reading Literacy Study. Status reports prepared for the Board on International Comparative Studies in Education. April and September. 1991 IEA Reading Literacy Study. Status reports prepared for the Board on International Comparative Studies in Education. January, April, and September. 1991 IEA Reading Literacy Study: dissemination plan. Presentation to the Board on International Comparative Studies in Education. May. 1992 IEA Reading Literacy Study. Status reports prepared for the Board on International Comparative Studies in Education. January and April. 1993 IEA Reading Literacy Study. Status reports prepared for the Board on International Comparative Studies in Education. June and October. Binkley, Marilyn R., and Gary Phillips 1988 IEA Reading Literacy Study: plans for the U.S. national study. Presentation to the Board on International Comparative Studies in Education. November. 1989 IEA Reading Literacy Study. Status report prepared for the Board on International Comparative Studies in Education. April. Binkley, Marilyn R., and Trevor Williams 1992 IEA Reading Literacy Study: status report. Presentation to the Board on International Comparative Studies in Education. October. Elley, Warwick B. 1988 IEA Reading Literacy Study: International aspects of the study. Presentation to the Board on International Comparative Studies in Education. November. 1992 How in the world do students read? The International Association for the Evaluation of Educational Achievement, The Hague, The Netherlands. 1994 [title?] Main research report of the IEA Reading Literacy Study. Oxford: Pergamon Press. Elley, Warwick N., Editor 1993 International Report: The IEA Study of Literature: Achievement and Instruction in Thirty-two School Systems. Oxford: Pergamon Press. International Association for the Evaluation of Educational Achievement 1993 Activities, Institutions, and People: IEA Guidebook 1993-1994. IEA, The Hague, The Netherlands. 1993 IEA Reading Literacy Study: international database. IEA, The Hague, The Netherlands. 1993 The IEA Reading Literacy Study: Technical Report. IEA, The Hague, The Netherlands. 1994 IEA Reading Literacy Study: full study archive. IEA, The Hague, The Netherlands. Lundberg, Ingvar, and Pirjo Linnakylä 1993 Teaching Reading Around the World: IEA Reading Literacy Study. January. The International Association for the Evaluation of Educational Achievement, The Hague, The Netherlands. Postlethwaite, T. Neville, and Kenneth N. Ross 1992 Effective Schools in Reading: Implications for Educational Planners: An Exploratory Study. November. The International Association for the Evaluation of Educational Achievement, The Hague, The Netherlands. Wagemaker, H., Georgia Polydorides, and Michael Martin, Editors 1994 Gender Differences in Reading. IEA, The Hague, The Netherlands.

OCR for page 89
International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies ****** NOTE: This study summary was reviewed and edited by Marilyn Binkley at the National Center for Education Statistics in Washington, D.C. on June 22, 1994.