International Comparative Studies in Education: Descriptions of Selected Large-Scale Assessments and Case Studies (1995)

Purpose Education in languages is important everywhere in the world, and its importance increases as societies change, assess their expectations for what schools can do, and evaluate their economic positions internationally and human resources nationally. Major transformations in how languages are taught; used in work and daily life; and valued for all aspects of personal, academic, business, and societal opportunity have occurred within the past 20 years.

Education in foreign and second language is important throughout the world. Some global trends that make languages in education especially prominent at this time are:

• Restructuring of international economic and political agreements require common, facilitating languages as well as new priorities in the languages needed by specific nations to realize these relations through workers' skills, for labor mobility, cooperation, and awareness across cultures.

• Most academic, technical, scientific, and financial information is now being coded into a few languages, making knowledge of these languages necessary to realize social, intellectual and economic advances in all parts of the world.

• Increased global migration has created massive demographic shifts between so-called developed countries and from rural to urban areas, making language education for migrant or immigrant populations increasingly essential for social participation and harmony as well as occupational opportunities.

• The foregoing trends are prompting many societies to lose the language resources of indigenous or minority populations, resulting in increased calls for public institutions to accommodate and capitalize on ancestral heritages and multi-linguistic diversity.

Comparative information and research to assess these trends are needed by:

• policy makers who have little current data at their disposal to determine the effectiveness or impacts of existing programs

• business and industry that need reliable indices to determine or forecast current capacities for future needs

• educators who lack a systematic data base on language education to raise awareness of curriculum or instructional choices

• consumers of language education who lack choices about language options and instructional programs in the absence of clear-cut information

• educational researchers who lack empirical data to form a comprehensive international perspective of educationally-relevant theories and recommendations for teaching practices of school policies in language education

The IEA Language Education Study will address these issues systematically in order to develop a comprehensive international information base focused on four purposes:

• To determine the yield of language curricula in different countries. What percentage of students reach a basic, threshold level of proficiency in specific languages? What percentages reach an advanced, fluent proficiency? What differences in yield exist between and within countries?

• To describe the scope and content of language curricula in different countries. What are the key features of foreign and second language curricula (number of languages taught, levels of student participation, starting ages, duration, intensity)? What exemplary possibilities for foreign and second language education exist?

• To identify the factors related to differences in yield in language curricula. To what extent are the difference in yield at the national level, the school level, and the individual level related to differences in curricula for language education? What is the impact of out-of-school opportunities on the learning and uses of foreign and second languages? To what extent do such relations between yield, curriculum factors, and environmental factors vary within and between countries?

• To assess needs and promising options for changes in foreign and second language curricula. Is the yield of language curricula sufficient for societal purposes in specific countries and educational jurisdictions? Which aspects of foreign and second language curricula call for change?

There is no precise, authoritative information on what language education is accomplishing around the world. Few countries know accurately what their own systems for language education are doing, how successful their language programs are in terms of national priorities, how useful they are for which types of students or purposes, or how they may compare to those of neighboring countries. The IEA Language Education Study is an effort to gather this information, which will be useful for educational policy and practice planning, and for socio-economic planning as well.

This comparative survey of language education will:

• Determine what school systems around the world do and produce in language teaching and learning.

• Develop international assessment standards and tests to define basic and advanced levels of communicative competence in key languages.

• Describe the scope and content of school language curricula as well as societal conditions directly related to language learning.

• Identify optimal configurations for school language learning appropriate to national priorities and situations.

• Describe exemplary cases of school language teaching and learning, along with noteworthy innovations internationally.

• Assess needs and recommend promising options for change in participating countries.

This study will provide:

• overview information on national policies for education in foreign, second, and minority languages in about 60 countries

• descriptions of language curricula based on surveys of students, teachers, and schools representative of secondary education in about 30 countries

• descriptions, using internationally-validated tests, of the proficiency for communication that students in about 30 countries achieve in English, French, German, and other commonly

taught languages at (1) the end of the compulsory schooling and (2) upon completing secondary education

• case studies of exemplary practices in language teaching and learning in about 15 countries

• recommendations of needs and promising options for change in school language curricula for each participating country

The study will help participating countries to promote the value of functional communicative competence in key languages, capitalize on resources and the highest levels of research expertise coordinated internationally, as well as learn about and critically analyze their own language education systems and current conditions for language learning. Results will be useful for governments, educators, businesses and industries, and researchers, as well as language students and their families.

Organization and Management The National Foundation for Education Research in England and Wales has been designated as the international coordinating center. This designation was made through a process of sending the proposal out for bids by potential international coordinating centers. The international coordinating center has the responsibility to coordinate the study and for the final results and reports, which includes: research design (including the time line and analysis plan), instrument development, data analysis, and reporting. Additional responsibilities that are generally common to all IEA studies (fund raising, field operations [including sampling procedures, standardized manuals, data collection, training of national research coordinators' field monitoring], and data management) are the responsibility of the international coordinating center or the IEA headquarters, as determined by mutual agreement and approved by the Language in Education Study's International Steering Committee and the IEA Standing Committee.

The IEA Standing Committee will define how the international coordinating center will coordinate with IEA headquarters; negotiations between the ICC and headquarters will determine how and by whom data will be collected and who will have responsibility for other aspects of the study. Either the ICC or the IEA headquarters may contract out activities. Other responsibilities are to be in accordance with IEA by-laws (IEA Guidebook 1993, Section 8, Appendices pages 173-175).

The international coordinator is located at and manages the international coordinating center. He is responsible for the international aspects of the study, is the senior research officer of the study, is responsible to the General Assembly and Standing Committee, serves ex officio on the International Steering Committee, and oversees the work of national research centers. The coordinator has responsibility to maintain communication about the study with all participating countries, the International Steering Committee, and other interested persons; provide documentation on all basic components of the study; manage the international funds of the study according to budgets and other policy decisions of the General Assembly; arrange for and act as rapporteur of meetings of the International Steering Committee and the International Project Committee; arrange meetings of national project coordinators and other meetings associated with the study; ensure agreed deadlines for the study are met by all participants; arrange for conduct of all phases of the study; arrange for incorporation of basic cross-system data into suitable archives for secondary analysis; and arrange for planning and writing of all international project reports and publications.

The international coordinator has responsibility to appoint a project manager to manage all stages of the study and appoint a data manager to organize the international data processing centers. The international coordinating center has responsibility to: prepare all instruments, manuals, codebooks, and documentation and check all constructs at the pilot stage; monitor sampling in cooperation with the sampling referee; arrange for the continuous training of national project coordinators; prepare data entry, cleaning and merging programs to be used by the national research centers; prepare and test all programs required for international statistical analyses; check incoming data sets and documentation;

document all stages of data processing procedures; ensure printouts can be easily read and interpreted; prepare graphics and software and secure appropriate hardware services; and provide each participating center with appropriate documentation and software.

A Steering Committee has been established for this study. Its membership includes Alister Cumming (chair), OISE, Canada; Francis Debyser, CIEP, France; Kees de Glopper, University of Amsterdam, The Netherlands; Elana Shohamy, Tel Aviv University, Israel; John Trim, Council of Europe; G. and Richard Tucker, Carnegie Mellon University, U.S.A.

Participants Twenty-five IEA member countries have expressed interest in participating in the study. Some are interested in participating in regard to foreign languages widely used for international communication. Further interest exists for optional studies of education in second languages (education in a majority language for ethnic, immigrant, or regional minorities on a specific country). This interest focuses on education for social interaction and academic studies and will address diverse populations as well as complex factors in local societal contexts.

The United States (as a whole nation or selected states) might choose to participate in the whole study or some of four parts of the study (national language education profiles; survey of student language proficiency for two populations; survey of language teaching and learning; case studies of exemplary, successful schools). This decision would depend on the nature and focus of the study (for example, which language/s is/are included) and on available funding. Most countries want to focus on English; the United States would be interested in Spanish as a foreign language, and in English as a second language.

Sample Simple, but sophisticated sampling techniques will be used, to keep numbers of students and schools involved to a minimum while achieving representative sample populations. At the first stage of sampling, schools will be sampled with probability proportional to their size, at both population levels. At the second stage of sampling students will be sampled randomly within schools. There will be no sampling of intact classrooms; consequently school effects and classroom effects cannot be separated. The study will aim for an effective sample of 400 students for those proficiency measures that will be collected from all students (i.e., reading and listening comprehension). The precision of estimates of population values will thus be + or − 5 percent of a standard deviation for any given variable.

For other skill domains (typically writing and speaking) students will be sub-sampled (typically 5 students per school per skill); the effective N aimed for will be 200 students. Between 20 and 30 students will be tested per school. Stratification will be employed in the preparation of the sample designs, in order to enhance sampling accuracy. Students will be tested in one language only. Countries testing multiple languages may opt for separate samples of schools, one school sample for each language, or for sampling separate student groups within a common sample of schools.

Procedures and Summary of Content The study will assess language proficiency as the capacity to use a particular language appropriately and accurately to achieve social purposes in particular contexts (both inside and outside of school). It will address various competencies and domains of language use, rather than limiting itself to traditional testing of students' knowledge of grammar and vocabulary. Assessment will consist of focused and integrated tasks (tasks that require students to

communicate interactively, such as reading and writing in conjunction, or speaking and listening in ongoing conversations).

School policies for foreign language education can be readily identified and compared internationally, and they form the principal focus for the of this study. School policies for second language education are considerably more complex, interact with local social contexts, and vary in curriculum realizations, which makes them suitable as optional components of the study according to interests and situations of specific countries.

Previous research points the way to many learning and curriculum variables, design issues, test characteristics, and sampling requirements that this study can adopt. For example, empirical studies of students ' language performance have generally found variation according to such factors as situations of language use or communication tasks; previous education or literacy; age factors; time spent on language learning; structures or features of particular first and second languages; attitudes toward the society or culture of the language being learned; and the status and use of the languages in the local community.

Numerous curriculum and instructional variables have also been analyzed, though few have received rigorous, large-scale empirical evaluation: instructional approaches or didactic procedures; curriculum content and organization; and effects of specific media, technologies, or resources for language learning.

The core study and options design is as follows:

National profiles inventory: This first phase of the study will provide at-a-glance overviews of the general context and policies for language education in each country. Findings will identify key policy issues and factors in school contexts broadly influencing curricula and students' learning in specific

languages. A survey report form will be conceived, designed, pilot-tested, and then distributed to all participating nations, as well as other nations that (because of limited resources or interest) will engage in this part of the study only. Data collection (scheduled for January 1995) will be reported initially (scheduled for November 1995) in the form of an inventory booklet; then a more extensive compendium reference document will be published the following year.

Language proficiency assessment: A major portion of work during the study will be devoted to developing, conducting, and analyzing results from assessments of student language proficiency -- all coordinated centrally through the international coordinating center. The study will produce and validate a common set of tests to assess the proficiency that students achieve to communicate in speaking, listening, reading, and writing in key languages, as selected by each country. Results will indicate the yield of school systems overall in terms of the percentage of students achieving either (1) threshold or (2) fully proficient levels of communicative competence at (1) the end point of compulsory schooling and (2) the completion of secondary schooling.

Descriptions of language curricula: Work on questionnaires for school administrators, teachers, and students will follow a schedule similar to that for language proficiency testing. Questionnaires will survey representative samples of students, teachers, and schools to describe their characteristics and a wide range of resources and practices related to language curricula and learning -- assessing the fit between resources available and those most frequently used in practice in each country. After developing and pilot-testing instruments in 1996 and data collection in 1997, results will be reported in booklet form (scheduled for early 1998), followed by national reports and a full international report (later that year), incorporating results of language assessments as well.

Case studies: To assure that this aspect of the study is sensitive to the interests of participating countries, initial consultations will review options for case study inquiry (classroom observations, ethnographic interviews, narrative techniques, teacher action research), determine common analytic purposes and frameworks, and link the design to issues highlighted in the national profiles inventory for each participating country. Case studies will provide vivid, holistic portraits of exemplary practices in language teaching and learning. Illuminating in depth what actually happens in innovative, successful classrooms, this aspect of the study will provide educators with concrete, practice-based models of the experiences and conditions of language education. Building on initial results from surveys and proficiency assessments, case studies will be conducted in certain countries (scheduled for 1997), which will produce a booklet, then a detailed compendium of results (1988)

Language proficiency yield, optimal configurations of language curricula, needs and options for change: In the final year of the study, data analyzed from the national profiles inventory, language proficiency assessments, descriptions of language curricula, and case studies will be compiled and evaluated centrally to produce tables and reports on language proficiency yield, curriculum descriptions, and optimal configurations of language curricula. Project centers will prepare national reports as well as reports on needs and promising options, ensuring they are relevant to local situations. Analyses, preparation of major books, writing of articles, and public presentations will proceed intensively through this final year but continue, according to the interests of participating researchers, for several years following the completion of the project.

Information from the surveys, testing, and case studies will be synthesized to identify -- in respect to the goals and societal contexts of each participating country -- optimal configurations for school language education, specific needs in each country, and promising options for change. National reports are scheduled for publication in late 1988. An overview booklet will identify (1) key variables amenable to manipulation within educational systems to improve language achievement as

well as (2) major differences in societal contexts between countries that affect language acquisition and use.

Publications Case studies will be reported first in a booklet, and then in a detailed report of the results. The survey of students, teachers, and schools will be reported first in a booklet, and then in national reports and a full international report, which will include language assessments. National reports of a synthesis of information from the survey, testing, and case studies will be published.

Funding IEA headquarters is seeking funding from foundations, international organizations, and countries other than the United States. The United States will not be a principal or major funder of

the coordinating center or international activity. The National Center for Education Statistics may provide a small amount for the U.S. national center.

Peter Dickson, International Coordinator

National Foundation for Educational Research in England and Wales

The Mere, Upton Park

Slough, Berkshire SL1 2DQ ENGLAND

telephone: 44-7-53-74-12-3

facsimile: 44-7-53-69-16-32

Internet: EZUE100@VMSFE.ULCC.AC.UK

International Association for the Evaluation of Educational Achievement Secretariat 14 Sweelinckplein

NL 2519 The Hague, THE NETHERLANDS

telephone: 31-70-346-96-79

facsimile: 31-70-360-99-51

e-mail: IEA@SVO.NL

Gordon M. Ambach, U.S. Liaison to IEA

Council of Chief State School Officers

One Massachusetts Ave., N.W., Suite 700

Washington, D.C. 20001-1431

telephone: 202/ 408-5505

facsimile: 202/ 408-8076

e-mail: gambach@nas.edu

International Association for the Evaluation of Educational Achievement

1993 Activities, Institutions, and People: IEA Guidebook 1993-94. IEA, The Hague, The Netherlands.

1993 The IEA Language Education Study Proposal. July. For discussion, revision, and approval. 34th General Assembly, El Escorial, Spain, September 1993. IEA, The Hague, The Netherlands.

Ambach, Gordon M.

1993 IEA Language Education Study. Status reports presented to the Board on International Comparative Studies in Education. January 28, June 19, November 7. Council of Chief State School Officers, Washington, D.C.

1994 IEA Language Education Study. Status reports presented to the Board on International Comparative Studies in Education. February 6, May 16. Council of Chief State School Officers, Washington, D.C.

Selden, Ramsay

1993 Activities on Foreign/Second Language Study. Status report prepared for the Board on International Comparative Studies in Education, June 2. Council of Chief State School Officers, Washington, D.C.

******

NOTE: This study summary was reviewed and edited by Peter Dickson at the National Foundation for Educational Research in England and Wales (NFER), in Berkshire, England on July 11, 1994.