Cover Image

PAPERBACK
$34.75



View/Hide Left Panel

1
Introduction and Rationale

INTRODUCTION

The increasing scrutiny of earlier studies has revealed their limitations and the consequent need for improvement in the planning, execution, and dissemination of international comparative research…. The lack of an adequate system of education indicators to inform education policy making has become increasingly apparent. Data are not collected regularly, systematically, or with enough coordination either to satisfy natural curiosity about education systems around the world or to answer the questions of researchers and policy makers about changes over time in education in a variety of countries. Trend data are needed on many aspects of education.

A Collaborative Agenda for Improving International Comparative Studies in Education (National Research Council, 1993, hereafter the 1993 Agenda)

By the last half of the 1990s, many concerns described in this excerpt from the 1993 Agenda of the Board on International Comparative Studies in Education had been or were well on the way to being resolved. The proposed solutions, however, produced several new, somewhat overlapping problems. Previously, there was a scarcity of data sufficiently robust to support valid cross-national comparisons; today, a glut of good-quality data overwhelms the field and remains largely unanalyzed, even as new follow-on surveys are launched. Previously, large-scale cross-national education surveys were initiated sporadically, every few years; however, between 1999 and 2003, data collection for at least one and as many as three surveys was scheduled annually.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education 1 Introduction and Rationale INTRODUCTION The increasing scrutiny of earlier studies has revealed their limitations and the consequent need for improvement in the planning, execution, and dissemination of international comparative research…. The lack of an adequate system of education indicators to inform education policy making has become increasingly apparent. Data are not collected regularly, systematically, or with enough coordination either to satisfy natural curiosity about education systems around the world or to answer the questions of researchers and policy makers about changes over time in education in a variety of countries. Trend data are needed on many aspects of education. A Collaborative Agenda for Improving International Comparative Studies in Education (National Research Council, 1993, hereafter the 1993 Agenda) By the last half of the 1990s, many concerns described in this excerpt from the 1993 Agenda of the Board on International Comparative Studies in Education had been or were well on the way to being resolved. The proposed solutions, however, produced several new, somewhat overlapping problems. Previously, there was a scarcity of data sufficiently robust to support valid cross-national comparisons; today, a glut of good-quality data overwhelms the field and remains largely unanalyzed, even as new follow-on surveys are launched. Previously, large-scale cross-national education surveys were initiated sporadically, every few years; however, between 1999 and 2003, data collection for at least one and as many as three surveys was scheduled annually.

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education Previously, U.S. schools faced few mandated tests, and most were willing to participate in the occasional voluntary, internationally oriented tests; today, with increased requirements for mandatory testing, increasing numbers of schools are unwilling to add to their testing burden by participating in voluntary assessments. The infrastructure for conducting large-scale international studies that has developed over the past decade, which plays an important role in ensuring the quality of large-scale international education surveys, has become institutionalized, and the desire to keep this infrastructure engaged has played a role in decisions to support new and more frequent studies. Indeed, there is an increasing concern that international assessments are now conducted more frequently than reforms can produce change in the U.S. education system, which may discourage ongoing, longer term reform efforts. In addition, the results of large-scale domestic and international surveys are raising a host of questions that often are addressed best by smaller scale studies requiring a wide range of research methods, both qualitative and quantitative. For example, although a full series of more detailed thematic analysis of the data was commissioned before the completion of the first Programme for International Assessment of Student Achievement (PISA)1 international report, it was of necessity carried out by researchers closely aligned with the study. Few new initiatives have been launched either to cull insight from ongoing nonsurvey-based international studies or to support systematic new ones attuned to independent research agenda. Despite major investments in a half-dozen large-scale international surveys over the past decade, U.S. public discourse about education remains curiously untouched by international comparisons. Beyond the common knowledge that U.S. students are not first in the world in mathematics and science, educational rhetoric in the United States remains essentially one-dimensional, lacking the sense of rich possibilities that international perspectives can provide. Possible reasons for this deficiency include the general imperviousness of U.S. education policy to domestic or international education research (Lagemann, 2000), and widely shared assumptions that other areas of the world are simply not relevant to the United States. The lack both of interpretive international comparative education studies and of secondary analysis focused on issues of primary concern to the public and policy makers, however, certainly contributes 1   PISA is being conducted by the Organisation for Economic Co-operation and Development’s Directorate for Education. It is a triennial survey of the knowledge and skills of 15-year-olds in the principal industrialized countries. It assesses how far students have acquired some of the knowledge and skills that the study considers essential for full participation in society. http://www.oecd.org.

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education to the persistence of an inwardly focused approach to education studies in the United States. The Education Sciences Reform Act of 2002 places strong emphasis on using rigorous scientific methods to study education (U.S. Congress, 2002). This act has reorganized the U.S. Department of Education’s Office of Educational Research and Improvement by creating the Institute of Education Sciences, which includes three centers: the National Center for Education Research, the National Center for Education Evaluation and Regional Assistance, and the National Center for Education Statistics (NCES), one of the main U.S. funders of international comparative education research. This is an important moment to examine the nature of international comparative education research and to reaffirm its critical contribution to a well-rounded program of domestic education research in the United States. The purpose of this report—which is directed to federal sponsors of international comparative education research, domestic and international researchers, private foundations, and state and district officials who are eager to improve their part of the U.S. education system—is to lay out the rationale for such research; describe its scope, purpose, and potential impact; and make recommendations regarding future directions. Fundamentally, international comparative studies contribute to basic education research by documenting the existence of a much broader array of educational practices and outcomes than is available in the United States alone. International studies, however, can do much more than this. The rest of this chapter explores the current rationale for U.S. participation in international comparative studies and discusses the scope of such studies. Chapter 2 outlines the range of international comparative studies and their relative costs and presents recommendations for moving toward a more balanced research agenda for these studies. Chapter 3 draws on some recent studies to illustrate different ways that international comparative studies have—or, in some cases, have not—made an impact on the U.S. education system. Chapter 4 begins by offering suggestions for continuing to improve one type of study—large-scale, cross-national surveys— with which the board has been mainly involved since its inception, and to address key issues that persist or have emerged with those types of studies since the board’s 1990 report, A Framework and Principles for International Comparative Studies in Education (National Research Council, 1990, hereafter referred to as the 1990 Framework). It continues by addressing the pressing need for more public access to the findings of all types of international comparative studies and the consequent need for an array of studies addressing a wide range of questions that call for many different research methodologies. Chapter 5 examines the implications of recommendations from earlier chapters for supporting infrastructure, both fi-

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education nancial and organizational, for future international comparative studies of education. The final chapter provides a summary of the board’s recommendations. RATIONALE Although many features of international data collection in educational research have changed over the past decade, at least one has not: research that provides comparative information across nations continues to expand understanding of education as a social and economic institution and provide rich sources of ideas about how nations can strengthen teaching and student achievement. Throughout its history, the U.S. education system has benefited immensely from ideas borrowed and adapted from education systems in other countries. These ideas range from methods for early childhood education (France, Germany, and Italy), a model for the structure of higher education (Germany), and goals for mass urban education (England), to the Suzuki method of teaching music (Japan). Holmes (1985) traces the earliest efforts to observe and learn from foreign education systems to Plato’s reference to Sparta in The Republic. He dates the beginning of comparative education as a systematic study to the early 19th century. He mentions early reservations about the limitations of what is likely to be learned from such study. He cited one educator who claimed that “the practical value of studying other systems of education is that much can be learned about one’s own system of education.” His second claim was that “what goes on outside the schools matters even more than the things inside schools to an understanding of any system of education” (p. 866). U.S. interest in international education studies has waxed and waned over the decades, but it grew particularly keen after the National Commission on Excellence in Education issued its report, A Nation at Risk (National Commission on Excellence in Education, 1983). That report made extensive use of findings from then-current national and international comparative studies of student achievement, portraying them in provocative terms.2 The data cited in that report seized the interest of policy makers, who had little previous knowledge of or interest in comparative international education statistics but who subsequently evolved into strong proponents of comparative research at both state and cross-national lev- 2   For example, “International comparisons of student achievement . . . reveal that on 19 academic tests American students were never first or second, and in comparison with other industrialized nations, were last seven times” (p.8).

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education els.3 Some scholars, however, questioned the use of these particular international studies to judge the U.S. education system, given their imperfect sampling and other technical problems at that time.4 Nonetheless, by 1990, the president and the governors acknowledged the importance of international perspectives in formulating domestic education policy when they defined national education goals for the nation. The United States was challenged to be first in the world in mathematics and science achievement by the turn of the century, and to ensure that every adult “will be literate and will possess the knowledge and skills necessary to compete in a global economy” (Rothman, 2002). This focus on comparisons of achievement brought valuable attention to the potential benefits of learning about education in other countries. However, the country rankings that were so widely publicized did little to suggest the breadth of international research. Cuban (1988) has argued that one remarkable feature of U.S. schools is how alike they are. In contrast, education systems in many other countries encompass a far greater degree of diversity. For example, there tends to be great diversity across nations regarding what citizens expect of their schools, what roles teachers play in society, and what education services governments and private organizations provide. International comparative research in education can help to expand the repertoire of possible practices and policies in several ways. International education studies help to define what is achievable. How much can students learn and at what age can they learn it? How do different countries mix different amounts of pre-service and in-service professional development for beginning teachers at the early childhood, primary, secondary, and tertiary levels? How do different countries determine the optimal number of hours of schooling at each of these levels? What roles do parents with different levels of education play in governing and supporting schools? Most people would be reluctant to conduct controlled experiments with their children’s educations, but naturally oc- 3   For example, a July 1994 NCES strategy document noted, “Education policy makers and analysts now routinely request information about how American schooling compares to that found in other countries . . . . The effort to provide a quality education to all of America’s students has increasingly used international comparisons to assess our school’s effectiveness and to generate ideas about ways to reform our schools” (U.S. Department of Education, National Center for Education Statistics, 1994:ii,1). 4   For example, U.S. Department of Education, National Center for Education Statistics (1995:2) lists several follow-up activities addressing “the dubious quality of the data” in the studies cited in A Nation at Risk (National Commission on Excellence in Education, 1983).

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education curring variation in other countries can help develop more confidence in—or courage to consider changing—U.S. policies and practices. Studies such as PISA, for example, demonstrate that high average performance does not have to be associated with the wide disparities in performance found in the United States. International comparative research can help researchers and policy makers to observe and characterize consequences of different practices and policies for different groups, under different circumstances. Research can examine correlates of various approaches (Holmes, 1985; Postlethwaite, 1999) and explore the reasons for observed differences in student performance, thus enhancing confidence in the generalizability of studies (1990 Framework). It can also contribute to and possibly influence the content and direction of useful debate concerning public issues, such as teenage employment, and the terms of service of teaching, by enhancing the discourse through increasing knowledge about a wider range of alternatives and possible consequences. International comparative studies often bring to light concepts for understanding education that have been overlooked in the United States, helping U.S. educators to think in terms of new principles and categories. The Second International Mathematics Study helped to popularize the concept of the intended, implemented, and achieved curriculum and facilitated more nuanced discussions and studies about relationships between curriculum and student achievement. A recent book highlighting the expert, “profound” understanding characteristic of Chinese elementary mathematics teachers (Ma, 1999) casts new light on layers of understanding within subject matter knowledge. PISA’s efforts to measure “preparedness for life” have led to new ways to operationalize different types of literacy. International comparisons of education often lead us to identify and question beliefs and assumptions that are taken for granted. This contribution is sometimes characterized as making the familiar strange and the strange familiar (Kluckhohn, 1944). International comparisons help to raise questions about the universality of particular features of the U.S. education system and offer new insight into current disputes. For example, Japanese teachers can offer cogent reasons why classes of fewer than 20 students are more difficult to teach than larger classes, and why, at the preschool level, teachers often should not discipline a misbehaving student. Large-scale cross-national surveys have received much attention in the United States in the two decades since the release of A Nation at Risk. Many of the benefits of international comparative education studies, however, are achieved by relatively small-scale, low-cost, more open-ended studies. Such studies, in addition to contributing to our understanding of

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education the broader range of possibilities in education, are essential precursors to large-scale studies because they help to identify contextual features of school systems that are common to many countries and can be quantitatively measured. Similarly, questions raised by counterintuitive findings of large-scale studies are often best explored by smaller scale, targeted studies. International comparisons of education systems often produce outcomes that are not part of their original rationale but that nonetheless make valuable contributions to the improvement of U.S. education and international relations. In an increasingly interdependent world, they provide useful insights into the socioeconomic structure of other countries and cultures. For example, the insights of comparative education scholars who, in relative obscurity, had studied religious schools in Central and South Asia became more valued at the end of 2001, when graduates of those schools attacked the United States. The challenges posed by international studies can increase the educational research capacity of the United States as well as that of other countries (1990 Framework). For example, the Trends in International Mathematics and Science Study (TIMSS)5 Videotape Classroom Study helped to raise the technical sophistication of video research methods in the United States and elsewhere. Furthermore, questions of sampling, instrument design, data gathering, and data analysis that had to be addressed in the second and third international mathematics studies yielded results and experience that have been useful in national surveys of achievement. All these benefits do not flow automatically from every study. Rather, they are more likely to result from systematic investments in a variety of studies, differing in methodology, scope, and purpose, at least some of which try to test and build on earlier findings. Simply observing and measuring apparently effective practices in other countries is not sufficient to bring about desired improvement in U.S. schools. Ideally, promising practices would undergo several rounds of study in the context of their country of origin, and in the United States, in which practitioners and researchers attempt to construct and test hypotheses about the rela- 5   The Third International Mathematics and Science Study (TIMSS) was conducted in 1995. Subsequent iterations of the same study changed the word “Third” to “Trends.” Each iteration of the study is now referred to by the year it was conducted. Hence TIMSS becomes TIMSS 1995, TIMSS-R becomes TIMSS 1999, and TIMSS 2003 will remain TIMSS 2003.

OCR for page 4
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education tionship between the practice and desired outcomes in different settings. More often, informal experiments initiated by practitioners using innovations from other countries attract the attention of researchers post hoc; policy makers call on researchers to investigate promising practices; and, of course, researchers themselves may initiate exploratory studies. Instances of each of these cases are highlighted in boxes throughout the next chapter.