Cover Image

PAPERBACK
$34.75



View/Hide Left Panel

2
Range

Since the end of World War II, the U.S. government and private funders have supported a wide range of international education activities, including student and scholarly exchange activities in various disciplines, and the development of K-12 curricula to expand awareness and understanding of the rest of the world. By contrast, U.S. government funding for comparative social science research focusing on education in other countries—what we will refer to as international comparative studies in education—is a much more recent phenomenon, focusing to date on a relatively narrow range of studies.

This report redirects attention to the full range of international and comparative education studies. This range includes studies that involve one or many countries; collect large and small samples; employ a variety of quantitative and qualitative methodologies1; aim both to generate new hypotheses and test existing ones; and cover topics relevant to many levels, ranging from early childhood education to employment-related training for adults, and from governance issues for public and private schools to more practical issues, such as parent involvement in schools.

The breadth of these international comparative education studies and the way they spur interest in other research are illustrated by the summaries in the boxes scattered throughout this chapter. For the most part, these boxes highlight individual studies, rather than research programs

1  

For a more complete description of methodologies in international comparative education research see Rust et al. (1999:86-109).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education 2 Range Since the end of World War II, the U.S. government and private funders have supported a wide range of international education activities, including student and scholarly exchange activities in various disciplines, and the development of K-12 curricula to expand awareness and understanding of the rest of the world. By contrast, U.S. government funding for comparative social science research focusing on education in other countries—what we will refer to as international comparative studies in education—is a much more recent phenomenon, focusing to date on a relatively narrow range of studies. This report redirects attention to the full range of international and comparative education studies. This range includes studies that involve one or many countries; collect large and small samples; employ a variety of quantitative and qualitative methodologies1; aim both to generate new hypotheses and test existing ones; and cover topics relevant to many levels, ranging from early childhood education to employment-related training for adults, and from governance issues for public and private schools to more practical issues, such as parent involvement in schools. The breadth of these international comparative education studies and the way they spur interest in other research are illustrated by the summaries in the boxes scattered throughout this chapter. For the most part, these boxes highlight individual studies, rather than research programs 1   For a more complete description of methodologies in international comparative education research see Rust et al. (1999:86-109).

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education or reviews of specific topics. For example, books and studies by researchers associated with predecessors of TIMSS 1995 fed grassroots practitioner interest in Japanese lesson study, a teacher-driven process of professional development. Lesson study experiments are now being conducted in the United States as action research, with small collaboratives of researchers and teachers from both Japan and the United States using ethnographic methods in a few schools and districts. These studies are partly hypothesis generating, partly hypothesis testing studies; at present they do not aim to make generalizations beyond the cases they are studying, although at some later point they may. In contrast, the data from Liping Ma’s single-researcher study included interviews with only about 72 Chinese teachers, but Ma eventually used those data to articulate a kind of knowledge base for elementary mathematics teachers. Her work is now spurring domestic research in the United States on what Ma refers to as “the profound understanding of fundamental mathematics.” DIFFERENT PURPOSES In this report, we distinguish three types of international comparative education studies according to their initial, primary purpose. Type I studies typically include large-scale surveys that aim to compare educational outcomes at various levels in two or more countries. Type II studies are designed to inform one or more particular U.S. education policies by studying specific topics relevant to those policies and their implementation in other countries. Type III studies are not designed to make direct comparisons between the United States and other countries in terms of specific policies or educational outcomes. Rather, they aim to further understanding of educational processes in different cultural and national contexts. This typology avoids false dichotomies in educational research— large-scale versus small scale, qualitative versus quantitative—but it is not without difficulties. First, the domains of different study types are not mutually exclusive. TIMSS 1995, for example, was comprised of several somewhat independent studies, each using different methodologies. Some of these methodologies—such as case studies and videotaped classroom observations—are generally associated with more open-ended goals. However, all TIMSS components were undertaken to inform the overall goal of comparing achievement cross-nationally. The large-scale survey component of TIMSS is therefore an unambiguous Type I study. To the extent that the video study component of TIMSS was undertaken primarily to inform the survey, the video component is also a Type I study, but as a stand-alone study focusing on classroom interactions, it would be considered a Type III study.

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education Second, Type III includes the vast majority of international comparative studies in education, but Type I studies receive the vast majority of funding. Although cost is certainly not the most important factor distinguishing the three types of studies, it is certainly one of the most contentious and therefore figures prominently in the discussion below. To summarize, the typology used here is based upon the initial, primary purpose, rather than the ultimate use of a study. The typology creates overlapping rather than mutually exclusive domains. Furthermore, the types do not neatly divide the corpus of international comparative education research into three equal parts, either in terms of number of studies or funding. This typology would be inadequate for the study of international comparative education research as a discipline per se. Rather, it is intended to expand the discussion begun in earlier reports (1990 Framework and 1993 Agenda) to a broader range of studies and to a broader policy audience. Type I: Comparing Cross-Nationally Type I studies typically use survey data to compare outcomes—such as performance on standardized tests—in two or more countries. They also include rates-of-return analyses, social mobility studies, and many cross-national studies of comparative education development. Box 2-1 provides three examples of recent Type I studies. Many Type I studies to date have been large-scale quantitative studies with one of two primary goals. One goal is to monitor the status of a target population subgroup (such as 15-year-olds) in each country over time. These we refer to as indicator studies; the Programme for International Assessment of Student Assessment (PISA) of the Organisation for Economic Co-operation and Development is one example. The other goal is to understand the relations between education variables and to generate hypotheses about causal relations. Studies of this kind are referred to as research studies; of these, TIMSS is the best-known example and the Civic Education Study of the International Association for the Evaluation of Educational Achievement (IEA) is another (see Box 2-1). Although research studies of this kind may be very useful in testing hypotheses about correlations and generating hypotheses about the causes of these correlations, they are not well suited to evaluating causal hypotheses (National Research Council, 2002a). Trend studies covering a stable group of countries can strengthen the grounds for speculating on causes but cannot establish them. As noted previously, the quality of both indicator and research studies has improved dramatically over the past decade. Nonetheless, areas for improvement remain. For example, these studies require continued efforts

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education BOX 2-1 Comparing Cross-Nationally: Three Type I Studies The Programme for International Student Assessment (PISA) of the Organisation for Economic Co-operation and Development (OECD) is designed to assess how well 15-year-old students apply and use what they have learned both inside school and outside it. PISA is the capstone of a large OECD education indicators program (INES). Fifteen is the age at which compulsory schooling ends in most countries, and PISA’s outcome measure serves as an indicator of the quality of potential new entrants to a nation’s workforce that can be compared cross-nationally. PISA surveys mathematics, reading, and scientific literacy every three years, with one domain as a primary focus in each cycle. In 2000, PISA assessed over 250,000 students in 32 countries (including 28 OECD member countries). A second round of the PISA 2000 survey was administered in 2002 to 13 non-OECD countries. PISA also administers student and principal background questionnaires to explore the social and economic context of the learning environment and students’ attitudes toward learning. One of PISA’s more interesting findings was that high average performance levels were associated with relative small gaps between the highest and lowest performing students in comparisons with the United States. SOURCE: Organisation for Economic Co-operation and Development (2001). The Third International Mathematics and Science Study (TIMSS 1995), conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA) in 1995, assessed the mathematics and science achievement of a half-million students in over 40 countries at three age levels, which correspond roughly to the 4th and 8th grades and the last year of secondary school. TIMSS 1995 included several components: a curriculum analysis of 50 countries, a videotape study of 8th-grade mathematics classrooms in Germany, Japan, and the United States; a case study analysis of Germany, Japan, and the United States; and surveys of teachers and students to explore the context in which learning and teaching take place. The 4th- and 8th-grade cohorts achieved higher sampling standards than those achieved with earlier IEA studies.* The curriculum analysis related the mediocre performance of U.S. students to U.S. mathematics and science curricula, which it described as “a mile wide and an inch deep.” SOURCES: Peak et al. (2002); Schmidt, McKnight et al. (1997); Schmidt, Raizen et al. (1997); U.S. Department of Education, National Center for Education Statistics web site (http://www.nces.ed.gov/timss/); U.S. Department of Education, Office of Educational Research and Improvement (1998,1999); Wiseman and Baker (2002). *   The U.S. 12th-grade sample remains problematic for many reasons (Rotberg, 1998).

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education The IEA Civic Education Study examined school programs that promote civic knowledge, attitudes, and participation in more than 20 countries, in an effort to compare what young people around the world think about democracy. The study consisted of two phases that incorporated both qualitative and quantitative methods. In Phase I (1996-1999), 24 countries developed case studies to explore the context and meaning of civic education. Findings from these case studies then informed Phase II, an assessment of nearly 90,000 14-year-olds’ civic knowledge and a survey of their civic engagement in Phase II. Phase II also included an assessment of upper secondary students to investigate the influence of additional years of schooling on students’ civic knowledge as they approach the transition to adulthood. The assessment was an attempt to measure civic knowledge, skills in interpreting civic information, attitudes toward democratic institutions, and expectations regarding civic engagement as adults. The second phase of the study found that most 14-year-olds agree that good citizenship includes the obligation to vote. An area for secondary analysis is the gap between young students’ perceptions about the importance of voting and low actual voting rates of young adults. SOURCES: Amadeo et al. (2002); Steiner-Khamsi et al. (2002); Torney-Purta et al. (1999,2001). to improve measurement of important school and student background variables pertaining to socioeconomic status and to experiment with cross-national comparisons using jurisdictions closest to the level of education decision making—such as school districts or states in the United States. Note that this last area for improvement is likely to increase cost by increasing the number of samples and observations. For example, in PISA, national samples of around 5,000 were required, but to make interstate comparisons, Canada tested around 35,000 and Germany around 80,000. Finally, those who conduct and sponsor these studies are still struggling to find ways to communicate the difference between correlations and causality in releasing their findings to the public, and to make clear that surveys are more often the beginning, not the end, of a research process that must involve many different types of studies. Type I studies that collect primary data tend to be the most expensive, in terms of both direct costs, such as the diplomatic capital required to secure agreement at the national level for primary data collection and the cost of conducting surveys and employing experts to organize and direct them, and indirect costs, such as the time demands placed on student and

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education administrator participants. But even within this subset of Type I studies, costs vary dramatically. For example, the estimated direct costs to the U.S. government of TIMSS 1995 are almost $20 million over five years. Other direct costs, however, including support for the Board on International Comparative Studies in Education (BICSE),2 curriculum studies, and secondary analysis grants, totaled more than $3.5 million. In contrast, the U.S. contribution to the IEA Civic Education Study, much of it from private sources, was less than $1.7 million over three years. The high cost of Type I studies must be weighed against the particular benefits. These studies enable us to establish benchmarks for comparing the performance of students in the United States to that of students in other countries. They also stimulate hypotheses about the reasons for differences in achievement levels in the United States and those elsewhere. The scale of Type I studies justifies spending significant sums on publishing and disseminating a variety of reports aimed at many levels of the education system. With help from public relations firms, these reports can attract the attention of a broad spectrum of the public and practitioners to specific issues in education, such as the depth of the curriculum. This is a double-edged sword: results issued with much public fanfare may dominate public debate long after smaller studies with much smaller budgets call them into question. The relatively high cost of Type I studies increases the likelihood that attempts will be made to expand their potential uses by increasing the number of background variables selected, subgroups sampled, and topics studied, increasing cost at the design and data collection stages. As a result, two tightly focused Type I studies, each with a single purpose, may be less expensive than one expansive study, designed to serve many purposes. Funding for the largest Type I studies derives principally from national governments and dwarfs funding for all other types. This focus on surveys is consistent with the pattern of development for federally funded domestic education research. The collection and dissemination of domestic education statistics was mandated almost 90 years before Congress authorized the U.S. Office of Education to fund broader education research (National Research Council, 1992).3 Demand for Type I studies is often generated by policy makers. For the past 10 years, the U.S. Department of Education has worked from a draft multiyear strategy for U.S. 2   This amount only covers the period 1992-1994, when BICSE’s sole charge was “to support efforts to improve data collection of international studies of science and mathematics.” 3   Office of Education Reorganization Act, 1867.

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education participation in these studies (U.S. Department of Education, National Center for Education Statistics, 1994). Nonetheless, the choices of timing, topics, frequency, and target group for these types of studies are not entirely within the control of U.S. policy makers, since the studies can be undertaken only with close cooperation from other participating countries. Sometimes U.S. participation in a study of relatively low priority for the United States may be necessary in order to secure broader cross-national participation in a study of higher priority. Every study that is rigorously conducted improves the capacity of the participating countries to produce better quality statistics, and in turn those can provide better, more consistent comparisons with U.S. education statistics. Conversely, by bearing more than its share of the cost, the United States sometimes persuades other countries to participate in studies of particular interest to the United States. For example, U.S. costs for TIMSS 1995 were very high, in part because the United States wanted more rigorous technical standards and shouldered much of the international costs of the study necessary to achieve those standards. The U.S. share of funding of both TIMSS 1999 and TIMSS 2003 were expected to decrease over time, as more countries adopt those standards and participate in cost sharing. Type II: Informing Policy Type II studies are designed to inform U.S. education policy in a direct way, by examining specific policies and their implementation in other countries. Type II studies also include evaluations of attempts in the United States to implement similar policies and practices originating in other countries. Many issues—for example, innovation and change, teacher involvement in change, and teacher development—simply cannot be addressed by Type I studies. Such studies demand a mix of quantitative and qualitative, descriptive and interpretive studies. A study of the experience of U.S. schools that have implemented the Singapore mathematics curriculum falls into this category, as do school choice and high school tracking studies (see Box 2-2) and many of the studies produced by the Organisation for Economic Co-operation and Development, such as the 13-nation study of educational innovation and change in science and mathematics education (Black and Atkin, 1996; Raizen and Britton, 1997). These studies may be initiated by policy makers or by researchers, either anticipating or responding to U.S. interest in specific policies or practices in other countries. Type II studies tend to be smaller in scale and therefore less expensive than those Type I studies that collect primary data, although they may be more costly than Type III studies (described below). Studies that aim to

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education BOX 2-2 Informing Policy: Three Type II Studies Singapore Mathematics Curriculum Study. Singapore’s mathematics curriculum has attracted attention in the United States because of Singapore’s high achievement results on international assessments. Approximately 100 U.S. schools have adopted Singapore’s mathematics textbooks as part of their reform efforts in mathematics education. The U.S. Department of Education has funded a joint study with the Singapore Ministry of Education to assess the implementation of the Singapore mathematics program in U.S. schools. This 27-month study will include observations of Singapore classrooms by U.S. district and school staff; site visits to U.S. schools using the Singapore mathematics approach, including observations by Singapore experts; surveys of district mathematics coordinators, principals, and teachers; and electronic networking among study participants. SOURCE: U.S. Department of Education, Planning and Evaluation Services (2002). School Vouchers. Private school vouchers are widely debated as a potential means of addressing the quality of the public education available to some students in the United States, with proponents insisting that private schools can deliver education more effectively at a lower cost and opponents arguing that vouchers take much-needed resources away from public schools. To inform this debate, McEwan and Carnoy (2000) assessed the relative effectiveness and efficiency of Chile’s voucher system, which has been implemented on a large scale and has prompted an increase in private school supply. The researchers defined effectiveness as higher academic achievement, holding student background constant, using data from Chile’s national assessment of mathematics and Spanish achievement. They defined efficiency as producing the same achievement with less funding, using multiple data sources to construct a proxy of the annual per-student cost of each school in 1996. Although not the first analysis of Chile’s public and private school achievement, this study used a more complete set of student achievement data, divided the voucher schools into three categories rather than considering them as a whole, and provided the only comprehensive analysis of costs and efficiency. As the researchers point out, the findings from the study “are probably not satisfying for either voucher advocates or opponents” (p. 227). Their results suggest that nonreligious private schools are marginally less effective than public schools—even less effective when located outside Santiago—but more efficient at producing achievement at a lower cost. Catholic schools are more effective than public schools, but because of the additional resources consumed in producing that achievement, are relatively similar in efficiency. McEwan and Carnoy draw inferences from these findings for the U.S. debate on school choice. They suggest that the United States may want to rethink existing comparisons of Catholic and

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education public schools as direct evidence of the potential impact of a large-scale voucher plan. SOURCE: Adapted from McEwan and Carnoy (2000). High School Tracking. Tracking, the controversial practice of assigning students to groups or classes for instruction based on their prior academic performance, has been extensively studied in the United States, but where international studies are available, they have interesting implications for U.S. findings. U.S. findings clearly demonstrate that achievement inequality between students assigned to different tracks (e.g. college preparatory versus vocational, or honors versus remedial classes) widens over time, but because almost all high schools are tracked, there is relatively little U.S. evidence on the effects of tracking versus the absence of tracking. An important study of tracking in Britain by Alan C. Kerckhoff (1986) indicated that students in elite schools and high tracks gained from being placed in such educational settings, compared with similar students in comprehensive schools and mixed-ability classes. Meanwhile, students assigned to low-status schools and classes within schools fall further and further behind, compared with similar students in untracked schools and classes. Kerckhoff’s results were based on a unique dataset consisting of all children born in England and Wales during the first week of March 1958 and provided compelling evidence for the effects of tracking on achievement inequality. Many American studies of tracking follow students over time but lack evidence on changes in systems of tracking over time. Adam Gamoran (1996) examined four waves of survey data on high school students in Scotland during a time in which the secondary education system became less stratified (1984-1990), to determine whether the reduction in tracking led to lower levels of social inequality in educational outcomes. Not only were outcomes less unequal after the reform, but also achievement levels on a national examination were higher overall. This study showed that reducing tracking can lead to less inequality without harming achievement levels. A third international study, this one comparing Israel and the United States, found that while schools with more intense tracking systems produce more inequality in the United States, that was not the case for Israel (Ayalon and Gamoran, 2000). A key difference between the United States and both Israel and Scotland is that the latter countries have national examinations that provide incentives for student performance in lower-level academic classes as well as for those in elite classes. On the basis of these results, one may speculate that the current emphasis on testing in American schools may lead to better performance among low achievers—if their teachers are well prepared, and if the students have adequate opportunities to learn the material on the tests. SOURCES: Ayalon and Gamoran (2000); Gamoran (1996); Kerchoff (1986).

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education make inferences relative to policy at decentralized levels may require larger sample sizes for primary data collection than broader, more generalized studies. Type II studies, however, do not necessarily require primary data collection. Meta-analyses of existing datasets, or syntheses of existing studies across countries, are also valuable; both the 1990 Framework and the 1993 Agenda reports specifically encouraged the United States to participate in more of this type of research, in order to get the most out of existing studies. Like Type I studies, Type II studies must draw on a limited pool of experts; they also require modest diplomatic capital. Neither the federal government nor states have established formal priorities or budgets for this type of study, although they offer the most direct means to explore causal links between public policies and the performance of the education system, often for a fraction of the cost of Type I studies. Type III: Understanding Education Broadly Type III studies are designed to increase general understanding about education systems and processes. Such studies are not designed to have immediate policy relevance, though many are of relevance to policy makers. Rather these studies are designed to bring to light new concepts, to stimulate interest in educational issues, to generally deepen understanding of education as a practice and as a social phenomenon and, most generally, to establish the foundation on which all other comparative education research is based. These studies, like Type II studies, may be qualitative or quantitative, large- or small-scale, incorporating positivist, interpretive, or critical perspectives (Heck and Hallinger, 1999:143). However, unlike the other two types, Type III studies do not fall neatly under one purpose. Donmoyer (1999) identifies at least five potential purposes for qualitative Type III studies. These include efforts to answer the questions “What is the correct answer (assuming a particularly clear question)?” or “How do the people studied interpret the phenomenon?” or “How does an organization, individual, or group of individuals change over time?” or “How can the researchers simultaneously learn about and change educators and/or educational organizations?” Some quantitative Type III studies also relate to Donmoyer’s purposes and others’. To designate all these studies as one type does not do them justice, yet to disaggregate them systematically is a task larger than the charge of this report. As described in Box 2-3, the work of Catherine Lewis, in her book Educating Hearts and Minds, and the work of Robin Alexander, using videotapes to explore classroom culture and pedagogy in five cultures, are examples of this type of study. Although relatively large in scale, the Alexander study qualifies as Type III since it was not driven by the desire

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education BOX 2-3 Understanding Education Broadly: Two Type III Studies Educating Hearts and Minds. The research of Catherine Lewis (1995a, 1995b) provides insights into the Japanese approach to elementary education, which is designed to meet the needs of the whole child, for belonging, contribution, and intellectual competence. She argues that Japanese schools are successful in promoting students’ academic achievement because by meeting these basic human needs, schools help children develop a positive attachment to schooling. Lewis bases her analysis on observations and interviews spanning 10 years in more than 30 elementary schools in three Japanese cities. She observed and interviewed teachers to explore why they used particular instructional techniques. Lewis outlines nine qualities of Japanese elementary education central to understanding Japanese achievement: whole-child education, including nonacademic subjects such as art and music; values-rich education, focusing on friendship, cooperation, and other aspects of social and emotional development; a caring, supportive community, with opportunities for students to get to know each other and to collectively shape classroom values and practices; learning to live in groups, through the use of han, or longer-term family-like groups with diverse abilities mixed together; reflection, or hansei, on goals either privately or collectively (in small groups or as a class), formally or informally; methods of discipline that promote a personal commitment to values; children’s thinking helping to drive instruction and classroom life; “wet learning,” a term used by the Japanese to describe approaches that are personal, emotional, and interpersonal (as opposed to “dry learning,” which is logical, rational, and unemotional); and a standardized curriculum supportive of child inquiry. SOURCES: Lewis (1995a, 1995b). Culture and Pedagogy in Five Countries. Robin Alexander (2001) explores the effects of primary education on children’s sense of empowerment using classroom videotapes from five countries: England, France, India, Russia, and the United States. Alexander’s study compares four aspects of primary education in each culture: the structure and purposes, the thinking of teachers and the classroom experiences encountered by students, the relationship between the classroom and the world of educational and social policy, and the impact of culture and history. The multimethod analysis draws on data gathered in 1994-1998 at the system,

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education school, and classroom levels, using interviews, observation, and videotape and audiotape, supplemented by school and country documentation, photographs, and daily journal entries. The notion of cultures, as “the web of inherited ideas and values, habits and customs, institutions and world views which make one country, or one region, or one group, distinct from another,” is paramount throughout the study (Alexander, 2001:5). Alexander acknowledges that although there may be cross-cultural similarities, educational policy and practice cannot be fully understood without reference to these distinctions. For example, he describes the nature of interactions between the levels studied, where classrooms, schools, and systems are microcultures in their own right, with inherent values and customs that also respond to external influences, and that offer a window onto the larger culture. The international nature of the study also enables him to study the processes by which ideas and practices migrate from one culture to another, such as when one country imposes practices on another through colonialism or when one country borrows practices from another. SOURCE: Adapted from Alexander (2001). either to compare achievement or to inform a specific policy. Such studies may be conducted by U.S. scholars in foreign countries or by foreign scholars in the United States and provide new ways of looking at familiar practices. Private foundations pay for many of these studies, the cost of which is generally measured in tens or hundreds of thousands of dollars. In terms of indirect costs, Type III studies are undertaken only as expertise is available; they are often arranged between individual scholars or research groups and require minimal diplomatic capital. Because they deal with many fewer schools and smaller samples, they have relatively little impact on the time of students, teachers, and administrators. Type III studies that involve in-depth analysis of a single aspect of an education system can be relatively inexpensive in terms of both direct and indirect costs; unlike more expensive Type I and II studies, they often do not produce either generalizable findings or research methodologies capable of being precisely replicated in other jurisdictions. Instead, many offer rich descriptions of context in particular settings; these rich descriptions can provide clues to the variables influencing the phenomenon. As a result, although they vastly outnumber Type I and Type II studies, Type III studies often do not come to the attention of policy makers or

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education the public. This is a loss, since many are rich in narrative detail and paint a more engaging and provocative portrait of education in other countries than do the summary bar charts and graphs typical of many larger studies. Ethnographic and case studies, in particular, can explore cultural context in depth and, in turn, help elucidate the way education is organized and understood in different cultures. The detail these studies offer, however, renders them difficult to summarize, and their complexity often leads researchers to focus on just one or two countries, making their findings difficult to compare cross-nationally. The U.S. government currently has no formal priorities or budget for this type of study. MOVING TOWARD A MORE BALANCED RESEARCH AGENDA Establishing funding priorities for such a wide range of studies poses a challenge to all funders of international comparative education research. A balanced research agenda involves some support for basic statistics and indicators, other investments to address short- to medium-term policy concerns, and also significant investment in studies that explore broader issues in education over a longer period of time. Here the board suggests several criteria for developing a balanced, coherent research agenda. These criteria include investing in international studies that can help to address high-priority issues on the domestic research agenda; areas neglected to date by Type I studies; areas in which international research has a comparative advantage over domestic research; studies that respond to questions from a variety of positions and levels in the education system; and studies of the relative effectiveness of different methodologies in answering different types of questions and in communicating with different constituencies in the U.S. education system. A part of the international comparative agenda should address some of the high-priority issues for domestic education research. Federally funded studies to date do this, to a limited extent, by focusing on Type I studies of achievement, in core curriculum areas, in formal K-12 schools, in countries perceived to be economic competitors of the United States. To the extent that these issues remain priorities for domestic education research and these international studies indeed are structured and timed in such a way that they are able to inform domestic research and policy, this current de facto international agenda is on target. The domestic research agenda, however, is broader than the issues and methodologies evident in the de facto international research agenda. The list of high-priority domestic issues that could be illuminated by thoughtful research with international dimensions includes alternative assessment methods, finance, governance, teacher education, education at either end of the K-12 continuum, aspects of school outcomes other

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education than achievement, closing the achievement gap between students from differing family backgrounds, and the school-to-work transition. To take one example: the National Educational Research Policy and Priorities Board in the U.S. Department of Education identified as a priority finding ways to better serve the growing number of English language learners in U.S. schools (U.S. Department of Education, National Educational Research Policy and Priorities Board, 2000). This particular domestic research priority should be informed by international studies comparing other countries’ experiences with second-language students and understanding of education systems in such geographic regions as Africa, Central and South America, and South Asia, which constitute the points of origin for large immigrant communities and many English language learners in U.S. schools. The weak state of education statistics in many of the countries in these regions means that many cannot participate in Type I studies in a meaningful way; fortunately, there is much to be learned about the children and parents who emigrate from these countries by means of other research methodologies. Informing the domestic research agenda, however, should not be the only concern driving the international and comparative education research agenda. We should also reflect on how international comparative work can make unique contributions to basic understanding of the relationship between education and society. Because international research is so well positioned to reveal variation across societies, international comparative studies in education offer rich promise in deepening understanding of school-society relations and links between culture and schooling, religion and schooling, and the home and the school. In addition to those areas unevenly addressed by research (described above), these additional areas point to questions worth pursuing in future involvement in international studies. Recommendation 1: Funding for international comparative education research should reflect a balance among the three types of international comparative education studies and should encompass a broad array of methodologies, scale, purposes, and topics. Specifically, the United States should increase investments in studies that focus on understanding the education experiences of other countries in their own context (Type II and Type III) to provide a broader context for U.S. experiences and efforts to innovate. Yet another criterion for any research agenda for U.S. involvement in international comparative education studies is that it needs to include a range of questions generated from different originating points: address-

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education ing policy concerns, growing out of discipline-based theoretical concerns, accommodating and responsive to issues raised by other countries, and flowing from the experiences of practitioners. As elaborated in the following chapters, serious research is needed to uncover new ways of understanding education and good ideas for improving education should come from a variety of sources and flow in all directions among researchers, practitioners, parents, and students. Recommendation 1.1: U.S. funders should foster closer links among practitioners and researchers so that both participate in the formulation and conduct of research, and both take responsibility for creating effective ways to use international education research. One way to ensure that international comparative education studies support and inform the domestic education research agenda is to encourage major domestic education research efforts to include an international component when that would add value to the domestic research findings. As do multimethod studies, international components work best when they are not simply contemporaneous add-ons to relatively complete domestic studies, but rather are executed in time to provide input at specific stages of the design, data collection, and analysis of domestic studies. Recommendation 1.2: U.S. funders should routinely support international components in domestic (state, local, and national) education policy and practice studies that draw on experiences in other countries. MULTIPLE METHODOLOGIES No single type of international comparative education study can answer all of the most pressing questions about how education works in other countries and how it might be improved in the United States. The most fruitful studies, however, are often those that combine multiple methodologies. For example, in the IEA Civic Education Study, case studies informed the design of frameworks for large surveys, which in turn generated correlations; these can now be explored by a range of other Type II and Type III studies tailored to specific geographic locations and contexts. At present, however, this sequencing of different types of related studies seems to be the exception rather than the rule. LeTendre (2002) argues that when multiple methods are used, an overarching framework for analysis that integrates results of the different research compo-

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education nents strengthens the overall analytic power of any study. (We return to this topic in Chapter 5.) Many scholarly debates about the validity, relevance, and generalizability of findings of international comparative education studies focus on the extent to which context has been recognized and properly taken into account in various aspects of the study. While randomized field trials and quasi-experimental designs are sometimes held up as the gold standard in domestic education research on effects of interventions, such methods can work only when in-depth knowledge of specific contexts allows researchers to model and control independent variables. In many areas of international comparative studies, those who wish to pursue randomized field trials will need to invest in much exploratory, open-ended work in order to identify salient variables, comparable populations, and critical differences in environment. Can we understand the function of school exit exams or university entrance exams in different countries if we do not understand the income distribution and the returns to school attainment in those countries? Can we understand the early school performance of children without understanding the child-care and pre-school policies of a country? Patterns of income distribution and returns to school attainment must first be identified and the means to measure them must be developed before researchers can “control” them sufficiently to apply experimental methods. More support is needed for qualitative and historical studies, which, in addition to being valuable in and of themselves, can build the necessary foundation for further quantitative work. Recommendation 1.3: U.S. funders should evaluate proposals for qualitative or historical studies and for quantitative studies by somewhat different criteria, conforming to fundamental principles of sound research for both and accommodating the different canons of systematic inquiry and different warrants for generalization in each discipline. Although there is no single method of analysis that is agreed on by all international comparative education researchers, there is room for improvement in the methodologies of all three types of studies, and some component of any international comparative education research agenda should include studies to compare and improve research methodologies. Specific goals for improvement include organizing studies that use multiple methodologies in more effective ways. Designers and secondary data users could collaborate more effectively throughout the life of large-scale research studies to ensure greater use of the data. The use of video-

OCR for page 12
Understanding Others, Educating Ourselves: Getting More from International Comparative Studies in Education taped studies of classrooms in other countries could be better understood. Findings from international comparative education studies could be used to stimulate better-informed discussions about education among U.S. policy makers and the public. Recommendation 1.4: U.S. funders should encourage multicomponent research studies with longer time horizons, using a variety of qualitative and quantitative methodologies. In summary, because policy makers and educators in the United States cannot know in advance which studies will be critical, a prudent approach to developing a research agenda for international comparative studies in education is to support and encourage a broad range of study types and topics, with increased attention to those methodologies, topics, and geographical regions that have received relatively less investment in recent years, incorporating more international components into domestic studies, with some formative evaluations to study how domestic and international components can complement each other. Finally, the scope of international comparative education studies means that no single set of methodological criteria will be adequate to evaluate the quality and promise of proposals for international comparative research and finished studies. As noted above, there is a continuing need for more qualitative and historical work that, while conforming to fundamental principles of sound research, is subject to different canons of systematic inquiry and different warrants for generalization than are large-scale surveys.