Part II
Culture and Context



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Part II Culture and Context

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement This page in the original is blank.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement 5 Cultural-Cognitive Issues in Academic Achievement: New Directions for Cross-National Research Janine Bempechat, Norma V. Jimenez, and Beth A. Boulay* CULTURAL-COGNITIVE ISSUES IN ACADEMIC ACHIEVEMENT: TOWARD AN UNDERSTANDING OF CROSS-NATIONAL DIFFERENCES The past quarter century has seen a burgeoning interest in cross-national comparisons of student achievement. The fascination with achievement in different nations has been fueled by rapid technological advances that have changed the face of the global economy. Increasingly, nation states are expressing concern with their ability to compete in a world that is becoming ever smaller. Major cross-national investigations of academic achievement have been undertaken systematically by the International Association for the Evaluation of Educational Achievement (IEA) of which the Third International Mathematics and Science Study (TIMSS) and TIMSS-Repeat (TIMSS-R) are the most recent (Beaton et al., 1996; Husen, 1967; McKnight et al., 1987). In addition to these IEA investigations, a large body of research has compared the achievement outcomes of American students with their peers in other nations. The research programs of Stevenson and Hess are particularly notable in this regard (Hess & Azuma, 1991; Hess, Chih-Miei, & McDevitt, 1987; Stevenson, Chen, & Lee, 1993; Stevenson & Stigler, *   The authors are from the Harvard Graduate School of Education. They wish to acknowledge the generous feedback of Kurt Fischer, Susan Holloway, Julian (Joe) Elliott, and Neil Hufton on earlier drafts of this chapter.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement 1992). The interpretations brought to the results of this body of research have tended to conclude that American students perform poorly in all aspects of mathematics and science, relative to their peers in other industrialized nations. What can we do with this knowledge? And, having an established database of computational achievement across many nations, what are the next steps that researchers can take to expand our current understanding? The purpose of this chapter is to take a constructively critical view of what we know about the achievement of children and youth across nations, and to suggest fruitful directions for the next steps in cross-national research. We have organized this chapter around two core themes—culture and methodology—through which we will examine two central domains of study: Social cognitive factors in learning. In addition to observing and documenting the range of cognitive goals that cultures have for members of their group, it is critical to understand the social and cultural beliefs about learning that give rise to these values. Beliefs and attitudes about learning and achievement form the core of achievement motivation research today. There is a very important benefit to studying how students are motivated. Motivation research has established that achievement beliefs (e.g., implicit beliefs about effort and ability) are critical to school success (Nicholls, 1989). Indeed, in many cases, achievement beliefs appear to be better predictors of school performance than are IQ or achievement tests (see Dweck & Bempechat, 1983). By anchoring our review in achievement motivation theory, we will show the ways in which this body of knowledge can help fill gaps apparent in current cross-national research on achievement. Cognitive psychology. The assessment focus in cross-national investigations has been on computational skills. However, cognitive psychologists and mathematics educators have been arguing that in order for students to become technologically competent, they need to engage in learning that fosters a deep conceptual understanding of mathematics and science. This argument raises important questions for cross-national investigations—questions that we will explore in this chapter. We will examine the extent to which the available cross-national data shed light on how students come to a deep conceptual understanding of mathematics and science. We will ask: Does this understanding mean the same thing in different countries? Indeed, there may be different pathways to deep conceptual understanding within and between countries. Having considered these issues, we will propose directions that cross-national studies can take to investigate these questions.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Why Study Academic Achievement Across Cultures? The accumulated work on cross-national achievement generally has been praised for bringing attention to the state of underachievement in the United States, especially as it relates to technical knowledge (Bempechat & Drago-Severson, 1999). The existing research has led educators to take a closer look at factors that may be contributing to the underachievement of American students, such as pedagogical practices (Stigler & Hiebert, 1999), students’ and parents’ beliefs about learning (Bempechat, 1998), and school structure (Beaton et al., 1996). This kind of comparative self-examination gives us a clearer picture of our own approaches to education. Indeed, examining how other cultures educate their children challenges us to look at our own system with a more critical eye. There is a way in which our own familiar pedagogical beliefs and practices become unfamiliar when set beside those of other nations (Spiro, 1993). Cross-national research on mathematics and science also has the potential to reveal the rich and varied ways in which students, teachers, and parents conceptualize the meaning and value of learning. In this way, it can help us to understand the cognitive goals that each culture has for its students. Seen in this light, cross-national research on achievement can reveal much more than a simple rank ordering of nations according to technological competence. Comparative studies of achievement provide us with a window through which we can view culture in action. Inasmuch as culture serves as a guide for the socialization of children, cross-national research allows us to see how culture guides the socialization of achievement. In addition, cross-national studies of achievement give us some insight into how the logic of individuals’ beliefs influences their behavior. For example, Stigler and Perry (1988) note that teachers in Asian cultures (Japan and Taiwan) routinely ask students to display their answers to mathematics problems with which they are experiencing difficulty. In contrast, mistakes and difficulty more often are experienced privately in American classrooms. Indeed, many American teachers and parents would view this Japanese practice as humiliating and cruel. Stigler and Perry (1988) attribute this differential view of a particular pedagogical practice to a cultural difference in beliefs about the nature of mathematics intelligence. On average, Japanese mothers and teachers are less likely than their American counterparts to believe that mathematics ability is innate. Therefore, with the appropriate amount of effort, all children can solve a problem. In this context, mistakes are not something to be ashamed of, but something to work through. The general European-American view of mathematics ability as innate contributes to the notion that mathemat-

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement ics errors are the consequence of low ability, over which students have no control. To send students to the board, then, is to ask them to admit publicly that they have low ability. In the U.S. context, this practice might foster concerns about the potential to erode students’ self-esteem. Theoretical Tensions One can readily see that each strategy for dealing with struggle may be appropriate, given the social and cultural context in which it has arisen. Herein lies the key to expanding our knowledge and understanding of cross-national differences in achievement—in realizing that we, as a research community, can move forward only if we situate culture and context at the center of our investigations. At the same time, however, we need to remain cautious about making assumptions about entire nations without considering the variation in beliefs and practices that exist in all cultures. For example, there may very well be a great many students in Japan who do experience the public display of their mistakes as humiliating. Similarly, there may be many U.S. students who would experience such a practice as educational and helpful. Yet many of us who study cultural influences in social cognition tend to rely on cultural models that speak of nations as if they were monolithic, when, in fact, there is a great deal of variation in a given society’s cultural models. Shore (1996) has discussed the tension between cultural anthropology and cultural psychology, noting that both disciplines view the construction of meaning as an ongoing, active process that is influenced by culture. He has encouraged scholars to view culture not as one “cultural narrative,” but rather as a collection of cultural models which present competing views and interpretations about a society. Shore has argued for the integration of cultural psychology and cognition through his notion of an “ethnographic conception of mind,” in which cultural knowledge would be viewed as rich and diverse and shared through various cultural models. While cultural psychologists do indeed endorse this view, much of the work we discuss in this chapter tends to characterize nations as being at one or the other end of a dichotomy. In this regard, Japan and the United States have come to epitomize the comparisons that are made between “Eastern” and “Western” societies. Japan has been characterized as a culture that fosters interdependence, while U.S. culture fosters independence (White, 1987). Japanese people are said to be oriented around collectivist concerns, in which group loyalty and harmony lead individuals to subjugate their individual needs to those of the group, for the sake of the group’s well-being (Mouer & Sugimoto, 1990). In contrast, Ameri-

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement cans are characterized as individualistic and concerned with fostering personal goals (Greenfield, 1994). We view this tension between the disciplines as representing different layers of inquiry. A line of research may reasonably begin with a large-scale survey, through which one might uncover interesting, general differences between groups of individuals. These differences might then be examined in increasingly detailed fashion, through multiple methods, including experiments, targeted questionnaires, indepth interviews, and ethnography. It is thus that research becomes more nuanced and reveals the varied and complex ways in which cultural beliefs are contested in a society. Social-Cognitive Factors in Learning As Bruner (1990) argues in Acts of Meaning, it is no longer sufficient to explain what children do. It has become imperative to study what children “think they are doing and what their reasons are for doing it” (p. 49). Studying children’s achievement beliefs has opened a window into why students engage in behaviors that either promote or inhibit their academic achievement. Any study that measures achievement without concurrently examining the context in which this achievement occurs will yield results that may be limited in their use. The integration of achievement motivation theory with social cognition has resulted in a much deeper understanding of the motivational factors that underlie academic achievement. We have gone from viewing academic achievement as originating from an innate need or drive to the realization that achievement cognitions, such as attitudes, expectancies, and beliefs about ability, mediate the relationship between achievement behavior and achievement outcomes (Bempechat, 1998; Dweck, 1999; Eccles, 1993; Nicholls, Cobb, Wood, Yackel, & Patachnick, 1990; Weiner, Russell, & Lerman, 1979). For example, research has established that students who believe their intelligence is relatively stable (entity theorists) tend to avoid challenging tasks, and have been shown to sacrifice opportunities to learn new material in order to show that they are “smart” (Dweck & Bempechat, 1983). These students tend to succumb to learned helplessness when faced with a difficult task. In contrast, students who believe that intelligence is a malleable quality (incremental theorists) prefer challenging over nonchallenging tasks and tend to display mastery-oriented behavior in the face of difficulty or challenge. Achievement beliefs include students’ attributions for success and failure, their beliefs about the malleability of intelligence, their confidence, expectations and standards for performance, and affect in the face of

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement difficulty or challenge (Ames & Archer, 1987; Bempechat, Graham, & Jimenez, 1999; Eccles, 1993; Marsh & Shavelson, 1985; Nicholls, Cheung, Lauer, & Patachnick, 1989). These motivational factors have been shown to be critical elements in students’ achievement-related behavior. For example, Weiner and his colleagues have carefully documented that children and young adults tend to attribute success and failure to three basic categories of attribution—effort, ability, and external factors, such as luck or task ease/difficulty. Individuals interpret these attributions along three primary causal dimensions—locus (internal/external), stability, and controllability. Weiner has painstakingly shown that each attribution is linked to an emotion (e.g., lack of effort is linked to feelings of embarrassment), and it is the emotion that predicts future achievement behavior (Weiner et al., 1979). In other words, through a process of implicit self-evaluation, a student may decide that he failed a mathematics test because he is not smart (lack of ability). According to Weiner’s theory, ability is perceived by the vast majority of students as internal, stable, and uncontrollable. Given that there is little remedy for lack of ability, the student would probably feel ashamed, and this feeling would likely predict maladaptive achievement behavior, such as little or no preparation for the next test. Weiner’s theory views ability as a stable entity that does not change. However, Nicholls (1978, 1989) and Dweck (Bempechat, London, & Dweck, 1991) have demonstrated that under certain circumstances, children can be influenced to perceive intelligence as a malleable quality that changes as a result of disciplined effort. For example, classrooms that are oriented around cooperative learning rather than competition tend to minimize students’ concerns about their abilities and foster a greater tendency to take academic risks (Nicholls, 1989). This view of ability as mastery through effort focuses children’s attentions on the process of learning. However, as Covington has shown, the view of ability as capacity fosters the conundrum of effort as the “double-edged sword” (Covington & Omelich, 1979). Many students come to believe that if they have to try hard, they must be “dumb.” In short, effort becomes an implicit condemnation of ability (Nicholls, 1978). The Origins of Achievement Beliefs Students’ beliefs about learning do not develop in a vacuum. They are very much influenced by the achievement beliefs of their parents, peers, and teachers, as well as the social and cultural environment in which they are growing (Ames & Archer, 1987; Ogbu, 1986; Peak, 1991). In the context of cross-national comparisons of academic achievement, the issue becomes one of integrating the sociocultural contexts of education with social-cognitive aspects of learning. Although the majority of the research

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement on beliefs about learning has been done at the individual level, these beliefs are indeed culturally and socially constructed, and therefore contribute to a nation’s mindset about education (Bruner, 1990; Schurmans & Dasen, 1992). How, then, can we compare the academic achievement of students from different cultures when those cultures differ in their pedagogical goals? The fact that culture guides socialization implies that important influences in academic achievement, such as parent attitudes about learning, teacher expectations, and cultural construals of schooling, will differentially dictate how students understand their educational experiences. Cross-National Studies of Achievement Indeed, international studies of achievement are measuring much more than what students have learned (Holloway & Minami, 1996; LeVine, 1977; Markus & Kitayama, 1991; Munroe & Munroe, 1997). In a sense, they are measuring a nation’s pedagogical goals. The early IEA cross-national investigations (Husen, 1967; McKnight et al., 1987) were focused primarily on performance and provided us with rank orderings of achievement outcomes in the various domains of mathematics and science. Explanations for these differential outcomes tended to focus on aspects of curriculum, teacher preparation, and system structure, such as the number of days in each nation’s school year. Although these factors clearly play a role in student outcomes, these studies left the research community with a gap in our understanding of the extent to which social-cognitive factors may have influenced the academic outcomes that were documented. The latest investigation, TIMSS (Beaton et al., 1996), represents a major advance in how we study and interpret academic achievement across nations. Through case studies and classroom observations, rich portraits were painted of school systems within countries (Germany, Japan, and the United States). For example, students were asked in individual interviews to speak about the relationship between effort and ability in academic achievement, giving us deeper insights into how they conceptualize achievement within the context of their own cultures. However, less attention was paid to variation within culture. The result is that we know little about how these students’ beliefs may differentially influence their achievement. Achievement Beliefs and Culture To consider culture in education means that we have to study education in the cultural context in which it takes place (Bruner, 1996). As

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Bruner and others have argued, all of us, including teachers, have implicit theories, or “folk theories,” of how children’s minds work. These folk theories are embedded in cultural construals of what it means to be an educated person; how one understands the role of innate ability, effort, or luck in learning; and the like. Therefore, we need to have a very strong understanding of what these folk theories are as we continue to conduct large-scale cross-cultural comparisons of academic achievement. Following Bruner, we believe we need to examine what nations think they are doing and what their reasons are for doing it. In other words, folk pedagogies drive educational policy and practice, and we need to understand these if we are to be able to draw reasonable and pragmatic conclusions from cross-national comparisons of academic achievement. Indeed, a culture’s socialization goals shape its pedagogy. What a culture defines and requires of its citizens shapes what they are taught and the ways in which they are taught (Cole & Scribner, 1973; Spiro, 1993). Indeed, Bruner (1990) has eloquently argued that we must place culture in a central role in the study of human development. Because each of us develops in a culture, we cannot hope to understand the human psychology at the individual level. Each of us is an active participant in our culture, through which our understandings evolve. In addition, meaning making is negotiated in culture—“By virtue of participation in culture, meaning is rendered public and shared” (p. 12). Finally, a folk theory of mind is a very powerful influence on individual and collective meaning making. In Japan, for example, some parents identify the ability to endure hardship as a quality they wish to foster in their children as they grow (Lee, 1987). The ability to endure hardship is discussed in the national school policy in the following way: “[I]t is desirable that, in the lower grades, one should learn to bear hardship, and in the middle grades, to persist to the end with patience, and in the upper grades, to be steadfast and accomplish goals undaunted by obstacles or failures” (White, 1987, p. 17). This concern with the development of resiliency is ongoing. For example, college entrance requirements in Japan are grueling and arduous. Yet those who succeed are not said to be the “smartest”; they are believed to have the strongest will and character. According to White and LeVine (1987), a major goal of child rearing in Japan is to encourage children to be “committed to and positively engaged in disciplined effort” (p. 59). The child-rearing beliefs of many Japanese parents illustrate a commitment to fostering strengths of character that are essential for school success. For example, Japanese parents believe that character is molded by ki—the will to live; tamashi—the determination to overcome obstacles; and seishin—the mental attitude that helps a person to embark on a task. Parents also believe that character is

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement shaped by experiences of hardship, endurance, effort, and sustained struggle. In contrast, the United States is a nation committed to the individual’s right to pursue happiness, as stated in the Constitution. Many parents and educators have become concerned with ensuring that children have high self-esteem (Elkind, 1988). Generally speaking, our “ego-ideal” is of a child who is intelligent, athletic, social, musical, and creative—in short, we value the child who is “well-rounded” (Elkind, 1994; Kagan, 1989). The response of many educators and psychologists to the societal and economic upheavals that marked the past three decades has been to place children’s salvation in high self-esteem. Rich or poor, the new thinking is that if we can get youngsters to feel better about themselves, we can chip away at the problems that threaten their development into healthy and productive citizens. Many parents want their children to develop their skills in many domains, including those outside of school (Bempechat, 2000). These are but two examples of contrasting cultural models that are by no means characteristic of all parents in Japan and the United States. Indeed, these notions of enduring hardship and fostering self-esteem are contested within each society. Further, we cannot know the extent to which these models play any role in the achievement differences observed between Japanese and American students. At the same time, however, these different models illustrate how culture can guide pedagogical beliefs and goals. The Differential Meanings of Achievement Beliefs As mentioned earlier, cultural psychologists and psychological anthropologists agree that culture guides socialization practices, including those related to education (Roopnarine & Carter, 1992; Serpell & Hatano, 1997). An enduring concern for cross-cultural researchers is the differential meaning that students, parents, and teachers bring to the same or similar educational concepts. For example, much has been made of Japanese students’ adherence to effort as a means to ensure school success, and American students’ beliefs in innate ability as the driving force behind achievement (Stevenson et al., 1993). We do not know, however, what these concepts mean to the Japanese and U.S. students who have been studied. Nor do we know how these students, within their own cultural group, may vary in the meanings they bring to achievement cognitions such as effort and ability. We have argued elsewhere that this lack of attention to meaning making between and within cultures has led some researchers to draw generalizations about nations’ performances in cross-national assessments (Bempechat & Drago-Severson, 1999).

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement goals of development that are valued in different cultures, as well as the varied means used to attain them. For example, it is a common occurrence in some, but not all, cultures for children to accompany an adult to a grocery store, assist in the selection of items from a list, and help put purchases away in cupboards. Such a task emphasizes sorting and categorizing skills and reveals how adults teach children to do an essential household task. It would be inappropriate, however, to conduct this kind of ethnographic study in different nations, given that sorting and categorizing skills may not be scaffolded universally via a grocery shopping trip. While it is the case that cultural models of learning and achievement are affected by the global adoption of certain structures, such as mass education (Meyer, Ramirez, & Soysal, 1992), implicit in Rogoff’s argument is that a culture would not promote a cognitive outcome that had no use in that culture. Just as “necessity is the mother of invention,” we train our youth toward the cognitive skills they will need to survive in their culture. Thus, if we continue to focus on only a small portion of the range of cognitive goals for which people around the world strive, we will have an incomplete theory of human cognitive development. Therefore, cognitive psychologists are quite interested in observing both the goals of development and the strategies we use to help children reach those goals. Only then will we have a more complete sense of the cognitive skills we are capable of, and the myriad of ways for attaining proficiency in those skills. Relatedly, cultural psychologists would want to know how students transfer knowledge of mathematics acquired formally through schooling to contexts in which they have to apply their knowledge. They would also want to understand the extent to which students can be flexible in their application of mathematical principles. In a certain way, this echoes the concern of mathematics educators, who have been distressed over the enduring tendency of many students to view mathematics as a domain that requires no creativity and in which success can be gained through rote memorization (see Cobb et al., 1991). The result is that many students are uncomfortable with mathematics problems when they diverge even slightly from problems they have previously encountered. Researchers in the field have argued that students need to be able to develop a “deep conceptual understanding” of mathematics in order to become comfortably numerate (Lampert, 1990). One way to do this is to teach in ways that foster a more constructive view of the domain (Pirie & Kieren, 1992). In sum, comparative cognitive psychologists would seek a greater understanding of learning in culture and context than is currently available in cross-national studies, such as TIMSS. We realize this involves conducting a series of smaller ethnographic studies of situated learning in many nations, a task that is formidable indeed. Yet we need to bear in

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement mind that the overall goal in this work is to gain a deeper understanding of thought in action. It is important, therefore, to complement large-scale surveys of mathematics achievement with smaller case-based studies of the practical applications of knowledge in everyday life. This may concern survey researchers, who are accustomed to generating large enough samples for statistical generalizability. Yet, as Robin Alexander, the noted scholar of comparative pedagogy has argued, there is an important distinction between statistical and “cultural” generalizability. Statistical Versus Cultural Generalizability Alexander (2000) recently argued that it is possible to derive valid and reliable “cultural” generalizations about national educational thought, practice, and outcomes from qualitative field studies centered in a small number of sites of educational practice. Alexander has contrasted “statistical” and “cultural” generalization, and views the claim for “cultural generalizability” as resting on two conditions. First, he argues that researchers need to accept the “proposition that the culture in which the schools in a country, state or region are located, and which teachers and pupils share, is as powerful a determinant of the character of school and classroom life as are the unique institutional “dynamics, local circumstances and interpersonal chemistries which make one school, or classroom different from another.” Alexander adds, “the research methods used [must be] sufficiently searching and sensitive to probe beyond the observable moves and counter-moves of pedagogy to the values and meanings which these embody.” (Alexander, 2000, p. 266, italics added). Cross-national research can indeed yield cultural generalizations by adhering to guidelines Alexander has posited. First, investigators can operate under the working assumption that the beliefs, commitments, and practices of those we are researching are influenced by extra-personal systems of belief, commitment, and action with which they are acquainted. Second, mixed methods of inquiry can allow researchers to become familiar with the potentially influential wider systems of beliefs, commitment, and normative practices with which our participants may be familiar. Third, fine-grained qualitative field work can be used to uncover the beliefs, norms, and commitments and understand the rationale of practices amongst the participants. Fourth, researchers can consider relations between potentially influential wider systems of beliefs and commitment and normative practices and any system found in the beliefs, commitments and rationales of practice amongst those they are researching.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Learning More from the Current Cross-National Data One of the misuses of cross-national comparisons that has emerged since large-scale cross-national comparisons were first conducted in the early 1960s has been the rank ordering of countries. The value of cross-national comparisons lies not in the ranking of nations to see which educational systems are superior to others, but rather to investigate why some countries differ in their achievement levels. The IEA, established in 1959, strongly emphasized that “horse race” analyses that rank ordered countries were just first and necessary steps toward understanding cross-national differences (Keeves, 1995). The move from educational tourism (pre-1960), where visitors from one country formally observed the educational systems of foreign countries and offered rich descriptions of teaching practices, students’ behavior and learning opportunities, and school structures, to large-scale international comparisons (post-1960) was only feasible because sophisticated methods of inquiry were developed. Prior to the 1960s, educational researchers did not have the tools to make such cross-national comparisons, but improved methods in survey design and statistical analyses such as inference statistics and sampling, coupled with technological advances in the use of computers for data analyses, opened the door for cross-national comparative research. Interestingly enough, cross-national data, in one form or another, moved from detailed ethnographic accounts across countries and the use of qualitative inquiry to a somewhat sole reliance on quantitative methods, given the introduction of internationally valid standards of such inquiry by the IEA. Although quantitative methods allowed cross-national data to be compared on equivalent measures, such large-scale inquiry must be coupled with qualitative methods, as Husen (1967) has argued, in order to give us a rich portrait of the factors associated with educational achievements. Cross-national comparisons offer an awareness to nations that they “cannot borrow wholesale from each other [but rather] by looking at the other systems one can get a perspective that provides insights into how one could go about improving one’s own system” (Keeves, 1995, p. 169). The purpose of cross-national data should not solely be to list factors that are related to educational achievement. Such research also should be focused on the processes, in addition to the products, involved in educational systems in order to develop a greater understanding of how educational systems work. For example, how do teachers, students, parents, administrators, and others in the educational system make meaning of their experiences, and how does this influence issues of learning and teaching? The goal of cross-national research should not only be to construct models of teaching and learning processes in school, but it should

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement also include the testing of these models against observed data in order to confirm or reject model structures. Furthermore, we, like others (Husen, 1967; Keeves, 1995), firmly believe that the time is long overdue for an integration of more sophisticated statistical methods, such as multilevel analyses along with qualitative methods for in-depth inquiry and analyses of how individuals make meaning of their experiences within educational systems. In all likelihood, there is probably more variation within nations than between nations. When we focus exclusively on between-nation differences, we fail to detect the rich variation that exists within a nation of learners. Although it is interesting to look cross-nationally at achievement, it is difficult to simply look at the average achievement of a nation. Comparing averages can begin to give us a sense of the variation in achievement between nations, but it will not reveal whether there is substantial variation within a nation. This question can be easily addressed using current data collected by such cross-national examinations as TIMSS. The data gathered by TIMSS can be conceptualized as hierarchical in nature—students were sampled within countries. The question then becomes, given that there is variation in test scores overall, how much of this variation is attributable to differences between countries (Level 2) and how much is attributable to differences in students (Level 1). Intraclass correlations (calculated by fitting a multilevel model with no predictors) would help address this question. Although it is interesting to know that Country X has higher test scores, and that students in Country X spend more time in class, it also would be interesting to show that there is a correlation between these variables across countries. Although neither correlation nor regression can establish causal relationships (e.g., more time in class causes higher test scores), they would at least allow us to begin to determine if and where these variables covary. IEA has begun to conduct such multilevel analyses of cross-national data (Keeves, 1995) due to the expansion of statistical methods that allow for hierarchical linear modeling, for example, and there is a call for more of the same in future cross-national comparisons. The Future of Cross-National Research I have no objection in principle to creating better measuring instruments in order to find out how well our students are doing in science, in mathematics, in literature, in reading.... Of course we need standards and resources to make our schools work well in solving the myriad tasks they face. But resources and standards alone will not work. We need a surer sense of what to teach to whom and how to go about teaching it in such a way that it will make those taught more effective, less alienated,

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement and better human beings.... What we need is a school reform movement with a better sense of where we are going, with deeper convictions about what kind of people we want to be. (Bruner, 1996, pp. 117-118) As Bruner points out, we need to go beyond cataloging the different ways in which education is delivered and move toward a deeper sense of the purposes of education. We believe that the current data on cross-national achievement and school systems are insufficient for us to make informed recommendations about how to improve our own educational system. We, and others, have argued that we cannot simply adopt the educational methods of those nations whose students are performing well. We need to understand how education is viewed, valued, and understood by the citizens of the culture whose outcomes we admire. If we are to adopt some of their methods, we must do so in a culturally sensitive way. Without information about the meaning of education, we cannot begin to translate any of the methods used by other nations in a culturally sensitive way—the exercise would be akin to translating a text without knowing the language in which it has been written. Integrating Survey and Qualitative Methods of Inquiry Cross-national survey research can lead us to pertinent qualitative research questions. Qualitative inquiries, in turn, can shape the questions we attend to in future large-scale quantitative surveys. We believe that it is time for qualitative methods of inquiry to once again be present in cross-national data, as they were in the pre-1960s era. When integrated with surveys, the more sophisticated methods of qualitative inquiry that are now at our disposal can better serve our purposes of taking an indepth look at other countries’ educational systems so that we may gain insights about improving our own system. A promising approach is found in Li’s (2000) comparison of Chinese and U.S. conceptions of learning using prototype research methods. In general, prototype research methods require that the researcher interview individuals to elicit words and phrases that are used to describe the domain in question. This elegant method uses emic concepts, yet employs sophisticated quantitative analyses to understand the data. In this case, Li obtained a cultural prototype of “intelligence through a five-step process in which she asked increasingly larger groups of American and Chinese students to relate any words or phrases related to learning. This culminated in a hierarchical cluster analysis of groups of ideas that represented the students’ conceptions of learning. Li found little conceptual overlap between Chinese and American conceptions of learning. American conceptions of learning did not include any words or phrases related to actual achievement. For the U.S.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement students, the major focus is on thinking, which can be considered a hypercognized domain, one that is well developed in U.S. society (Levy, 1973). In contrast, Chinese students used words and phrases that related achievement as representing breadth and depth of knowledge, extraordinary ability, and the unity of moral development and knowledge. In short, American students appear to be hypercognized for the process of learning while the Chinese students appear to be hypercognized about attitudes for learning. Li’s contribution to our sociocultural understanding of achievement and motivation is significant, because she rises to the call for combined methods (e.g., Shweder, 1997). Yet Li goes one step further, in the sense that her sophisticated analyses of data were derived entirely from emic or qualitative understandings of learning in each culture. A second positive approach is found in Stigler and Hiebert’s (1999) use of qualitative methods to analyze the TIMSS video study, which included data on classroom learning and teaching. One of the most interesting findings that emerged from this analysis is that teaching, and not teachers, is a critical factor in the teaching and learning of mathematics when comparing Japan and the United States. American mathematics teaching tends to focus on procedural skills rather than conceptual understandings. A second finding is that among U.S. and Japanese schooling processes, there are large differences in teaching between cultures, but not within cultures. That is, in comparison to the within-culture variation in teaching, there are much larger gaps between different countries in terms of teaching processes. Along this vein, Stigler and Hiebert took advantage of qualitative data to observe and document that teaching is very much a cultural activity that has embedded in it notions of learning that stem from cultural beliefs and practices. In their analyses, they state that teaching is very difficult to change given the cultural underpinnings. Such insights would have been very difficult, nearly impossible, to arrive at if qualitative methods were not used for such inquiry. Conclusion We believe that researchers conducting cross-national investigations need to be aware of their own culture and context and the extent to which it influences both their investigations and their interpretations of their findings. As Sir Michael Sadler stated in a now famous lecture delivered in Guildford: In studying foreign systems of education, we should not forget that the things outside the schools matter even more than the things inside the

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement schools, and govern and interpret the things inside. We cannot wander at pleasure among the educational systems of the world, like a child strolling through a garden, and pick off a flower from one bush and some leaves from another, and then expect that if we stick what we have gathered into the soil at home, we shall have a living plant. (Sadler, 1979, p. 49) REFERENCES Alexander, R. (2000). Culture and pedagogy. London: Blackwell. Ames, C., Ames, R., & Felker, D. W. (1977) Effects of competitive reward structure and valence of outcome on children’s achievement attributions. Journal of Educational Psychology 69(1), 1-8. Ames, C., & Archer, J. (1987). Mothers’ beliefs about the role of ability and effort in school learning. Journal of Educational Psychology, 71, 409-414. Beaton, A., Mullis, I., Martin, M., Gonzalez, E., Kelly, D., & Smith, T. (1996). Mathematics achievement in the middle school years: IEA’s Third International Mathematics and Science Study. Chestnut Hill, MA: Boston College, Center for the Study of Testing, Evaluation, and Educational Policy. Bempechat, J. (1998). Against the odds: How ‘at risk’ students exceed expectations. San Francisco: Jossey-Bass. Bempechat, J. (2000). Getting our kids back on track: Educating children for the future. San Francisco: Jossey-Bass. Bempechat, J., & Abrahams, S. (1999). “You can’t oppress yourself”: Conceptions of achievement and opportunity in post-apartheid South Africa. Teachers College Record, 100, 841-859. Bempechat, J., & Drago-Severson, E. (1999). Cross-national differences in academic achievement: Beyond etic conceptions of children’s understandings. Review of Educational Research, 69, 287-314. Bempechat, J., Graham, S., & Jimenez, N. (1999). The socialization of achievement in poor and minority students: A comparative study. Journal of Cross-Cultural Psychology, 30, 139-158. Bempechat, J., London, P., & Dweck, C. S. (1991). Children’s conceptions of ability in major domains: An interview and experimental study. Child Study Journal, 21(1), 11-35. Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press. Bruner, J. (1996). The culture of education. Cambridge, MA: Harvard University Press. Cobb, P., Wood, T., Yackel, E., Nicholls, J., Wheatley, G., Trigatti, B., & Perlwitz, M. (1991). Assessment of a problem-centered second-grade mathematics project. Journal for Research in Mathematics Education, 22, 3-29. Cole, M. (1988). Cross-cultural research in the socio-historical tradition. Human Development, 31, 137-151. Cole, M. (1996). Cultural psychology: A once and future discipline. Cambridge, MA: Harvard University Press. Cole, M., & Scribner, S. (1973). Culture and thought. New York: John Wiley & Sons. Cole, M., & Wertsch, J. V. (1996). Beyond the individual-social antimony in discussions of Piaget and Vygotsky. Human Development, 39, 250-256. Covington, M. V., & Omelich, C. L. (1979). Effort: The double-edged sword in school achievement. Journal of Educational Psychology, 71(2), 169-182.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Dweck, C. (1999). Self-theories: Their role in motivation, personality and development. New York: Psychology Press. Dweck, C. S., & Bempechat, J. (1983). Children’s theories of intelligence: Consequences for learning. In S. G. Paris, G. M. Olson, & H. W. Stevenson (Eds.), Learning and motivation in the classroom. Hillsdale, NJ: Lawrence Erlbaum Associates. Eccles, J. (1993). School and family effects on the ontogeny of children’s interests, self-perceptions, and activity choices. In J. Jacobs (Ed.), Nebraska Symposium on Motivation: Vol. 40. Developmental perspectives on motivation (pp. 145-208). Lincoln: University of Nebraska Press. Elkind, D. (1988). The hurried child: Growing up too fast too soon. New York: Addison-Wesley. Elkind, D. (1994). Ties that stress: The new family imbalance. Cambridge, MA: Harvard University Press. Elliott, J.G., Hufton, N.R., Hildreth, A., & Illushin, L. (1999). Factors influencing educational motivation: A study of attitudes, expectations and behaviour of children in Sunderland, Kentucky and St. Petersburg. British Educational Research Journal, 25, 75-94. Elliott, J.G., Hufton, N., Illushin, L., & Willis W. (2001). The kids are doing all right: Differences in parental satisfaction, expectation and attribution in St. Petersburg and Sunderland, Kentucky. Cambridge Journal of Education, 31, 179-204. Greenfield, P. (1994). Independence and interdependence as developmental scripts: Implications for theory, research, and practice. In R. Cocking and P. Greenfield (Eds.), The cross-cultural roots of minority child development (pp. 1-37). Hillsdale, NJ: Lawrence Erlbaum Associates. Harkness, S., & Super, C. (1992). Parental ethnotheories in action. In I. E. Sigel, A. V. McGillicuddy-DeLisi, & J. J. Goodnow (Eds.), Parental belief systems: The psychological consequences for children (2nd ed., pp. 373-391). Hillsdale, NJ: Lawrence Erlbaum Associates. Haste, H. (1994). The sexual metaphor. Cambridge, MA: Harvard University Press. Hess, R., & Azuma, H. (1991). Cultural support for learning. Educational Researcher, 20(6), 2-8. Hess, R. D., Chih-Miei, C., & McDevitt, T. M. (1987). Cultural variations in family beliefs about children’s performance in mathematics: Comparisons among People’s Republic of China, Chinese-American, and Caucasian-American families. Journal of Educational Psychology, 79, 179-188. Holloway, S. (1988). Concepts of ability and effort in Japan and the United States. Review of Educational Research, 58, 327-345. Holloway, S. D. (2000). Contested childhoods: Diversity and change in Japanese preschools. London: Routledge. Holloway, S. D., Kashiwagi, K., Hess, R. D., & Azuma, H. (1986). Causal attributions by Japanese and American mothers about performance in mathematics. International Journal of Psychology, 21, 269-286. Holloway, S., & Minami, M. (1996). Production and reproduction of culture: The dynamic role of mothers and children in early socialization. In D. Shwalb & B. Shwalb (Eds.), Japanese childrearing: Two generations of scholarship (pp. 164-75). New York: Guilford Press. Hufton, N. R., & Elliott, J. G. (2000). Motivation to learn: The pedagogical nexus in the Russian school: Some implications for transnational research policy borrowing. Educational Studies, 26(1), 115-136. Hufton, N. R., Elliott, J. G., & Illushin, L. (in press). Motivation to learn: The elusive role of culture: Qualitative accounts from three countries. British Educational Research Journal, 27(5). Husen, T. (1967). International study of achievement in mathematics. New York: John Wiley & Sons.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Kagan, J. (1989). Unstable ideas: Temperament, cognition, and self. Cambridge, MA: Harvard University Press. Keeves, J. P. (1995). The case for international comparisons. In J. J. Lane (Ed.), Ferment in education: A look abroad (pp. 169-189). Chicago: University of Chicago Press. Kitayama, S. (2000). Collective construction of the self and social relationships: A rejoinder and some extensions. Child Development, 71, 1143-1146. Kitayama, S., Markus, R., Matsumoto, H., & Noraskkunkit, V. (1997). Individual and collective processes in the construction of the self: Self-enhancement in the United States and self-criticism in Japan. Journal of Personality and Social Psychology, 72, 1245-1267. Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching. American Educational Research Journal, 27, 29-63. Lave, J. (1977). Tailor-made experiments and evaluating the intellectual consequences of apprenticeship training. The Quarterly Newsletter of the Institute for Comparative Human Development, 1, 1-5. Lave, J. (1988). Cognition in practice. Cambridge, England: Cambridge University Press. Lawrence-Lightfoot, S., & Davis, J. H. (1997). The art and science of portraiture. San Francisco: Jossey-Bass. Lee, C. (1987). China’s integrated approach to child care and socialization. Paper presented at the Society for Research in Child Development, Baltimore. LeVine, R. (1977). Child rearing as cultural adaptation. In P. Leiderman, S. Tulkin, & A. Rosenfeld (Eds.), Culture and infancy: Variations in the human experience (pp. 15-27). New York: Academic Press. Levy, R. I. (1973). Tahitians: Mind and experience in the Society Islands. Chicago: University of Chicago Press. Li, J. (2000). What do U.S. and Chinese college students think learning is? Exploring learning models of American and Chinese. Paper presented at the 16th biennial meeting of the International Society for the Study of Behavioral Development, Beijing, China. Li, J. (in press). Chinese conceptualization of learning. Ethos. Markus, H., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98, 224-253. Markus, H. R., Mullally, P. R., & Kitayama, S. (1997). Selfways: Diversity in modes of cultural participation. In U. Neisser & D. A. Jopling (Eds.), The conceptual self in context: Culture, experience, self-understanding (pp. 13-61). Cambridge, England: Cambridge University Press. Marsh, H. W., & Shavelson, R. J. (1985). Self-concept: Its multifaceted, hierarchical structure. Educational Psychologist, 20, 107-125. McKnight, C., Crosswhite, F., Dossey, J., Kifer, E., Swafford, J., Travers, K., & Cooney, T. (1987). The underachieving curriculum: Assessing U.S. school mathematics from an international perspective. Champaign, IL: Stipes. Meyer, J., Ramirez, F., & Soysal, Y. (1992). World expansion of mass education, 1870-1980. Sociology of Education, 65, 128-149. Miller, P. (1996). Instantiating culture through discourse practices: Some personal reflections on socialization and how to study it. In R. Jessor, A. Colby, & R. Schweder (Eds.), Ethnography and human development: Context and meaning in social inquiry (pp. 183-204). Chicago: University of Chicago Press. Mouer, R., & Sugimoto, Y. (1986). Images of Japanese society: A study in the social construction of reality. London: Routledge and Kegan Paul. Mouer, R. E., & Sugimoto, Y. (1990). Images of Japanese society: a study in the social construction of reality. London; New York: Kegan Paul International.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Munroe, R., & Munroe, R. (1997). A comparative anthropological perspective. In J. Berry, Y. Poortinga, & J. Pandey (Eds.), Handbook of cross-cultural psychology: Vol. 1. Theory and method. Boston: Allyn & Bacon. Nicholls, J., & Hazzard, S. (1993). Education as adventure: Lessons from second grade. New York: Teachers College Press. Nicholls, J. G. (1978). The development of the concepts of effort and ability, perception of own attainment, and the understanding that difficult tasks require more ability. Child Development, 49, 800-814. Nicholls, J. G. (1989). The competitive ethos and democratic education. Cambridge, MA: Harvard University Press. Nicholls, J. G., Cheung, P. C., Lauer, J., & Patachnick, M. (1989). Individual differences in academic motivation: Perceived ability, goals, beliefs, and values. Learning and Individual Differences, 1, 63-84. Nicholls, J. G., Cobb, P., Wood, T., Yackel, E., & Patachnick, M. (1990). Assessing students’ theories of success in mathematics: Individual and classroom differences. Journal for Research in Mathematics Education, 21(2), 109-122. Ogbu, J. (1986). The consequences of the American caste system. In U. Neisser (Ed.), The school achievement of minority children: New perspectives. Hillsdale, NJ: Lawrence Erlbaum Associates. Peak, L. (1991). Learning to go to school in Japan. Berkeley: University of California Press. Pirie, S., & Kieren, T. (1992). Creating constructivist environments and constructing creative mathematics. Educational Studies in Mathematics, 23, 505-528. Quihuis, G., Bempechat, J., Jimenez, N. V., & Boulay, B. A. (in press). Identifying students’ implicit theories of intelligence: A study of meaning making in context. In J. Bempechat & J. Elliott (Eds.), Achievement motivation in culture and context: Understanding children’s learning experiences (Vol. 96 [Summer]). San Francisco: Jossey-Bass. Reese, L., Balzano, S., Gallimore, R., & Goldenberg, C. (1995). The concept of “educación”: Latino family values and American schooling. International Journal of Educational Research, 23, 57-81. Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in a social context. New York: Oxford University Press. Roopnarine, J., & Carter, D. B. (1992). The cultural context of socialization: A much ignored issue! In J. Roopnarine & D. B. Carter (Eds.), Annual advances in applied developmental psychology: Vol. 5. Parent-child socialization in diverse cultures (pp. 245-251). Norwood, NJ: Ablex. Sadler, M. (1979). Selections from Michael Sadler: Studies in world citizenship. Liverpool, England: Dejall & Meyorre. Schurmans, M., & Dasen, P. (1992). Social representations of intelligence: Cote d’Ivoire and Switzerland. In M. V. Cranach, W. Doise, & G. Mugny (Eds.), Social representations and the social bases of knowledge (pp. 144-152). Lewiston, NY: Hogrefe & Huber. Scribner, S. (1983/1992). Mind in action: A functional approach to thinking. The Quarterly Newsletter of the Laboratory of Comparative Human Cognition, 14, 103-110. Serpell, R., & Hatano, G. (1997). Education, schooling, and literacy. In J. Berry, P. Dasen, & T. S. Saraswathi (Eds.), Handbook of cross-cultural psychology: Vol. 2. Basic processes and human development (pp. 339-375). Norwood, NJ: Ablex. Shore, B. (1996). Culture in mind: Cognition, culture, and the problem of meaning. New York: Oxford University Press. Shweder, R. (1990). Cultural psychology—What is it? In J. Stigler, R. Shweder, & G. Herdt (Eds.), Cultural psychology: Essays on comparative human development (pp. 1-43). New York: Cambridge University Press. Shweder, R. (1997). The surprise of ethnography. Ethos, 25, 152-163.

OCR for page 115
Methodological Advances in Cross-National Surveys of Educational Achievement Spiro, M. (1993). Is the Western conception of the self “peculiar” within the contexts of world cultures? Ethos, 21, 107-153. Sternberg, R. J. (1990). Prototypes of competence and incompetence. In R. J. Sternberg & J. Kolligian, Jr. (Eds.), Competence considered (pp. 117-145). New Haven, CT: Yale University Press. Stevenson, H., Chen, C., & Lee, S. (1993). Mathematics achievement of Chinese, Japanese, and American children: Ten years later. Science, 259, 53-58. Stevenson, H., & Lee, S. (1990). Contexts of achievement: A study of American, Chinese, and Japanese children. Monographs of the Society for Research in Child Development, 55 (12, Serial No. 221). Stevenson, H., & Stigler, J. (1992). The learning gap. New York: Simon and Schuster. Stevenson, H. W., Lee, S., & Stigler, J. W. (1986). Mathematics achievement of Chinese, Japanese and American children. Science, 231, 693-699. Stigler, J., & Hiebert, J. (1999). The teaching gap. New York: Free Press. Stigler, J. W., & Perry, M. (1988). Mathematics learning in Japanese, Chinese, and American classrooms. In G. B. Saxe & M. Gearhart (Eds.), Children’s mathematics (Vol. 41). San Francisco: Jossey-Bass. Stigler, J. W., & Perry, M. (1990). Mathematics learning in Japanese, Chinese, and American classrooms. In J. W. Stigler, R. A. Shweder, & G. Herdt (Eds.), Cultural psychology: Essays on comparative human development (pp. 328-353). Cambridge, England: Cambridge University Press. Tobin, J., Wu, Y., & Davidson, D. (1989). Preschool in three cultures. New Haven, CT: Yale University Press. Tudge, J., Hogan, D., Snezhkova, I., Kulakova, N., and Etz, K. (2000). Parents’ child-rearing values and beliefs in the United States and Russia: The impact of culture and social class. Infant and Child Development, 9, 105-121. Vygotsky, L. (Ed.). (1978). Mind in society. Cambridge, MA: Harvard University Press. Weiner, B. (Ed.). (1972). Theories of motivation: From mechanism to cognition. Chicago: Markham. Weiner, B. (1984). Principles for a theory of student motivation and their application within an attributional framework. In R. E. Ames & C. Ames (Eds.), Research on motivation in education: Student motivation. New York: Harcourt Brace Jovanovich. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548-573. Weiner, B., Russell, D., & Lerman, D. (1979). The cognition-emotion process in achievement-related contexts. Journal of Personality and Social Psychology, 37, 1211-1220. White, M. (1987). The Japanese educational challenge. New York: Free Press. White, M., & LeVine, R. (1987). What is an “ii ko” (good child)? In H. Stevenson, H. Azuma, & K. Hakuta (Eds.), Child development and education in Japan (pp. 55-62). New York: Freeman.