Page 1

EXECUTIVE SUMMARY

Video technology has evolved into a powerful methodological tool for international comparative research in education. It provides a lens through which to view and record classroom practices. International video studies generate data that can create audiovisual glossaries of teaching strategies and skills that expand the repertoire of possible teaching approaches. This audiovisual glossary provides a reference point for teaching practices that are difficult to describe in words, particularly when foreign languages and cultural contexts create barriers to interpretation and communication. Carefully selected videotapes can introduce teachers to a variety of practices, to help them to rethink what they might otherwise take for granted, to consider the pros and cons of different approaches, and, in general, to become more reflective practitioners.

International videotapes serve as a record of teaching in a particular time and place, and make that teaching available for multiple reexaminations; they facilitate collaboration among researchers from diverse perspectives that traditional forms of data collection limit in cross-national studies. Recent advances in storing and coding large volumes of footage permit researchers to move quickly through digitized videotapes for specific events or words. Ancillary data, such as teacher questionnaires and student work, can be stored with videotaped footage to augment the video data with contextually rich background data. Coded video data can help track the myriad interactions within a classroom, such as the amount of time spent in teacher-student interactions. Quantitative analysis of coded images may clarify broad trends and variations, and qualitative analysis can facilitate deeper understanding of quantitative phenomena, such as how teacher-student interactions take place. Archived video data can be reexamined in the future by researchers with new research questions.

Video technology offers a number of important potential benefits to researchers and policy makers interested in international comparative research. However, a number of practical and methodological issues remain to be addressed, including sample sizes and the confidentiality of research participants. In light of the potential benefits and recognizing the unresolved issues, the Board on International Comparative Studies in Education (BICSE) offers four recommendations to researchers, funding agencies, and policy makers.

Recommendation 1: The international comparative education research community should pursue projects that appropriately use video technology as a research tool.

Such research will help scholars build a body of work that can contribute fundamental new understandings of educational practices, while at the same time resolving some of the important methodological challenges discussed in this report.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Page 1 EXECUTIVE SUMMARY Video technology has evolved into a powerful methodological tool for international comparative research in education. It provides a lens through which to view and record classroom practices. International video studies generate data that can create audiovisual glossaries of teaching strategies and skills that expand the repertoire of possible teaching approaches. This audiovisual glossary provides a reference point for teaching practices that are difficult to describe in words, particularly when foreign languages and cultural contexts create barriers to interpretation and communication. Carefully selected videotapes can introduce teachers to a variety of practices, to help them to rethink what they might otherwise take for granted, to consider the pros and cons of different approaches, and, in general, to become more reflective practitioners. International videotapes serve as a record of teaching in a particular time and place, and make that teaching available for multiple reexaminations; they facilitate collaboration among researchers from diverse perspectives that traditional forms of data collection limit in cross-national studies. Recent advances in storing and coding large volumes of footage permit researchers to move quickly through digitized videotapes for specific events or words. Ancillary data, such as teacher questionnaires and student work, can be stored with videotaped footage to augment the video data with contextually rich background data. Coded video data can help track the myriad interactions within a classroom, such as the amount of time spent in teacher-student interactions. Quantitative analysis of coded images may clarify broad trends and variations, and qualitative analysis can facilitate deeper understanding of quantitative phenomena, such as how teacher-student interactions take place. Archived video data can be reexamined in the future by researchers with new research questions. Video technology offers a number of important potential benefits to researchers and policy makers interested in international comparative research. However, a number of practical and methodological issues remain to be addressed, including sample sizes and the confidentiality of research participants. In light of the potential benefits and recognizing the unresolved issues, the Board on International Comparative Studies in Education (BICSE) offers four recommendations to researchers, funding agencies, and policy makers. Recommendation 1: The international comparative education research community should pursue projects that appropriately use video technology as a research tool. Such research will help scholars build a body of work that can contribute fundamental new understandings of educational practices, while at the same time resolving some of the important methodological challenges discussed in this report.

OCR for page 1
Page 2 Recommendation 2: The international comparative education research community should support not only large-scale studies that make use of video technology, such as the Third International Mathematics and Science Study (TIMSS), but also other kinds of video-based research. Research studies with a variety of sizes, goals, and methodologies can benefit from the application of video technology in important ways that have the potential to stimulate progress in both methodological and substantive issues. Recommendation 3: The international comparative education research community should undertake initiatives, such as the support of a working group, to help clarify and develop solutions to the privacy and confidentiality issues in using video technology in such research. The very nature of video technology creates problems for and challenges to confidentiality that cannot be easily handled by simple extrapolation from existing procedures for other research methods. Thus, serious and focused consideration of confidentiality issues in video research, especially in international settings, is needed to develop creative solutions and to foster discussion and consensus building around such solutions. Recommendation 4: The international comparative education research community should undertake initiatives, such as the support of a working group, to explore the creation of a video archive or archives for international comparative research in education. Video technology can be of significant benefit in expanding the accessibility and application of comparative research and in serving as a unique historical resource. Given the substantial costs associated with both international comparative education research and video technology, wide distribution and archiving will contribute to its cost effectiveness.

OCR for page 1
Page 3 INTRODUCTION Throughout the history of educational research, scholars have used a variety of methods to study classroom interaction in order to analyze the complexities of teaching and learning—ethnographic case studies, interviews, and questionnaires to analyze content, pedagogical strategies, classroom cultures, and teacher-student interactions. More recently, the potential contribution of film and video technologies have expanded the repertoire of tools to provide rich qualitative and quantitative data for analysis of classroom environments (Bogdan and Biklen, 1992; Jordan and Henderson, 1995; Stigler, Gallimore, and Hiebert, 2000). As the technology advances rapidly, however, scholars must confront fundamental issues about both its possibilities and limitations in educational research. The Board on International Comparative Studies in Education (BICSE) held a workshop to consider the benefits and complexities of using video technology in comparative education research. Participants included scholars with expertise in contemporary ethnography, teacher education, cognitive science, international comparative education, and videography in educational research and teacher professional development (see the Appendix for the workshop agenda and participants). BICSE invited several participants to write brief responses to the following set of targeted questions on the use of video technology in comparative educational research and professional development: What are the strongest arguments for and against the use of video technology in international comparative studies of education? If you were asked to advise the planners of such a study, what recommendations would you make about its design? How should it be conducted? How should results be analyzed and disseminated? How would you address methodological issues, such as the ethics of the data collection and handling? What particular challenges or opportunities would conducting such a study internationally pose? Can you point to studies—not necessarily comparative or largescale ones—that might inform our thinking about the use of video technology? Responses to these questions served as a starting point for a day-long discussion of the advantages, barriers, and possible future directions for the use of video technology in international comparative research. BICSE structured the workshop around three particular uses of video technology. One discussion focused on the use of videotapes to systematically collect and aggregate images of classrooms in order to record and portray trends or patterns of classroom practice across different countries. A second discussion explored the use of videotaped images to support the professional development of teachers to improve classroom practice. The third discussion considered efforts

OCR for page 1
Page 4 to link the variation in teaching practices captured on videotape to achievement differences identified within and across countries. The workshop discussions clearly illustrated that video technology has evolved into a powerful tool for use in international comparative education research. The workshop also generated rich discussions of a variety of both methodological and analytical questions that relate to the role video technology can play in such research. 1 Over the course of several meetings, the board explored further the issues raised during the workshop to synthesize lessons learned from the international comparative studies using video as a methodology. The board developed several conclusions and recommendations for researchers and policy makers regarding the use of video in future international comparative education research. This report presents highlights from the workshop discussions and from the subsequent board work on this topic; it is intended to provide an overview of the issues, not to provide specific methodological procedures for using international video. Video in international comparative research in education has lately received a great deal of attention, most notably in light of the public release of the TIMSS Videotape Classroom Study. The use of video in educational research has been evolving in many fields, from anthropology to qualitative research traditions in education, ethnomethodology, sociolinguistics, and interactional analyses. 2 The next section provides an overview of the historical context of video in international comparative research and therefore highlights selected works from international perspectives. 3 BRIEF HISTORICAL PERSPECTIVE ON INTERNATIONAL VIDEO RESEARCH Video technology is emerging as an important ethnographic research tool in the fields of educational anthropology and psychology. Ethnographers use a variety of methods to describe and interpret “events that occur within the life of a group, with special regard to the social structures and the behavior of the individuals with respect to their group membership . . . and the meaning of these for the culture of the group” (Taft, 1985:1729). Early fieldwork required researchers to observe and interview participants, to take copious notes during or 1Many issues raised in this workshop, such as the relationship between cross-cultural versus within culture studies, have been fundamental to comparative and cross-cultural research for many years (see, e.g., Campbell, 1961). 2For more detailed analysis of these qualitative traditions, see Erickson (1986, 1992), Jordan and Henderson (1995), and McDermott and Roth (1978). 3The workshop and board deliberation did not include discussions of the history of video in international comparative research. The board has added this useful overview as useful background for the reader.

OCR for page 1
Page 5 after the observations, and to translate their findings into written accounts. Cameras enabled ethnographers to expand their data collection efforts to record real-time images for subsequent detailed analysis (Henley, 1998). Anthropologists Margaret Mead and Gregory Bateson were pioneers in the use of film for ethnographic research. They first used cameras—both still and motion picture—in their work in Bali in 19361938. They used film to record “the types of non-verbal behavior for which there existed neither vocabulary nor conceptualized methods of observation” (de Brigard, 1995:26). For 2 years, Mead and Bateson lived in the mountains at Bajoeng Gede, filming and photographing family life in villages. We tried to use the still and the moving picture cameras to get a record of Balinese behavior, and this is a very different matter from the preparation of a “documentary” film or photographs. We tried to shoot what happened normally and spontaneously, rather than to decide upon the norms and then get the Balinese to go through these behaviors in suitable lighting (de Brigard, 1995:27). Mead and Bateson later spent 6 months collecting comparative data among the Iatmul in New Guinea. From the 25,000 still photographs and hundreds of hours of film footage, they prepared Balinese Character and edited several films in the Character Formation in Different Cultures Series for cross-cultural comparisons of behavior patterns, as in Bathing Babies in Three Cultures (de Brigard, 1995; Bateson and Mead, 1952). Mead and Bateson’s innovative use of film technology in Bali has been described as “by far the most significant ethnographic research use of visual media in the first half of this century” (Henley, 1998:44). Mead’s work in early childhood development was a precursor to the field of educational anthropology, which emerged in the middle of the twentieth century (Spindler and Spindler, 1992). Leading educational anthropologists such as George and Louise Spindler focused their ethnographic research on classrooms as cultural contexts. Their comparative work in two schools (in Schoenhausen, Germany and in Roseville, Wisconsin) was a groundbreaking use of video technology as both a means to collect cross-cultural classroom data and as “evocative stimuli” for later discussion about cultural differences (Spindler and Spindler, 1992). This long-range study examined the influence of culture on the role of the school in the preparation of children for an urbanizing environment and changing world. The Spindlers aimed to capture a more complete record of activities in the classrooms, playgrounds, and on field trips than had previously been possible. We filmed in Schoenhausen and in Roseville, and we showed the teachers, children, and administrators the films from both places. We conducted interviews about what they saw in their own classrooms and in those of the “other” and how they interpreted what they saw (Spindler and Spindler, 1992:80).

OCR for page 1
Page 6 The Spindlers coined the term “cultural screens” to describe the way viewers interpreted the images they saw of school life. In describing their research in Schoenhausen and Roseville, the Spindlers explained that the “greatest utility of films as ‘records’ is that we can ‘return’ to the classroom years later” (Spindler and Spindler, 1992:78). They described how reexamining the images recorded from 1977 to 1985 revealed new insights. One phenomenon, for example, that came to our attention through repeated viewings of the films was that despite great variations in the explicit aspects of teacher style in the management of classroom activity, all of the teachers in the Schoenhausen school were in constant charge of their classrooms. . . . Although they might take a position in the back or along the side of the room and seemingly be quite relaxed about what was going on, we saw that teachers were giving signals, sometimes as subtle as pursed lips or raised eyebrows, to reinforce or intervene in student behavior (Spindler and Spindler, 1992:78). The Spindlers described the value of recorded images to educational anthropologists in terms of two important issues: archiving data for secondary analysis at a later time and stimulating reflective thinking by viewers. The use of film and video technology has enriched qualitative descriptions of school environments as cultural contexts; see Box 1. By the end of the 1980s, researchers were looking for a way to integrate the qualitative richness of small-scale video studies with the representative sampling of large-scale quantitative approaches in cross-national studies. The TIMSS Videotape Classroom Study provided such an opportunity. TIMSS was one of a series of mathematics and science achievement studies conducted under the auspices of the International Association for the Evaluation of Educational Achievement. TIMSS tested and gathered contextual data from students in 45 countries at three age levels. Funded by the National Center for Education Statistics (NCES) of the U.S. Department of Education, the TIMSS Videotape Classroom Study had the goal of clarifying some of the contextual factors that might help explain differences in achievement. TIMSS was the “first large-scale study to collect videotaped records of classroom instruction in the mathematics classrooms in different countries” and the first “to attempt observation of instructional practices in a nationally representative sample of students within the United States” (Stigler et al., 1999:2). The TIMSS Videotape Classroom Study drew from a randomly selected subsample of German, Japanese, and U.S. eighth-grade mathematics classrooms already participating in TIMSS; it used a national probability sample from each of the three countries to create a comparative picture of eighth-grade mathematics teaching. In the United States, researchers also planned to examine the effects of reform policies on U.S. mathematics teaching practices (Stigler, Gallimore, and Hiebert, 2000). The work on the TIMSS Videotape Classroom Study

OCR for page 1
Page 7 BOX 1 Using Videotapes as Cues for Reflective Thinking In the Preschool in Three Cultures Project, Joseph Tobin, David Wu, and Dana Davidson (1989) used video technology as a tool for analyzing the cultural meanings of preschool in Japan, China, and the United States. In their study, videotapes were used not as data, but as cues for reflection. Tobin, Wu, and Davidson videotaped days in one preschool in each culture and then edited the tapes down to 20 minutes. These videotapes became cues for interviews they conducted with the classroom teachers. They showed the teachers the videotape of their classroom and asked them to explain the thinking behind their actions. To address the question of typicality, they showed the videotapes to teachers, administrators, and parents associated with six other preschools in each country, asking them to describe their reactions to the practices on the videotape. Another feature of their method is that they asked informants in Japan, China, and the United States to comment on the videotapes made in all three countries. This method produced understanding of some very interesting findings, including, for example: Japanese teachers’ tendency to hold back from intervening in children’s disputes; Chinese teachers correcting the over-indulgence that single children receive at home; and American teachers teaching young children to express their feelings in words. led to important breakthroughs that have earned video studies a new place in international comparative studies. The techniques developed for digitizing and coding marked a major advance in the use of video technology as a research tool. Researchers found that combining quantitative and qualitative analyses allowed a more comprehensive examination of classroom practice across cultures. “Quantitative coding is necessary to validate insights gained from close qualitative analysis. . . . . On the other hand, qualitative descriptions are essential because they lend substance and coherence to the results of quantitative coding” (Stigler, Gallimore, and Hiebert, 2000:95). Research in video ethnography continues to stimulate new technology in the storing, coding, and sharing of video images. Ricki Goldman-Segall, at the Multimedia Ethnographic Research Lab (MERLin) at the University of British Columbia, has been developing tools for video analysis and annotation on the Internet. “Web Constellations is the first server-side, Web-based database system designed to enable a community of researchers to catalog, describe, and meaningfully organize data accessible on the Web” (Goldman-Segall, 1998:145). Using this technology, Goldman-Segall has posted videotaped data on the Internet from her comparative study of computer cultures. In the study, Goldman-Segall used video technology to examine the influence of computers on elementary and middle school students’ understanding of their own thinking as they explore science. Her web site allows visitors to view the video images and to discuss the nature of

OCR for page 1
Page 8 teaching and learning through on-line communication. Goldman-Segall’s work exemplifies the rapid innovations in video technology in the last decade and its influence on ethnography as a research tool. International video technology offers a number of important potential benefits to scholars, practitioners, and policy makers interested in educational research and practice. It also raises a number of practical and methodological issues about the early planning stages of video research. This section of the report describes the primary benefits of—and caveats associated with—using video technology in international comparative studies in education. POWER OF AN IMAGE Early in the workshop discussions, participants focused on a topic that seems almost self-evident: the compelling nature of visual images themselves is the prime advantage of video technology. James Hiebert, Catherine Lewis, and Frederick Erickson helped workshop participants explore some of the reasons video images are so powerful and the uses and misuses of that power. All three agreed that videotapes capture more of what happens in a classroom than do other forms of data collection, such as self-reported data from teachers collected through interviews or questionnaires. Erickson explained that the difficulty in collecting valid data on classroom practice from teachers is that no teacher can take in the myriad interactions in his or her classes: “[Teachers] can only report very globally their recollections about the ‘how’ of classroom practice.” Erickson asserted that the video record serves as a “resource for the illustration of instructional and learning behaviors through an audiovisual real-time record of the real-time enactment of those behaviors.” Videotaped images provide both a lens through which to view classrooms and a tool to develop a shared language with which observers can discuss what they see. Of particular importance is the value of this shared language in building a common professional language of teaching. The problem of defining “good” teaching is extremely complex, but the difficulty of finding words to refer to a specific aspect of teaching and being perfectly understood exacerbates the problem. Video technology, especially video from another country, with a mix of familiar and unfamiliar practices, heightens the possibilities of providing fresh insights. By providing an audiovisual record of countless teaching approaches, international video studies provide an audiovisual glossary of teaching tools, strategies, skills, styles, pitfalls, and mistakes. For example, a conversation in which a teacher’s videotaped actions can be freeze-framed and viewed repeatedly can help to establish some common understandings about and terms to describe classroom practices. Such a common professional language of teaching would be very useful to both practitioners and researchers in minimizing linguistic differences in describing observed

OCR for page 1
Page 9 behaviors and focusing instead on significant classroom practice; see Box 2. Such an addition to the discourse has specific benefits for crosscultural and cross-national work. The practical problem of describing classroom instruction in words is further exacerbated when these interactions take place in a foreign language and an unfamiliar culture. While videotape does not eliminate the need for translation and discussion of a classroom’s cultural context, visual images provide a reference point that can make cross-cultural differences and similarities more readily apparent. James Hiebert offered an example from the TIMSS-Repeat (TIMSS-R) Video Study. TIMSS-R was conducted in 1999 to measure the mathematics and science achievement of eighthgrade students (ages 13 and 14) and to measure trends in mathematics and science achievement in countries that participated in TIMSS. The TIMSS-R Video Study videotaped national samples of mathematics and science teaching in seven countries. Research collaborators from the participating countries met to develop a coding scheme to interpret the teaching practices in the videos and to compare practices across countries. The international group developed a coding scheme to analyze four dimensions of classroom instruction compa- BOX 2 Developing Shared Language of Practice Through Video Analysis In Learning from Mentors, a comparative study conducted through the National Center for Research on Teacher Learn ing, Lynn Paine and colleagues examine mentoring practices for novice teachers in China, the United Kingdom, and the United States, and how novice learning is shaped by institutional and social contexts. Videotaped lessons and mentoring sessions in one site are shared with mentors in each of the other sites. The results of using this process have proven useful in unexpected ways. One videotape that showed a Shanghai beginning elementary teacher teaching a lesson and then debriefing with her mentor afterward drew vehement responses from the majority of U.S. mentors. The U.S. mentoring teachers voiced criticism about the seemingly intrusive approach used in China to show a novice teacher how to teach particular content. Discussion about the video sessions stimulated the U.S. mentors to examine their own assumptions about mentoring practices, which they had not been able to clearly articulate to researchers in initial data collection. Analysis of concrete and unfamiliar practices captured on video helped researchers and mentors create a common understanding about theretofore vague generalities, such as mentors playing the role as “guides” and their efforts to “support” novices’ learning. The use of video technology for discussion about mentoring stimulated mentors to examine what they really meant by guidance and support and how they believed such guidance and support are best provided. The chance to examine practices concretely, but to do so at some distance from one’s own practice, afforded both participants and researchers insights into unexamined assumptions about learning to teach.

OCR for page 1
Page 10rable across countries: content, actions of participants, discourse, and climate. Hiebert highlighted an example from a German-speaking community in Switzerland, where some mathematics lessons are devoted to an activity eventually labeled “working through.” Prior to the video study, the nature of this activity had been difficult to translate. By looking at how the four dimensions were coded and comparing them to lesson activities in other countries, however, the research group eventually came to a common understanding of “working through.” Hiebert explains: “Video data permit researchers from many countries to collaborate around concrete examples of classroom processes and to sort out superficial and linguistic differences from significant classroom practice differences.” This example also illustrates another point that several workshop participants emphasized: the importance of truly collaborative interaction between international partners. Video technology creates an opportunity for researchers from diverse perspectives to examine and interpret concrete examples of teaching behaviors in a way that is typically not possible through more traditional forms of data collection in cross-national or comparative research. This type of collaboration can enhance communication among researchers about different methods of video analysis. Several workshop participants expanded this point: the research community needs to actively create international participation in every phase of a study to avoid a single nationcentered perspective. As Hiebert and others noted, each member of an international research team should be considered a valuable resource and be committed to sharing the meanings they make of videos from their country and other countries. A second potential advantage of video technology in international and cross-cultural research is that videotapes allow viewers to witness a volume and variety of classroom lessons that may not be possible any other way, and to see them in juxtapositions that can generate valuable insights. For example, depending on the nature of the material that has been archived, a researcher can, in the span of a day or a month and without leaving home, become immersed in the elementary mathematics teaching of classrooms thousands of miles away. Alternatively, the researcher could examine treatments of a particular concept, age level, or other element across numerous countries. For U.S. researchers, policy makers, and educators, these external reference points allow for deeper insights into U.S. teaching practices, both in terms of providing new ideas and in creating greater clarity about their own practices. Hiebert maintains that video allows U.S. educators to “[h]old a mirror alongside contrasting pictures from other countries to see our own practice more clearly; [and] uncover concrete examples of alternative practices not imagined within our own culture.” Is It Too Powerful? The convenience with which videotapes can be shared and reviewed, however remarkable, relates to what has been perhaps the

OCR for page 1
Page 11 most prevalent concern expressed about video research. Viewing even one videotaped lesson is a very powerful experience, sometimes deceptively so. Seeing one—or ten—mathematics lessons in Japanese classrooms cannot transform an observer into an expert on teaching in Japan, but it may make him or her feel like one. An observer who did not already have considerable understanding of Japanese culture and the structure of education in Japan could easily make unfounded and possibly incorrect inferences about the lessons, the teachers, the students and what they learned, the schools, and many other things. Joseph Tobin referred to this exaggerated sense of confidence about what observers think that they know about a classroom after they have observed only a few videotapes as the “problem of video seduction or verisimilitude.” Tobin pointed out that as a society “we are gullible watchers of video,” that audiences have a tendency to give themselves over to the authority of the researchers and their video data. Heidi Ross illustrated this point in her description of a colleague’s interpretation of the TIMSS Videotape Classroom Study: [The TIMSS videotape] confirmed everything he believes he knows about why many American students fear and are not generally high achievers in mathematics. . . . . The vivid and seemingly bounded lessons [that videotapes] convey can easily obscure the complexity of teaching and learning contexts, and be used to solidify, rather than open to sensitive investigation, previously held assumptions about learning and teaching. Erickson also noted that relatively little research has explored people’s perceptions of videotapes or, indeed, of other means of recorded events, such as written narratives or field notes. Addressing the problem of overgeneralization remains a major issue in broadening the use of video technology. How Important Is Contextual Information? The compelling nature of video images also raises another question: Can videotapes stand alone as data, independent of any contextual information? Erickson argued that the videos record behavior but not the meaning behind that behavior. Information about meaning lies in understanding the thought processes that the teacher uses (Erickson, 1986). For this reason, David Berliner and others argued that to make sense of videotaped images requires contextual information. “I learned very quickly that, unless I understood the purposes of teachers, I really didn’t understand the behavior I was coding,” Berliner noted. The significance of contextual information becomes greater in international video research in which data are collected about unfamiliar practices and cultural meanings are less well known and not explicit. Catherine Lewis agreed that collecting a variety of data to supplement videotapes is crucial:

OCR for page 1
Page 18 that can track or zoom in on particular interactions or respond in other ways to the idiosyncrasies of a particular lesson. No camera, of course, can record teachers’ intentions or students’ real-time understanding, reactions, or learning. Moreover, decisions about placing and handling the camera may reflect unconscious assumptions about what will happen during the lesson—an expectation that the teacher will remain in the front of the room, for example—and may indeed subtly influence the actions of teachers or students. These framing decisions are complex in international studies because of cultural, political, and gender-based differences. Heidi Ross and Ricki Goldman-Segall reinforced the message that the complexity of these framing decisions in international video has implications for what viewers see and how they make sense of it. While a variety of factors will undoubtedly influence every research design, the inclusion of contextual material will enhance the usefulness of video data that is archived or intended for use by multiple researchers. PRIVACY AND CONFIDENTIALITY Videotapes are easy to share; indeed, many interested in this methodology speak enthusiastically about the possibility of using the Internet as a means of sharing digitized footage that can be used by researchers anywhere for a wide variety of purposes. This possibility leads to the question of obtaining informed consent from the participants of such research. Assuming that anonymity cannot be guaranteed and that the videotapes will be placed in archives, how can researchers protect research participants? The difficulties surrounding informed consent present immediate and pressing practical obstacles for researchers already involved in international video research projects who must identify statistically sound samples of participants and obtain their cooperation in a fair manner. Issues of privacy are connected to deep cultural meanings and assumptions about public teaching. Several workshop participants spoke about the variability across cultures of ideas about privacy and the social context in which teachers’ performance is viewed. The presence of a video camera in the classroom may have a very different meaning for the students, teachers, and administrators in one country than in another. Indeed, the sampling of Japanese classrooms for the TIMSS Videotape Classroom Study was complicated in part by the desire of education officials to put the teachers they considered the very best in the spotlight. In some cultures, teachers might be reluctant to be taped or very uncomfortable in front of the camera, while in others taping might be commonplace. The possible consequences of judgments about teachers’ performance will vary by country as well—and likely affect teachers’ views about being taped—but disentangling cultural differences from individual variation is often tricky. Government agencies, some of which have faced distrust from citizens because of inappropriate data collection efforts in the past,

OCR for page 1
Page 19 are now major funders of video research and remain particularly concerned about setting standards for professional conduct in this area. The public release of the TIMSS videotapes in 1998 offers insight into issues of confidentiality and consent. TIMSS researchers planned to use videotapes to help communicate the results of the study to the general public. However, because the survey participants had been guaranteed confidentiality, the actual survey footage could not be released to the public. Consequently, another set of videos was filmed for use in public discussions at such forums as PTA meetings, professional conferences, and teacher training events. The participants of these tapes provided explicit permission for this dissemination, but as the TIMSS researchers noted: “It is not easy to find teachers who will agree to being videotaped for public viewing” (Stigler et al., 1999:14). Many of the workshop participants noted that discussions of publicly released videotapes have often focused on the negative and, at times, deteriorated into teacher blaming. John Frederiksen described this tendency toward criticism as “normative negativity.” The viewing public focuses on what appears to be wrong in the lessons shown and not on what appears to be working effectively. Participants discussed the TIMSS Videotape Classroom Study as an example of how normative negativity might influence teachers’ willingness to allow a video record of their teaching to be made public. A major theme in the discussion of the TIMSS results was the perceived inadequacy of U.S. schools, and of U.S. teachers in particular. As Joseph Tobin summarized, “The study is so thick with the sense of despair about the quality of American math education that, of course, there are a lot of problems of confidentiality especially for the Americans.” Public discussions that lead to comparisons between teachers, with some teachers’ performance being cited as examples of inferior practice, could also have a chilling effect on teachers. Because evaluation and judgment have become an almost inevitable aspect in video research on classrooms, the traditional roles of researchers and research participants have been somewhat altered. Tobin, Magdalene Lampert, and others highlighted how video technology as a tool for observation brings some troubling connotations from other contexts. Video cameras are used for surveillance in stores, banks, and even prisons. They are sometimes used by parents to monitor the performance of their children’s day care centers or nannies. These uses of video technology all place subjects in a vulnerable position because the observer is in a position to intervene to prevent a bad outcome, and in possession of legal evidence of any actionable wrongdoing. When calls for school accountability often mean a direct connection between test scores and job security for administrators and teachers and when a misunderstood phrase can lead to disciplinary action, it should come as no surprise that videotaping in classrooms seems ominous to some. Practical solutions for addressing the tension between protecting

OCR for page 1
Page 20 the privacy of participants, scholarly access, and research need to be developed. For example, facial features might be disguised, although current technology renders doing so difficult for large volumes of tape. Furthermore, important information would surely be lost. Different levels of confidentiality might be guaranteed for videos collected for different purposes, ranging from very strict measures for tapes to be posted on the Internet for general public access, to very limited measures for tapes to which only registered scholars would have access. Permission for broader dissemination may result in lower participation by research participants, with implications for sampling size and representativeness. Lack of informed consent from one or two participants in a large class also creates complications for video data collection. Restrictions placed on video data by university Institutional Review Boards (IRBs) charged with protecting research participants may also hinder researchers’ use of video. In some extreme cases, in the interest of confidentiality, IRBs might require that videos be destroyed rather than archived. Clearly, these issues are very complex and require continuing attention from educators and ethicists, as well as researchers who have used video technology for a variety of purposes, to develop guidelines for the research community. Issues of confidentiality are further complicated in the case of international video because of cross-cultural differences in perceptions about privacy and teaching, as well as by the potential power of international video to reach and affect large and disparate audiences. PROFESSIONAL DEVELOPMENT In addition to being a useful methodological tool for research, video technology can also support and improve the practice of teaching. As noted by many workshop participants, videotaped lessons have proven very useful in stimulating conversations about teaching. Videotapes can be used to help teachers to imagine new approaches, to rethink what they might otherwise take for granted, to consider the pros and cons of different approaches, and, in general, to reflect on their practice in new ways. Videotaped footage from cross-national studies in education is particularly useful for provoking reflections on practice and prompting new ways of envisioning education. Frederick Erickson observed: Teaching has been such a secret local practice, that we always assume that what we have figured out how to do is the way it has to be. Seeing something that is really very different from far away can open up the possibility that there are lots of different roads to Rome. That opens up, I think, readiness for inquiry to change that can be very powerful. He noted that looking at the practice of a teacher from the next classroom can lead to new insights about one’s own practice. Looking at the practice of a teacher from another country can cause an even more

OCR for page 1
Page 21 profound rethinking of assumptions. Erickson’s insights focus attention on a question for international comparative education research: How does the effect of viewing a teacher from another culture differ from viewing a teacher from one’s own culture? The potential for international videotapes to stretch people’s thinking about familiar topics is only beginning to be explored. Further research into what and how people learn from watching international videos will help guide teacher educators in identifying the best uses of video technology. Workshop participants described the ways in which they have used video to improve teacher professional development to shed light on current practices and understanding. The flexibility of the new technology has clearly inspired considerable creative thinking about what happens in classrooms and, in BICSE’s view, has helped focus both the public and the research and policy communities on teaching in a way that seems both novel and constructive. Lampert observed that the TIMSS videos have been influential in helping people see teaching as a process that can be studied and have helped educators isolate some of its component parts: “There is a lot to suggest that teachers don’t teach on the basis of what they see happening in their classrooms. They are not reflective practitioners, on the whole.” She identified several applications of video technology that have potential for teacher development to help teachers learn and improve through that reflective thinking: learning a particular teaching technique; using evidence to analyze the relationship between particular teaching and evidence of learning; exposing teachers to new ideas, alternatives, or inspiration; and using videotapes to discuss and understand variations in teaching practice, to establish a more precise language of teaching that goes beyond simple characterizations of “good” and “bad.” Lampert argued that developing a shared professional language about teaching through the interpretation of video would constitute professional development, “. . . in the sense that it would enable teaching to become more of a practice-based profession.” John Frederiksen also provided insights into using reflection to transform teaching into a practice-based profession. He discussed ways that teachers can improve their cognitive and social skills in the process of viewing and interpreting video data collaboratively. He described a model in which teachers view and interpret classroom video coverage together in order to share and discuss ideas about instructional practices—video clubs. Eventually, the teachers develop a shared set of criteria for evaluating teacher effectiveness in accomplishing instructional goals, such as “mathematical thinking is going on” or “participants in the class are showing mutual respect.” Such criteria are not used as a basis for judging appropriate teaching be

OCR for page 1
Page 22 haviors; “[r]ather, they must facilitate recognizing in a video when teaching moves are meeting particular teaching goals in the particular teaching situations shown in the video,” Frederiksen said. By focusing on function, he argued, teachers are better able to concentrate on how instructional goals are being achieved rather than on specific forms of classroom organization or pedagogical strategies. Teachers are thus engaged in the process of reflecting about the practices of others, as well as their own practices, and encouraged to investigate the extent to which they are achieving their own goals. Frederiksen noted that this reflective process fosters important professional skills, such as an “evaluative judgment” of efficacy and “an inquiry attitude towards classroom teaching, innovation, and changing of one’s practice.” This model of video interpretation, he argued, can help teachers create a language of practice that directs attention to a broad range of teaching goals and methods for learning. Teachers can view and discuss many styles and situations and encounter practices they can experiment with in their own classrooms. Frederiksen described an example of how video clubs “proved to be a powerful catalyst for improving teaching practice” in his research on video portfolio assessment (Frederiksen et al., 1998:276). A member of one of the clubs gave a video presentation on her use of collaborative groups in mathematics. Her approach was very different from the teacher-centered classrooms that the rest of the video club members used. As a result of this meeting, three members took the initiative to change their teaching styles to incorporate more group work and then shared videotapes of themselves using this approach in subsequent meetings. “These club members were in essence carrying out design experiments . . . in their classrooms, using the video club as a research group to help them interpret the outcomes of their experiments” (Frederiksen et al., 1998:277). Drew Gitomer said that one of the challenges of cross-national studies is encouraging teachers to see the relevance of classroom practices from another country to their own professional experience. Teachers can easily dismiss research findings if the context of the teaching depicted in a video is very different from their own. High school teachers may consider portrayals of elementary level instruction irrelevant to their own work; teachers in rural schools may not see the relevance of videos from urban districts; and U.S. teachers may see little relevance in videotapes of lessons in Germany or Japan. Heidi Ross explained the value in using the sometimes radical differences across cultures that are evident in international video research as a catalyst for reflective thinking. She argued for the value of using classroom images from other countries to begin discussions and raise awareness among preservice teachers about the complexity of teaching practice. International video research can help them develop a critical understanding about how they have been socialized and how that socialization will affect what they do in their classrooms once they become teachers. Videotapes of practices in other countries

OCR for page 1
Page 23 can help U.S. teachers explore cultural values and what is important to them as teachers in comparison with what might be important to teachers in other cultures. Videotapes can also be used to present models of effective practice for the purpose of asking teachers to model their own practice on it. However, the board believes that using international videotapes to present exemplary practice and train teachers to adopt it is a particularly problematic enterprise that deserves more careful scrutiny than it has received thus far. At least two major drawbacks are evident. First, using videotapes to suggest specific changes in teaching practice is a higher stakes enterprise than simply using videotape as a point of departure for discussion. One of the risks is creating a misconception about a standard that does not take into account other contextual factors affecting teacher practice. Second, identifying the precise elements of teaching that should be imitated is complicated; specifically, it requires the establishment of an empirical link between a particular teaching method or approach and improvements in student learning. International videotape studies have yet to make this link. LINKS BETWEEN ACHIEVEMENT AND TEACHING PRACTICES Participants in the workshop agreed that empirical links between specific teacher practices demonstrated in videotaped lessons and learning outcomes have not yet been established. TIMSS serves as an example, since many researchers were interested in linking observations made in the TIMSS Videotape Classroom Study with TIMSS achievement scores. James Stigler explained in his written contribution to the workshop how the design of the TIMSS Videotape Classroom Study “precluded any causal inferences on the relationship between teaching and learning, either at the level of nation or at the level of teacher/class” for several reasons. At the national level, the sample size of countries in the TIMSS videotape study was three, “and the potential causes of achievement differences are many.” At the teacher level, researchers videotaped only one lesson per teacher, “which does not give a reliable indication of any particular teacher’s practice.” Stigler also pointed out that even if multiple lessons by the same teacher were videotaped, this approach measures teaching and achievement at just one point in time and does not account for students’ previous learning experiences. Participants at the workshop differed over whether such links between teaching and learning are likely to be established in the foreseeable future. From BICSE’s perspective, resolving these differences would require a large-scale study, incorporating a large sample of teachers and many background variables to capture their diversity. Deep understanding of the classroom interactions studied and the cultural contexts in which the lessons were conducted are just two of

OCR for page 1
Page 24 the components that would be necessary to make such a link persuasive. Workshop participants noted several issues that a persuasive study would probably have to address. Some noted that achievement data, at least in the United States, are generally relatively unstable; ideally, multiple measures of achievement should be used to establish valid links to instructional practice. One might videotape teachers teaching a single common lesson in various ways and compare the learning outcomes. Other participants, however, suggested that it would be difficult to distinguish the learning attributable to teacher practice from the learning attributable to previous learning, motivation, and other factors students bring to the classroom. While some participants argued that being able to characterize the achievement outcomes of the teaching that is taped is critical to making use of the observations, others noted that the link itself may be a misleading goal. For Joseph Tobin, for example, the comparison of achievement scores may have little relevance to the insights about teaching he would seek from comparative videos since variations in learning have so many other sources. Catherine Lewis seconded that view, noting that “whatever it is that’s causing achievement may not be represented in videos” and that deep ethnographic descriptions are necessary to ascertain the ingredients for learning in a particular setting. Ray McDermott noted that a focus on achievement should include how different countries define achievement, given different cultural contexts. The Japanese definition of achievement might be very different from the U.S. definition, so it is important to examine the cultural organization of what achievement is in cross-national comparative studies in education. Other participants suggested that cross-national video studies might best be used to generate hypotheses about effects of teaching on learning, while large-scale video studies within countries might be better suited to testing the hypotheses that are generated through cross-national comparisons. CONCLUSIONS AND RECOMMENDATIONS The Board on International Comparative Studies in Education concludes that international videotapes of students and teaching are a powerful tool for learning about and improving education. Videotapes of classrooms in other countries are particularly powerful in creating opportunities for learning from cross-national and cross-cultural comparisons. Video images of educational settings from around the world stimulate reflection and expand understanding of the potential range of instructional practices. Despite its novelty and its power to capture attention, however, this technology is a tool, not an end in itself. Researchers continue to grapple with complex questions regarding both the methodology and practical applications of this tool. Many such questions are not yet resolved, among them:

OCR for page 1
Page 25 How feasible will it be for future researchers to return to archived videotapes and recode them according to new schemes? How will the privacy of research participants be protected? What are the possibilities for using videotape data to link achievement to instructional practices? BICSE offers four recommendations to guide researchers, funding agencies, and policy makers in the judicious application of video technology as a tool for future international comparative studies. Recommendation 1: The international comparative education research community should pursue projects that appropriately use video technology as a research tool. Such research will help scholars build a body of work that can contribute fundamental new understandings of educational practices, while at the same time resolving some of the important methodological challenges discussed in this report. Recommendation 2: The international comparative education research community should support not only large-scale studies that make use of video technology, such as TIMSS, but also other kinds of video-based research. Research studies with a variety of sizes, goals, and methodologies can benefit from the application of video technology in important ways that have the potential to stimulate progress in both methodological and substantive issues. Recommendation 3: The international comparative education research community should undertake initiatives, such as the support of a working group, to help clarify and develop solutions to the privacy and confidentiality issues in using video technology in such research. The very nature of video technology creates problems for and challenges to confidentiality that cannot be easily handled by simple extrapolation from existing procedures for other research methods. Thus, serious and focused consideration of confidentiality issues in video research, especially in international settings, is needed to develop creative solutions and to foster discussion and consensus building around such solutions. Recommendation 4: The international comparative education research community should undertake initiatives, such as the support of a working group, to explore the creation of a video archive or archives for international comparative research in education. Video technology can be of significant benefit in expanding the accessibility and application of comparative research and in serving as a unique historical resource. Given the substantial costs associated with both international comparative education research and video

OCR for page 1
Page 26 technology, wide distribution and archiving will contribute to its cost effectiveness. The board hopes that this powerful technology will continue to be harnessed for research that informs international comparative understanding of education. The board sees a bright future for research that capitalizes on the strengths of this important tool while working within its limitations. REFERENCES Bateson, G., and M. Mead 1952 Bathing Babies in Three Cultures . Videocassette . University Park, PA : Pennsylvania State University Audiovisual Services . Bogdan, R., and S. Biklen 1992 Qualitative Research for Education: An Introduction to Theory and Methods . 2nd ed. Boston, MA : Allyn and Bacon . Campbell, D. 1961 The mutual methodological relevance of anthropology and psychology. In F. Hsu, ed., Psychological Anthropology: Approaches to Culture and Personality ( pp. 333-352 ). Homewood, IL : The Dorsey Press, Inc. de Brigard, E. 1995 The history of ethnographic film. In P. Hockings, ed., Principles of Visual Anthropology . ( pp. 13-43). : New York : Mouton de Gruyter . Erickson, F. 1986 Qualitative methods in research on teaching. In M. Wittrock, ed., Handbook of Research on Teaching . 3rd ed. ( pp. 119-161 ). New York, NY: MacMillan Publishing Company . 1992 Ethnographic microanalysis of interaction. In M. LeCompte, W. Millroy, and J. Preissle, eds., The Handbook of Qualitative Research in Education ( pp. 201-225 ). New York, NY : Academic Press/Harcourt Brace . Frederiksen, J., M. Sipusic, M. Sherin, and E. Wolfe 1998 Video portfolio assessment: Creating a framework for viewing the functions of teaching. Educational Assessment 5(4) : 225-297 . Goldman-Segall, R. 1998 Points of Viewing Children’s Thinking: A Digital Ethnographer’s Journey . Mahwah, NJ : Lawrence Erlbaum Associates . Hall, R. 2000 Videorecording as theory. In A.E. Kelly and R.A. Lesh, eds., Handbook of Research Design in Mathematics and Science Education ( pp. 647-664 ). Mahwah, NJ : Lawrence Erlbaum Associates . Henley, P. 1998 Film-making and ethnographic research. In J. Prosser, ed., Image-based Research: A Sourcebook for Qualitative Researchers ( pp. 42-59). Bristol, PA : Falmer Press, Taylor & Francis, Inc. Jordan, B., and A. Henderson 1995 Interaction analysis: Foundations and practice. The Journal of the Learning Sciences 4(1) : 39-103 . Linn, M., C. Lewis, I. Tsuchida, and N. Songer 2000 Beyond fourth-grade science: Why do U.S. and Japanese students diverge? Educational Researcher 29(3) : 4-14 . McDermott, R., and D. Roth 1978 The social organization of behavior: Interactional approaches. Annual Review of Anthropology 7 : 321-345 .

OCR for page 1
Page 27 National Research Council 1999 Next Steps for TIMSS: Directions for Secondary Analysis . Board on International Comparative Studies in Education , A. Beatty, L.W. Paine, and F.O. Ramirez, eds. Board on Testing and Assessment. Commission on Behavioral and Social Sciences and Education . Washington, DC : National Academy Press . Spindler, G., and L. Spindler 1992 Cultural process and ethnography: an anthropological perspective. In M. LeCompte, W. Millroy, and J. Preissle, eds., The Handbook of Qualitative Research in Education ( pp. 53-92 ). New York, NY : Academic Press/Harcourt Brace . Stigler, J., R. Gallimore, and J. Hiebert 2000 Using video surveys to compare classrooms and teaching across cultures:Examples and lessons from the TIMSS video studies. Educational Psychologist 35(2) : 87-100 . Stigler, J., P. Gonzales, T. Kawanaka, S. Knoll, and A. Serrano 1999 The TIMSS Videotape Classroom Study: Methods and Findings from an Exploratory Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States . National Center for Education Statistics . Washington, DC : U.S. Department of Education . Taft, L. 1985 Ethnographic research methods. In T. Husen and T. Postlethwaite, eds., The International Encyclopedia of Education: Research and Studies ( Vol. 3, pp. 1729-1733 ). New York : Pergamon Press . Tobin, J., D. Wu, and D. Davidson 1989 Preschool in Three Cultures: Japan, China, and the United States . New Haven, CT : Yale University Press .

OCR for page 1
Page 28